Jump to content
jethro

Add Rotating Offsite Backup to Current Configuration

Recommended Posts

Hi,

We are currently backing up 4-5 machines to a single HD media set. Backups run M-Thurs each week at night (we turn the drive off on weekends to save some wear & tear). The media set is almost 6TB now (across 5 hard drives), and spans 5+ years of incremental backups.

 

But we now would like to have an offsite copy just to be safe (one of the previous drives from our current set went bad while in storage, losing a year's worth of older backups :(. We're thinking we could take a drive/media set home once a week realistically. So just trying to understand the best or most efficient way to set this up.

 

1) Should we create an entirely new media set and set the current scripts to backup to each set (e.g., A & B) for one week, then rotate to the next? This would mean we'd lose at worst a week's worth of backups if something happened.

 

1a) If so, would this new media set just start backing up from the current state and then be incremental going forward, thus not being in-synch with our 5-6 year-old media set? Or is there a way to 'synch' the sets first, so they are at least covering the same backup data & dates?

 

2) Or would it be more efficient to copy the existing media set to a new media set once a week? I can imagine we'd bring the other media set drive(s) in on a Friday and hope it could complete all copying from the week while we were here, then it would go back offsite.

 

2a) What exactly is this type of operation called, and where would I find specifics about setting it up?

 

2b) I'm guessing we'd need enough space to copy the entire media set (6TB) initially, then it would just do incremental, correct? Could this destination set span multiple drives like our current media set??

 

 

Thanks a lot. Let me know if there are better ways to accomplish what we're after that I'm not aware of.

Share this post


Link to post
Share on other sites

As you found, having only a single media set is risky. Having a 6 TB media set with years' worth of data will also make a complete restore (as opposed to retrieving specific files one or two at a time) potentially more cumbersome.

 

We use three media sets. Set C is a single-member disk media set that is groomed regularly and is used to perform daily backups. It's also the primary source for restores, since it's relatively small. Sets A and B are multi-member tape media sets that are rotated on a weekly basis, with the inactive set being taken off-site. We use a Copy Backups script to copy daily from set C to whichever tape set is currently active. (In our scheme, sets A and B could also easily have been multi-member disk media sets, had we chosen to do that.)

 

When media sets A and B become inconveniently large, we perform a New Media backup to each set. You do this by selecting the media action "Start new Media Set" when the backup script is run. The script you use to perform the new media action can be very simple; for example, backing up a small favorite folder. The advantage of a New Media backup is that all of your scripts will automatically change their destinations to the new set; there's no need to edit the scripts manually. Your media sets will also have a clear nomenclature history: for example, if the initial media set is named "Set A," the next one will be named "Set A [001]," and so on.

 

We retain all of our tape media sets in case we ever need to retrieve a file from long ago. The set A series are all kept on-site and the set B series are kept off-site.

 

With the above arrangement, you would lose at most one day's worth of data unless both set C and set A or B (whichever is on-site) were destroyed or became unreadable, in which case you would lose at most a week's worth of data.

  • Like 1

Share this post


Link to post
Share on other sites

Hi, thanks for the reply and example. I can see how this might be a very complete way of backing up, not only for recovering old/overwritten files, but for doing a complete restore as well. In the years of using Retrospect, we have only ever used it to recover files that were overwritten, corrupted, deleted, or lost in some way. But we are aware that catastrophes can happen, and know our setup might be a bit cumbersome or time-consuming to restore from (our file storage drive is a 4-member RAID 5, so 'fairly' safe, though).

 

We'll consider your example, but have to just consider the costs (additional media needed and additional time to mange the system - adding/removing drives at the right time, etc.).

 

- When you say you use a 'Copy Backups' script to copy C to A or B, is there a reason to choose that over a 'Copy Media Set' script? We would want to be sure to have all past backups of files in case they are overwritten or corrupted at a point in time.

 

- When you do a 'New Media' backup for A or B, does that just continue the current incremental backups onto new media? Or does it do a 'fresh start', backing up all of the current files even if versions of them exist on the previous Media Set?

 

We haven't done a new set due to how large the first backup would be (all current files), and concerns about how to search for a specific version of a file which may exist across either of the Media Sets. More insight here might be helpful.

 

- Any other replies on our initial questions??

Share this post


Link to post
Share on other sites

- When you say you use a 'Copy Backups' script to copy C to A or B, is there a reason to choose that over a 'Copy Media Set' script? We would want to be sure to have all past backups of files in case they are overwritten or corrupted at a point in time.

"Copy Media Set" is for when you want to duplicate an entire existing media set. When you're looking at an ongoing process, "Copy Backups" is best. Unless you include parameters in your script that would exclude certain backups, you will copy over all backups.

 

- When you do a 'New Media' backup for A or B, does that just continue the current incremental backups onto new media? Or does it do a 'fresh start', backing up all of the current files even if versions of them exist on the previous Media Set?

A "New Media" backup is a fresh start onto the new media set with the advantage of retaining your existing scripts and schedules.

 

We haven't done a new set due to how large the first backup would be (all current files), and concerns about how to search for a specific version of a file which may exist across either of the Media Sets. More insight here might be helpful.

I presume that "all current files" is smaller than "all versions of any files that were ever backed up," but yes, the first backup to any new media set will typically be much bigger than subsequent incremental backups. Assuming you don't delete the catalog for your previous media set, searching for a specific file across multiple media sets is a simple task. The more files each media set contains, though, the longer the search will take.

  • Like 1

Share this post


Link to post
Share on other sites

For reassurance jethro might first want to view the video linked to in this post, which was created by Mayoff.  Note that it is entitled "Staged Backup with Retrospect for Macintosh", which is exactly what jethro now wants to do—except that the video  shows disk-to-disk-to-tape and he wants disk-to-disk-to-disk.  You will note that Mayoff made the post in a thread that jethro should have some slight familiarity with (insert appropriate smiley here), since he started that thread and made several posts in it last spring.

 

In other words, what jethro wants to do now is just a variant on the disk-to-disk-to-cloud that he wanted to do last spring.  Therefore I suggest that he consider basically following the steps in section C) of this post in the same thread.  All jethro would need to do is replace all mentions of "Cloud" with mentions of "Disk", eliminate mention of Grooming in steps C1) and C2) unless he still wants to cut his off-site backup down to the most recent year, eliminate steps C3) through C5), and not check No Verification in step C6)—he would be doing a Copy Backup to fallible Disk rather than ultra-reliable (thus sayeth Retrospect Inc.) Cloud.  If jethro wants to have both a Media Set A and a Media Set B on off-site Disks, he should repeat steps C1) and C2) to a second portable Disk and change the last sentence in step C6) to alternate the scheduled weekly Copy Backup between Media Set A and Media Set B.

 

In return for all this additional help that he shouldn't really have needed, I'd like jethro to give us in this thread an answer to one question:  Why didn't he implement the disk-to-disk-to-cloud plan he wanted to do last March?  Was it because cloud storage turned out to be too expensive for his monthly budget?  If I point out that Retrospect Mac 13.5—released 14 September—would allow him to maintain a Cloud backup of up to 1TB on Dropbox Pro for only US$99/year, would that change his mind?  Or was it because he realized that, with "seeding" and "restore-to-door" eliminated by all U.S./Canada cloud backup providers except pricey Amazon and Google, initial population to and restores from the cloud would take an unreasonably long time?

Share this post


Link to post
Share on other sites

Thanks for the ongoing help. It's becoming a bit clearer what we need to do.

 

To answer David about why we never followed through with cloud (and are considering an alternative now), it honestly was getting too complicated to figure out how to properly groom, seed, and estimate both current and future storage needs, JUST so we could figure out how much it was going to cost. We didn't want to dive in only to figure out that in the end the monthly costs were just not reasonable or it was too time-consuming.

 

I think we could get one big 6TB drive now ($200-300 total) and just copy our entire media set over, then keep it offsite in a safe environment. We would then only need to update that once a week with what's changed. That, plus maybe another 3-6TB drive when it fills up, would probably last us a few years at our current data rates. And I would hope that with a robust enterprise drive, only being used once a week, that it would be reliable enough (it IS just the backup-backup anyway).

 

-----

 

A) But to get back to original questions if possible, can I get pros/cons of using 2 separate media sets that rotate every other week vs. our current single media set plus a 'Copy Backup' set once a week??

 

B) And still a hair fuzzy on whether to use 'Copy Media Set' vs. 'Copy Backup' script. I believe we would want the main onsite backup and the offsite 'backup-backup' to be identical so one could be swapped for the other in the case of emergency.

 

C) Our current set is almost 6TB and goes back 5+ years. The catalog file for it is 120GB. At what point to we REALLY need to consider doing a 'New Media' backup to 'start over'? We do not use compression, and we really need to have as many iterations of a file, and as far back as possible, available to restore at any time.

 

Thanks!

Share this post


Link to post
Share on other sites

I can answer A:

When you have two rotating media sets, you can (and SHOULD) have one set off site AT ANY TIME.

With a copy, you must have both (all( backups on site at least once a week, risking to lose both (all) backups AND the originals if a fire starts (for instance).

 

So that gets B out of scope, really.

 

I can also comment on C:

I say "right now", before any of the members goes bad. And make sure you make another copy of the entire set before retiring it. Store both sets off site at different locations.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×