Suggestions for upgrading backup system...

[theboyk]

Hello,

Currently I run an alternating backup system (1 set is Tuesday/Thursday and the other Monday/Wednesday/Friday/Saturday/Sunday) allowing me to keep 1 set off-site nightly. Right now this is being done via FireWire from a G5 tower via a FireWire AIT-2 Turbo tape drive & tapes. I've been running this setup for 4+ years now and we've outgrown it (in terms of cost of tapes vs storage capacity), and I have a feeling the drive is on its last leg.

So, I'm wondering, what would people suggest for replacing this system? I will have the option of setting whatever the new system is on either an Xserve (G5) or possibly an Xserve (Intel). Again, I'd like to keep it to portable media (like tape or cartridge) that I can keep off site each night.

Advice would be appreciated!

Thanks,

Kristin.

[rhwalker]

You haven't provided any information regarding the volume of data being backed up or your backup policy or your backup needs. No suggestions are possible.

 

You might want to review this:

What should a good backup policy address?

 

Russ

[theboyk]

The backup policy works on an alternating set/cycle. Set 1 runs nightly on Monday/Wednesday/Friday/Saturday/Sunday and the other set runs nightly on Tuesday/Thursday. This allows me to take the previous days tape off site each night. We require off-site storage for all backups (even though a day behind current work as backups take place post-workday when everyone has gone home/computer are no longer in-use). Each client machine has a "work in progress" folder and this is the folder that gets backed up each night. Once a set reach 4-5 tapes in size, the tapes are archived (ie. not re-used) and a fresh cycle begins with a new set of tapes (new catalog files, etc.). And so the same work that was backed up in the previous cycle doesn't just get backed up again (filling the tapes with things that have already been backed up and not changed), all work that had been previously backed up moves to a different folder (outside of "work in progress" folder) and only new jobs, or older jobs (that aren't closed/complete and moved to an archiving server for archiving on and off site) that are being revised, get put back into the "work in progress" folder to be included in the new cycle. It's a bit of work when a new cycle begins, but it prevents catalog files from getting too big (and thus becoming a huge pain to rebuild if they ever become corrupt, etc.).

 

As for volume being backed up, it's impossible to give a number on a daily basis as it all depends on the projects being worked on (small if web, large of print and really large if video). Mainly though, we work on print. On average though, we're going through 10 (includes both sets, so 5 tapes per set) AIT-2 Turbo tapes (80GB native/208GB compressed per tape) every 3-4 months (costing us just over $600/10 tapes). Now, this backup strategy was put in place when the studio was 1/3 of the size it is now, so back then we'd go almost a year or longer before needing to order new tapes. Now, we're going through about 30 tapes a year. As well, the tape drive is getting old as it's been in constant use for a number of years now.

 

So, I'm looking for a new backup system (in terms of hardware) to transition to. Basically, things that are important are:

 

1. Storage Media Capacity/Cost - it'd be nice to not go through as many tapes as we're going through now. Or, at the very least, the same amount of tapes, but at a cheaper cost.

 

2. Archival Storage Media - since we archive the tapes when a set/cycle is complete (full), we need to be able to store them for future reference (incase we ever need to go back and pull a job — if, for example, all other archived medium have failed, etc.). Basically, a last resort of all jobs ever worked on.

 

3. Portability of Storage Media - Because media is removed from premises each night, the media needs to be portable (especially when we get into multiple tapes, though, depending on new storage capacities, this many not be as big an issue, though they're still need to be portable).

 

And for some more information (updating what I'd posted before), whatever the new system is, it'll be running on a G5 Xserve (not an Intel). Basically, we're putting in a new Xserve (Intel) to replace the old Xserve (G5) and thus the old Xserve (G5) will become a Retrospect Server.

 

So, hopefully this information (which I should have originally posted — sorry!) will be of more help in getting some suggestions!

 

Thanks,

Kristin.

[rhwalker]

Ok. Still, without daily backup volume, it's hard to make a suggestion. Also hard to see how you are getting that much compression (factor of 2.6), because we seem to see about 1.8, but your files may be big and highly compressible, as compared to our many small files, not very compressible.

 

As an average, though, you seem to be doing about 5 tapes @ 208 GB every 3.5 months (to each backup set), let's say about 21 working days per month, or about 1040 GB / 73.5 days = about 14 GB per day, which is a low number (except on your initial backup for each backup set, and you don't indicate the total volume of data in that initial session, so it's hard to guess the length of your backup window). While your exact backup strategy hasn't been specified, I assume that you are doing backups on alternate days to the alternating backup sets, which means that each backup session is backing up two days' worth to each of the sets, which still gives these numbers.

 

I'd suggest that you move to LTO3 (or LTO4) (and also consider an autoloader), if you want to cut your media costs.

 

You indicate that you are paying $60 per AIT2 tape, which seems about right. That's $0.29 cents per GB using your compression numbers.

 

LTO3 tapes (400 GB uncompressed) are about $40 each. With your compression ratio, that would give about 1040 GB per tape, or just under 4 cents per GB. At your current rate, with two backup sets, you would need two tapes every 3.5 months, or $80 as compared to $600 every 3.5 months.

 

Your backup window would also be much shorter because LTO3 is faster than AIT2, which also would shorten the restore time.

 

Physically, the tapes are about the same size.

 

You will also need a SCSI HBA card for the Xserve G5 handle the drive. Hopefully, you have a free slot (the Xserve G5 only has two PCI-X slots, and competition for those two slots can include the Apple Hardware RAID card, a graphics card, a FiberChannel card, a network card, etc.). The only SCSI card to consider is the ATTO UL4D; others aren't supported or don't work.

 

Do the numbers and figure out how long it would take for such an investment in new hardware to pay off.

 

Tape seems to make sense with your data amounts and restore times.

 

Oh, and do consider getting an autoloader and barcoding your tapes. It will change your life. Really.

 

Russ

[theboyk]

Thanks for the info! I'd been looking at LTO drives as a solution, so very nice to get confirmation on that!!!

 

And by no means was I saying we were actually getting that kind of compression on the tapes, I was just listing the min/max — we're defintely on the min side (80GB) as all the files we work with are generally already compressed (compressed TIFs, JPGs, etc.). The only time we really see a benefit in the compression (tape) is with video. Either way though, the LTO vs AIT is cost-effective.

 

And I have a PCI-X slot free in the G5, so that's perfect!!!

 

Now, I'm coming LTO with very basic knowledge — is there one brand that's better than another, etc.? Anything you could suggest?

 

Thanks again, this has been a huge help!

Kristin.

[rhwalker]

Now, I'm coming LTO with very basic knowledge — is there one brand that's better than another, etc.? Anything you could suggest?

They are pretty much commodities except on the subject of Mac support. I wouldn't advise getting one from a vendor that didn't have a diagnostic that runs on the Mac, and that provides a means of updating firmware using a Mac.

 

Tandberg recently bought Exabyte, and Exabyte has quite a few LTO drives and autoloaders. Exabyte's support has been very good for us, and they have diagnostics that run on the Mac (vxatool for their VXA drives, ltotool for the LTO drives, etc.) and which also update the firmware. They have also done a good job of releasing firmware updates, at least for our VXA-2 drive and autoloader. That's another possibility for you to consider (VXA-2 or VXA-3), but the VXA tapes are more expensive (about $70 for rated 160 GB compressed VXA-2, 320 GB compressed VXA-3). We see less than the "marketing speak" rated compression amounts - about 115 to 120 GB using Retrospect on a VXA-2 tape, which is rated at 160 GB, with our files and data.

 

Some people reported DOA VXA-2 and VXA-3 units from Exabyte, which, of course, Exabyte swapped out, but ours has always worked OK.

 

Russ

[theboyk]

Oh, and do consider getting an autoloader and barcoding your tapes. It will change your life. Really.

 

Any chance you could give me some ammo on why the auto-loader would benefit vs manual loading? The way I see it, with auto-loading, the device could have blank tapes on-hand for when a tape fill mid-way through a backup session (and there's no one here to change the tape)? How else would it come into play?

 

Also, in regards to the barcoding — how does that work? Is the reading of barcodes built into the LTO autoloader unit? Are they just regular barcodes we'd produce through barcode software/printed to labels on a laser printer?

 

Thanks again!

Kristin.

[rhwalker]

Any chance you could give me some ammo on why the auto-loader would benefit vs manual loading? The way I see it, with auto-loading, the device could have blank tapes on-hand for when a tape fill mid-way through a backup session (and there's no one here to change the tape)? How else would it come into play?

It would also let you have multiple backup sets in the autoloader for changing from one to the other. For example, rather than having two backup sets, you might have three, one offsite and two in the autoloader. That way, if the person assigned to swap out the tapes is sick or forgets one day, the backups still happen to the rotating sets. Extends naturally to more backup sets, and you can also move to next backup set in the sequence ("New Media") on a schedule.

 

What we do is to have alternating backup sets that are sequence [001], [002], etc., that automatically move to the next one in the sequence, similar to what you have, never reusing tapes. The past few N sequences from one are kept in the autoloader, other set is taken offsite always. That way, even if doing a remote VPN into our network from home, we can restore past files if needed.

 

Also, in regards to the barcoding — how does that work? Is the reading of barcodes built into the LTO autoloader unit? Are they just regular barcodes we'd produce through barcode software/printed to labels on a laser printer?

Most autoloaders have a barcode reader built in, usually standard, that reads the barcode on each tape cartridge when it is inserted. If you get an autoloader with an ethernet admin interface (that's what we have on our Exabyte VXA-2 1x10 1u autoloader), you can also see what's in the autoloader and where, from a remote network connection. Very useful if you are debugging Retrospect issues, and if you have to interact with someone over the phone from a remote location.

 

As to the barcodes themselves, well, yes, I guess that you could print them up yourself, but really, everyone just buys sheets of them preprinted to your specification. They are very cheap. The vendor we use is Tri-Optic, probably the standard vendor that has been around a long time, and you just enter the specifics of how you want the barcode printed, and they send you the sheets. Good customer service, good price.

Tri-Optic barcode labels

 

We have our barcodes printed to match the member, Backup set name, and sequence that Retrospect uses:

 Barcode        Backup Set              Tape Member Name
1-A 0001         VXA Set A [001]      1-VXA Set A [001]
2-A 0001         VXA Set A [001]      2-VXA Set A [001]
3-A 0001         VXA Set A [001]      3-VXA Set A [001]
    . . . .
1-A 0002         VXA Set A [002]      1-VXA Set A [002]
2-A 0002         VXA Set A [002]      2-VXA Set A [002]
3-A 0002         VXA Set A [002]      3-VXA Set A [002]
    . . . .
1-B 0001         VXA Set B [001]      1-VXA Set B [001]
2-B 0001         VXA Set B [001]      2-VXA Set B [001]
3-B 0001         VXA Set B [001]      3-VXA Set B [001]
    . . . .
1-B 0002         VXA Set B [002]      1-VXA Set B [002]
2-B 0002         VXA Set B [002]      2-VXA Set B [002]
3-B 0002         VXA Set B [002]      3-VXA Set B [002]
    . . . .
etc.

Similar backup set names for our older (DAT) backup sets, etc. The barcode doesn't have to encode the type of media because that's obvious from looking at the tape.

 

Autoloaders typically have a way for the software to interrogate what barcoded tapes are in what slot, so that Retrospect can go directly to that slot when it needs a tape rather than having to load each tape in sequence to find the tape with the right header.

 

FYI, Retrospect has long had a dangerous bug that I reported years ago and which was fixed in the Windows version years ago but never in the Mac version, and is my biggest negative rant about Retrospect. The danger is caused by the interaction of one "feature" of some tape drives (or the Retrospect driver) with this particular bug.

 

The "feature" is that Retrospect reports a tape as erased if a tape error is seen at BOT (as happens with a tape that is truly erased - no header, no logical EOT mark, etc., just blank tape). I don't know whether this is a feature of Exabyte VXA-2 drives or Retrospect's driver, and frankly, I don't care.

 

The bug is that, if Retrospect needs a new tape, whether by a "new media" backup or the filling up of a tape in the autoloader, it will [color:red]randomly[/color] choose what it believes to be an erased tape from within the autoloader, even if another erased tape exists in the autoloader that has been barcoded and pre-erased and pre-named using Retrospect to have the correct tape member name that Retrospect wants.

 

Because of this bug, if Retrospect incorrectly believes that a tape in the autoloader is erased (even if it's your valuable data from the past year), perhaps because the heads on your tape drive needed cleaning, perhaps because of a bad tape insertion into the drive, or any other error that would cause a tape error at BOT, Retrospect might (again, it's random) choose that tape when it needs an erased tape, and could overwrite that tape with data from the current backup.

 

This bug makes management of pre-barcoded tapes very difficult with Retrospect, and forces us to keep all tapes in the autoloader write protected, wait for Retrospect to fail in a backup session and demand a new tape, so that we can insert the right tape for the next day. Grumble, Grumble. I've been making this rant for years now, and I genuinely hope that this bug is fixed in Retrospect 8 (or X, whatever) because that version is based on the Windows code, not the current Mac code. Sigh.

 

Russ

Edited January 27, 2009 by Guest

[theboyk]

 Barcode        Backup Set              Tape Member Name
1-A 0001         VXA Set A [001]      1-VXA Set A [001]
2-A 0001         VXA Set A [001]      2-VXA Set A [001]
3-A 0001         VXA Set A [001]      3-VXA Set A [001]
    . . . .
1-A 0002         VXA Set A [002]      1-VXA Set A [002]
2-A 0002         VXA Set A [002]      2-VXA Set A [002]
3-A 0002         VXA Set A [002]      3-VXA Set A [002]
    . . . .
1-B 0001         VXA Set B [001]      1-VXA Set B [001]
2-B 0001         VXA Set B [001]      2-VXA Set B [001]
3-B 0001         VXA Set B [001]      3-VXA Set B [001]
    . . . .
1-B 0002         VXA Set B [002]      1-VXA Set B [002]
2-B 0002         VXA Set B [002]      2-VXA Set B [002]
3-B 0002         VXA Set B [002]      3-VXA Set B [002]
    . . . .
etc.

 

Thanks, that's some great info...and geez, that "feature" is horrible!!! Lets hope it's fixed in the new version!

 

As for your backup sets — much more advanced than we have here! Ours is more like...

 

Backup Set     Tape Member Name
MWFSS-1       MWFSS-1 (and when that tape fills, we move on to...)
MWFSS-1       MWFSS-2 (and when that tape fills, we move on to...)
MWFSS-1       MWFSS-3

(alternating with...)

TT-1            TT-1 (and when that tape fills, we move on to...)
TT-1            TT-2 (and when that tape fills, we move on to...)
TT-1            TT-3

 

and then, when tape 3 fills (well, nearly fills), we start a new backup set...

 

Backup Set     Tape Member Name
MWFSS-2       MWFSS-1 (and when that tape fills, we move on to...)
MWFSS-2       MWFSS-2 (and when that tape fills, we move on to...)
MWFSS-2       MWFSS-3

(alternating with...)

TT-2            TT-1 (and when that tape fills, we move on to...)
TT-2            TT-2 (and when that tape fills, we move on to...)
TT-2            TT-3

 

So, the autoloader could really help out!!!

 

Thanks again!

Kristin.

[rhwalker]

Note that, if you have your backup sets named as:

Backup Set Whatever [001]

then, when you do a "New Media" scripted backup (schedule every month or two, depending on your needs, or do manually), Retrospect will automatically create a new backup set with the same name but incremented number in the brackets, and all script destinations will be automatically changed. See the Retrospect User's Guide beginning at the bottom of page 23 ("New Media Backups").

Retrospect Mac 6.x Users Guide

 

The tape members for each backup set are automatically numbered with prefixed "1-", "2-", etc.

 

One more thing - make sure you name your backup sets and barcodes properly from the start. The names of backup sets cannot be changed once they have been created. The association of barcodes to tape members can be changed, but it is more painful than you want to know. Basically, because the barcode database is not editable, and because there is no GUI to make changes (even if it were editable, which it is not), you have to wipe out the entire barcode to tape member mapping, then reinsert EVERY TAPE YOU HAVE EVER BARCODED, run a verify on each tape (so that Retrospect is forced to read the header - you can stop the verify after that happens without having to verify the entire tape) so that the barcode is reassociated with the tape member name. Been there, done that, wasted hours and hours and hours and hours.

 

I made a feature request years ago to either:

 

(1) have the barcode database be editible by a GUI in Retrospect; or

 

(2) have the barcode database be an editable text file; or

 

(3) publish an API for the barcode database so I could write my own code to manage it.

 

That request went into the proverbial black hole - apparently I'm the only user who cares about this. Sigh.

 

Russ

Edited January 27, 2009 by Guest