Nightly Full Backups

[sbn]

I want to be able to do full backups everynight. No differential or incremental backups. Exactly how should I configure the jobs? (I'm using the eval version and have no documentation)

 

Thanks

[mcswgn]

That will depend on what you want to do with the tapes the morning after. Are you planning to archive all tapes, some tapes, or no tapes? If you want to archive everything, then do a "New" backup each night. This will rename the backup set each time ("Monday Set" -> "Monday Set [001]", etc) and will allow you to put the previous tape(s) off somewhere. Most people won't want to do this because of the amount of media that gets used/stored. If you don't want to archive everything, then you will be planning on reusing some of the tapes. To reuse a tape you would do a "Recycle" backup. This deletes all of the information from the previous use of that tape and allows it to be used as if blank media for another backup. A recycle backup will not rename the backup set, so you could have one tape that you use each Monday called "Monday Set" and recycle it each time--every Monday reusing the same tape. Most people will do some sort of combination of Recycle and New.

 

Now, however, I feel obliged to digress from your actuall question to ask why you plan on doing a full backup every night? Maybe you have a valid reason, but in most instances this is a Bad Thing. I think it's an easy trap to fall into thinking that surely full backups are "better" than incrementals, and since you want to do the best thing, you'll stick with fulls. The assumption that fulls are "better" than incrementals is not valid--in a sense it's quite the other way around.

 

When designing a backup strategy there are two points people need to consider:

1) Latency -- For how long does a file need to exist before it's caught in a backup?

2) Time To Live (TTL) -- Once a file is in a backup, how long will it stay there?

 

Here "file" means a particular version of a file. Clearly the smaller the latency the better, and the longer the TTL the better, but there are practical limits on both and everyone has a different assessment of what is "good enough" for them. I'm at a university and my backup system has a latency of 24 hrs and a TTL of about 3 months. My reasoning is that if someone loses their excel gradebook we need to make sure we have a very recent backup, so short latency is very important. However, what are the odds that someone will delete an old exam and then not realize they still need it until at least three months later? Not high, and if it does happen, then they will just have to rewrite it! There is nothing that is so critical that it couldn't be redone, so at this point we keep no perminant archives. (This would be very different if we at the departmental level were responsible for all of the archiving of student grades, but that is handled by the university.)

 

OK, now lets look at a strategy of nightly full backups. If all tapes are archived it's great on both counts. It has a latency of only 24 hours and a TTL of forever. However, this will generate tons of archived tapes, so it's not very practical. So maybe instead you only archive the Friday tapes. This cuts down the number of tapes consumed by a factor of 5, but what has it done to backup? It has effectively turned it into two independent backup systems. One has a latency of 24 hrs, so that's good, but a TTL of only one week! The other has a latency of one week and a TTL of forever. Suppose you create some big project and then your boss tells you it needs go in a completely different direction. So you work on it for a week and bring it back and now he says you were right the first time. Well, you're out of luck because your old version wasn't there on Friday when the archived backup was run and it's already been overwritten on the daily sets.

 

Suppose instead a full backup was done on Monday night (I prefer Friday as it can extend on into the weekend if it takes a long time) and then incrementals for the rest of the week with the weeks' worth of tapes sent off to be archive on Monday morning. You now are back to a latency of 24 hrs and a TTL of forever. What's more, the amount of media typically required for an incremental backup is so small compared to a full, that the media used won't be significantly different than a single full backup. So you have the best of both worlds. The latency and TTL of a nightly archive, but the media usage of a weekly archive. Moreover, you don't lose any of the convenience of a full backup like you would with an old-style "dump" type full/incremental system. In fact, the whole concept of full vs incremental doesn't really apply to Retrospect. You have a stack of tapes that is your backup set. If you want to restore to a previous state, you just select the correct snapshot and let Retrospect have at it. There is none of this restoring the full backup first and then working your way back up the incrementals.

[sbn]

 

Thank you for your insightful response. Our backup needs are far more modest than what you deal with at a university. We have only a few gigs to backup and a limited budget for tapes.

 

If I understand your definitions: Latency should be within 24 hours. TTL, a week.

 

My primary concern in using an incremental strategy is having to step through multiple tapes should a full restore be needed. Had a bad experience once on a Unix box when we encountered media errors on an incremental archive while restoring after a major upgrade. Wasn’t using Retrospect at that time…

 

[fgallagher]

I like the cut of your jib! Can I be like you? Can you pass along some instructions on how to configure the script to achieve the scheme you have? Your 3 month TTL and 24 hr latency would be perfect for my environment and I seem to be stuck in the "Full Backup" trap. How many tapes does it take for you scheme? What type drive do you use? I have a DLT now but am thinking of going to an LTO 2 if I can get the funds from upper management.

 

Regards,

Frank