Easy question....or so I hope...Retrospect 4.2

[coot]

I've just taken over the mac side of support and need some help with retrospect. It seems a little too easy to use - too easy that I'm wondering how do I do differential backups?

 

Here's my situation:

I'd like to do a full backup on a monday - requires 3 DLT tapes. I would then like to run differential backups (or something similar) from tuesday to friday. The only options are Recycle backup, new media, and normal. Can anyone point me in the right direction?

 

Thanks in advance.

[mcswgn]

Just leave the last of your three tapes in the drive (unless you suspect that it has very little space left, in which case you could tell Retrospect to skip to new media) and then do "Normal" backups, which is the default, for the rest of the week. There is no need to distinguish between "full" and "differential" backups. Retrospect's standard behavior for each backup is to (1) record the exact contents and status of the disk as of that instant (this is called a "snapshot") and then (2) append to the current storage set (your DLT tapes) any files in the snapshot that are not already written there (effectively your "differential"), making note of where in the storage set ALL the files in the snapshot were located.

 

The result is that unlike a classic "dump level" style backup, where you need to restore in layers--first the lowest (oldest) level dump, then the next most recent, etc. (and if you want to restore a single file, you need to search through all of the dumps to find the one that has it)--with retrospect you can just select a snapshot and tell it to restore the disk (or any part) to its exact state as of that date.

 

You will probably want to "flesh out" your backup strategy--rotate through 2 or 3 independent storage sets in case anything happens to the tapes in one of them, possibly keep some of these off-site, etc.

[coot]

So are you saying select "New Media" for my Monday backup (the full one requiring three tapes) and the select a "Normal" backup for the rest of the week to backup changed files?

[mcswgn]

"New media" is used when you have an existing storage set (in your case consisting of several DLT tapes) and you want to *archive* the existing tapes and start over with a brand new set of tapes. The new storage set uses a modified name from the original (E.g., "StorageSet A [001]" in place of "StorageSet A") so that if need be, you can come back to these old tapes and pull off archived files without confusing the tapes with the current set.

 

Since you probably don't want to achive your tapes every week (a bit excessive, not to mention expensive), you would mostly want to select "Recycle" on Mondays. This erases the existing catalog and removes all of the tapes from the storage set (effectively marking the tapes as unused). The name in this case does not change, so you are just restarting the backup from scratch. Unlike the New Media option where the tapes are archived, here the tapes are reused so all previous backup information is lost. (That is why you want to rotate through several storage sets with maybe occational archiving, so that you don't wipe out your entire backup history every time you reset a storage set. You want to think about a TIL--time to live--for files in your backup set. What if someone comes to you and says, "I deleted a file a month ago and now I find I really need it." How far back is appropriate for your circumstances? Maybe someone discovers this week that they were infected with a virus last week and needs to restore to a date 2 weeks ago.)

 

For the rest of the week you don't need to "select" anything, because Normal is the status quo. Unless you explicitly tell it to do something else first, when Retrospect does a backup, it is a normal backup. (The "Normal" button in media options is functionally equivalent to "Cancel".) This includes when creating a Storage set for the first time. You should not be going into Media Options when you are first creating a storage set. The one exception to using "normal" for the rest of the week would be if from past experience you expect the current tape to not be able to hold all of the coming night's backups (and it is supposed to be running unattended, so no one will be there to supply a new tape). In that case, you might want to select "Skip" as the media option. This will ignore the blank space at the end of the current tape and continue the backup on an empty tape.

 

Now if you will permit me, let me put in my 2c on the basic assumptions behind your backup strategy (restarting every week). This is the sort of strategy that is used in "dump" type backups. By the end of a week you already have 5 independent backups to search through to find files, etc. and it is starting to get unwieldy. This is not true with a Retrospect backup set. There is really no reason to restart every week. If you are filling 3 DLT tapes, you are backing up a lot of data, which stresses your network (if these are remote clients) and your tape drive. It does not improve the security of your backup sets (it reduces it because the tapes will wear out faster since they are written to so much more often) and it reduces the TIL for files in the set. Suppose you have 4 sets A, B, C, D that you rotate through and recycle every 4th week. (So every week starts with a recycle, but each successive week uses a different set until they repeat after 4 weeks.) In this case you have redundency in that if any set goes bad you have another that is at most a week out of date, BUT the TIL is only 3 weeks. (If I find out that a file is needed just after I recycled set D, then the earliest I can go back to is the beginning of set A which was 3 weeks earlier.) If I did the same weekly rotation between sets but did NOT recycle on Mondays, maybe only recycling every 3 months, then I have lost nothing in the form of security or redundency, but I have gained a TIL of almost 3 months. I've also written individual tapes 1/3 as often, and I've put a lot fewer hours on my drive.

 

My personal opinion (which others may disagree with) is that:

1) Sets should only be reset when either the number of tapes in the set is becoming unwieldy (remember that when restoring you can anticipate needing to read from every tape, so if the number of tapes is large, this can take a very long time), the number of "sessions" (individual backups) in the catalog is sufficiently large that matching is starting to take an inordinate amount of time, or a predetermined time is reached (that is consistent with the first two) that allows keeping to a regular schedule (E.g. recycling once a quarter). Exactly how frequently this should be must be based on each person's experience with their system. Just don't let yourself be locked into the old "dump" mindset.

 

2) one should NOT let the script do the recycling for you (as is the default used by Dantz). Recycling actually causes you to lose data. This is not something you should entrust to a fixed schedule. It takes all of a half a second to recycle a storage set manually, it can be done anytime prior to the start of the script, and there is no downside risk if you forget. I lost a valuble file storage set in the past (which can't be rebuilt like a tape storage set can be) because when it was created I accepted the default script which automatically recycles the set periodically. The machine had been offline long enough to go beyond this period and then I opened Retrospect specifically to get at this backup set. Before I realized what it was doing it had launched the script and deleted all the data. Needless to say, I won't make that mistake again.

[coot]

OK...Well that is pretty much what I thought at the beginning. Here's what I tried this week.

 

On Monday I did a recycle backup. this spanned 3 DLT's. Tuesday came along next (as it usually does ;-) ) and I had a Normal backup script ready to execute. Unfortunately it had backed up on 3 DLT's as well (near enough the same size backup). This isn't what I wanted to happen.

 

I wanted a 4 week schedule where on Monday I'd get a full backup of everything. Then on tuesday-friday I'd like to backup any changed files. the following monday would be a full backup again and so on. The Monday full of the 4th week would then be stored off site.

 

I was told that a "Normal" backup would only backup changed files. It seems as though this is not the case. Can anyone steer me in the right direction.

[mcswgn]

I wonder if you haven't actually started two separate backup sets. You can't just put in blank tapes and have retrospect continue an existing backup set. It will want to continue on with the last tape it was working on. If you tried to do that, it would kick the blank tape out and ask you to insert the one it was still working on. (Unless this is a tape library and both the original 3 and the second set of 3 were all available at the same time, or it's also possible to override this behavior with a preference item, so maybe that's what's going on.)

 

Go to the Configure tab and select Backup Sets. You should see a list of current backup sets. I think in this window you also see the date each set was last used. (I would just check mine, but it is currently busy backing someone up... Boy, I sure would like it if retrospect was multithreaded.) Does this match what you expect? Then double click on the backup set you've been using. It should open a window with several tabs at the top. Click on Members. How many members does it show that it currently has (and does the volume on each member seem right)?

 

In the script you were using, what/how many backup sets are listed that it is supposed to be using?

[mcswgn]

After rereading your last post, I'm going to anticipate your answers to the questions I asked. I suspect that, since you want a 4 week rotation, your script lists 4 backup sets. (Let's say A, B, C, D.) Monday you did a Recycle to A. Tuesday you put in new tapes expecting it to continue on with A, but when you go to the Configure Backup sets window, you will find that on Tuesday night it started B from scratch. The reason is that when running a script, Retrospect will always write to the backup set that was last used the LONGEST time ago, if it is given a choice. It doesn't have a choice if the current tape already belongs to a backup set, so in that case it must continue on with that backup set. But if you have a storage set with no members yet (like B would be if you just set this all up), and you have a BLANK tape in the drive, then since B has never been used and A has, it will start working on B.

 

For that reason I never provide completely blank tapes to an automated script. I always force the tape to be joined to the storage set I want by doing an immediate backup of something small. (For me, that's always my dedicated backup server. It doesn't have anything on it besides the OS, Retrospect, and the catalogs.) Note, this doesn't apply if Retrospect is prompting you for new media in the middle of a backup.