Disk to Disk to Tape-How does Retrospect know where to restore from?

[showe]

We are planning a config where we dump from disk to disk to tape. Because of space limitations we are thinking about recycling the disks once a week after we dump to tape. After that is done and we need to do a restore, does Retrospect know where to pull the restore from? Does it make two catalogs?

[Lennart_T]

No, you have to keep track of where to restore from.

Yes, it makes two catalog files, since you are using two media sets.

 

Considering the price of hard drives today, get a larger one and do grooming instead of recycling. Then you can (almost) always restore from the disk media set.

[showe]

After you dump from disk to tape, you can't then pull a restore from tape? I don't understand what the point of dumping to tape is then. Im not understanding something here.

 

Basically we cannot lose any backups at all so I don't think grooming is an option.

[Lennart_T]

Of course you can restore from the tapes. It just faster/easier to restore from the Disk Media set (as long as it is available).

 

The tapes should be stored off site. So if the disk breaks down or catches fire, you still have the tapes.

 

Yes, you can groom the oldest backups. They are still on the oldest tapes.

[showe]

Thanks for your suggestions. I appreciate it. Here's what we're planning to do...maybe you can comment on it:

 

We have 16TBs of disk space. To use the 16 threads Retrospect has, we're planning on dividing those into 1 TB media sets named by OS and then use tags to designate the clients. We need to see how quickly the sets fill up to determine how often we need to recycle the sets. Grooming is a good idea but I don't know what that will buy us as we have so many clients that we need to squeeze onto those sets we may not be able to go back more than 2 backups or so so we dont really gain anything. My original question relates to how we would do a restore if we have to recycle the disk sets often. So say I go to do a restore and ask to restore from a certain date. How does Retrospect know where to pull the restore from? Will it return two options, one for the disk set and one for the tape set? How would we differentiate the two?

 

Thanks for your help. I hope I dont sound too stupid.

[Lennart_T]

Haha, no you are not stupid. There are no stupid questions if you don't know the answer. And how would you find out the answer without asking?

 

How many clients do you have?

A reasonable amount of clients is 10-15 per Media Set. That way Retrospect can backup all of them during one day. At least after the initial (full) backup.

Retrospect never backs up an identical file more than once per Media Set. So if you use too many Media Sets, you would waste storage space.

 

If you recycle a backup, you will not be able to restore anything from it until after the next backup. The next backup will be a full backup, since you haven't saved anything from the old backups. I think this is a waste of time and a waste of network bandwidth.

I still suggest grooming. We keep the last 5 backups of each client on disk. Once a week we transfer to tapes that are stored off site. All tapes are stored at least two months. One set of tapes each month are stored one year. The April and October tapes are stored ten years.

Back to grooming instead of recycling. A typical client has 30-60GB of files here. Many OS files and Program files are identical across many clients, so they are backed up only once. So 10-30GB are actually stored as unique files for each client. Every new backup adds typically 1-2GB per day (we backup once per day). So each new backup doesn't add much data to the Media Set. You can set grooming to keep just two backups if you are short on disk space. If not, set it to ten backups. We are a bit short of disk space, so we have set it to five backups.

 

Now for your restore question. When initiating a restore, you have to select the proper Media Set yourself. Retrospect doesn't (really) keep track of where the backup of each client is stored. You can make an educated guess of which Media set the backup is on by comparing the "certain date" with today's date. That will tell you if the backup is still on the Disk Media set or if you have to hunt for old tapes. Which tape, then? Here is a screen shot of our selection of tapes from Retrospect Windows. You can see when we last used each set of tapes.

So it's easy to see which Media Set you would use to restore.

[showe]

We have about 200 clients. This is why I'm leery of grooming because I don't think we're going to be able to save that many backups. My guess is two at the most. I need to decide how often to dump to tape too. Do we do once a week or once a month? I guess it depends on how quickly the disk sets fill up, right? Even for grooming because we don't want to lose ANY backups (we are a government facility and it's law that we archive ALL backups.)

 

As far as naming media sets...I'm still confused a bit. Would we have to change the name of the media set before we dump to tape to differentiate between the two catalogs so when we do a restore we know which media set to choose? In other words, when we go to do a restore, how would we know which media set is tape and which is disk?

 

Thanks again.

[twickland]

We have about 200 clients. This is why I'm leery of grooming because I don't think we're going to be able to save that many backups. My guess is two at the most. I need to decide how often to dump to tape too. Do we do once a week or once a month? I guess it depends on how quickly the disk sets fill up, right? Even for grooming because we don't want to lose ANY backups (we are a government facility and it's law that we archive ALL backups.)

In terms of both the time needed to back up and the space needed in your media set, the total number of unique files and how often they are modified matters more than the number of clients. If you have difficulty accommodating everything that needs to be backed up in the time or disk space you have available, you may want to consider creating and using several media sets, and backing up different groups of clients to each media set.

 

If you're planning to use disk-to-disk-to-tape (D2D2T), you need to be sure that you have sufficient disk capacity in your disk media set(s) to accommodate all the files that will be backed up in the intervals before you copy from the disk media set(s) to the tape media set(s). You also need to perform that copy operation before the grooming action is performed (or before you recycle the disk media set). In general, grooming is better than recycling because it's more efficient and because it retains more file versions at the ready for performing a quick restore. (After a recycle, all of the files in the media set are gone until they are backed up again.)

 

As far as naming media sets...I'm still confused a bit. Would we have to change the name of the media set before we dump to tape to differentiate between the two catalogs so when we do a restore we know which media set to choose? In other words, when we go to do a restore, how would we know which media set is tape and which is disk?

When you create your tape media set, it will be completely separate from the disk media set; each media set will have its own catalog. In D2D2T, you are backing up to the disk media set, and are then copying the data to the tape media set. You can choose to restore files from either the disk or the tape media set. The idea is that eventually you will groom earlier versions of files from your disk media set so that you will retain only the most recent N versions of each file in this media set; Lennart suggests N=5, while we use N=30. Assuming that your time interval for copying from the disk media set to the tape media set is sufficiently frequent, every version of every file you have backed up will be on the tape media set, but only the more-recent versions will be on the disk media set. You would probably rely on the disk media set for performing most of your restores, using the tape media set only for retrieving older files.

 

How you choose to name these media sets is completely up to you. You can also decide whether you want a one-to-one match from your disk media sets (assuming you have more than one) to your tape media sets, or if you will copy all the disk media sets to a single tape media set.

 

Given the potential for tape damage or loss, I would actually recommend that you maintain at least two tape media sets for redundancy, stored in different locations. You would regularly copy each disk media set to both of these tape media sets at appropriate intervals.

[showe]

Thanks. Great suggestions. Im still a little unclear on the naming thing though...how will I know if Retrospect is referring to the disk media set catalog or the tape media set when I do a restore unless I name them differently? Would I have to, say, name the catalog for the disk media set something like "July2012-Disk" and the tape media set catalog "July 2012-Tape" to differentiate the two? I ask because it would be great to just have one script that runs monthly instead of having to create new scripts every month (or week, depending) with a different name to differentiate the catalogs. I think I can leave the disk media set with the same name all the time as that will just be being groomed or recycled anyway.

[twickland]

When I said you can name your media sets whatever you choose, it did not occur to me that anyone would consider giving two different media sets the exact same name (I guess I should have thought of George Foreman). No, that would be a bad idea, and if you're storing all your catalogs in the same place, Retrospect will not even allow you to use the same name for two media sets.

 

If you're planning to use D2D2T, it would make sense to name the disk media set something like "Disk Backup." If there's enough disk capacity to go for a month without grooming or recycling, a monthly copy to tape would work. There's no need to create a new tape media set every time you copy to it, though, unless that's what you really want to do. A copy media set script by default compares the two media sets and copies only new or changed files. IMHO, it might make sense to rotate in new tape media sets yearly rather than monthly. (Note that I said "sets;" I really recommend two or three tape sets that are essentially redundant, and which you copy to on an alternate basis.)

 

There is also no need to write a new script when shifting to a new media set. At the time you want to have that shift occur, simply select the media option "Start new Media Set" in the Schedule pane for your copy media set (or backup) script. The new media set will have the same name as the original media set but with an incremental number appended. For example, of the original was named Media Set A, the new one would be named Media Set A [001]. The best thing is that all the scripts that used to write to the original media set will automatically change so they begin writing to the new media set.

[showe]

Thanks to everyone for their help with this. Much appreciated.