Jump to content
Sign in to follow this  
technothrakon

Restore entire set? Restore entire tape?

Recommended Posts

Hi all. I'm hoping someone can help. I am not a Retrospect user, except that I've been told to find a way to restore all of our art department's legacy backups so it can be moved to alternate storage (DVD's probably). Retrospect 4 is running on a PowerMac G3.

 

The first of their backup sets has about 44 tapes, and probably a couple of dozen 'snapshots' in the set. I did a complete restore of the first snapshot in the set, and was dismayed at amount of tape switching. The restore seemed to pull a few files of of many many tapes before finally pulling the bulk of the restore off of some of the later tapes in the set. It's well over an hour into a 40Gb restore, most of which was spent swapping tapes in and out.

 

So, my question. Is there a way to simply take tape one 1 pull everything off of it, regardless of what snapshot its from, then move to tape 2, etc. The DAT drive this system uses is making noise that suggests it's not going to be operating much longer, so I'm looking for the most expedient way possible to restore the entire backup set, all snapshots. I'm going to want to seperate out the restored files into folders on an external hard drive.

 

Does this make any sense? Am I stuck with my current method? Thanks for any help or advice. If I've not been clear, please tell me so and I will do my best to clarify.

 

Thanks much,

 

Aaron Hutton

Share this post


Link to post
Share on other sites

You are stuck with the current method UNLESS you want to "transfer" the backup set to another backup set, perhaps a file backup set, etc. (the choice of destination set type for the transfer will require you to understand the differences between "file" and "removable media" backup sets. Retrospect would then make a big pass through the tapes, transferring the data to another backup set on another set of media, and you could then do the retrieval from that new backup set.

 

To understand what is happening you have to understand the "snapshot" paradigm that Retrospect uses. Under the hood, Retrospect pretty much does the usual "full backup" followed by "incremental backups" to catch things that changed from the "full backup" and previous "incremental backups" (to use the terminology of other backup programs, with which you may be more familiar). A "snapshot" presents a view into the disk at the time it was backed up, and that view may include some files from the "full backup" (that have never changed since), and a scattering of files from every incremental backup since. That's why Retrospect has to pick and choose from all member tapes of the backup set. It's a backup strategy tradeoff (time to backup versus time to restore) that your predecessor may not have properly made if you've got so many tapes in each backup set.

 

As a slightly off-topic digression:

 

Personally, we hold the number of tapes in each backup set to a quite small number, and never recycle/reuse tapes and never delete files, for two reasons: (1) makes the restore faster in time of crisis, and (2) if any member tape goes bad (it happens, especially if you have tapes going back 15 years or more as we do), there is the high probably that the file you need is in a later backup set.

 

It's all a question of how valuable your data is. The right first step, before pumping out backup tapes, is to develop a backup strategy that suits your needs. Some of the issues that might need to be addressed are here:

What should a good backup policy address?

 

Russ

Share this post


Link to post
Share on other sites

Thanks for the advice Russ - you explained it perfectly. If I can clear enough space on the network drive, I may transfer the set to disk as you suggested and go from there. If anything, it should be faster to restore from! Also, if I understand you correctly, the last snapshot is the latest of all the data in the set. I will speak with the Art Director and see if she /really/ needs to have all the previous versions of these files. If not, then the last snapshot may suffice in the end.

 

However, since I'm now off the hook as far as sitting there and doing the actual restores (my boss says just show them how to do it and it's their responsibility), they may want to restore each snapshot individually.

 

Right now I think they take all the files required for each month's publication - as we're a publisher - and burn them onto DVD. So for magazine X, Aug 2007, here's everything that went into that 'zine. Probably the most effective way since 99% of the time the request for archived documents (for the artists anyway) comes from the editors and they ask 'Remember in Aug 2007 that ad we had on page 25? Use that this month.' Anyway, probably more info than you wanted. Thanks again!

 

--Aaron

Share this post


Link to post
Share on other sites

Quote:

Also, if I understand you correctly, the last snapshot is the latest of all the data in the set.

 


No, it's the latest changes to the data. There can be data for the last snapshot from any of the previous backups (full plus incrementals). That's why it takes so long to rebuild the catalog from scratch, and why Retrospect has to think so long between the scanning phase and the writing phase during a backup.

 

Depending on how your software works, you may need to keep more than just the final pages. For example, component elements that are massaged to create the final pages may be needed to, for example, recreate page 25 with different ads, or to extract specific ads from page 25.

Share this post


Link to post
Share on other sites

OK, think I got that. And the problem is I'm not sure these were backed up effectively, because most of what they want is 'archiving'. So they'll fill up the 20GB hard drive with data, then back it up, then delete it off the hard drive. Another reason why it's spread out all over the tape sets and why just restoring the last won't do it.

 

The first snag I've run into is Retrospect is complaining that my disk storage set can't be made large enough - it says there's a 2GB maximum. So... I may be stuck just restoring each snapsnot individually, or there a way to override this?

 

Thanks for your help and advice,

 

--Aaron

Share this post


Link to post
Share on other sites

Quote:

The first snag I've run into is Retrospect is complaining that my disk storage set can't be made large enough - it says there's a 2GB maximum. So... I may be stuck just restoring each snapsnot individually, or there a way to override this?

 


I'm not trying to offend, but I think there is a bit of confusion here.

 

That message from Retrospect should only occur when you are trying to back up to a Windows-formatted disk, which has a 2 GB maximum file size. I bet if you checked the formatting of that disk, it is formatted for Windows. Reformat the disk for a Macintosh file system (to HFS+ (preferably HFS+ Journaled)) using standard Macintosh tools (Disk Utility), and you should be good to go.

 

See also my first post above regarding the backup set type:

 

Quote:

You are stuck with the current method UNLESS you want to "transfer" the backup set to another backup set, perhaps a file backup set, etc. (the choice of destination set type for the transfer will require you to understand the differences between "file" and "removable media" backup sets. Retrospect would then make a big pass through the tapes, transferring the data to another backup set on another set of media, and you could then do the retrieval from that new backup set.

 


You probably need to study the manual for a discussion of the different backup set types. I would suspect that you would want a "file" backup set (not a (removable) disk backup set, as you seem to be trying).

 

I'm also a bit concerned, re-reading your original post, about the fact that you are using Retrospect 4 (current version is 6.1.126). There may be some limitations that you are hitting because of that old version, which we last ran years ago on our ASIP server under Mac OS 8.6.

 

Russ

Share this post


Link to post
Share on other sites

LOL... yeah Russ, I have concerns as well! Not the least of which is that this project is becoming a quagmire! laugh.gif

 

Well, I did not know that limitation with respect to Windows file systems. The disk in question is NTFS and is sitting off of a Windows 2003 server using the Mac file sharing service. It's a firewire and USB2 external WDC 'My Book', but unfortunately this G3 has no USB and no Firewire.

 

So... it appears now my options are to -

 

a) just go with plan a and do each restore in sequence (still makes me shudder)

 

B) find a large enough hard drive lying around I can put inside the G3 and get the storage local

 

c) get an external scsi enclosure and attach an external hard drive via the scsi interface on the machine.

 

Also, thanks for double-checking - I did see the options for 'removable' and 'file' storage, and I was attempting to create the file storage. I also had problems with Retrospect erroring out when scanning the backup snapshots for files to move to the new storage set - there are options like 'use only the most recent version of a file' and 'put all the files from mutiple snapshots into one snapshot' that seem to be precisely what I want and need. But since there are so many files to scan through, Retrospect complains about running out of memory - but I was able to work around this by specifying a date range.

 

Anyway, I still feel like I'm fumbling in the dark but at least getting somewhere! Thanks much!

 

--Aaron

Share this post


Link to post
Share on other sites

Um, these facts seem to keep changing:

 

Quote:

The disk in question is NTFS and is sitting off of a Windows 2003 server using the Mac file sharing service.

 


This is the first that you have mentioned that the disk is a network share using the Mac file sharing service. Up until now, the disk appeared to be a disk attached to the Mac.

 

Mac OS X can't write NTFS, but can read NTFS. But you aren't trying to write/read NTFS, but are using Mac file sharing service of Windows 2003. That may provide some compatibiliity with NTFS, but I suspect that an older version of Apple File Protocol (AFP) is being supported rather than AFP 2.0, which is probably the cause of the 2 GB issue. When I was referring to Windows file system, I was referring to a FAT-32 (non-NTFS) disk, which Mac can read and write but which has a 2 GB limit, but my comments are irrelevant now that it's revealed that this isn't an attached disk, but a network share.

 

I suspect that either your option (B) or © is the best course of action. There is also the possibility of an option (d), which I just throw out as a possibility. That would be to install Mac OS X on the G3 (or to use another computer with Mac OS X), update Retrospect to 6.1.126 (the current version) with current RDU (Retrospect driver update). If you use the G3, you'd have the additional difficulty of installing OS X on it if it doesn't have a DVD drive (you would have to go through the media exchange mess with Apple to get OS X on CDs). But if you use a different (faster) computer, it might make the process go faster, and, if you use Mac OS X, you might be able to read/write big files on your Windows server, perhaps by mounting an SMB share because Mac OS X is more compatible with Windows than Mac OS 9.

 

Whatever you do, you really should chose a path that lets the tapes be read on the same tape drive used to write them - that will give you the best chance of readability for legacy tapes.

 

Just a suggestion.

 

Russ

Share this post


Link to post
Share on other sites

Russ - thank you for all of your help and my apologies if I wasn't clear.

 

I think I have figured out the best way to accomplish what I need. After you suggested a backup set transfer, I noticed that the software ran out of memory when trying to transfer the entire set, and so I had to apply a date-range filter to narrow the selection so it didn't overflow available memory.

 

That got me thinking about the 'select files and folders to restore' option. Previously I had been trying to restore a whole backup set or a whole 'disk'. Because of the way files were backed up then deleted, and the backup method of one full plus many incrementals - this didn't lend itself to efficient restores. However, with the 'select files and folders to restore', I can search across mutliple snapshots, much like I could in the backup set transfer selection screen. Again, I can't choose every file in the backup set, because it overflows, but I am now restoring all files backed up before a certain date, about 70GB worth, and they are restoring fine to the network drive I mentioned previously (it doesn't have a problem with large files). After that's complete, I expect I can then choose files between two dates and continue that process until all the files are restored.

 

Again, sorry for the confusion - this was really not a problem except that I didn't know how to use the software. Thank you for the help, I'll let you know how it turns out in a few days once I write up instructions for the artists to finish these restores themselves.

 

--Aaron Hutton

 

P.S. I think the reason why the situation seems to be changing is because it has. I've been trying various solutions over the last week to get all the parts of this restore to work: 1) an external drive purchased to hold the restored data that uses an interface (firewire and usb2) that isn't available on the old G3. 2) an legacy backup program on a legacy Mac. 3) nontechnical staff who have been doing backups with this system and not knowing exactly how they are backing things up. It was trial-and-error, definitely. Again, my apologies if that made me a difficult person to help. smile.gif

Share this post


Link to post
Share on other sites

No apologies necessary. I've been in your situation before, and yours is a thankless job with impatient people wondering why it is taking so long. And wait until you get old - you will know how easy it becomes to get confused, as I often get.

 

One reason to restore an entire disk is that, for reasons that I have never understood for over 15 years since we began using Retrospect, the permissions and ownership are only restored correctly if an entire disk is restored. It's a Retrospect "feature" and is documented. Go figure.

 

The reason I didn't think about the "running out of memory" issue is that it's never been an issue since Mac OS X and our big server memory. I seem to recall that there's some odd hack to increase the amount of memory for Retrospect in Classic, and some old tech note that indicated exactly what the formula was for memory needed versus number of files, but it's been a long time.

 

You are no more difficult to help than I am to put up with. Ask my wife.

 

Russ (newly married this month)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×