Jump to content

Backup dilemma - files being skipped…


Recommended Posts

Alright - hopefully someone will have some insight and suggestions for me. I'm not a new comer to Retrospect, but this has me pulling out my hair.

 

First our set up - we've relegated Retrospect to only archiving completed jobs to tape. We have one onsite set for retrieval and one offsite set. Completed projects are moved to a Drobo share named "Completed" via AFP - where they are put in client coded folders. This share is mounted on a G4 (also via AFP) running 10.4.11, with Retrospect 6.1.2, with a VXA 320 via SCSI (ATTO Card). It's worth noting that we have one person in charge of this. They mount each designer's (three of them) client folder on their desktop. They then move the completed projects to the Drobo Completed share, also on their desktop - where they are then sorted into client folders. On Monday evenings, the newest files are added to the onsite tape, on Tuesday evenings, the offsite tape.

 

We've been running this set up since the first of the year. Up to this point I've only been checking logs for errors. I started going through the sets contents report after one of the designers mentioned that the fonts folder was missing from a job they had pulled from tape. Turns out we're missing folders and files from a variety of projects when I compare it to the contents on the Drobo. Fortunately we haven't suffered any permanent data loss that I've found. The Drobo was set up for mass storage of these completed jobs so that we only pull the oldest jobs from tape. The newer ones are stored on the Drobo. As the Drobo fills up we do some house cleaning, knowing that if a project was on the Drobo, it has also been moved to tape. The Drobo is currently configured to 2.6 TB.

 

I spent a few hours yesterday trying to determine what would cause a single folder and it's contents to be skipped - but the rest of the content from the enclosing folder to be present. I know permissions can be an iffy thing - but I'm not certain what steps to take to test this. From the Finder on the G4, the files and folders on the Drobo have the same permissions, even those that Retrospect skipped. I noticed that some (but not all) folders also had folder labels - which I turned off (note that there are no rules in the script to skip anything). I'm stumped.

 

BUT, we have Retrospect 8 on a G5 that I was never able to get set up to access our clients. However, now we aren't using clients, just backing up the locally mounted Completed share from the Drobo. IF I can get the SCSI card in the G5 to see the tape drive, and IF I can get Retrospect 8 running (a week long fiasco I gave up on last year) might this at least improve the reliability of our back up? One caveat, this G5 is also used to run our Studiometry server - though its overhead is fairly light. I don't think I'll have issues running both - particularly considering we only back up to each set once a week. The G5 is also running 10.5.8 - I could upgrade it to 10.6 if there was any benefit.

 

Of course, I might run into the same issues in Retrospect 8 - or I may be opening up another can of worms.

 

Any input and feedback are appreciated. Thanks.

Link to comment
Share on other sites

How were you retrieving the files from the tape? Was it via some kind of search or did you use a backup snapshot? If you haven't selected the option to "match only same location" in your backup script, Retrospect will not back up files it has previously backed up from some other volume or folder. If you examine the report of a backup session, or if you attempt to retrieve files by searching a particular path, any files that were backed up from a different location will not appear, even if they are in the backup set. These files will only appear if you perform a restore using a snapshot.

Link to comment
Share on other sites

Do you use a custom selector? Or the supplied "All files" or "All files except cache files"?

 

I recommend against using Retrospect 8 or 9 on a PowerPC Mac. Retrospect 8 and 9 was designed for the little endian on an Intel processor, while PowerPC is big endian. So there will be a LOT of byte swapping load on the CPU.

Link to comment
Share on other sites

I'm not too concerned with the retrieval method. I have been viewing the Contents Reports - looking at the data that was moved on a particular date, to a particular set. There are many instances of missing files when compared to what is on the Drobo. For example we have a convenient store client - the path to a job may look like this:

 

Completed/QKT/12-QKT-122 Red Bull Ice Mountain Promo Door Decal/

 

Inside 12-QKT-122 are project files, usually consisting of a folder containing the final collected artwork (which will have fonts, images and art files - .ai, .psd, .indd) and a misc. folder containing various files (unused stock, extra layouts, etc). In this instance, on the Contents Report, the "Fonts" folder is missing from the collected projects folder. But everything else is accounted for. See attachments.

 

In some cases it's a folder of images, sometimes it's the actual art files - illustrator files or what not. I've attached screens regarding a couple of instances of this.

 

If I can just determine why Retrospect is skipping the files I can stick with out current set up - a new Mac Pro isn't in the budget. Though we do a fourth designers workstation that is only used for admin and light design work. We have a part-timer there at the moment and they are out of here by 3:30 everyday. I suppose we could put the tape drive and R8 on this system. This is the person that manages the archiving anyway. As long as R8 plays nice with this persons day to day activities.

 

Please review the screens and let me know if you have any recommendations. Thanks.

post-2106-0-24353800-1337267246_thumb.png

post-2106-0-28929400-1337267263_thumb.png

post-2106-0-87921400-1337267280_thumb.png

Link to comment
Share on other sites

I'm not too concerned with the retrieval method. I have been viewing the Contents Reports - looking at the data that was moved on a particular date, to a particular set.

The problem with looking at the Contents Report is that it will only show which files were copied in a given backup session. If the files were already copied on a previous date or from a different source or a different folder, they will not show up in the Contents Report.

 

The only way to view all the files that were on a given volume at a particular time and have been backed up is to initiate a Restore using the desired snapshot. You don't have to actually restore any files, but you do need to get to the point where you are able to view and select the files.

 

To save time and space on the backup media, Retrospect intentionally does not back up files it has already backed up. If your preference is to back up everything from a given source whether or not the files have already been backed up, you can uncheck the options for matching and not adding duplicates in your backup script.

Link to comment
Share on other sites

The problem with looking at the Contents Report is that it will only show which files were copied in a given backup session. If the files were already copied on a previous date or from a different source or a different folder, they will not show up in the Contents Report.

 

 

Eureaka - I had thought of this too. However, I assumed that since the files weren't touched again after being moved that there would be no reason for Retrospect to append any data. I wasn't restoring from a snapshot, but I was restoring via a search for the job number folder and all enclosed items. This was showing the same results as the Contents Report, thus freaking me out a bit.

 

I think the problem is in the .DS_Store files. For example, the project with no files in the production folders has a .DS_Store file that was modified at a later date. Thus when doing a search via enclosing folders, only returns the latest version of the folder - the one only showing the newer .DS_Store files (correct?).

 

So now it would appear that the only way to positively restore a full project from tape is to do it from a snapshot? The search appears to be a rather unreliable method if modified .DS_Store files can muck things up. Would it be possible to add a selector that ignores .DS_Store files?

Link to comment
Share on other sites

So now it would appear that the only way to positively restore a full project from tape is to do it from a snapshot?

Yes, that is exactly what snapshots are for. It is a snapshot of the source volume's state at the time of the backup. Restoring a snapshot restores that state completely.
Link to comment
Share on other sites

I wasn't restoring from a snapshot, but I was restoring via a search for the job number folder and all enclosed items. This was showing the same results as the Contents Report, thus freaking me out a bit. I think the problem is in the .DS_Store files.

Your results would indicate that the missing files existed on (and were backed up from) some other folder. Searching for files by enclosing folder will only display files that were actually backed up from that folder; not by whether they ever existed in that folder. The only sure way to view (and restore) all the files that were in a folder as of a particular backup date is to use a snapshot.

 

Retrospect does not refer to the contents of the .DS_Store file (or of any file, for that matter) when performing a search or a restore.

 

So now it would appear that the only way to positively restore a full project from tape is to do it from a snapshot? The search appears to be a rather unreliable method...

If you don't want to take advantage of Retrospect's incremental backup feature to save time and tape capacity, you could deselect all of the matching options in your backup script. This would ensure that each Contents Report always matched the actual contents of the folder at the time of backup, as well as facilitating your search by folder. Alternatively, you could select all three matching options (Match to catalog, Don't add duplicates, Match only same location), which would also ensure that your search by folder would show everything.

 

The potential problem with performing any kind of restore using a search by folder is that Retrospect will include multiple copies of any file that has been backed up more than once, plus files that used to exist in the folder at one time but which were subsequently deleted. The best way to restore a folder, and as I said above, the only sure way to restore it as of a given backup date, is to use a snapshot.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...