Amnesiac Catalogue

[ShadeTek]

After some initial firmware issues with our Quantum L700 Superloader 3 LT04 which were updated to get it to see the tape drive correctly, We are now using it to do proper back-ups. The 5.7TB back-up I did took all 8 slots in the Superloader. I was under the assumption you could eject a full tape from a back-up job, say Tape 1, Slot 1 and then insert a tape for a different back-up job. Wrong! After ejecting seven full tapes of job 1, I then did a quick test to see if job 1 still behaved properly by identifying the fact that it had no extra media added to it's source and would need no further backing up. It should have recognized this, but instead it now wants to back-up 4TB of the 5.7TB media that was already backed up. It seems that R8 sees all the tapes in the Superloader as a hard disc drive and has lost a large chunk of the media......or am I doing something wrong here? Surely you can eject a full tape from a media set and the system will carry on without it, using the current empty tapes in the media set?

[twickland]

I was under the assumption you could eject a full tape from a back-up job, say Tape 1, Slot 1 and then insert a tape for a different back-up job. Wrong! After ejecting seven full tapes of job 1, I then did a quick test to see if job 1 still behaved properly by identifying the fact that it had no extra media added to it's source and would need no further backing up. It should have recognized this, but instead it now wants to back-up 4TB of the 5.7TB media that was already backed up. It seems that R8 sees all the tapes in the Superloader as a hard disc drive and has lost a large chunk of the media......or am I doing something wrong here? Surely you can eject a full tape from a media set and the system will carry on without it, using the current empty tapes in the media set?

You're using a lot of non-Retrospect terminology here, and have made other statements that I'm trying to make sense of (Is "job 1" a backup session or a media set? Is the "next job" another backup session or sessions to the same media set or to a different media set? Retrospect seeing the tapes as a hard drive volume--as what, a source? A lost chunk of media, or is that supposed to be data?)

 

Back to basics for a moment: Retrospect backs up to media sets. A tape media set consists of the various tapes that are members of the media set, where the backup data is actually stored, plus the catalog that is an index of all that data and that is stored on a hard drive volume (by default, inside the folder /Library/Application Support/Retrospect/Catalogs of the engine computer). You perform a backup by writing and running a backup script. Each time a source is backed up, a separate backup session (typically abbreviated in Retro 8 to just "backup" or "past backup").

 

 

Once a backup or series of backups has been performed, the data is on the tape members and the listing of that data is in the catalog. Whether or not a particular tape member is in the tape library or tape drive makes no difference in the catalog. The number of tape members used depends on the capacity of each member, whether data compression was used, and how compressible were the files being backed up.

 

If what you're trying to say is that you performed a backup of one or more sources, and then when you tried to back up the source again, that Retrospect wanted to back up more files than you expected, that would typically be because the data on the source had changed in some way between the time of the first backup and your second attempt. Such a change might be only in the metadata, such as ACLs. It would certainly not be due to what tape members happened to be in the tape library, though it would also occur if Retrospect thought one or more of the tape members was lost. (Go to Media Sets:Members to confirm that all the members of your media set are listed, and that the "Lost" box is unchecked.)

 

If you don't need to be able to restore ACLs, be sure to uncheck that option in the Options:Source section of your backup script.

 

If/when you reply back, give us step by step information about what you did, using standard Retrospect terminology.

[ShadeTek]

OK, let me explain in more detail. Back-up Job using a script from a network drive to LT04 tape, lets call it job1, which has 8 tapes in it's media set, and it has to back-up 5.7TB of Media.

 

Job1 script runs well completing 5.7TB of back-up, then another job comes in to back-up to LT04 tape, let's call that job2 script with two tapes in it's media set. Now because the Loader has 8 tapes from job1, there is no room at the inn for job2 script tapes. So thinking that job1 tapes 1 to 7 are full, and no longer needed, I eject tape 1 to 7 from job1 media set out of the loader into my hand and then put them on the shelf for safe even temperature controlled enviroment storage that won't damage the tapes over time.

 

I then think, now that tapes 1 to 7 are no longer in the loader and and no longer needed as they are at capacity, let me just check that this software is going to complete the job1 script without any flakey behaviour, so I run job1 script using only one remaining empty tape from the job1 media set, I.E. tape 8 of the job1 media set which is the only tape from the job1 media set left in the loader, and see if it backs-up correctly to tape 8. Well, it didn't. It now says it has to back-up 4.1TB of data using the job1 script onto the job1 media set! Now, I have to tell you that there is no way 4.1TB of data has suddenly appeared or moved on our job1 script source location and no tape memebers are lost, so I am thinking that Retrospect 8 has gotten awfuly confused or some other problem has occured. I post this issue up in the hope others have encountered this and also that if they themselves experience such a dilemma that it will be of re-assurance they are not alone.

Edited April 29, 2011 by ShadeTek

[f30e6e60-a263-4d69-8f60-c92703669308]

What does the removal of tapes 1-7 have to do with how much data Retrospect is matching for backup?

 

If you do not remove any tapes and simply run the job1 Script again, how much data is matched?

 

That would be the issue that needs explanation or resolution, with the part about tapes for job2 simply a confusing red herring.

 

 

It seems that R8 sees all the tapes in the Superloader as a hard disc drive and has lost a large chunk of the media

 

No it doesn't. Retrospect sees tapes as Members, each capable of holding not only its portion of the source data but also catalog information for emergency rebuilding. For users without an autoloading tape library, Retrospect is perfectly satisfied with having access to only one tape at a time.

 

 

Dave

Edited April 30, 2011 by CallMeDave

[ShadeTek]

The problem of suddenly needing to back-up 4+TB of already backed up data only occured when I ejected tyapes 1-7. It seems to me there is a direct link there. It had worked for 3 weeks with no problems until that exact point. When I run it even now, it says it wants to back up 4TB of media, so it is only matching up a 1.7TB of 5.7TB. This is a problem.

 

We have a client paying for this operation which will be the final back-up of a multi-million pound project. I cannot rely on this software to safely carry this function out.

Edited May 2, 2011 by ShadeTek

[twickland]

The problem of suddenly needing to back-up 4+TB of already backed up data only occured when I ejected tyapes 1-7. It seems to me there is a direct link there. It had worked for 3 weeks with no problems until that exact point. When I run it even now, it says it wants to back up 4TB of media. This is a problem.

If you go to Media Sets, does the summary page for job 1 media set show the correct number of members and a reasonable total space used? If you click on the Members tab, are all tape members shown, with a reasonable figure for space used for each member? In the Backups tab, are all the backups you actually performed listed correctly?

 

I would suspect that the need to back up 4+ TB again could well be real; that the files have changed in some way, perhaps only in their metadata. Are the sources you are backing up Mac volumes? If so, try changing the options for your backup script. Go to the Scripts page, hit the Options button and Sources disclosure triangle for the script in question and uncheck the Macintosh option "Use attribute modification date when matching." Does Retrospect still want to back up 4+ TB?

 

If it does, I would compare a few files in your backup with their versions on the source. Are their sizes exactly the same, byte for byte; are the creation and modification dates and times identical?

[f30e6e60-a263-4d69-8f60-c92703669308]

The problem of suddenly needing to back-up 4+TB of already backed up data only occured when I ejected tyapes 1-7. It seems to me there is a direct link there.

 

 

Cum hoc ergo propter hoc; correlation does not imply causation.

[ShadeTek]

If you go to Media Sets, does the summary page for job 1 media set show the correct number of members and a reasonable total space used? If you click on the Members tab, are all tape members shown, with a reasonable figure for space used for each member? In the Backups tab, are all the backups you actually performed listed correctly?

 

I would suspect that the need to back up 4+ TB again could well be real; that the files have changed in some way, perhaps only in their metadata. Are the sources you are backing up Mac volumes? If so, try changing the options for your backup script. Go to the Scripts page, hit the Options button and Sources disclosure triangle for the script in question and uncheck the Macintosh option "Use attribute modification date when matching." Does Retrospect still want to back up 4+ TB?

 

If it does, I would compare a few files in your backup with their versions on the source. Are their sizes exactly the same, byte for byte; are the creation and modification dates and times identical?

 

Thanks for the input Twickland. It is a Mac OSX server the files are sitting on (via 10GE network) and the attribute check box was ticked. I have now unticked it but will have to wait till tomorrow until another back-up finishes before testing. There seems to be a uniform 25GB left on each tape member (members 1 to 7). The final member, tape 8, had 700GB.

Edited May 2, 2011 by ShadeTek

[ShadeTek]

OK, I unchecked the attributes box and ran the script and it still wants to back-up media....see update below.

Edited May 3, 2011 by ShadeTek

[ShadeTek]

Well, things get more interesting as time goes on. The Media Set of Job1 now says it has 50TB capacity and 43TB of free space! Even with software compression on, which it isn't, the amount of free space would only come to 12.8TB, so something has gone wrong in the Media set capacity. What I did last night was run job1 script without any of the media set in the loader. So, what should have happened is it would have scanned the sources and then decided how much media was required to upadate the media set using the catalogue on the local HDD and then ask for media to be inserted which it was this morning.

Edited May 3, 2011 by ShadeTek

[twickland]

So, what should have happened is it would have scanned the sources and then decided how much media was required to upadate the media set using the catalogue on the local HDD...

Actually, no. With tapes, the capacity that Retrospect lists is only an estimate. Retrospect will write to a tape member until the drive reports that the tape is full, after which Retrospect will then look for the next tape member (if you have added that tape to the catalog) or for a blank tape.

 

In earlier version of Retrospect, you could manually enter a number for tape capacity based on your experience of how much was likely to fit on a tape. Retro 8 doesn't have this manual option as far as I can tell. I'm not sure where Retrospect pulls its tape capacity figure from, but clearly something has gone wacky in your case. (It's interesting that this capacity error seems to have been applied only to the first seven members.) If you quit the console, stop and restart the engine, and relaunch the console, do you still see the bizarre capacity figures?

 

I'd suspect something has gone wrong with the Config80 configuration file, and/or possibly with the catalog for this particular media set. Regarding the need to back up 4+ TB again, have you checked and compared the files in the backup with those on the source to see whether anything is different between the corresponding files? You may be dealing with multiple problems here, as well as some things that may not actually be a problem.

[GMRMacBackup]

The capacity bug has been around for a long time, I had a couple of posts in the EMC forums about it. As far as I've found there is no way to adjust or reset the capacity unless you recycle the media set.