Restore / Retrieve Ghost Files

[ignprint]

Hello.

 

Last month, one of the servers died that we backed up with Retro 6.1.230.

 

We were able to bring this back online and move the files that were on that to another server. The dead server data was backed up to June 4.

 

Some of the files appear 'lost' - they were not on the server that we brought back and they are not on the new server. But, they should be on tape (the files are from mid-May).

 

Now I am looking for files that were on that server before the crash. I am doing a 'search-retrieve' on them but they do not actually restore. When I do a retrieve on the files, there is an asterisk next to the files. I cannot find those files on a restore...only a retrieve. Does that mean that those files were corrupt in the first place (the server was dying far sooner than the crash)?

 

Any help would be appreciated!

 

Thank you!

 

Addendum - The purple asterisk indicates a missing backup set member. At the time, a tape had its name changed by Retro from LTO A - 13 to LTO A - 14. Does that mean all that data was eaten by Retro or does it still exist on 14?

 

Addendum 2 - How do I go all the way back to that time period in the Operations Log? The Op Log only goes back to the end of June...

Edited July 12, 2010 by Guest

[Daniels]

Have you tried restoring to a different directory instead of doing an inplace restore? Depending which option you choose a restore Retrospect will only replace files that are newer.

 

As for the log you would need to increase the number of lines stored as well as the size. Once you reach the line value it erases the earlier entries to make room for new entries so you would lose the older information. Once these entries are erased the only way to view them is to look at an older log file.

[ignprint]

Daniels, thank you for your response! And, for the log info.

 

I think that a perfect storm of things happened all at the same time - the server dying, Retrospect getting updated (which I found out happened around this time), and the tape getting overwritten.

 

I think that that tape is gone and is now known as 14 (from 13).

 

I do have a question though -

 

If the tape is overwritten, does Retro not back up that data? What I mean is - the data is still on the servers. Does Retro not backup that data and only backs up the new data as it is adjusted/placed on that server? FYI - it is an incremental backup (full backup of the server took place earlier in the year).

 

Thank you again for your time in reading this and imparting your experiences and knowledge. I very much appreciate it.

 

Anthony

Edited July 13, 2010 by Guest

[twickland]

I am doing a 'search-retrieve' on them but they do not actually restore. When I do a retrieve on the files, there is an asterisk next to the files.

As you note, the asterisk indicates that the files in question are on a tape member that has been marked "missing." Whether you are able to eventually retrieve the files depends on whether the tape is truly missing; if it is, you are out of luck.

 

To check on the status of your tape members, go to Configure> Backup Sets> Configure [your_backup_set]> Members. If any tapes are marked "missing" when you still have them in your possession, you can change their status in this window.

 

At the time, a tape had its name changed by Retro from LTO A - 13 to LTO A - 14. Does that mean all that data was eaten by Retro or does it still exist on 14?

Retrospect will not change the name of a tape on its own. In addition to when a tape member becomes full, Retrospect will ask for a new member if the previous member cannot be written to for any reason. However, unless you have an autoloader with an available blank tape, somebody will need to manually respond to Retrospect's request.

 

Note that Retrospect will always want to move on to a new blank tape. Unless someone has subsequently chosen to erase the tape, has thrown it away, or the tape itself is too damaged to read, you should always have your old data.

 

How do I go all the way back to that time period in the Operations Log? The Op Log only goes back to the end of June...

You may have the log size limit set too small. However, unless you changed the size, the default limit of 10MB should allow well over 100,000 lines in the log. Bizarrely, though, the Retrospect app will not be able to display all the lines in this default limit. The "solution" is to first close Retrospect and then open the operations log in an application that can read text files. Unfortunately, that method will make visual hash of the special Retrospect symbols in the log, but you should still be able to make sense out of the rest.

[Daniels]

Incremental backups backup any data that was changed so any existing files that were changed do get backed up and depending how you have it set up it should append this to end of the media set.

 

Usually when a tape goes from 13 to 14 it means the media set was recycled and used an additional tape and the process usually erases the content of the tapes. If the backup set has not been recycled you might be able find the files on the tape.

[twickland]

Usually when a tape goes from 13 to 14 it means the media set was recycled and used an additional tape and the process usually erases the content of the tapes.

Your comment doesn't make sense to me, but does raise a potential issue. IGNPRINT/Anthony did not use the standard nomenclature for backup sets/members and I jumped to conclusions about what he meant. Let me back up a bit.

 

When a tape backup set is created (for example, "Backup Set A"), the first member (i.e., physical tape) is named "1-Backup Set A." If other tape members are appended to the backup set, the subsequent members are named "2-Backup Set A," "3-Backup Set A," etc.

 

If the user later performs a New Media backup, Retrospect will generate a new backup set. In my example, the first new backup set will be named "Backup Set A [001]," with the first backup set member being named "1-Backup Set A [001]." However, the old tapes are not reused by Retrospect, and the old backup set catalog is not reset; the old data remains intact. That is in fact the whole purpose of performing a new media backup.

 

On the other hand, if the user performs a Recycle backup, the backup set catalog is reset and Retrospect will reuse the old tapes. However, in this case the backup set name remains the same (in my example, it will still be named "Backup Set A"); there will be no increment to any number at the tail of the backup set's name as you seem to suggest in your comment, Daniels.

 

At this point, it's not clear what Anthony (or someone else at his firm) did, nor what type of a backup structure they have in place. I think we'll need to hear from him with some more details if we're to be of help.

[Daniels]

Lets say that you are backing up more data than before which requires additional tapes you could possibly have a tape number change from 13 to 14.

 

The old 13 became 14 because there was a new tape used before the old 13 was added to the backup set. Since it was 14th tape used for backup set it would be named LTO A - 14 and lose it spot as the 13th member. If this is the case then the data on the tape is probably gone as well as in the catalog file.

[twickland]

Lets say that you are backing up more data than before which requires additional tapes you could possibly have a tape number change from 13 to 14.

 

The old 13 became 14 because there was a new tape used before the old 13 was added to the backup set. Since it was 14th tape used for backup set it would be named LTO A - 14 and lose it spot as the 13th member.

Daniels, have you ever observed such a tape number change? I doubt it, because your discussion of working with tapes bears no resemblance to how Retrospect actually behaves with tape backup sets.

 

Go back and reread my description of the two different numbering schemes that Retrospect uses to number tape members and to number subsequent backup sets in those cases where a new media backup has been performed. That's how Retrospect works.

 

The only time that Retrospect will erase data without the operator issuing a manual command to do so is during a recycle backup. Even then, Retrospect is picky. If a tape already has data on it, it has to be the correct tape member of the correct backup set. In other words, Retrospect will only reuse tapes with data on them from the backup set being recycled, and will only accept the tape members in the proper order (1-Backup Set A, 2-Backup Set A, etc.).

 

Retrospect will accept blank or erased tapes anytime it wants to create a new backup set member. This would be the only circumstance I know of where the tape member numbering could "change." However, before this could happen, the operator would have needed to erase the tape manually. In this case, it would be the manual erase that changed the tape numbering, and not some action by Retrospect. (When a tape has been erased, Retrospect no longer cares what name the operator may have given the tape during the erase process.)

[Daniels]

I have seen the following situation happen: Doing a recycle backup Retrospect grabs a blank tape and uses that for the next media member so this blank media becomes tape number 2. If this scenario happened and then the old 13 was added to backup set it would have become 14.

 

The other possibility is that Retrospect grabbed a blank tape because the last member was missing from media set and then old tape 13 was added back to the media set which would make it 14.

 

My other question is what would happen if you are doing a recycle backup and tape 2 is missing, would the backup just stop backup and wait for tape 2 or start using blank media to continue the backup?

[twickland]

I have seen the following situation happen: Doing a recycle backup Retrospect grabs a blank tape and uses that for the next media member so this blank media becomes tape number 2.

Yep, that's entirely possible.

 

If this scenario happened and then the old 13 was added to backup set it would have become 14.

 

The other possibility is that Retrospect grabbed a blank tape because the last member was missing from media set and then old tape 13 was added back to the media set which would make it 14.

While it is possible for you to do either of the above, it cannot occur without operator intervention.

 

If neither the correctly named/numbered tape member nor a blank tape is available during a recycle backup, Retrospect stops the backup and puts up a window asking for help. To continue with the wrong tape, the operator would manually have to select the tape in that window and tell Retrospect that that is the tape to use. If you haven't selected "Minimal erase confirmation" in the Preferences, Retrospect will actually ask you twice.

 

My other question is what would happen if you are doing a recycle backup and tape 2 is missing, would the backup just stop backup and wait for tape 2 or start using blank media to continue the backup?

A blank tape is as good as the correctly named/numbered tape member when performing a recycle backup.

[Daniels]

I am beginning to wonder if the old 13 actually had data on it or if it was an extra member of the old backup set?

 

I have never tried doing a recycle backup with a media set member marked as lost so I don't what would happen in this case.

[twickland]

I am beginning to wonder if the old 13 actually had data on it or if it was an extra member of the old backup set?

I think speculating here is useless without more input from Anthony.

 

I have never tried doing a recycle backup with a media set member marked as lost so I don't what would happen in this case.

The information about whether a media member is missing or not is included in the backup set's catalog. A recycle backup resets the catalog, so the status of a media member becomes largely irrelevant. However, to avoid having to continually OK the "wrong" media members during the recycle backup, it's best to insert a blank tape in place of the missing member when that missing member's place in the backup sequence arrives.