Jump to content
cmcfarling

Error -105 unexpected end of data during catalog rebuild

Recommended Posts

Due to Retrospect's imperfect grooming feature, disk based backup sets don't always get groomed thoroughly causing disk backup sets to fill up. Once this happens the catalog must be rebuilt in order to perform a successful groom operation. I've had to go through this procedure several times in the last 18 or so months. Normally the rebuild completes with no problems.

 

However, I have a catalog now that will not rebuild. I'm using the Repair Catalog >> Recreate from disks command to recreate the catalog from the disk based backup data. Once Retrospect gets to the point where it begins to read the backup data from the disk, the following error come up:

 

Device trouble: 1-Production2, error -105 (unexpected end of data)

 

The disk is good working order. It is used for backup data for several other backup sets as well, all of which are functioning without problems. I've run chkdsk on the disk also and it checks out fine.

 

Does anyone have any experience with this particular problem? Any suggestions for getting this catalog to rebuild at this point without having to wipe out all of the existing backup data and start over from scratch?

Share this post


Link to post
Share on other sites

I have seen this once and I had to delete the backupset and recreate the backupset.

 

Recreating the catalog file did not help.

 

I believe this must be a error in Retrospect. Somehow backupsets can get corrupted and I personal think it´s caused by Retrospect itself.

 

Robert

 

Share this post


Link to post
Share on other sites
I personal think it´s caused by Retrospect itself.

 

It isn't caused by Retrospect. It can happen when a tape contains a bad end of data marker or with a corrupt data file.

 

This sounds like you have a corrupt .rdb data file. If the rebuild always stops at the exact same spot, then the .rdb file being read is your problem. You can turn on device logging to 7 (ctrl alt-p-p in Retrospect) and do the rebuild again. The operations log will show you the .rdb number that is last being used when it fails.

Share this post


Link to post
Share on other sites

I changed the logging level to 7 but did not get any additional info. I then tried changing the log level of Engine, Trees and Volumes, Backup Sets and Catalog Files and Networking to 7 as well, which still did not generate any useful info in the log. The only additional entry that appeared was

 

xipSetDevice: can't seek, error -105 (unexpected end of data)

 

Any other suggestions?

 

 

Share this post


Link to post
Share on other sites

Does it tell you which .rdb file it is reading when the error happens? That would be the corrupt data file.

Share this post


Link to post
Share on other sites

I have the exact same problem; the error happens immediately when trying to do the recatalog, and there is nothing listed about any rdb files when the extra logging is enabled. Note that this is a *disk* backup set, not tape.

 

Do I have any option besides deleting and recreating the backup set? With 3 of us having this problem within the last couple months, it sure seems like a new bug in version 7.6.

 

Share this post


Link to post
Share on other sites

I don't think you can transfer from a backup set when you can't create a catalog file for it(?). :(

 

Since I got no response, I trashed the backup set and recreated it.

 

Share this post


Link to post
Share on other sites

i am really impressed to see the official response to this error is 'nuke your backup and start again' -- i just lost my entire backup and our company just lost about 100 hours of work because of this error, which is the SECOND time retrospect has trashed its own data.

backup exec never had this problem, and a backup system that trashes its own backup data is worse than useless.

being burned twice is enough to convince me to move back to backup exec, and i am just sad that i paid for tech support for this product, which so far has been your techs telling me to nuke all my backup data and start again, great product.

 

Share this post


Link to post
Share on other sites
i am really impressed to see the official response to this error is 'nuke your backup and start again

 

As the only EMC employee posting to this thread, I have never said this. I did say you may have some corrupt backup data, which would normally be limited to a single .rdb file of 600 MB in size.

 

The error is coming from the file system and not Retrospect because it is having trouble reading data.

 

Does the Windows Event log have any errors that correspond with the Retrospect reported error?

Edited by Guest

Share this post


Link to post
Share on other sites
i am really impressed to see the official response to this error ...

... your techs telling me...

It seems you are not aware that this is a user-to-user forum, not the official feedback or support channel.

So nothing here is "official", even if there are employees posting here. And giving good advice, I must say. It may not be what posters WANT to hear, but that's life...

 

Share this post


Link to post
Share on other sites

ok, having calmed down a bit now, i would like to ask the following:

 

1. is there ANY way to recover files from a 'corrupt' backup set?

 

2. if not, is there a better way to do a backup set for the purposes of recovery, i.e. instead of using a full disk set, should i do smaller sized file sets, etc, something where i could minimize the amount of data lost if this happens?

 

i think what is causing this corruption is that retrospect is hanging, not properly shutting down, when the system is rebooted. i will stop the backup jobs and all pending jobs, the night before, then schedule a reboot for early a.m. when nobody is using it, the next day when i check it, sometimes the day after, is when i find that the backup jobs are hung on corrupt backup sets...?

is there any way to tell if retrospect is still trying to access a backup set and some way to force it to stop?

 

Share this post


Link to post
Share on other sites

Just posting on this again to see if anyone has any suggestions for Retrospect destroying its backup sets? Is there any way to mitigate the damage caused by this bug, will it be fixed anytime soon, is there a recovery or repair tool for the data files? Or is it just sort of a russian roulette style backup system?

 

Share this post


Link to post
Share on other sites

I have asked support about the same thing and the answer was:

 

"We will be looking at the grooming feature, but I don't expect bug fixes to happen until the next major release at the earliest. "

 

How did EMC get so big?

 

Robert

 

 

Share this post


Link to post
Share on other sites

Here's a log excerpt from a Grooming session within a backup session where I got the same error as everyone else:

 

 

- 2/20/2010 10:56:47 PM: Copying /state/partition1 on goddard (rhel4)

2/21/2010 12:04:57 AM: Grooming Backup Set goddard_disk...

2/21/2010 2:58:08 AM: Groomed 2,916.3 GB from Backup Set goddard_disk.

File "/state/partition1/temp/Gau-29565.rwf": can't read, error -1101 (file/directory not found)

File "/state/partition1/temp/Gau-29565.scr": can't read, error -1101 (file/directory not found)

2/21/2010 3:02:09 AM: Snapshot stored, 446.5 MB

2/21/2010 3:02:17 AM: Comparing /state/partition1 on goddard (rhel4)

Device trouble: "1-goddard_disk", error -105 (unexpected end of data)

2/22/2010 8:27:05 AM: Execution incomplete

Remaining: 19 files, 28.4 GB

Completed: 0 files, zero KB

Performance: 16.0 MB/minute (624.0 copy, 0.0 compare)

Duration: 1 09:30:16 (03:18:59 idle/loading/preparing)

 

I deleted the last day's worth of rdb files, about 15, and did a Recatalog that is working now.

 

In my backup set, I have over 9500 rdb files, about 5.1TB, and Groom took about 3 hours, then 3 minutes later, I get the "end of data" warning.

 

I am getting file/folder not found as client is searching for temp files that are no longer there as this is Linux high-performance cluster running jobs that generate many temp files that go away.

 

So, maybe I got lucky and my corrupt rdb files were on my last backup/grooming day??

 

BTW, my disk set is on a 12.2TB RAID6 array of 20 disks, and I'm running 7.6 MultiServer on x64 Windows 2003 R2.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×