cmcfarling Posted January 14, 2009 Report Share Posted January 14, 2009 Due to Retrospect's imperfect grooming feature, disk based backup sets don't always get groomed thoroughly causing disk backup sets to fill up. Once this happens the catalog must be rebuilt in order to perform a successful groom operation. I've had to go through this procedure several times in the last 18 or so months. Normally the rebuild completes with no problems. However, I have a catalog now that will not rebuild. I'm using the Repair Catalog >> Recreate from disks command to recreate the catalog from the disk based backup data. Once Retrospect gets to the point where it begins to read the backup data from the disk, the following error come up: Device trouble: 1-Production2, error -105 (unexpected end of data) The disk is good working order. It is used for backup data for several other backup sets as well, all of which are functioning without problems. I've run chkdsk on the disk also and it checks out fine. Does anyone have any experience with this particular problem? Any suggestions for getting this catalog to rebuild at this point without having to wipe out all of the existing backup data and start over from scratch? Quote Link to comment Share on other sites More sharing options...
robvil Posted January 16, 2009 Report Share Posted January 16, 2009 I have seen this once and I had to delete the backupset and recreate the backupset. Recreating the catalog file did not help. I believe this must be a error in Retrospect. Somehow backupsets can get corrupted and I personal think it´s caused by Retrospect itself. Robert Quote Link to comment Share on other sites More sharing options...
Mayoff Posted January 16, 2009 Report Share Posted January 16, 2009 I personal think it´s caused by Retrospect itself. It isn't caused by Retrospect. It can happen when a tape contains a bad end of data marker or with a corrupt data file. This sounds like you have a corrupt .rdb data file. If the rebuild always stops at the exact same spot, then the .rdb file being read is your problem. You can turn on device logging to 7 (ctrl alt-p-p in Retrospect) and do the rebuild again. The operations log will show you the .rdb number that is last being used when it fails. Quote Link to comment Share on other sites More sharing options...
cmcfarling Posted January 19, 2009 Author Report Share Posted January 19, 2009 I changed the logging level to 7 but did not get any additional info. I then tried changing the log level of Engine, Trees and Volumes, Backup Sets and Catalog Files and Networking to 7 as well, which still did not generate any useful info in the log. The only additional entry that appeared was xipSetDevice: can't seek, error -105 (unexpected end of data) Any other suggestions? Quote Link to comment Share on other sites More sharing options...
Mayoff Posted January 19, 2009 Report Share Posted January 19, 2009 Does it tell you which .rdb file it is reading when the error happens? That would be the corrupt data file. Quote Link to comment Share on other sites More sharing options...
cmcfarling Posted January 19, 2009 Author Report Share Posted January 19, 2009 No, there is no reference to any .rdb file. Quote Link to comment Share on other sites More sharing options...
spawn Posted February 13, 2009 Report Share Posted February 13, 2009 I have the exact same problem; the error happens immediately when trying to do the recatalog, and there is nothing listed about any rdb files when the extra logging is enabled. Note that this is a *disk* backup set, not tape. Do I have any option besides deleting and recreating the backup set? With 3 of us having this problem within the last couple months, it sure seems like a new bug in version 7.6. Quote Link to comment Share on other sites More sharing options...
emulator Posted February 20, 2009 Report Share Posted February 20, 2009 This might get you the same error, but have you tried the "transfer backup sets" function to copy the data from the bad backup set to a new one? Quote Link to comment Share on other sites More sharing options...
spawn Posted February 20, 2009 Report Share Posted February 20, 2009 I don't think you can transfer from a backup set when you can't create a catalog file for it(?). Since I got no response, I trashed the backup set and recreated it. Quote Link to comment Share on other sites More sharing options...
hapbt Posted April 1, 2009 Report Share Posted April 1, 2009 i am really impressed to see the official response to this error is 'nuke your backup and start again' -- i just lost my entire backup and our company just lost about 100 hours of work because of this error, which is the SECOND time retrospect has trashed its own data. backup exec never had this problem, and a backup system that trashes its own backup data is worse than useless. being burned twice is enough to convince me to move back to backup exec, and i am just sad that i paid for tech support for this product, which so far has been your techs telling me to nuke all my backup data and start again, great product. Quote Link to comment Share on other sites More sharing options...
Mayoff Posted April 1, 2009 Report Share Posted April 1, 2009 (edited) i am really impressed to see the official response to this error is 'nuke your backup and start again As the only EMC employee posting to this thread, I have never said this. I did say you may have some corrupt backup data, which would normally be limited to a single .rdb file of 600 MB in size. The error is coming from the file system and not Retrospect because it is having trouble reading data. Does the Windows Event log have any errors that correspond with the Retrospect reported error? Edited April 1, 2009 by Guest Quote Link to comment Share on other sites More sharing options...
Lennart_T Posted April 2, 2009 Report Share Posted April 2, 2009 i am really impressed to see the official response to this error ...... your techs telling me... It seems you are not aware that this is a user-to-user forum, not the official feedback or support channel. So nothing here is "official", even if there are employees posting here. And giving good advice, I must say. It may not be what posters WANT to hear, but that's life... Quote Link to comment Share on other sites More sharing options...
hapbt Posted April 3, 2009 Report Share Posted April 3, 2009 ok, having calmed down a bit now, i would like to ask the following: 1. is there ANY way to recover files from a 'corrupt' backup set? 2. if not, is there a better way to do a backup set for the purposes of recovery, i.e. instead of using a full disk set, should i do smaller sized file sets, etc, something where i could minimize the amount of data lost if this happens? i think what is causing this corruption is that retrospect is hanging, not properly shutting down, when the system is rebooted. i will stop the backup jobs and all pending jobs, the night before, then schedule a reboot for early a.m. when nobody is using it, the next day when i check it, sometimes the day after, is when i find that the backup jobs are hung on corrupt backup sets...? is there any way to tell if retrospect is still trying to access a backup set and some way to force it to stop? Quote Link to comment Share on other sites More sharing options...
hapbt Posted April 16, 2009 Report Share Posted April 16, 2009 Just posting on this again to see if anyone has any suggestions for Retrospect destroying its backup sets? Is there any way to mitigate the damage caused by this bug, will it be fixed anytime soon, is there a recovery or repair tool for the data files? Or is it just sort of a russian roulette style backup system? Quote Link to comment Share on other sites More sharing options...
robvil Posted April 17, 2009 Report Share Posted April 17, 2009 I have asked support about the same thing and the answer was: "We will be looking at the grooming feature, but I don't expect bug fixes to happen until the next major release at the earliest. " How did EMC get so big? Robert Quote Link to comment Share on other sites More sharing options...
bnorum Posted February 24, 2010 Report Share Posted February 24, 2010 Here's a log excerpt from a Grooming session within a backup session where I got the same error as everyone else: - 2/20/2010 10:56:47 PM: Copying /state/partition1 on goddard (rhel4) 2/21/2010 12:04:57 AM: Grooming Backup Set goddard_disk... 2/21/2010 2:58:08 AM: Groomed 2,916.3 GB from Backup Set goddard_disk. File "/state/partition1/temp/Gau-29565.rwf": can't read, error -1101 (file/directory not found) File "/state/partition1/temp/Gau-29565.scr": can't read, error -1101 (file/directory not found) 2/21/2010 3:02:09 AM: Snapshot stored, 446.5 MB 2/21/2010 3:02:17 AM: Comparing /state/partition1 on goddard (rhel4) Device trouble: "1-goddard_disk", error -105 (unexpected end of data) 2/22/2010 8:27:05 AM: Execution incomplete Remaining: 19 files, 28.4 GB Completed: 0 files, zero KB Performance: 16.0 MB/minute (624.0 copy, 0.0 compare) Duration: 1 09:30:16 (03:18:59 idle/loading/preparing) I deleted the last day's worth of rdb files, about 15, and did a Recatalog that is working now. In my backup set, I have over 9500 rdb files, about 5.1TB, and Groom took about 3 hours, then 3 minutes later, I get the "end of data" warning. I am getting file/folder not found as client is searching for temp files that are no longer there as this is Linux high-performance cluster running jobs that generate many temp files that go away. So, maybe I got lucky and my corrupt rdb files were on my last backup/grooming day?? BTW, my disk set is on a 12.2TB RAID6 array of 20 disks, and I'm running 7.6 MultiServer on x64 Windows 2003 R2. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.