twickland Posted January 4, 2011 Report Share Posted January 4, 2011 Does anyone know what Retrospect does during and after a fast catalog rebuild? Here's the background: In a problem reported in the bugs section, Retrospect listed a number of files in one tape media set's catalog that had actually been backed up to a different tape media set. Because of this error, the first media set's catalog needed fixing. As an experiment, I told Retrospect to "repair" (rather than "rebuild") the catalog. I inserted the last tape member and happily Retrospect determined on its own that a rebuild was the correct action to take and began performing it. During the rebuild, Retrospect presumably performed a fast catalog rebuild. As I understand it, a fast rebuild means obtaining the catalog covering the earlier tape members from the beginning of this last member and then updating the catalog from that point on. The results were ultimately successful. Here's my question: There was just under 100 GB of data on the last tape member. The rebuild initially began quickly and completed 69 GB on this member in about 20 minutes, but I noticed then that the process was rapidly slowing down. Over the next half-hour, only a few GB were added, and the tape drive was now being accessed only sporadically. I went to Activity Monitor and noted that Retrospect Engine was using only a small CPU percentage and relatively small amount of RAM (around 70 MB). Overall, the CPU was largely idle and there was plenty of free RAM. What I did notice was a lot of writing to disk (writes, not reads). In Finder, the size of the rebuilding catalog was rapidly bloating. Its original size had been just under 1.8 GB on disk, but it was now marching towards 4 GB. At this point I left for the day, having quit the Console in the meantime, so I don't know exactly what happened during the progress up to the end. Launching the Console and checking this morning, I found that the remaining 25 GB or so of data on the tape had taken an additional 4 hours to catalog, which seems absurdly slow. The final size of the catalog, as indicated in Finder, was just under 6 GB, more than triple the size of the original catalog (which, if anything, should already have been larger than expected, since it included files that were not actually on the tape). Here's where things really got weird: After the rebuild, Retrospect no longer knew where the catalog was located. I pointed it to the correct location and then I accessed the catalog (by performing a dummy restore) to see if I could determine if there was anything funny about the file listing that could explain the bloated size of the catalog. The files looked just fine, so I checked the catalog file again in Finder; the file had now shrunk back to the perfectly reasonable size of 1.77 GB! Is it just me or does all of this seem odd? "All" as in the drastically-slowing recatalog process, seemingly-excessive disk writes, bloated rebuilt catalog, losing track of the catalog's location, and finally, the catalog's shrinking only after it had been reaccessed. Quote Link to comment Share on other sites More sharing options...
Maser Posted January 4, 2011 Report Share Posted January 4, 2011 Do you have the catalog set to be compressed? IIRC, when you do a catalog rebuild, the catalog will *not* be compressed until you take some action on the catalog *after* the rebuild has finished. I think I filed a feature request about this one as it's not necessarily a bug, but a design choice. As for the speed of the rebuild -- how many backups were on the set? It might be that the first 69G was the *first* backup and then everything after that was incremental -- which will take longer to create the snapshots in the catalog file. Quote Link to comment Share on other sites More sharing options...
twickland Posted January 4, 2011 Author Report Share Posted January 4, 2011 Do you have the catalog set to be compressed? No, which is what makes the change in size so odd. As for the speed of the rebuild -- how many backups were on the set? It might be that the first 69G was the *first* backup and then everything after that was incremental -- which will take longer to create the snapshots in the catalog file. There are a total of 619 backups in the set with probably 40, all incremental, on the last tape member. I would guess that the first 69 GB are not significantly different from the remaining 28 GB on this member. Are you saying that the snapshots are not stored on the tape, but that during a rebuild, Retrospect is recreating them on the fly? That doesn't seem to jibe with the (relative) ease of retrieving earlier snapshots from the media during a restore. Quote Link to comment Share on other sites More sharing options...
Maser Posted January 5, 2011 Report Share Posted January 5, 2011 Do you have grooming off for your media set, but it was on previously? The size of the catalog is somewhat proportional to how many "Past Backups" you can see for the media set, too. Unfortunately, I don't use tape. If tape media sets are like disk media sets, the snapshots are (I think) stored in the media set, so the only thing recreated during a rebuild is the *catalog* file. But, I'm guessing about tape behavior. Earlier snapshots, though, are faster to retrieve as there's not as much "matching" that needs to be done to create the snapshot to restore -- that's fairly typical in my experience with disk media sets, so I wouldn't think tape would be that much different in this case. Quote Link to comment Share on other sites More sharing options...
twickland Posted January 5, 2011 Author Report Share Posted January 5, 2011 Grooming is not an option for a tape media set, so that doesn't apply. When retrieving an earlier snapshot from a tape media set, one simply has to insert the correct tape member that holds the corresponding past backup. I've always assumed that that snapshot resides on the tape--and is retrieved--as a single file. Quote Link to comment Share on other sites More sharing options...
Maser Posted January 5, 2011 Report Share Posted January 5, 2011 That's probably the case. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.