Suddenly, very slow MD5 verification

[baweeks]

-All of a sudden, MD5 verification of backups has slowed from 2+ GB/min to ~300 MB/min. No idea why. It was not the transition to 10.6.3; I have had (usual) fast verifications after update to 10.6.3.

-MacBook Pro 4,1, 4GB RAM, engine and console on same 1-person Mac, direct-connect backup to a USB2 drive, Retro 8.1.626 with the driver update that came out a month or so ago, OS 10.6.3, SimpleCrypt, software data compression. Write speeds remain the same, ~500MB/min (is that about right, considering my configuration?). I tested trying a backup of just my docs folder onto a new Media Set, and it still has slow MD5 backups. Speed of verification varies now from 300MB/min to ~1GB/min; far slower than formerly usual ~2GB/min.

-Have run utilities to clean caches, have done the Combo 10.6.3 updater, run Disk Warrior, permissions, stopped and restarted the Engine, all to no avail.

-Ideas? Completely remove and reinstall Retrospect? Could it be a config file corruption? I could compress and save my config files, and try freshly-created config files (ouch if I have to rebuild all scripts, etc). I do have stored and saved config files from months ago I could restore and try using. Other ideas for suddenly slow verification?

[Mayoff]

have you tried backups with MD5 turned off to compare performance?

[baweeks]

Dear Robin:

-Thanks for your reply.

-I can turn off MD5 verification. I would imagine it would speed up the write phase of the backup a small amount, from the ~500 MB/min it is now. IIUC, that speed increase would be because there would no longer be the processor overhead of creating the MD5 key concurrently as it's writing the backup file. I've never had problems with the write phase. (Although, is the ~500GB/min I'm getting, about what's expected for backing up to a direct-connect USB2 drive with SimpleCrypt and software compression on? That's about double of what 6.1 got, BTW).

-Your response makes me wonder if I was unclear. The sudden slowdown in speed is not in the write phase, it's in the verification phase. The verification phase is somewhere around 90% slower.Used to be ~2GB/min. I could turn MD5 verification off, but would that not just kill verification entirely, leaving me with a write phase only? What I'm looking for is for the verification phase to speed back up to where it once was.

-Willing to try anything you suggest that can help.

-Probably too late for the soon upcoming release, but I'd have a few interface suggestions. How to communicate them?

-baweeks

[baweeks]

-I meant to ask one more thing, Robin. With every MD5 verification, Retrospect always says "execution successful," but gives a warning triangle, and looking at the log it says that about 5% or so of the files that were to be verified never got verified, they are "remaining." Is this a bug to be fixed, that it does not verify all the files that need verification? Would those un-verified files get verified the next time that backup script is run?

-Finally, is there any way in 8.1 to see exactly what files got backed up in one certain (to use a deprecated 6.1 term), "session." I ask this because, if one does an archive-and-reinstall of the OS, would Retrospect ignore those new OS files, since they all have the same size, creation date, etc? Or is there some way that Retro sees the re-written OS files as fit to be backed up again? I'm curious how this scenario would work out.

-baweeks

[Maser]

-I meant to ask one more thing, Robin. With every MD5 verification, Retrospect always says "execution successful," but gives a warning triangle, and looking at the log it says that about 5% or so of the files that were to be verified never got verified, they are "remaining." Is this a bug to be fixed, that it does not verify all the files that need verification? Would those un-verified files get verified the next time that backup script is run?

-baweeks

 

 

Examine the log closely -- it's entirely possible that the files were backed up, but then were *not* on the computer during the verification pass (this happens often if you back up an "in-use" system and it's indicating certain cache files/mail messages, etc. weren't able to be verified. That's pretty common.

[Daniels]

I run into this error on Retrospect 7.6 on Windows especially when the anti-virus updates the virus definitions while the backup is running. I schedule all my backups at night so I only occasionally run into this problem. The only other error I see during comparison is the date/time changed message.

If you backup while people are working this error could be very common.