Curious (spurious?) verify error?

[jhg]

I have a large (2.5TB) library of images on disk, for which I keep two separate Retrospect disk archives.  Occasionally (once or twice per year) run a Retrospect media verify on both archive drives.  This time around I got a single verify error on each drive, flagging a different file on each drive.  Here's one of them

Here's the same file in Windows Explorer

 

And in the Backup Set

The only difference is that Retrospect seems to think that the file in the backup set is newer than the one on disk.

I restored that single file to a temporary location and compared it to the original (still on disk at its original location) with WinMerge and the Windows "comp" command.  Both WinMerge and "comp" report that the files are identical. 

Then I ran "chkdsk /r" on both drives, neither of which flagged any bad sectors or other filesystem errors.

So I have 2 questions:

1. What exactly does the error message mean?  When was the "Generated MD5 digest" calculated (at backup time or during the verify)? Where is the "stored MD5 digest" actually stored?
2. Since the original and restored copy of the file compare equal, I assume this is a problem in the backup set and/or catalog.  How do I get Retrospect to "fix" this error and get itself back in sync?

[Lennart_T]

My best guess is that you got a (temporary) read error from the disk during the verify. An error that is not repeatable.

The MD5 digest is stored either in the catalog file or in the backup set itself. It is created/stored during the backup.

What happens if you run the verify again?

How is the disk connected to the computer? (USB isn't one of the most reliable types of connection.)

[jhg]

The disk is USB3 and normally has worked flawlessly.

I ran the verify again on one of the archives and it flagged the same file again.

Is there a way to make Retrospect archive that one file again?

[Nigel Smith]

On 3/9/2020 at 6:16 PM, jhg said:

The only difference is that Retrospect seems to think that the file in the backup set is newer than the one on disk.

Stupid question but, given the lack of column headers, one that I need to ask -- in the backup set screenshot, is that the file modification date that's listed or the backup date?

[DavidHertzberg]

On 3/9/2020 at 7:51 PM, jhg said:

The disk is USB3 and normally has worked flawlessly.

I ran the verify again on one of the archives and it flagged the same file again.

Is there a way to make Retrospect archive that one file again?

jhg,

In regard to the question that Nigel Smith asked, is it possible you backed up that one file again on 8 November 2017 with the Generate MD5 Digests During Backup Operations (page 402 of the Retrospect Windows 17 User's Guide) preference disabled?  That way the stored MD5 digest (in the Catalog File for the archive Backup Set) wouldn't match the one generated during the Verify run.

If that didn't happen, let's consider the fact that the file in question is a .dng file.  The Wikipedia article says DNG "mandates significant use of metadata".  I don't know anything about digital photography, but it's possible there's some aspect of the metadata that Retrospect doesn't handle correctly.  Is the other file that gives a Verify error also a .dng file?

To make Retrospect back up that one file again, first create a Subvolume (pages 420-422 of the UG) for the directory containing that original file.  Then create a Backup run from that Subvolume to your archive file, specifying the file name in the Include section of a custom Selector (pages 432-448 of the UG).

[x509]

@jhg.  Picking up on what DavidHertzberg wrote, were you running Lightroom and doing DEVELOP edits or updating metadata in LIBRARY while the backup operation was running?

 

x509

[jhg]

@DavidHertzberg no, I never alter the settings, this is from a standard archive script I run regularly.

@x509 no, I don't run the archive while Lightroom is running... although it's conceivable I accidentally left Lightroom in the background during an archive run.

Since the file on disk compares correctly with the restored copy, and

chkdsk /r

  on the drive turns up no errors, I'm not going to worry about this one.

[x509]

On 3/12/2020 at 12:10 AM, DavidHertzberg said:

jhg,

In regard to the question that Nigel Smith asked, is it possible you backed up that one file again on 8 November 2017 with the Generate MD5 Digests During Backup Operations (page 402 of the Retrospect Windows 17 User's Guide) preference disabled?  That way the stored MD5 digest (in the Catalog File for the archive Backup Set) wouldn't match the one generated during the Verify run.

If that didn't happen, let's consider the fact that the file in question is a .dng file.  The Wikipedia article says DNG "mandates significant use of metadata".  I don't know anything about digital photography, but it's possible there's some aspect of the metadata that Retrospect doesn't handle correctly.  Is the other file that gives a Verify error also a .dng file?

To make Retrospect back up that one file again, first create a Subvolume (pages 420-422 of the UG) for the directory containing that original file.  Then create a Backup run from that Subvolume to your archive file, specifying the file name in the Include section of a custom Selector (pages 432-448 of the UG).

David,

(catching up a bit here).

DNG is a photo format introduced by Adobe.  It's different from other "RAW" photo file formats in that it is OK for an application to write metadata into the DNG.  The best practice for other photo RAW formats, such as Canon's CR2 and Nikon's NEF, is to never write any metadata into these files, except date corrections.  Otherwise, there is the real risk that some photo editors may not be able to read a CR2 or NEF file.  So whatever happened with the OP's situation, metadata isn't the issue.

 

Phil Burton