Incremental Backup: Unmodified Files Backed up Multiple Times

[gnoelken]

I'm running a proactive backup to a disk backup set. The backup is incremental and will backup the client every 24 hours. Recently the backup set filled up and was prompting me for new media. Looking closely at the backup set sessions, I noticed that all files on some drives were getting backed up in subsequent sessions. Because these files may add up to 100 Gb, the backup set fills up quickly when the same files are backed up multiple times.

 

To illustrate. A 1.3 Gb file in one folder on the client was originally backed up on July 14, 2010:

 

 

Note the 'Created' and 'Modified' Dates.

 

Then, for some reason on October 5, 2010, this file, along with the rest of the files on the drive, were backed up again:

 

 

Note that the 'Created' and 'Modified' date did not change.

 

Then, one week later on October 12, 2010, the drive was backed up again(!):

 

 

Again, the 'Created' and 'Modified' dates did not change.

 

Notice it does not happen every night.

 

The 'Don't add duplicates to Backup Set' for the backup set is checked.

 

The user of this system did tell me he uses 'Robocopy' to mirror this drive to a USB drive. He's been doing it for several months.

 

The Retrospect client is 7.6.106 running on Windows Server 2003 (NTFS files system)

 

Retrospect Server: 7.7.325 (64-bit)

Driver Update and Hot Fix, version 7.7.3.102(64-bit)

Running on Windows Server Standard 2008 SP2 64-bit OS

4 Gb Memory

 

Any suggestions?

 

Greg

[Lennart_T]

Retrospect uses several matching criteria to find new or changed files. If one of the criteria has been changed, Retrospect will back up the file again. On Windows, Retrospect looks at creation date and time, modified date and time, size and name. If match only in same location option is set, Retrospect matches on the path, volume name and drive letter also.

 

By default, the archive attribute is not used as a matching criteria in Windows, allowing for true and reliable backups to multiple backup sets.

 

Is this a multiple boot client? We're running Retrospect 7.6 and we have a multiple boot client with a data volume that gets fully backed up twice, once when booted XP and once when booted in Vista. So different versions of Windows presents different metadata to the Retrospect Client.

 

Maybe it's a Windows Update that causes the files to be backed up again?

[gnoelken]

Lennart,

 

Thank you for your prompt reply.

 

Retrospect uses several matching criteria to find new or changed files. If one of the criteria has been changed, Retrospect will back up the file again. On Windows, Retrospect looks at creation date and time, modified date and time, size and name. If match only in same location option is set, Retrospect matches on the path, volume name and drive letter also.

 

By default, the archive attribute is not used as a matching criteria in Windows, allowing for true and reliable backups to multiple backup sets.

 

That makes this more perplexing, since neither the creation date or modified date has changed. Maybe I should set the 'match only in same location' option.

 

 

Is this a multiple boot client? We're running Retrospect 7.6 and we have a multiple boot client with a data volume that gets fully backed up twice, once when booted XP and once when booted in Vista. So different versions of Windows presents different metadata to the Retrospect Client.

 

This is not dual boot -- just the Win Server 2003 OS.

 

Maybe it's a Windows Update that causes the files to be backed up again?

 

That is a good thought. I checked the event logs. Packages were downloaded on October 4, but not installed until October 13 at 1:52PM. It is odd that the duplicate backups occurred within this Oct 4-13 period.

 

Again, thanks for any suggestion you may have.

 

Greg

[rhwalker]

Maybe I should set the 'match only in same location' option.

No, that addresses a different issue - whether two apparently-identical copies of a file at different locations (different directories or computers) should be backed up multiple times.

 

That setting shouldn't have any affect on this particular issue, which is simply a question of what metadata is checked when Retrospect decides whether changes have been made.

 

It's possible that the actual NTFS timestamp (which is much more specific and more granular than the timestamps displayed in the GUI) isn't matching what is reported in the backup. The way to test this would be to see if this replication occurs on a different filesystem.

 

Another possibility is if "Robocopy" somehow changes metadata that is not being reported. If you have some utilities to view the full metadata for the file(s) at issue, you might investigate in that direction, or perhaps suspend "Robocopy" operations for a few days while you observe the results.

 

Russ

[Cygnis]

(Sorry, clicked the wrong "Reply" button: my question is directed at the original poster, not rhwalker):

 

Do you have "Back up file system security information from servers" enabled (under "Security" in the Backup options)? Have you tried disabling it?

 

We were having the same issue at work (on a Win 2003 server) before disabling that option. (We've left the "Back up FOLDER security information from servers" option enabled without any problems.)

 

I would assume that failing to back up file security information could have side-effects when restoring certain types of data, so I can't guarantee it will be an ideal solution for your situation.

Edited October 28, 2010 by Guest

[gnoelken]

Do you have "Back up file system security information from servers" enabled (under "Security" in the Backup options)? Have you tried disabling it?

 

I do have it enabled. I've disabled it and will monitor the backup set.

 

Thank you for your suggestion.

 

Greg

 

[gnoelken]

 

Do you have "Back up file system security information from servers" enabled (under "Security" in the Backup options)? Have you tried disabling it?

 

We were having the same issue at work (on a Win 2003 server) before disabling that option. (We've left the "Back up FOLDER security information from servers" option enabled without any problems.)

 

I would assume that failing to back up file security information could have side-effects when restoring certain types of data, so I can't guarantee it will be an ideal solution for your situation.

 

Well...It's been 2 weeks since disabling 'Back up file system security information from servers' and no duplicate backups! Let's hope your suggestion solved the problem.

 

Thank you Cygnis!

 

Greg

[Cygnis]

No problem! Glad to hear it worked out for you.