Data deduplication query

[jelockwood]

Retrospect is supposed to be able to do deduplication and in fact has had this feature since before that term was invented. It can compare a file backed up by a single server against previous backups - a simple incremental comparison, and it can compare the file backed up by multiple servers and in the same location on those servers, and it can compare the file in any location on multiple servers and if identical only needs to store a single copy.

 

In Retrospect 8 or higher this option is located in 'Options' for a backup script under the sub-heading Matching under the heading Backup. I have my script configured to Match source file agains media set, to Don't add duplicate files to the Media set, and not ticked to Match only in the same location/path.

 

There is another option in the parent level of Backup which says Block level incremental backup. This to me implies that data deduplication should look at each block of the file and only backup blocks that are changed. This would for example be particularly useful for backing up files like a virtual machine disk image which are large files but most of the content will not change.

 

My problem is that I recently applied an ACL permissions change to a lot of files. This change would not actually change the content of the file but would change the permissions of the file and might change the modification date as well. I therefore did not expect this to result in a lot of extra storage space being used for the backup. However when the backup did run it seems to be doing a back of the full file again because the amount of space used jumped significantly far more than a normal incremental backup.

 

So it seems that something as minor as a file permission change is resulting in the data deduplication being useless.

 

Has anyone else seen this sort of problem? Any suggestions? Should this be filed as a bug?

 

Note: I am running Retrospect 13.5.0 (173) on El Capitan 10.11.6

[Scillonian]

Block Level Incremental Backup only applies to files over 100MB in size. Additionally if the time since the last backup of the file is more than thirty days the whole file will be copied. These rules can be changed and there is a Retrospect Knowledge Base article that covers this.

 

Unless you tell Retrospect to ignore changes to security settings on a file any change will cause the file to be copied even if the content has not changed. (This is an option on Retrospect for Windows and I assume Rectrospect for MacOS has the same.)

[jelockwood]

Block Level Incremental Backup only applies to files over 100MB in size. Additionally if the time since the last backup of the file is more than thirty days the whole file will be copied. These rules can be changed and there is a Retrospect Knowledge Base article that covers this.

 

Unless you tell Retrospect to ignore changes to security settings on a file any change will cause the file to be copied even if the content has not changed. (This is an option on Retrospect for Windows and I assume Rectrospect for MacOS has the same.)

 

Thank you for this, it does at least seem to explain things. I am not sure I found the same KB article you are referring to so if you have a link to it that would be appreciated. I have not seen any similar options in the Mac version but I would want the security changes backed up as these would of course need to be available to be restored.

 

I am currently recycling the affected backups.

[DavidHertzberg]

The Mac KB article for Block Level Incremental Backup, reachable from the Release Notes for Version 11.0.0, is here.  It has now been "disappeared" from the directory of KB articles shown under Support, but is still hiding on the Retrospect Inc. website.  The article, as is typical for KB articles, omits a step—that you have to click the Pencil icon at the bottom of the window to get the dialog.  I found that trick out by watching this Tutorial; Heavens to Betsy, Mayoff, the video is over 3 minutes long—think of the children's attention span!   (derek500 doesn't like me to use smilies in these forums)

 

I only have Mac version 12.5, but the dialog—or anything else in the Version 13 UG—doesn't have the security settings option Scillonian describes.  I guess the option is Windows-only.