How to FORCE a file backup on each run?

[MrPete]

Due to security requirements, certain files may NOT get a timestamp update when they change. (Consider files that contain encrypted filesystems.)

Right now, by default Retrospect ignores all files with an "old" timestamp.

QUESTION: is there a way to get Retrospect to back up a file even if the timestamp has not changed?

[DavidHertzberg]

MrPete,

How about putting all such files in a single folder defined as a Subvolume, and backing up that Subvolume separately with a Recycle backup?  You might want to exclude those files from your other backups using a custom Selector that Excludes files with that single folder's Path (Windows)—page 365 in the Retrospect Windows 17 User's Guide.

[MrPete]

Excluding is not an issue.

Using Recycle Backup would essentially remove the value of Retrospect: no versions, no block-level-change efficiency. OUCH! Might as well just do a file-copy.

Hmmm... that would be interesting:

 - Script a file copy of the file(s) of interest, ensuring the copy has a new timestamp.

- Since they ARE big, Retrospect would take advantage of block-level efficiency to only back up the blocks that have changed!

- Script deleting the copy (for security) after the RS run is complete.

I'll try it!

[DavidHertzberg]

MrPete,

This 2019 thread says that having hard links to files causes those files to be backed up every time.  Unfortunately for your purposes, the next-to-last post in that thread implies Microsoft may have done something  to fix that—which was a problem for the OP in that thread but which would be exactly what you want.

[Nigel Smith]

17 hours ago, MrPete said:

- Script a file copy of the file(s) of interest, ensuring the copy has a new timestamp.

Can you not just play with the original's timestamp? Pre-run script to store original timestamp and set timestamp to "now", do the backup, post-run script to restore timestamp to original. Easy to do on Mac/Linux clients with "touch", and I believe Win PowerShell has [Get|Set]-ItemProperty to do similar. Much quicker without the overhead of the copy and delete.

But is any of that really necessary? As I understand it, all you need to do is turn off the "Don't add duplicate files to the Media Set" option in the "Matching" section of you script's "Options" and files will be backed up every time, regardless of metadata matching. You might want to set up another script, narrowly aimed at the files you want this behaviour to apply to, so you get "normal" matching across the rest of your data. Don't know if you'd get your block-level efficiency though.

[MrPete]

Nigel, I am cautious about mucking with original-file metadata. (Example: every file has more than one timestamp. Do we really want to be writing scripts that somehow are maintained to understand all metadata? I would rather not be in that line of work anymore )

I don't want a comprehensive copy every time. If this "don't add duplicate" switch still takes advantage of block-scanning etc... that WOULD be interesting! I'll check it out

THANKS!

[DavidHertzberg]

MrPete,

Look at the sub-section "Matching Execution Options" on pages 301-302 of the Retrospect Windows 17 User's Guide.  It seems that if you turn off "Match source volumes to Catalog File" and "Don’t add duplicates to Backup Set" for a Backup script, the source files accessed by that script will be backed up on every run.  Of course the script source will have to be one or more Subvolume(s) containing only the files that do not get a timestamp update when they change, as I suggested in this up-thread post.

I see now that Nigel Smith suggested this in his second paragraph here up-thread.  I agree with him that turning off "Don’t add duplicates to Backup Set" (he used the Retrospect Mac option name) would be sufficient.

It so happens we have a probable test case for block level incremental backup, identified by this post in a Macintosh 9+ Forum thread and its predecessors.  The OP Joriz is a Belgian administrator with a penny-pinching boss (who is impeding Joriz's effort to add disk-to-tape backups to go off-site); he is currently doing disk-to-disk backups of one server disk that contains Veeam backups of one or more VMWare VMs.  He complained that those backups are creating excessive Retrospect Mac "churning".   Nigel Smith suggested that Veeam might be doing block level backup—I verified it has the capability, but Joriz evidently hasn't enabled block level backup in Retrospect.  Unfortunately at last report Joriz is out with illness (not COVID-19, he says), so he won't be able to immediately enable block level backup in Retrospect. 

We may have helped Joriz, but I think you're going to have to forgo the benefits of block level backup for those files if you turn off "Don’t add duplicates to Backup Set".  How would Retrospect Windows know to not backup the unchanged blocks of a file if it's adding duplicates to the Backup Set?  But come to think of it, how are you getting block level incremental backup for those files now, if their timestamps don't change? 🙄  Read the second paragraph on page 458 of the UG, with careful attention to the word "subsequent".

Did I disclose that I receive a sales subsidy from the World Association of Disk Manufacturers? 🤣  Buy bigger Backup Set disks, or a tape drive.

P.S.: Considering all the foregoing, I think your idea—outlined in this up-thread post—of making short-lifetime timestamped copies of the non-timestamped files and letting Retrospect do block level incremental backups of the copies is the best approach.  Buy bigger source disks instead of Backup Set disks. I ask, in naive ignorance: how does having files not be timestamped make them more secure?