Scanning incomplete, error -1101 (file/directory not found)

[Malcolm McLeary]

I'm using a new install of Retrospect for Windows v17.0.2 and I have created a Custom Selector so as to NOT backup files in some specific directories because I know the content changes every 5 minutes and I don't need a backup of them.

Linux path /backup/FileMaker/Progressive/ and subfolders

Linux path /home/FileMaker/Progressive/ and subfolders

Linux path /opt/FileMaker Server/FileMaker/Data/Progressive/ and subfolders

Depending on my configuration only 1 of these will be in play and contain files.

The backup runs but records hundreds of errors because my progressive backups "change" during the backup.

--Log--
+Normal backup using Immediate - Lenovo at 07/07/2020 9:09 AM (Execution unit 1)
*    Resolved container Lenovo to 3 volumes:
        /backup on Lenovo
        /home on Lenovo
        / on Lenovo
        07/07/2020 9:09:41 AM: Finished scanning backup set data files
    To Backup Set Lenovo 002...

-    07/07/2020 9:09:40 AM: Copying /backup on Lenovo
    While scanning volume backup,
        Folder /backup/FileMaker/Progressive/IncrementalBackup_2020-07-07_0900/Additional/RC_Data_FMS/PacificStone/Files/PacificStone/Secure/01/,
                Scanning incomplete, error -1101 (file/directory not found)
        Folder /backup/FileMaker/Progressive/IncrementalBackup_2020-07-07_0900/Additional/RC_Data_FMS/PacificStone/Files/PacificStone/Secure/02/,
                Scanning incomplete, error -1101 (file/directory not found)

       (snip)

          07/07/2020 9:13:38 AM: Found: 102,106 files, 395,298 folders, 59.7 GB
          07/07/2020 9:13:42 AM: Finished matching
          07/07/2020 9:14:04 AM: Selector "All Files Except Progressive Backups" was used to select 81,592 files out of 102,106.
          07/07/2020 9:14:16 AM: Copying: 0 files (zero KB) and 0 hard links
          07/07/2020 9:14:29 AM: Building Snapshot...
          07/07/2020 9:14:29 AM: Copying properties for 395,298 folders
          07/07/2020 10:54:17 AM: Finished copying properties for 395,298 folders and 0 files
          07/07/2020 10:54:27 AM: Copying Snapshot: 2 files (132.5 MB)
          07/07/2020 10:54:30 AM: Snapshot stored, 132.5 MB
          07/07/2020 10:54:30 AM: Comparing /backup on Lenovo
          07/07/2020 10:54:32 AM: 257 execution errors
          Duration: 01:44:52 (01:44:43 idle/loading/preparing)

It seems that the File Selector is being applied too late in the process.  I would have thought that if I've identified paths NOT to be included in the backup then by definition they'd be excluded from the scan.  Why waste time scanning paths (and recording errors for files) which are going to be excluded?

 

[Nigel Smith]

Retrospect doesn't do a UNIXy tree-walk, not bothering to look at anything "/backup/FileMaker/Progressive/" or lower. Instead it scans *every* file of a volume and applies its selectors to decide what to do. I'd assume from the errors that it is getting partway through scanning those directories' contents when, suddenly, they vanish.

Whilst annoying in a simple case like you describe, it's also part of what makes the selectors so powerful -- for example, being able to exclude files on a path *unless* they were modified in the last 2 hours -- and why all the metadata needs to be collected via the scan before a decision can be made.

Two ways round this. If you want to exclude most paths, define the ones you want as volumes and only back those up -- we only back up "/Users" so that's what we do, which also greatly reduces scan time. If you want to back up most but not all, which I guess is what you're after, use the "Privacy" pane in the client to designate those paths to exclude.

[DavidHertzberg]

Malcolm McLeary,

When Nigel Smith says "define the ones you want as volumes", he probably means Retrospect-specified Subvolumes.  Described on pages 349–351 of the Retrospect Windows 17 User's Guide, they were renamed Favorite Folders in Retrospect Mac 8.  I use a Favorite Folder in a Backup script; it works.

However Retrospect Windows also has defined-only-in-Retrospect Folders, which are described on pages 348–349 of the UG as a facility for grouping source volumes.  The description doesn't say so, but you can possibly move defined Subvolumes—even on different volumes—into a Folder.  Since the Folders facility was removed in Retrospect Mac 8, I didn't know it even existed until I read about it 5 minutes ago.  That's to say Your Mileage May Vary (YMMV), as we say in the States (in a phrase originally used in auto ads).  If they work as groups of Subvolumes, they may simplify your backup scripts.

[Malcolm McLeary]

I have often said ... "the only backup you ever really need is the one you don't have". Hence I tend to backup everything, except the things I know I don't need, rather than backup just the things I think I may need or may get "lost".

In this case I'm being overly cautious (and just testing things out) as realistically the only stuff I need to backup are;

/home/

/backup/Backups/

Database backups are stored in /backup/Backups/ so short of complete disk failure everything I need is on a disk in the machine.  Retrospect's job is to maintain backups off the machine.

Its not unreasonable to define the sub volumes I really want to backup to elsewhere as presently I use SAMBA to publish appropriate "shares" and back them up as if the machine was a fileserver.  Its just that using the Retrospect client I can backup the whole machine quite easily.  Whether I should is a different question.

To rebuild the machine I just boot from the CentOS install image, install, update, add a few packages stored in /home/sysadmin/ configure FileMaker Server and restore the most recent backup from /backups/ (which is a separate drive which may have been restored).