Selector doesn't work for only 1 Backup Script

[Nigel Smith]

12 hours ago, x509 said:

Why you say that drive letter will be mandatory?  So far it isn't.  I'm running Retrospect 15.61.  Now that I think about it, the way Windows networking works, Windows Explorer doesn't actually know about drive letters on networked systems.  Or, maybe I should say, drive letters on networked systems aren't exposed in Explorer.  I sure hope that Retrospect isn't going to screw all this up in a future release.

I'm mainly going by (possibly wrong) logic and (possibly incorrect) Retrospect instructions...

Windows Explorer and Windows Open/Save dialogs have a "current working directory" and "default working drive" (the C drive) -- for example, in an Explorer window address bar you can use full paths to go to a directory ("C:\Logs") but you can also use relative paths ("..\" will take you up one level from the currently shown directory). If you have a directory called "Data" on both C and D drives, typing "\Data" from any location on your system will always take you to C:\Data and never to D:\Data because there's an implicit "C:" before any path starting with "\".

Retrospect Client (assuming you aren't defining volumes/favourites) runs at the "machine" level -- it can "see" all drives so, to reference a particular one, you must state which. I can do no better than quote (again) the tip from Retrospect's "Windows Path" filter dialog:

Paths for FAT/NTFS volumes start with a drive letter and use backslashes (\) as separators, as in "C:\Data\".

That might, perhaps, be clearer if it was written as:

Retrospect path filters for FAT/NTFS volumes...

Drive letters aren't exposed on networked systems because the "server" doesn't present them. In a UNC path, e.g. \\MyServer\FirstShare\TheFolder, you could consider the share name to be analogous to the local drive letter.

It's just differences in the way Windows abstracts and presents absolute paths to directories/files accessible via various services. It's perhaps more obvious on Mac OS X, where every mounted volume including the system disk can be reached via /Volumes/<volume_name>.

I'm trying to find a PC client to test against, but I wouldn't be surprised if you got the results you did because "match" requires a prefixed drive letter, while "match pattern" has an implicit "?:" at the start and so will match C:<pattern>, D:<pattern>, etc.

[Nigel Smith]

2 hours ago, Nigel Smith said:

I'm trying to find a PC client to test against...

As promised.

The Test

Client: "NIG-PC" -- Windows 10 VM, Retrospect 15.6.0, with C drive and external E drive. Both drives had "Test_Data" and "Test_Data2" directories, each with a couple of text files inside.

Server: Windows Server 2016, Retrospect Multi Server Premium v15.6.1.104

Client was added by Direct IP with "Volumes" tab "Client Sources" set to "Client Desktop" -- both C and E drives were visible

A new disk-based Backup Set called "Filter_Test" was created

A new filter called "Filter_Test" was created, initially blank and then edited as per the following screenshots

After the filter was edited, an Immediate Backup was created: Source -- "NIG-PC"; Destination -- "Filter_Test"; Selecting -- "Filter_Test". The Preview button was clicked and, once the results generated, the screenshot taken. The Immediate Backup was then cancelled to clear any cached Preview and to force a re-scan for the next test

The Results

"Windows folder path exactly matches \Test_Data\" -- no drive letter, so nothing is matched:

 

"Windows folder path exactly matches E:\Test_Data" -- no trailing backslash, so nothing is matched:

 

"Windows folder path exactly matches E:\Test_Data\" -- drive letter and trailing slash included, matches only with "Test_Data" on E and not E:\Test_Data2 or C:\Test_Data:

 

So, what about "matches pattern"? We know from the filter dialog tip that "* matches any or no characters and ? matches any single character", but x509 had no special characters in his filter yet still got matches. Let's see what we can find out, starting with a filter similar to x509's...

"Windows folder path matches pattern \Test_Data" -- matches Test_Data and Test_Data2 on both drives:

 

"Windows folder path matches pattern \Test_Data\" -- matches Test_Data only on both drives:

 

"Windows folder path matches pattern E:\Test" -- matches Test_Data and Test_Data2 only on the E drive:

 

"Windows folder path matches pattern st_Data" -- matches Test_Data and Test_Data2 on both drives:

 

Conclusion

Exact folder matching requires a full path, including the drive letter, and a terminating backslash, i.e. "E:\Test_Data\".

The "matches pattern" condition includes invisible "*"s, both prefix and postfix, i.e. if you enter "st_Data" it is actually "*st_Data*". That may be what you want, e.g. the same folder name at different levels of the directory structure across multiple drives on a client, but could also greatly widen the matches beyond what was expected.

As always, the more explicit you are the closer the filter results will match your wishes. Vagueness on "includes" can massively increase backup resource requirements, vagueness on "excludes" can result in important data being missed. So test, test, test -- and be careful out there!

Note

Yes this was all done with "includes" whilst x509 was having trouble with "excludes" -- that's simply because I think it is much easier to see none, one or two ticked boxes amongst a column of unticked in a fuzzy screenshot than to spot the gaps in a line of selected items. But the conclusions above also apply to "excludes", and it's simple enough to verify for yourself if you doubt it.

Hope that helps someone!

[x509]

Nigel,

Thank you for doing all these tests.  I got dragged into this morass only when scripts that apparently had been working, stopped working.  So I was in "break-fix" mode, or just trying to solve the problems of broken scripts, so I could move things along.  I appreciate all your testing here.  I'm going to download this thread so I can save it in my folder of "informal" Retrospect documentation.

x509

[x509]

On 2/27/2019 at 9:27 PM, Lennart_T said:

Until the bug is fixed, you could try this:

Exclude files whose size is larger than 40 GB

AND whose name includes a curly left brace

AND whose name includes a curly right brace

AND whose name includes a dash (minus)

That is, of course, not suitable for the scripts where you backup a bootable disk.

Replying months later to Lennart_T's post, I finally (never mind why!) got around to doing a grooming operation.  It was a success.  Here is the selector I used in the groom job:

Note that the S-1 ... folder never should have been backed up.  The System Volume folder was explicit.y excluded in my universal "Always Exclude" selector. Just in case the System Volume exclusion didn't work, I also excluded files matching pattern 32{* and *{380* that are either 12 GB or 50 GB in size.

Before the groom operation I did a Find Files operation on the Media dataset, to identify files and folder to groom out.  AFter the groom operation, I did another Find Files operation to confirm that all those files were in fact groomed out, except for an S-1* folder inside the $RecycleBin, which should never have been backed up.  (But I'll clean that up.)

The groom operation recovered 345 GB and 15479 files.  After I get some more experience with this grooming selector, I will incorporate it into the backup script.

 

x509