Jump to content
x509

Selector doesn't work for only 1 Backup Script

Recommended Posts

I'm running Retrospect 15.6.1 on a Windows 10 Pro 64 system.  I use Always Excluded Selector for all the different types of files that I want to exclude from any backup script. Here is the relevant part.

image.png.db6608a85b7b80e9384db7693231d3b2.png

The issue here is with the name System Volume Information folder.   I use this Always Excluded selector for 7 different backup scripts for 5 different backup sets covering Windows 4 drives C-F.  For 7 days now, my MEDIA script does back up a 50 GB file contained in the System Volume Information folder.  None of the other scripts back up this folder.  I can't figure out how to stop my MEDIA script from backing up this folder.  It appears that every day, there is a different 50 GB backed up.

image.png.4651b9d79a87c906520771b566dcad65.png

I thought something might be wrong with the drive, so I reformatted the drive, and then ran the MEDIA script on the now empty drive. Windows never displays this System Volume Information folder.  If you use a cmd window, the system shows this directory as empty even though it does contain files.

image.png.9b0ecbd37aa10d600d50011707f64a93.png

However, the Session Report for the most recent MEDIA script does show files in this folder.

image.png.01f04965852710fbbe80363eefaf756a.png

So what can I do to stop MEDIA script from including files in this System Volume Information folder?  It's a big waste of time and disk space.  Again, No other script backs up this folder on Drive C, D, or F, and no other Always Excluded files are backed up by this script or any other script.

I believe I can do selective grooming to recover the 350 GB wasted on these files, but that's not a real answer.

Very frustrated at this point.  In a few days, I will post how apparent bugs in Selector Include definitions have caused a bunch of my scripts to stop working properly when I upgraded from Retrospect 12 to 15.6.

 

x509

 

 

 

Edited by x509
typos and additional information

Share this post


Link to post
Share on other sites

You should run the cmd window as Administrator.

Then, in order to see hidden files, you should type:

dir /a:h "system volume information"

 

Share this post


Link to post
Share on other sites

I also found some more information here:

https://fossbytes.com/system-volume-information-folder-windows-shrink/

Quote

This folder is used by Windows for system-level functions. That’s why you can’t access it. Apart from Restore Points, System Volume Information folder also stores the content indexing service databases that are used for faster file searches, etc. 

 

Share this post


Link to post
Share on other sites

x509,

I am a Retrospect Mac administrator, as you know, so the suggestion in the next  paragraph may not be worth anything.  This 2009 thread has something to say about System Volume Information, and this 2018 thread has something to say about the Program Files directory—which seems to be somewhat analogous.

What IMHO makes the directories analogous is that they both sound like what Retrospect may define as Special Folders (Windows).  That is among the Windows Selector Conditions defined on page 437 of the Retrospect Windows 15 User's Guide.  I don't know how to use such conditions in a Selector, so someone more familiar with Retrospect Windows is going to have to advise.

 

Share this post


Link to post
Share on other sites

Lennart,

Thanks for the link.  In the latest version (1809) of Windows 10, this System Information Volume is not displayed even though I have selected the View option for Hidden Items

image.png.517ac49fed254af7d486329a845e719c.png

But that's OK.  I don't want to ever back up this folder.

Share this post


Link to post
Share on other sites

DavidHertzberg,

I used the Operating System and Application selector as a source for ideas for my AlwaysExcluded selector.

My frustration here is that the AlwaysExcluded selector has been set-it-and-forget-it for several years now, and works as expected in all my scripts except this one MEDIA script starting about a week ago.  I have not made any changes to the backup script or to the selector that selects files for backup or to this AlwaysExcluded selector.

Here is the full selector.  The last four lines are application-specific.

image.thumb.png.cfbafa84913f5d92cee7eab204514516.png

 

Share this post


Link to post
Share on other sites
12 hours ago, x509 said:

Here is the full selector.

That ends with "Exclude nothing" !!!!

You specifically include "System Volume information" in what you want to backup.

 

In fact, you seem to backup "junk" files ONLY. You must have reversed the meaning of the selector.

Share this post


Link to post
Share on other sites
On 2/25/2019 at 10:52 PM, DavidHertzberg said:

x509,

Think about what changed about a week ago.  Did you update Windows to a new version?  Some other change?

No explicit changes in Retrospect. 

Desktop APO and Laptop are running Windows 1809.  Desktop APH (older system) still running Windows 1803.

Share this post


Link to post
Share on other sites
On 2/26/2019 at 9:44 AM, Lennart_T said:

That ends with "Exclude nothing" !!!!

You specifically include "System Volume information" in what you want to backup.

 

In fact, you seem to backup "junk" files ONLY. You must have reversed the meaning of the selector.

Ooops.  I didn't document well enough.

All my scripts have an "include" selector and an "exclude" selector, which is always AlwaysExclude.  Here is the MEDIA script:

image.png.17aa158c3f52bdd19b0a57cae0ad3332.png

The source  here is the Media drive (always Drive E:) on all three currently active systems:

image.png.42cd4792ec655bd0139f16301275edb8.png

Also I need to add that I back up to 4 different Windows volumes, C-F.  This system, APO, and DELOS contain all four volumes. System APHRODITE contains only volumes C-E.

The unwanted System Volume Information folder is backed up only on systems APO and DELOS, which are Windows 1809.  The unwanted folder backups started on Feb. 12.  I didn't make any explicit changes on that date, but Microsoft updates version 1809 frequently in the background or overnight.  This folder is not backed up on system APHRODITE, which is Windows 1803.

x509

 

Share this post


Link to post
Share on other sites

It would not be the first time that a Windows update breaks Retrospect. It often takes months for Microsoft to correct the problems.

Is there anything else but System Volume Information that is backed up, that should not be backed up?

Have you checked that everything you want to backup is actually backed up?

Share this post


Link to post
Share on other sites
42 minutes ago, Lennart_T said:

It would not be the first time that a Windows update breaks Retrospect. It often takes months for Microsoft to correct the problems.

Is there anything else but System Volume Information that is backed up, that should not be backed up?

Have you checked that everything you want to backup is actually backed up?

As far as I can tell from a quick inspection of each volume, the only file/folder that is backed up, but should not be, is the System Volume Information folder, and only on the systems running Windows 10 1809.  I guess I should report that as a bug to Retrospect.

To reply to your second question, I have had an ongoing exchange with Retrospect support about file/folder selector conditions that are part of my Include selector, but that don't in fact automatically select new files to back up.  I'm still sorting through all that, and I will both submit to Retrospect and do some postings here when I have successfully gotten all folders properly backed up.  In this case, it appears that some selector conditiions are not being implemented correctly.

As an example, I have top-level folders 2018 and 2018.  The selector "path matches \2018" does not automatically select files or subfolders to back up.  Change matches to matches pattern and suddenly files in folder 2018 do get backed up, so it seems.  I still want to run some more tests.

In a different script, I have my normal include selector, plus a few specific folder excludes, as well as the exclude AlwaysExcluded selector.  Once I finish these tests, I may need to adjust those folder excludes.

 

x509

Share this post


Link to post
Share on other sites

Until the bug is fixed, you could try this:

Exclude files whose size is larger than 40 GB

AND whose name includes a curly left brace

AND whose name includes a curly right brace

AND whose name includes a dash (minus)

That is, of course, not suitable for the scripts where you backup a bootable disk.

Share this post


Link to post
Share on other sites
On 2/27/2019 at 10:31 PM, x509 said:

As an example, I have top-level folders 2018 and 2018.  The selector "path matches \2018" does not automatically select files or subfolders to back up.  Change matches to matches pattern and suddenly files in folder 2018 do get backed up, so it seems.  I still want to run some more tests.

Isn't that to be expected? IIRC, the tip for Windows paths implies you have to include both the drive label and, for directories, the trailing slash. So it should be "path matches E:\2018\".

Share this post


Link to post
Share on other sites
On 3/8/2019 at 2:01 AM, Nigel Smith said:

Isn't that to be expected? IIRC, the tip for Windows paths implies you have to include both the drive label and, for directories, the trailing slash. So it should be "path matches E:\2018\".

Nigel,

I tested out your suggestion in Windows 10 Explorer.  The trailing backslash is not necessary for Windows to understand that I am referring to a directory and not to a filename.  Same is when I do a "Save As" with different applications such as Microsoft Word or Adobe Lightroom.

It's a damn selector bug, methinks.

x509

Share this post


Link to post
Share on other sites
12 hours ago, x509 said:

Nigel,

I tested out your suggestion in Windows 10 Explorer.  The trailing backslash is not necessary for Windows to understand that I am referring to a directory and not to a filename.  Same is when I do a "Save As" with different applications such as Microsoft Word or Adobe Lightroom.

It's a damn selector bug, methinks.

x509

Surely, x509, I don't need to tell you why and how to file a Support Case for a bug.

Share this post


Link to post
Share on other sites

David,

I didn't just file a support case.  I have written to one of the senior execs, several times.  I got replies each time, including one (from Robin Mayoff) that is absolute BS that just completely underscores your point about poor documentation in recent releases.  I'm still a bit underwater and I have another business trip soon, so i may not get to posting my list of issues until around May 15.  But trust me i have "found my voice" with Retrospect.

Sorry to dribble all this out.

x509

Share this post


Link to post
Share on other sites
On 2/27/2019 at 9:27 PM, Lennart_T said:

Until the bug is fixed, you could try this:


Exclude files whose size is larger than 40 GB

AND whose name includes a curly left brace

AND whose name includes a curly right brace

AND whose name includes a dash (minus)

That is, of course, not suitable for the scripts where you backup a bootable disk.

Lennart,

Sorry for being so very late in replying.

My backup dataset MEDIA has been filling up rapidly with these 50 GB files and my backup drive was dangerously close to being 100% full, so I thought the first thing I needed to do was groom that dataset.

I used this Selector with the Groom job:

image.png.facf1b483a891adb13739318e965b975.png

Before running the groom job, I ran a Find Files operation on the MEDIA dataset.  Here is the result.  There are over 20 of these crazy 50 GB files backed up:

image.png.63e0651033942ab17a276fd74fc072ff.png

After running the Groom Media job, there were still some 50 GB files left.  This is the backup log for that grooming job:

image.png.6b7b8c442499220aa3f22824903e473c.png

Unfortunately, some of the 50 GB files remain:

image.png.97743f1ae8448eb986fb4376cb448a26.png

I tried several different variations on the Selector, but I couldn't get the rest of these 50 GB files deleted.

Also, I noticed after running the groom job that there were a lot of $Recycle Bin folders backed up, like this:

image.png.b4c98c9f313ef238db3a2e036a649c64.png

I tried all sorts of Selector conditions to get these folders groomed out, but nothing worked.  Here is but one example:

image.png.b92029f2a075966a591943860452ea9b.png

All this is very frustrating, because I'm both trying to recover wasted space on my backup drive AND work out Selector conditions for the broken backup script.

Do you have any suggestions, outside of buying a biggger backup drive?

x509

 

 

 

 

Share this post


Link to post
Share on other sites
2 hours ago, x509 said:

Do you have any suggestions, outside of buying a biggger backup drive?

Try grooming optimized for storage (instead of optimized for performance).

I have noticed that optimized for performance leaves too many files left in the backup set. On the other hand, I don't have 50 GB files in my backups so I can't guarantee it will work for you. :) 

Share this post


Link to post
Share on other sites

Lennart,

I can't thank you enough for this seemingly simple suggestion.  Once I changed the dataset preferences to storage optimized grooming, I got this result:

image.png.f695a7051fbc5ed07427f594a9c8b595.png

All the 50 GB files are gone, and so are all the folders in $Recycle Bin. 

Your suggestions together have enabled me to recover over 1 TB of wasted storage.

 

x509

  • Thanks 1

Share this post


Link to post
Share on other sites
2 hours ago, x509 said:

I can't thank you enough for this seemingly simple suggestion.

I'm glad I could help. :) 

Share this post


Link to post
Share on other sites
On 4/30/2019 at 4:27 PM, x509 said:

I tested out your suggestion in Windows 10 Explorer.  The trailing backslash is not necessary for Windows to understand that I am referring to a directory and not to a filename.  Same is when I do a "Save As" with different applications such as Microsoft Word or Adobe Lightroom.

I'm sure they don't -- but they aren't Retrospect!

The Retrospect tool tip tells you how to how to do "path matches" -- you must include the drive letter and the trailing slash if you are matching a directory. I'm not saying it *isn't* buggy, just that you shouldn't assume it is if you haven't done things the way the software tells you to...

Share this post


Link to post
Share on other sites

Nigel,

Thanks for this reply. 

On 5/13/2019 at 8:46 AM, Nigel Smith said:

I'm sure they don't -- but they aren't Retrospect!

The Retrospect tool tip tells you how to how to do "path matches" -- you must include the drive letter and the trailing slash if you are matching a directory. I'm not saying it *isn't* buggy, just that you shouldn't assume it is if you haven't done things the way the software tells you to...

I can only comment that it's not exactly great software design to vary from standard Windows practice in specifying a path.  There are several different SAVE AS dialogs in Windows, I'm not sure why, but the key point is that none of them require a trailing backslash.

In my scripts, I have not found it necessary to include a drive letter, but maybe that's because I use Source Groups in Volumes.  I have four systems, and another being added next week, and it would greatly complicate matters if I had to specify fully qualified paths in the scripts, e.g. \\APOLLO\DATA or \\DELOS\DEL-DATA or \\APHRODITE\APH-DATA.  (APOLLO is the Retrospect host system.)

All my systems are organized the same way:

C - Windows and Programs, related settings.  NO user data.

D - User data, other than music, photos, videos.

E - Music, photos, videos

F - software downloads (only two systems)

x509

Share this post


Link to post
Share on other sites
20 hours ago, x509 said:

I can only comment that it's not exactly great software design to vary from standard Windows practice in specifying a path.  There are several different SAVE AS dialogs in Windows, I'm not sure why, but the key point is that none of them require a trailing backslash.

I can only comment that, as a Mac user, the total lack of consistency in Windows itself -- never mind other software houses' Windows apps -- gives me the heebie-jeebies... 😉

To be fair, I've just fired up Windows RS Server and checked, and the tip states:

Paths for FAT/NTFS volumes start with a drive letter and use backslashes (\) as separators, as in "C:\Data\".

...so it doesn't say it requires a trailing backslash, but does include one in the example.

However, you originally said:

On 2/27/2019 at 10:31 PM, x509 said:

As an example, I have top-level folders 2018 and 2018.  The selector "path matches \2018" does not automatically select files or subfolders to back up.  Change matches to matches pattern and suddenly files in folder 2018 do get backed up, so it seems.

...which didn't include the drive letter. That will be mandatory since, unlike your PC's Explorer Address Bar or an app's Open/Save dialog, the Retrospect filter has no concept of "current drive" or "current working directory", so a "match" or "starts with" must begin with drive-letter-colon-backslash. And there's no reason not to include the trading slash, just in case...

Still doesn't explain why it suddenly works when you change to "matches pattern" though (I had confused that with "contains", my bad...).

If I get time later I'll try a couple of tests.

Share this post


Link to post
Share on other sites
6 hours ago, Nigel Smith said:

I can only comment that, as a Mac user, the total lack of consistency in Windows itself -- never mind other software houses' Windows apps -- gives me the heebie-jeebies... 😉

To be fair, I've just fired up Windows RS Server and checked, and the tip states:

Paths for FAT/NTFS volumes start with a drive letter and use backslashes (\) as separators, as in "C:\Data\".

...so it doesn't say it requires a trailing backslash, but does include one in the example.

However, you originally said:

...which didn't include the drive letter. That will be mandatory since, unlike your PC's Explorer Address Bar or an app's Open/Save dialog, the Retrospect filter has no concept of "current drive" or "current working directory", so a "match" or "starts with" must begin with drive-letter-colon-backslash. And there's no reason not to include the trading slash, just in case...

Still doesn't explain why it suddenly works when you change to "matches pattern" though (I had confused that with "contains", my bad...).

If I get time later I'll try a couple of tests.

Nigel,

Why you say that drive letter will be mandatory?  So far it isn't.  I'm running Retrospect 15.61.  Now that I think about it, the way Windows networking works, Windows Explorer doesn't actually know about drive letters on networked systems.  Or, maybe I should say, drive letters on networked systems aren't exposed in Explorer.  I sure hope that Retrospect isn't going to screw all this up in a future release.

What I do compulsively, you might say, is that I have a uniform usage of drive letters, as I explained in an earlier post, plus the use of Storage Groups.  In my DATA storage group, I have the D drives from all systems.

About your comment about total lack of consistency in Windows itself, I agree.  Compared to the dark old days of DOS, Windows has enforced a lot of uniformity in the interface, even though there are many annoying inconsistencies.  There is no question that Apple does a better job in the interface area.  However, for my iPhone/iPad, Apple makes these strange changes from one iOS release to the next, and I'm forced to do Google searches. 

 

x509

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×