Jump to content
gnoelken

'Restore --> Find Files' yields 0 files found

Recommended Posts

Using Retrospect Desktop version 16.6.0.133 on a Windows 10 Pro PC (10.0.18363) with all latest updates installed.

I'm trying to restore some images from a particular directory. In particular all images ending in *.nef that are found in two subdirectories. The image below shows my backup set (Archive_Drive) and the path to those 'nef' files from the most recent snapshot (6-27-2020):

138236111_ArchiveBackupSet2016-01-04-WatercolorPath.thumb.jpg.91eb59773c98995c9648f9cae5f7a22d.jpg

The system path is 'D:\Data\Greg\Pictures\2016\2016-04-01 - Watercolor'.

The system path is 'D:\Greg\Pictures\2016\2016-04-01 - Watercolor'.

I used the option 'Restore --> Find Files' to select the 'nef' files in subdirectories within the above path. Here is my search criteria:

1139648806_2016-01-04Watercolorsystempath-nefselector.jpg.e30f52462250579cf0145fa703456bfd.jpg

I have the option to check this selector against my "D:\" drive and I do:

212760201_2016-01-04Watercolorsystempath-nefselectorresults.thumb.jpg.20ddf823c97959183e08445ffcb14273.jpg

This is the same path that is found in the 'Archive' backup set in the first image. It finds 69 'nef' files in 2 folders.

I click on 'OK' in my searching and retrieval dialog box. Retrospect searches my Archive_Drive backup set and 0 files are found:

871228304_Restore-FindFiles0filesfound.jpg.1a5d07bf6d4e6035fb24ae19d5276410.jpg

I also tested this search by creating the same selector as above and the selector finds the 'nef' files.

I've rebuilt the catalog file from disk. I've checked (windows) permissions on my drive and folders. 

I don't think it would, but would an upgrade to 17 fix this? 

Are there any other options to 'fix' the backup set?

Thanks for any help.

Greg

 

 

Edited by gnoelken
Wrong path.

Share this post


Link to post
Share on other sites

gnoelken,

I'm a Retrospect Mac administrator, and remember very little about Windows (mostly Windows 95) from my last job 15 years ago.  However, shouldn't your path start with "D:\Data\Greg"?  Or is "Data" some kind of a synonym for "D:\" on your system?  The line under the first screenshot in your OP says:

Quote

The system path is 'D:\Data\Greg\Pictures\2016\2016-04-01 - Watercolor'.

 

Edited by DavidHertzberg
Appended quote from the OP about system path

Share this post


Link to post
Share on other sites

The path I had in in my original post was in error. It was actually "D:\Greg\"

I have corrected the post.

Sorry for the error but thanks for the quick response. You are right in thinking it was a synonym. For years I had a drive set as a folder named Data on my "D:\" drive like Linux drives could be set. It was natural habit for me to type "D:\Data\..." Also, the label of my current drive is "Data".

Greg

Share this post


Link to post
Share on other sites

I haven't tested this, but using the "Condition" dialog can be very tricky.  I think instead of "Starts with" I would instead use "Match".  Keep experimenting in that dialog because that's where the problem lies.

In fact, I would delete the other conditions and do this as a building block.  Start with one line and make sure it gets the results you expect, then add one more condition, test the results, then another until you see the only results you want.

Also, when the restore starts pay close attention to any prompts, because there may be a question in there with a "Do you want to overwrite" with a reverse logic default answer.

Newer versions haven't changed this that I know of.

Share this post


Link to post
Share on other sites
2 hours ago, mbennett said:

I haven't tested this, but using the "Condition" dialog can be very tricky.  I think instead of "Starts with" I would instead use "Match".  Keep experimenting in that dialog because that's where the problem lies.

When I use the 'Check Conditions' of my conditions, the nef files are found. It would seem that they are configured correctly.  I'll play around with other configurations though.

2074126104_2016-01-04Watercolorsystempath-nefselectorCheck.jpg.c069be2d44c0823ac96d262c3ee3cc64.jpg

2 hours ago, mbennett said:

In fact, I would delete the other conditions and do this as a building block.  Start with one line and make sure it gets the results you expect, then add one more condition, test the results, then another until you see the only results you want.

That's the way I built this query.

2 hours ago, mbennett said:

Also, when the restore starts pay close attention to any prompts, because there may be a question in there with a "Do you want to overwrite" with a reverse logic default answer.

Thanks for the heads up. I've been burned by that before!

Thanks for your suggestions and help.

Greg

Share this post


Link to post
Share on other sites

Change that first match to "path starts with D:\Greg\" <- note the trailing slash, and see if that helps.

Previous testing showed that exact folder matching required that terminating backslash, and I suspect that "path starts with" does too, as implied in the "Tip"  under where you type the path for the selector.

Share this post


Link to post
Share on other sites

I did find that if I put the entire path in as a selector:

'D:\Greg\Pictures\2016\2016-04-01 - Watercolor'

1176895702_2016-01-04Wholepathmatchespattern.jpg.fed5dcd2db2d781a7db0c0a569a61baa.jpg

The folder is found but the 'nef' files within the folder are not included!

466676882_noneffiles.thumb.jpg.cdd6370f3bdf02c73a62c11f9ac47f63.jpg

Here is a view of the most recent snapshot in the backup set  of this folder:

1764628727_Archivesnapshotwithnefandxmp.thumb.jpg.598f0968c48e073abe688e189a553684.jpg

I'm really confused as to why my single 'path' selector would not include all files in that path.

Any selector I choose so far (such as 'folder name ends with Watercolor') finds the folder but does not list the 'nef' files. There are two folders in the '2016-04-01 - Watercolor' folder that have 'nef' files and both are not showing those files.

If I choose a single selector to find all 'nef' files several thousand are found including the folders under '2016-04-01 - Watercolor'!

Thanks for any help.

Greg

Share this post


Link to post
Share on other sites

I did another test using a folder under the '2016' folder that contained 'Pickle Springs' (D:\Greg\Pictures\2016\2016-05-05 Pickle Springs):

1349434506_picklespringsselector.jpg.aa7ae20af8f6da6d697565f77e4c6d33.jpg

Selecting 'Files Chosen' in the 'Searching and Retrieval' dialog box shows that the 'nef' files are included in the search results:

1245200502_PickleSpringssearchresults.jpg.69ba13ca52c92ff855f3828be3b61753.jpg

I repeated the same search for folders that contain 'watercolor'. Several folders are found but only two that contain images. What is interesting is that the '2016-04-01 - Watercolor' folder does not show the 'nef' files as I have found. But another folder (2018-05-01 - Watercolor), containing 144 'raf' image files among dng, tif, and jpegs image files, displays all files but the 'raf' files:

1679454917_2018watercolornonef.thumb.jpg.7028e8dcce4f761f32b37c901745c797.jpg

nef files are raw image files created by Nikon cameras. raf files are raw image files created by Fujifilm cameras. I point this out because they are similar "raw image file" formats. But retrospect backup doesn't care what the file is. This experiment has no 'Exclude' criteria. 

It seems to me the 'Watercolor' search criteria is the issue. So I took a look at my folders in windows explorer. A while back I migrated from the raw image editing application Adobe Lightroom to another raw image editing application Capture One. Once I started using Capture One I started using a different folder format for storing my digital images. These two 'Watercolor' folders are a case where I first edited the raw files in Lightroom, then after switching to Capture One went back and copied the raw files to new folders in a different directory.

In the case of the 2016 folders, I deleted the Lightroom folder after I was happy with my Capture One edits. When I set the search for "Files or Folders" that contain "Florida",  several (>13000) files are found. When I drill down to the 2016 image folders, I see the Capture One 'Watercolor' and the (deleted) Lightroom 'Florida' folder. Those 'Florida' folders display the 'nef' raw images files:

1068705505_DeletedFlorida-2016folder.thumb.jpg.38d7e831bc7d5d751e5458f1fb940fb0.jpg

When I browse the 'Recent Snapshots' in my backup set, none of them have the 2016-04-01-Florida folder displayed in them.  The same is true for the raf files in the 2018 Watercolor folder and 2018 Florida folders.

Is Retrospect using 'Deduplication' here? Even if so shouldn't my nef files appear in both directories?  I can restore the entire 2016 Watercolor folder, including nef files by using the Restore --> Restore dialog. But I really wanted to restore the 'nef' files which should be performed with a simple search in Retrospect.

At this point I will try manually grooming the disk and see if that resolves the issue.

UPDATE 3 HOURS LATER: Disk grooming did not make any difference.

Thanks again for all suggestions and any other thoughts.

Greg

Edited by gnoelken
Disk grooming results

Share this post


Link to post
Share on other sites

If I were you, I'd start again from scratch, but coming from the opposite direction. And remember that it pays to be as explicit as possible with selectors -- so files don't end with "nef", they end in ".nef".

So start with only the "filename ends with .nef" selector. If that picks up all you expect, add "and Windows file or folder path starts with D:\Greg\" (remember -- explicit! Include the trailing backslash). Then, maybe a "Windows path of folder contains..." to get just your subfolder. But you may not need to even go that far if you can manually remove/select what you want from the results of the first filter.

Selectors can be tricky beasts, which don't always behave the way you'd expect -- or the way selectors anywhere else would! But they work well once you master their own particular logic. If they then don't show what you expect it's usually a wrongly-chosen snapshot or similar, so search the whole set.

Share this post


Link to post
Share on other sites

Thank you for you help Nigel.

When I start with "filename ends with .nef" the deleted folder "2016-04-Florida" appears and my current folder "2016-04-01 - Watercolor" does not:

616945672_20200706endsin_nef.thumb.jpg.d9a0047ab53751c20278fdfc852963c2.jpg

I am searching the whole set but when I look at the Backup Set I can only browse snapshots within it. Every snapshot has the "2016-04-01 - Watercolor" folder in it and none of them have the "2016-04-Florida" folder:

540053711_20200706-snapshot.thumb.jpg.404dc8dd2ba833e34bf7e7aaa37abf9e.jpg

If I search "filename contains florida" both folders appear but no .nef files are found in the "2016-04-01 - Watercolor" folder.

2006141594_20200706containsflorida.jpg.573e85c4feaf64b100ba4740e99e684a.jpg

Thanks again,

Greg

Share this post


Link to post
Share on other sites
4 hours ago, gnoelken said:

When I start with "filename ends with .nef" the deleted folder "2016-04-Florida" appears and my current folder "2016-04-01 - Watercolor" does not:

So "Watercolor" is on your D:\ drive, contains "*.nef" files, but those files don't appear when you search your backups for them. Almost sounds as if they haven't been backed up -- have you checked for any exclusions in your backup scripts?

4 hours ago, gnoelken said:

I am searching the whole set but when I look at the Backup Set I can only browse snapshots within it.

You can also browse the entire Backup set by doing a search with no criteria -- IIRC, on Windows it defaults to "Include everything" and "Exclude nothing" -- then browsing the results. It'll probably be a long list, you'll get every backed version of every document, but you'll at least be able to drill down to "2016-04-01 - Watercolor" and see what's in there. If you still don't see the .nef files that strongly suggests they were never backed up for some reason.

I'd be inclined to do a quick test. Duplicate the backup script you've been using, define "2016-04-01 - Watercolor" as a volume, change the duplicate script to back up only that volume and ideally the destination to be somewhere new (stick, HD, share, cloud -- doesn't matter, it just pays to play safe and keep it separate from your "real" backups). Run it and see what happens -- do the .nef files get backed up?

Share this post


Link to post
Share on other sites

Thank you again for you help and suggestions.

2 hours ago, Nigel Smith said:

You can also browse the entire Backup set by doing a search with no criteria -- IIRC, on Windows it defaults to "Include everything" and "Exclude nothing" -- then browsing the results. It'll probably be a long list, you'll get every backed version of every document, but you'll at least be able to drill down to "2016-04-01 - Watercolor" and see what's in there. If you still don't see the .nef files that strongly suggests they were never backed up for some reason.

Thank you for the tip. When I search "all files" both folders appear, but no .nef files are found in "2016-04-01 - Watercolor".

2 hours ago, Nigel Smith said:

I'd be inclined to do a quick test. Duplicate the backup script you've been using, define "2016-04-01 - Watercolor" as a volume, change the duplicate script to back up only that volume and ideally the destination to be somewhere new (stick, HD, share, cloud -- doesn't matter, it just pays to play safe and keep it separate from your "real" backups). Run it and see what happens -- do the .nef files get backed up?

Another good idea! When I use the duplicate script the entire "2016-04-01 - Watercolor" folder gets backed up. I then search for "Ends with .nef" and the 69 nef files show up:

88676609_20200706Archive_testBackupSet-neffiles.thumb.jpg.b5b35d3fed4cf9d65a13f2764137f842.jpg

When I just do a simple restore and drill down to the "2016-04-01 - Watercolor" folder and restore it, the entire folder, including all of the.nef files, are restored. So they are getting backed up. And I can restore other folders using the "ends with .nef". The 2 folders that I "modified" with Windows Explorer are the '*Watercolor*" folders in which I copied the files from one folder to the new "*Watercolor*" folder. In both cases only the .nef (and .raf) raw files were copied. All the other files in the (new) folders (.xmp, jpg, .cos,...) were created as I edited the raw files in my image editing software and those all show up in every search.

I really appreciate your help. I can and did restore the .nef files using the simple restore. But, for me, it is now a matter of figuring out why the "Find Files" method does not work. 

Greg

Share this post


Link to post
Share on other sites

If you got the simple restore to work and saved the job, you should be able to toggle back over to the advanced restore and look for differences.  I don't do a whole lot of restoring.  The most fun I've had with this type of thing was to define my Downloads directory and exclude it from being backed up.  I took several runs at that before I could get it to work.

 

Share this post


Link to post
Share on other sites
14 hours ago, gnoelken said:

But, for me, it is now a matter of figuring out why the "Find Files" method does not work.

Only think left I can think of is an indexing issue with the set's database -- and that's assuming that there's a database, that the database is indexed, and the index is used when searching... I'm guessing all the "missing" .nef files were backed up in the same session?

You could always try a catalog rebuild and see if the problem persists -- but make sure your original catalog is safe and you don't overwrite it in the process!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×