Jump to content
roobieroo

Skipping many files that have been created/modified since last backup

Recommended Posts

This is pretty terrifying. I noticed that most of my clients are only getting about 30 or so files backed up each night which didn't seem right so I started looking at the backups to see just what was getting backed up. I found that many files that have been created or modified since previous backups are ignored by Retrospect. For example, it may back up their Mail program's Envelope Index file but skip all email messages that they've received for the past few days. On a different computer a user had saved a number of files to her desktop that are no where to be found in Retrospect. Basically these backups are useless.

 

If I perform a manual backup and preview what will get backed up it shows that the new mail files are checked for backup but the daily script is just skipping over them for some reason. I thought it was due to the ISA being screwed up but I disabled and removed it from all of the clients days ago. I'm using the exact same file selection rule for both the manual backup and for the scripted backup. I thought it might also be an issue with file matching but some of the files that it's skipping aren't anywhere on the backup to begin with.

 

Retrospect is on a machine running Mac OS 10.8.2 and is version 10.0.1(105). The clients are running 10.0.0(174).

  • Like 1

Share this post


Link to post
Share on other sites

If this bug is real, it's a Big Deal. FWIW, I have downgraded my clients again, so I am not running the "new" ISA clients anywhere. Despite this, there is data being collected by the engine, and I still see stuff in the log about "Using Instant Scan". Given some other posts in this forum, it is not clear to me what you have to do to "disable" ISA, which looks to me like it has bugs. Even if it doesn't, testing would be a lot easier if it could be easily and reliably toggled.

 

See http://forums.retros...during-backups/

 

See: http://forums.retros...-retrospect-10/

 

See: http://forums.retrospect.com/index.php?/topic/150163-backup-copies-no-files-with-v10-repeatedly/

Share this post


Link to post
Share on other sites

I followed the instructions of unloading using launchctl and then I deleted the launch daemon instead of moving it because I don't need it in my environments. I haven't downgraded clients because only version 10 seems to allow the backup to happen each time without getting a zero files need to be copied message 20 seconds after it connects to the client. Of course when the backup that does happen is skipping files seemingly at random it's not much better than the zero files problem.

Share this post


Link to post
Share on other sites

This morning I created a new folder and then saved a text file on six different computers to see if they would get backed up when tonight's backup ran and it wasn't backed up on any of them. Some other files were backed up so the backup did something, it just didn't backup my test files and who knows what else.

Share this post


Link to post
Share on other sites

I noticed something similar today as well. I have a proactive script that is backing up a PowerPC Mac OS X 10.5.8 client (retro client 6.3.029) favorite folder to a 10.6.8 engine - Retro 10.0.1 (105). It is backing up files that it should not, (have not changed), and failing to back up files that should be backed up. The proactive script says "every day", so I have data. I also see instances where there is a pattern - a few files from the source each day, followed by an instance of zero files, followed by a "bunch" of files the next backup, as though the zero-file instance triggered some bug that caused a bunch of files to show up as needing backup.

 

I have also seen instances of backups where the incremental is backing up more files than the initial backup of that source, which should not happen, unless there were lots of changes - not in this case.

 

Looks like a bug to me..... and concerns me.

 

I have not done the full boat chcking of exxactly what has been backed up, and comparing trees and the like. That would be a bunch of work, but I have checked enough to satisfy myself that there is something not right.

Share this post


Link to post
Share on other sites

Looks like a bug to me..... and concerns me.

 

I have not done the full boat chcking of exxactly what has been backed up, and comparing trees and the like. That would be a bunch of work, but I have checked enough to satisfy myself that there is something not right.

 

Will you be contacting Retrospect support about this? I'd be interested to hear what they have to say. I've got an open ticket and will post back if I get anywhere with them but the more they hear about it the better.

Share this post


Link to post
Share on other sites

I contacted support and was given a link to a prerelease version of Retrospect along with an updated client. I'll be installing these today and will let you know if they help.

Share this post


Link to post
Share on other sites

The update was still skipping lots of files seemingly at random. I could backup the same system with a new script and it would back up some files but still not others that had been modified since the last backup. The only way I could get Retrospect to backup all of the files was to unload the ISA launchd and delete all ISA related files from /Library/Application Support/Retrospect. Simply uloading and removing the launchd file was not enough. If you're using version 10, I'd recommend you do the same on each client since there's no great way to know which files ISA is causing Retrospect to skip.

Share this post


Link to post
Share on other sites

Is there something specific about the location/path name of the files that are being skipped? (Or is it an example of you have a folder with 100 files in them and only some of them are getting backed up?)

Share this post


Link to post
Share on other sites

Some files in the same location are being backed up while others are skipped. One client had an entire folder that was getting skipped along with some other random files in various folders. Below is the worst example of this but I've seen it on multiple clients. One of the easiest ways to check for missing files was by looking at email folders since I knew that there would be a decent number new files each day. I haven't been able to tell why instant scan backs up some folders and files but skips others.

 

Here is a log entry after performing a backup using the same selection rule to the same media set. The first used instant scan and the second is after unloading it and deleting all instant scan files from the client.

 

+ Normal backup using Testing at 1/26/13 (Activity Thread 1)

To Backup Set Week1...

- 1/26/13 7:34:42 PM: Copying Users on Alexis

Using Instant Scan

1/26/13 7:35:36 PM: Snapshot stored, 22.5 MB

1/26/13 7:35:38 PM: Execution completed successfully

Remaining: 3 files, 36 KB

Completed: 59 files, 129.4 MB

Performance: 970.4 MB/minute

Duration: 00:00:56 (00:00:47 idle/loading/preparing)

 

+ Normal backup using Testing at 1/26/13 (Activity Thread 1)

To Backup Set Week1...

- 1/26/13 7:40:12 PM: Copying Users on Alexis

1/26/13 7:48:25 PM: Snapshot stored, 24.4 MB

1/26/13 7:48:28 PM: Execution completed successfully

Completed: 7121 files, 10.9 GB

Performance: 1,631 MB/minute

Duration: 00:08:15 (00:01:23 idle/loading/preparing)

Share this post


Link to post
Share on other sites

So, if you have a folder that contains files where some are backed up and some are not...

 

What happens if you move that folder to another computer running the 10 client (with instant scan still enabled?)

Share this post


Link to post
Share on other sites

I haven't tried it and have disabled instant scan on every client. I probably won't be enabling it again either since I don't need the speed increase and just don't trust it anymore.

Share this post


Link to post
Share on other sites

I am also seeing this. Really horrible! Loads of files are not backed up. I have not gone through all the details, but here is what it looks like:

 

- I do not think this is related to the client, but to the Server.

- We did not install the version 10 client on many macs as it - on some macs - runs at 100% CPU 24/7.

- I even have a client running Mac OS 10.6.8 with version 6 client that now skips *a lot of* files on every backup

- This was, as far as I can tell, NOT an issue with Retrospect before version 10

- We have two separate Retrospect 10 servers. Both have this error, randomly skipping many, many files.

 

This really is a serious issue!

Share this post


Link to post
Share on other sites

Are you running the beta or the release version? There is a fix specifically for backing up non version 10 clients. Be sure to disable instant scan on all version 10 clients as well. http://kb.retrospect...icle/grx-assert

 

launchctl unload /Library/LaunchDaemons/com.retrospect.retroisa.plist

rm /Library/LaunchDaemons/com.retrospect.retroisa.plist

rm /Library/Application\ Support/Retrospect/ConfigISA.bak

rm /Library/Application\ Support/Retrospect/ConfigISA.dat

rm /Library/Application\ Support/Retrospect/retro_isa.ini

rm /Library/Application\ Support/Retrospect/retroISA_log.utx

rm -rf /Library/Application\ Support/Retrospect/RetroISAScans

rm -rf /Library/Application\ Support/Retrospect/RetrospectInstantScan.bundle

Share this post


Link to post
Share on other sites

Are you running the beta or the release version? There is a fix specifically for backing up non version 10 clients. Be sure to disable instant scan on all version 10 clients as well. http://kb.retrospect...icle/grx-assert

 

:snip:

 

I have so far avoided the pre-release version. (I am not very fond of the concept of using beta software in production.) I will however install it now as the current release is dangerous.

The problem happenes on all three of my client versions; 6, 9 and 10.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×