Retro 10 doesn't backup last 30 minutes of file changes?!

[amkassir]

(I posted this as a reply in a different thread but received no replies, so I'm starting a new thread on this topic.)

 

From the RetroISA FAQ:

Question

: If I change a file on my disk and immediately run a backup or trigger a Backup on Demand, will that file be included in my backup?

 

Answer

: It is possible that a newly saved file will not be included in a backup that runs within 30 minutes of a file change. If you were to copy a large number of files to a volume, that will result in a longer amount of time to scan and update the RetroISA scan files. The RetroISA process must have time to scan and update the Scan File before a backup for those file changes to be seen when the backup starts. A scan file may be updated in as little as 5 minutes after a file has changed, but on larger volumes it could take longer to update the scan files.

 

You may also see error -1101 (File/Folder not found) if a file was deleted from your disk between the last time a scan file was updated and when you have attempted a backup of a disk.

 

This seems like a potentially major problem. In the vast majority of cases, I perform a backup shortly after I've finished several hours of work, certainly less than 30 minutes later. According to this Q&A, there's no guarantee that the last 30 minutes of work will be backed up. For me, that includes the last 30 minutes of medical records that absolutely must always be backed up. Also, this doesn't seem like a good solution for backing up the active boot volume, where files are frequently changing. I can imagine dozens if not hundreds of files not being backed up if the backup window is 30 minutes.

 

Despite these limitations, I said to myself, "no sweat--I can still upgrade to Retrospect 10 if I disable the RetroISA function." The extra time to scan a volume is worth the guarantee of backing up every file.

 

Reading more about RetroISA, in order to disable the process, the user has to edit the retro_isa.ini file. What? You mean there's no preference in Retrospect 10's user interface to turn off RetroISA or exclude certain volumes? And if you're not comfortable mucking about in the retro_isa.ini file, you risk not backing up 30 minutes of work? This can't be good...

 

So far, Retrospect 10's landmark feature, RetroISA, seems more of a liability than an asset.

 

Am I stuck with a choice--sticking with Retro 9, or editing the retro_isa.ini file?

 

Thanks for any input.

[Maser]

I believe you could also get around this by sticking with the Retrospect 9 client -- only the 10 client adds the "retroisa" process that is looked for by the 10 Engine.

 

I went around with them on this during the testing process and you are essentially correct -- you'd need to stick with the 9 *client* or edit the retro_isa.ini file.

 

That said, if you watch the "Scans" folder in /library/application support/retrospect -- you'll see how often the .dat/.inf files are updated for your volume.

 

In general, though, -- unless you are installing an operating system upgrade (which touches thousands of files) -- the process seems to update every 5 minutes or so. I think they posted 30 minutes on the FAQ for outlier systems (such as those that have multiple hard disk partitions that are being touched consistently.)

[amkassir]

I believe you could also get around this by sticking with the Retrospect 9 client...

The Retrospect manual says, "Retrospect Clients extend the backup and restore capabilities of Retrospect to other computers on your network. "

 

This is probably a basic question, but I don't have any other computers on my network that I'm backing up, so I'm not sure if I have Retrospect 9 clients to begin with. I have Retrospect 9 installed on my boot volume and use that for backups of various sources to various media sets. How would I stick with the Retrospect 9 client after installing Retrospect 10?

 

(Is the Retrospect application the client software?)

 

Thanks for your help.

[amkassir]

Okay, so I installed Retrospect 10. In the /Library/Application Support/Retrospect/RetroISAScans/ folder, 2 files appeared. One ending in .dat, the other in .inf. So far, so good. After a few minutes, no other files appeared. My internal drive is partitioned into 4 partitions.

 

I attached a FireWire drive containing 3 other volumes, and mounted an encrypted disk image. A few moments later, 4 more files appeared in the RetroISAScans folder, 2 of each type.

 

Next, I duplicated my retro_isa.ini file.

 

Using TextEdit, I edited the "retro_isa.ini copy" file to exclude all of the volumes except for my "iMac HD" volume. I double-checked to make sure the exclusions were entered properly. For example, I have ExcludedVolume3=/Volumes/Media and Storage/

According to the instructions, the volume name is entered spaces and all, without "\" backslashes.

 

After I was done, I saved the file, trashed the original, and removed "copy" from the name of the edited file.

 

Using Activity Monitor, I quit the RetroISA process. After a few moments, it quit and restarted itself. At this point, I expected RetroISA to stop scanning the other volumes. Then, 2 more files in RetroISAScans appeared.

 

My questions:

1. How can I make sure my excluded volumes are not being scanned?
   
2. How can I tell which RetroISAScan files (of both types) correspond to which volumes?
   
3. Will Retrospect or RetroISA delete the unused Scan files in the RetroISAScans folder, after I excluded volumes? If not, how do I tell which files to manually delete?
   
4. If RetroISA is properly excluding the other mounted volumes, why would 2 more RetroISAScan files appear after restarting RetroISA?
   
5. Was I correct to use TextEdit to edit the retro_isa.ini file?
   

 

This would be so much easier and more fool-proof if the ability to exclude volumes and turn on/off RetroISA was in the Retrospect preferences.

 

Thanks for your assistance.

[Maser]

The .dat files will be named with the UUID of your volumes -- so you can compare those names with the UUID of the volumes as the show in Disk Utility. If you've excluded the right volumes, the .dat files will never get updated again (you could probably also *delete* the .dat/.inf pair and they should never get recreated.)

 

TextEdit should be fine to modify the file, but doing it in Terminal is probably best (and you have to stop/start the process to get the file to be reread -- which may be your issue.) Just reboot after you edit the file and you should be fine.

 

With your initial observation that only *one pair* of .dat/.inf files being created -- if you have multiple partitions, it creates the set of files one volume at a time -- and doesn't move on to the next volume until it's done with the current volume. So the "4 more files" -- may have been from the FW drive *or* may have been from one partition on the FW drive and one partition on the internal drive (or possibly two partitions on one of the drive, etc...) You *should* (eventually) have generated 7 sets of files (4 from the internal and 3 from the FW drive). It can take a while to generate these sets of files, though, depending on how many files are on each volume.

 

 

When you connect a FW drive, it will generate new .dat/.inf files for those drive (but not for a mounted disk image, IIRC). When you disconnect the FW drive, it *should* delete the files associated with those volumes (or it removes them after a restart). I think the last time I tested, they were gone the next time the "retroisa" process did a scan. (I just tried this -- I have a FW drive with 5 parititions on it and just one partition on my internal drive -- while typing this up, I now have 6 sets of files -- as expected. Once I disconnected my FW drive (ejecting all 5 partitions) -- only the internal drive set of files was still there...)

 

I don't think "quitting" the process actually deletes the files. I think you have to "unload" the process to stop it from running (otherwise it restarts itself automatically)

[amkassir]

Maser, thanks for the reply and insights.

 

Today I did my first incremental backups since upgrading to Retro 10. I usually backup my boot partition twice--one to a drive I keep in the office and one to a drive that I take offsite.

 

The RetroISAScans folder contained the pair of files pertaining to my boot partition, and when I did the first backup, the scan went very fast. A couple of hours later I got around to the second backup, but I noticed the RetroISAScans folder was gone. After a minute it was recreated, empty. I started the second backup of my boot partition and the scan took as long as it used to in Retrospect 9. The log showed the first backup used Instant Scan, but the log didn't show that for the second backup of the same partition. After several minutes, the two files were back in the RetroISAScans folder, but it was too late for my second backup scan to be "instant."

 

Questions:

1. Why does Retrospect delete the RetroISAScans folder/files after the first backup, preventing the second backup from being as fast?

 

2. Are the files supposed to be deleted and recreated after a given backup?

 

3. I'm also wondering how RetroISA works--it's supposed to keep track of what files have changed since the last backup, but what if you do several separate backups of the same volume? Does it keep track of the dates/times of each previous backup somewhere? I was also wondering if the .dat scan file just keeps growing arbitrarily large, as it keeps a running tally of every change on the volume, but the fact that the scan files were automatically deleted and recreated kind of answers that question.

 

4. How do you "unload" a process?

 

Thanks.

Edited November 16, 2012 by amkassir

[amkassir]

One other observation: Since installing Retrospect 10, about every 5 minutes my 2 processors are each being used at roughly 50% for a couple minutes. During this time my system slows down, typical activities lag, and I get the spinning rainbow more often. This is probably because RetroISA is doing its thing. Is there any way to reduce the load on the system? It interferes with my work.

 

Thanks.

[Maser]

In my testing, the RetroISAScans *folder* -- should never be deleted (unless you do it manually). The .dat/.inf file pairs for the volumes *might* be deleted if the process determines there was a problem with the files (after all, these files must be bullet-proof containing the data that the computer says should be backed up). If the .dat/.inf files are deleted (manually or automatically), it can take about an hour before the system realizes they are gone before they start being recreated (in which time a backup of the client would revert to the old behavior...)

 

In general, though, the .dat/.inf files for non-Firewire volumes -- should be there all the time (and update every 5-7 minutes automatically).

 

 

I agree about the 5 minutes CPU-load thing. Trust me, it was a *lot* worse before the GM release...

[amkassir]

Maser, thank you very much for the reply.

 

One question remains:

I don't think "quitting" the process actually deletes the files. I think you have to "unload" the process to stop it from running (otherwise it restarts itself automatically)

"How do you "unload" a process?"

[CallMeDave]

"How do you "unload" a process?"

 

 

man launchctl

 

https://developer.ap...aunchctl.1.html

 

(stoopid fourm truncating the text of my URL)

 

**https://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man1/launchctl.1.html**

[amkassir]

Thanks CallMeDave. Looks like invoking the launchctl command is necessary to permanently disable RetroISA (the setting in the retro_isa.ini file reverts to "=1" after a restart unless you use launchctl to unload RetroISA. For more detail, see: http://forums.retrospect.com/index.php?/topic/149942-version-10-comments-background-app-is-expensive-docs/page__p__252897&do=findComment&comment=252897