Jump to content

Exceptionally long "Updating catalog" phase?


Recommended Posts

I backed up a Macbook air. First backup to a recently recycled backup set.

 

Retrospect took perhaps an hour or two to backup, but then got stuck with the summary saying "1 file remaining", "Updating catalog" (or words to that effect). It then spent about several hours in that state before completing the backup.

 

[/size][/font][font=Helvetica][size=3]
Completed: 748479 files, 28.6 GB[/size][/font][font=Helvetica][size=3]
Performance: 265.1 MB/minute (196.5 copy, 407.4 compare)[/size][/font][font=Helvetica][size=3]
Duration: 07:04:17 (03:23:17 idle/loading/preparing)[/size][/font]

The Macbook Air was plugged in to a power adapter, left open, and set to not sleep.

What's up with that? 7 hours to backup 26GB is a bit pathetic.

Retrospect 10.0.1 (105), server and client running 10.8.2.

Link to comment
Share on other sites

What type of verification are you using? If you are using thorough verification it can take longer to verify than the actual backup as it has to compare each backed up file to the original file. Try either media verification or no verification and see if the time is reduced.

Link to comment
Share on other sites

What's the point of a backup without verification?

 

It is now doing a proactive backup to the MacBook air, again, plugged in, set to not sleep, and its bee "Building a snapshot" for three hours. Seriously! It's only got 30GB on the HD, and it's only got 500MB to back up. 3 hours to build a snashot for a laptop is untenable. I can't have the laptop plugged in and awake and open for 3 hours at a time (and more!) just to get an incremental update.

 

I really want to get back to loving Retrospect like I did 15 years ago when it was reliable, but this is ridiculous. Of my four Macs, the only Mac it will reliably back up is the server! The laptops take ages to do incremental backups, so long they would never complete if I don't actively leave them open, and it wont back up my desktop at all, failing with an error. Even though I've purchased Retrospect in the hopes I can make it work, I'm going to have to give up soon.

Link to comment
Share on other sites

750,000 files is not too bad. A MacBook Air has always been supplied with an SSD, which should be fast. Which Air is it? The first generation? The latest generation?

How much free RAM does the client have during the backup?

Is anything using a lot of CPU (on the client) when the backup runs?

 

 

Try a safe boot, then boot normally before trying again.

http://support.apple.com/kb/HT1564?viewlocale=en_US

http://support.apple.com/kb/ht1455

Link to comment
Share on other sites

I did a Safe Reboot on the MacBook air - not issues, and no change.

 

 

I tried another backup - 123 files backed up, 96MB, the backup had more or less finished after 10 minutes, and then 3 hours of "Building Snapshot".

 

It is a 13-inch mid 2011 model. It has 4GB of RAM, with (after the backup) 1.5GB free, 1GB wired, 1.3GB active, 0.3GB inactive. Nothing is using appreciable CPU.

 

I removed everything that was taking any CPU, and set off another backup. Again, it "backed up" in about ten minutes, and then went off to "Building Snapshot". It's been going for 40 minutes now, I presume it will take another few hours to complete. Nothing on the MacBook Air (including retrospect) is using more than 1% CPU. Network bandwidth is about 25KB/sec. Disk access on the MacBook Air is Zero.

 

I wouldn't mind so much it is had released the connection to the laptop, but laptops don't stay open for three hours at a time, and closing it while the backup is in process has (at least in previous tests) left retrospect in an unhappy state.

 

Notably, CarbonCopyCloner backed up the entire Mac (to a local harddisk admittedly) in 40 minutes, so I don't see how it can be justified to take 3 hours to build a snapshot of anything.

Link to comment
Share on other sites

So the school wiped the Mac and re-imaged it, which was, needless to say, a real pain to then restore the files on.

 

But the new backup now does an incremental backup in 11 minutes. So whatever the issue was, it appears to have been resolved, at least for the time being.

 

Unfortunately, this doesn't really result in any useful information to determine what was actually going on and how it could be avoided in the future.

 

Thanks for the assistance.

Link to comment
Share on other sites

  • 2 weeks later...

OK, so after the update to 10.1, this is now happening on my other laptop. 5minutes to backup, and then an hour to "update the catalog".

 

I've tried restarting the client and the server, but I don't know what else to do to resolve the issue - a backup from a laptop has to be fast enough that it is finished before the user closes the lid.

Link to comment
Share on other sites

I've had a support request going for a couple of months now with the same issue - 2 x MacBook Airs (latest 13 inch models), and the building snapshot process taking hours to complete (even if it only backs up a few files). The new version just released hasn't made any difference - in fact, it's actually taking longer to build the snapshots - around 2.5 hours. And this is only backing up a Users Library folder (excluding Cache folder) - not the whole system.

 

Still waiting to hear back from Support after submitting numerous log files.

Link to comment
Share on other sites

it's actually taking longer to build the snapshots - around 2.5 hours. And this is only backing up a Users Library folder (excluding Cache folder) - not the whole system.

 

Will check the support ticket history tomorrow. In the interim, a sanity check:

Are you using a Favorite as the backup source (instead of a Rule on the entire volume or entire computer as backup source) to backup the Users Library folder? If not, that will significantly shorten the snapshot building time, which depends on how many files and folders exist on the backup source, regardless of how many files are actually backed up.

 

Still waiting to hear back from Support after submitting numerous log files.

 

I am pinging the team for update.

Link to comment
Share on other sites

Are you using a Favorite as the backup source (instead of a Rule on the entire volume or entire computer as backup source) to backup the Users Library folder? If not, that will significantly shorten the snapshot building time, which depends on how many files and folders exist on the backup source, regardless of how many files are actually backed up.

 

Yes, it's a favourite. We have 2 favourites set per client Mac - Documents folder and Library Folder. Each favourite is tagged as "BU-clients" which is used as the source for the backup script.

 

All other clients are fine (mix of 10.8 and 10.6) but for whatever reason, 2 MacBook Airs we have are taking anywhere from 1 hour to 2.6 hours - below was today's log.

 

Generally all that's being backed up day-to-day are emails from the Library folder.

 

 

Completed: 430 files, 126.1 MB, with 24% compression

Performance: 194 MB/minute

Duration: 01:25:25 (01:24:46 idle/loading/preparing)

Link to comment
Share on other sites

Have you tried turning off compression as that can slow down the process?

No, I haven't, but the other 25 clients don't have the same issue backing up the exact same scenario, so I doubt compression would be an issue (it's also running on a top spec xServe / SSD / 12GB so I doubt speed would be an issue).

Link to comment
Share on other sites

In my experience, the script doesn't make any difference - I get the same result whether the Proactive Script (with two laptops) chooses the backup the laptop, or if I backup with a specific script for that laptop. And indeed, the same Proactive Script works fine and completes promptly for one laptop while taking an extra hour for the other (faster!) laptop.

Link to comment
Share on other sites

  • 3 weeks later...

Thanks to Disegno Group's and peternlewis' patience in working with us, we have reproduced this issue and is testing a fix. We expect the fix to be included in the next free update.

 

Background:

During the "Building snapshot" phase, (to ensure correct restore) Retrospect is backing up Mac HFS file system's ACLs and extended attributes for every folder in the backup source / favorite, regardless of selectors. Most folders don't have ACLs and extended attributes. Some versions of the Apple developer tools do create the /Users/_UserNameHere_/Library/Developer/Shared/Documentation/DocSets folder which may contain 10,000+ folders with ACLs and/or extended attributes.

 

Cause:

Compared to the Retrospect Windows Engine, the Retrospect Mac Engine has some degree of inefficiency specifically when backing the ACLs and extended attributes for folders from a Retrospect Client (not local HFS volume). This inefficiency is amplified when the Client is connected via WiFi.

 

Preliminary improvement:

After tuning the related code, on some internal test systems including my primary Mac, we are seeing significant improvements in preliminary testing specific to Mac Engine's "Building snapshot..." phase for Mac Client with large number of folders that have ACLs and/or extended attributes.

We are in the process of additional performance and regression testing. The improvement helps reduce overall backup time and targets the "Building snapshot..." phase specifically with folders meeting the mentioned criteria, and doesn't accelerate other phases of the backup process.

 

Applicable cases:

To see if your Mac Client would benefit from the upcoming improvement, run the following command in the Terminal window of the Mac Client computer. The preliminary estimate of time savings would be the result from the command line x 0.2 seconds for Client on WiFi. The savings when processing 10,000 folders meeting the mentioned criteria for Client on WiFi would be over half an hour. There will also be savings but to a smaller degree for Client on ethernet.

sudo find _YourBackupSourcePathHere_ -type d -exec ls -dlF {} \; | grep '^d.........[@+]' | wc -l

Link to comment
Share on other sites

Hi David,

 

Thanks for the news and update. I'd just like to mention that my issue with Building Snapshots never ending is when using Windows Clients and a Mac server. I'm not sure if that should get posted under the Windows or Mac subforum. Also, I haven't had this happen yet on the latest release of Mac server (10.1.0(221)) so I'm not sure if that issue was addressed or not.

 

Thanks

-Derek Cunningham

Link to comment
Share on other sites

Applicable cases:

To see if your Mac Client would benefit from the upcoming improvement, run the following command in the Terminal window of the Mac Client computer. The preliminary estimate of time savings would be the result from the command line x 0.2 seconds for Client on WiFi. The savings when processing 10,000 folders meeting the mentioned criteria for Client on WiFi would be over half an hour. There will also be savings but to a smaller degree for Client on ethernet.

sudo find _YourBackupSourcePathHere_ -type d -exec ls -dlF {} \; | grep '^d.........[@+]' | wc -l

 

 

So I ran this on my computer (which has xCode installed) and I backup the /Users folder as a Favorite Folder

 

 

maser$ sudo find /Users -type d -exec ls -dlF {} \; | grep '^d.........[@+]' | wc -l

3582

 

 

So, you are saying I should see 3582 *.2 = 716 seconds of savings here? Approximately 12 minutes when building the snapshot after each backup? Or not that much in savings as all of my backups are over Ethernet?

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...