lunadesign Posted October 5, 2006 Report Share Posted October 5, 2006 Incremental backups used to fly for me under Retrospect 6.5. I just moved to 7.5 and am finding that the backups spend 99% of their time in the "Building Snapshot" phase. What's amazing is when I'm watching the Activity Monitor during this phase, the display that shows the current file/folder is *really* slow (perhaps showing 1 or 2 items per second). Here's a specific example backing up to disk over the local network: Source: Windows XP with Retrospect 7.5.116 client 3.0 GHz Pentium 4, 2GB RAM, 74gb 10K RPM SCSI disk 18gb C partition with 153,000 files Incremental backup of 457 files (171MB) took 46 minutes, 28 seconds 45 minutes, 9 seconds of that time was in "Building Snapshot" phase Backup Server: Windows XP with Retrospect 7.5.324 Professional 2.0 GHz Core Duo, 2GB RAM, 500gb 7200RPM SATA disk Is there anything I can do to speed this up? Quote Link to comment Share on other sites More sharing options...
lunadesign Posted October 27, 2006 Author Report Share Posted October 27, 2006 Anyone? Bueller? Bueller? Quote Link to comment Share on other sites More sharing options...
RogerS Posted November 3, 2006 Report Share Posted November 3, 2006 I've given up on asking this question as well as any hope of receiving an answer... I first asked it 2 years ago. Last week, I created a new backup set for November 06, and a subsequent full backup for this new set. I run a Raid 5 array, 1 terrabyte. 3 partitions C, D, E. They're combined as a group going to a 3 member disk of 350 Gbt each. External Firewire 800. A total backup of apprx. 80 Gbt and 135K files. "File copy" takes about 15 minutes at an average 600Mbt/M. (Ver. 7.0) "Building Snapshot"... About 60 minutes. "Compare" About 8 minutes at apprx. 1000 Mbt/M. OK! Not bad. I can tolerate this. 3 minutes after completion, I ordered another backup. 12 Mbt, 52 files. "Copy" took about 2 seconds. "Compare" took about the same. "Building Snapshot" "Priceless"! 27 minutes! Nobody seems to want to talk about this... No good answers why building an .ISO file exceeds 700 Mbt's when using a retail XP-Pro sp2 CD as the key file source either! I used "Isobuster" to break open the Iso and spent several hours pouring over the files within. No "Big" files, No bevy of useless drivers, Just lots of system files placed there by Retro... Given that I've had my share of unmittigated disasters (Lightning bolts to my work station), Complete hard drive failure (Old system), Hurricane flood out of the one before that, and that Retro bailed me out 100% EVERY TIME!! Can I really beech? No. I just put up with it... But I still would like to know why! (It's the Engineer in me)! Quote Link to comment Share on other sites More sharing options...
markauk Posted November 4, 2006 Report Share Posted November 4, 2006 I think I remember reading somewhere that if you turn off the 'backup file security' option(s) for a backup then snapshot building is much quicker. Of course, that wouldn't help if you need to backup the security information... Quote Link to comment Share on other sites More sharing options...
lunadesign Posted November 4, 2006 Author Report Share Posted November 4, 2006 I've been interacting with EMC support on this for the past 4 weeks and am still awaiting a final answer. But so far, I've discovered the following (mostly on my own): 1) I'm running my backups with the default Security Options: Back up file security information from servers = checked Back up file security information from workstations = unchecked Back up folder security information from servers = checked Back up folder security information from workstations = checked If I un-check the folder security option for workstations, my "Building Snapshot" time almost disappears (I had a 45 minute snapshot build that would now happen in under 2 minutes) 2) If you're using the default Security Options, the amount of time in "Building Snapshot" is pretty much a linear relationship with the number of folders. I'm finding that every 4-5 folders equals one second of "Building Snapshot" time. 3) My "Building Snapshot" time is roughly 3x as long as it was when I was running Retro 6.5. Quote Link to comment Share on other sites More sharing options...
lunadesign Posted November 4, 2006 Author Report Share Posted November 4, 2006 For those that care, here's some 7.5 performance data from two machines with very different volumes: System 1, Volume 1: 154,107 files 11,739 folders 15.9 GB (Building snapshot takes 44:23) (Under 6.5, snapshot for this volume was 15:00) System 1, Volume 2: 256,086 files 26,401 folders 30.8 GB (Building snapshot takes 1:32:34) System 1, Volume 3: 31,737 files 1,683 folders 73.1 GB (Building snapshot takes 6:22) System 1, Volume 4: 4,912 files 1,661 folders 18.9 GB (Building snapshot takes 6:10) System 1, Volume 5: 67 files 26 folders 74.9 GB (Building snapshot takes 0:32) System 2, Volume 1: 80,392 files 8,155 folders 10.1 GB (Building snapshot takes 31:37) System 1 is the backup client I listed in my initial post. System 2 is a 2.0 GHz Pentium M laptop with 2GB of RAM and a 7200 RPM drive. If you do the math, you'll see each second of "Building Snapshot" time covers 4.3-4.8 folders on the volume (the only exception is the Volume 5 case, but that's a very atypical case). Number of files and GB seems irrelevant. Quote Link to comment Share on other sites More sharing options...
PEC Posted November 6, 2006 Report Share Posted November 6, 2006 I am just in the last couple days having the exact same problem with Pro 7.5 and the latest hotfix. I tried removing the program and doing a re-install ... went to the 'Admin Tools, Services' and made sure the entry for 'Retrospect Helper' was in 'Auto' and did a re-start... Nothing seems to work... A few days ago I installed the new IE 7.0.7530... and, that is about all I can figure. I have a email into EMC trying to figure it out. I also tried turning off all my Spyware, AntiVirus and Firewall Software to isolate this very new problem. Pete 11-05-2006 Quote Link to comment Share on other sites More sharing options...
Mayoff Posted November 7, 2006 Report Share Posted November 7, 2006 How big is the snapshot? The log should say. How big is the catalog file? How many sessions? I have a 500MB snapshot and it took about 75 minutes to complete. I have another snapshot that is 140MB and it took about 5 minutes. The more files, the longer it will take. Quote Link to comment Share on other sites More sharing options...
lunadesign Posted November 7, 2006 Author Report Share Posted November 7, 2006 Mayoff - If you're using the default security options, it doesn't seem to be the number of files, its the number of folders that affects the time. What options are you using and how many folders do you have in your 500MB and 140MB examples? Quote Link to comment Share on other sites More sharing options...
Mayoff Posted November 8, 2006 Report Share Posted November 8, 2006 What really is important is the total size of the snapshot. The larger the snapshot, the more files/folder/registry data that needs to be saved. The bigger the snapshot, the longer the process will take. If your snapshot larger or smaller then my example? Quote Link to comment Share on other sites More sharing options...
markauk Posted November 8, 2006 Report Share Posted November 8, 2006 The default security options don't backup security info from files on workstations but do backup security options from folders, so that would explain why number of folders has a bigger effect on snapshot time. If I remember correctly, it takes a lot more time for NTFS to look up and return the security details for a file/folder than the other file information recorded by the snapshot, hence building a snapshot takes much longer if you include security information. Quote Link to comment Share on other sites More sharing options...
lunadesign Posted November 9, 2006 Author Report Share Posted November 9, 2006 Mayoff - all of my snapshots are smaller in size than yours. System 1, Volume 1: 124.0 MB System 1, Volume 2: 67.8 MB System 1, Volume 3: 6.7 MB System 1, Volume 4: 2.2 MB System 1, Volume 5: 0.04 MB System 2, Volume 1: 77.5 MB I don't think the time is related to the size of the snapshot. If so, why does my 124.0 MB snapshot take 44:23 and my 67.8 MB snapshot take 1:32:34? Quote Link to comment Share on other sites More sharing options...
PEC Posted November 9, 2006 Report Share Posted November 9, 2006 Well, I fixed the Snapshot backup hangups by doing this... ( out of lack of any other ideas) a) Removed Pro 7.5.324 re-Installed Pro 7.5.285 c) Created new Backup Job Scripts d) The SnapShot worked ! e) Updated to Pro 7.5.324 f) The SnapShot Creation worked as before ! g) Did the mini upgrade to 7.5.8.101 ( HotFix) h) It still works ! i) So ... my time for a Pro 7.5 with default SecuritySettings & Snapshot on a 11.0 GB (apparent size) Job ... without Verify ... takes about 32 minutes. Life is good again.. Pete *** The bonus was that I learned how to do a small file restore without a Snapshot ! Quote Link to comment Share on other sites More sharing options...
lunadesign Posted November 10, 2006 Author Report Share Posted November 10, 2006 Hey Pete - Just curious, but how long did your "building snapshot" take before and after you made these changes? Also, how many files and folders on your volume being backed up? (I'm measuring the "building snapshot" time by looking at the "idle/loading/preparing" time in the log) Quote Link to comment Share on other sites More sharing options...
PEC Posted November 11, 2006 Report Share Posted November 11, 2006 I never completed a backup ... thought I had to figure it out ! Lets see ... I run about 11.3 GB per backup ... with Default Security, NO Verify ( about 67,000 Files ... 9,000 Folders ) ... just timed it at 23 minutes with The Snapshot. I also began using Diskeeper Pro 2007 to resort and Defrag my Backup Drive. I am pleased, again ! Pete 11-10-2006 Quote Link to comment Share on other sites More sharing options...
michaeln Posted November 13, 2006 Report Share Posted November 13, 2006 Hi, The reason why you didn't see long "building snapshot" times in 6.5 was because it didn't backup NTFS permissions on Desktop systems (non-Server OS'es). The default options in 7.0/7.5 do backup NTFS folder permissions (not files unless you specify) on Desktop systems which will increase snapshot creation times. That said, 7.5 has made significant changes to the way it backups security information from 7.0 to make it faster but I think you need to have a backup set created in 7.5. Is this a continuing backup set from 6.5? If so, can you create a brand new backup set in 7.5 and test this? Quote Link to comment Share on other sites More sharing options...
mauricev Posted December 27, 2006 Report Share Posted December 27, 2006 Quote: The reason why you didn't see long "building snapshot" times in 6.5 was because it didn't backup NTFS permissions on Desktop systems The slowness actually affects all clients, Mac and Linux, so it doesn't appear to be related to NTFS permissions (and even if it were, it shouldn't be this slow ). Quote Link to comment Share on other sites More sharing options...
ablend Posted December 28, 2006 Report Share Posted December 28, 2006 Quote: The slowness actually affects all clients, Mac and Linux, so it doesn't appear to be related to NTFS permissions (and even if it were, it shouldn't be this slow ). I agree with this point. Building a snapshot takes far longer on Macs than on any of our PCs. Both PC and Mac snapshots take longer than they did in Retrospect 7. I'm currently running Retrospect 7.5 Multi-Server. Quote Link to comment Share on other sites More sharing options...
rhwalker Posted December 28, 2006 Report Share Posted December 28, 2006 Quote: Building a snapshot takes far longer on Macs that on any of our PCs. That might be because of the ACL support that had to be added for Mac OS 10.4. Two incompatible permissions models (POSIX and ACL) existing in parallel. russ Quote Link to comment Share on other sites More sharing options...
bvaandr Posted January 30, 2007 Report Share Posted January 30, 2007 And I thought I was the only one! Snapshot creation is also very slow in Retrospect Pro v 7.0 for windows. This can take 30 - 40 minutes on the windozer with ~12 GB of used space. This is with a new backup set without historical baggage. I have an OS 10.4 mac at home that runs v 6.1. That hard drive has 8 times as much data and multiple users with more permissions and so forth than my work computer. Yet snapshots are created much faster - usually in a couple of minutes or so. Again, this is with a new backup set without baggage from system upgrades etc. This is obviously not consistent with the previous post. No reason for it is apparent. Quote Link to comment Share on other sites More sharing options...
rhwalker Posted January 30, 2007 Report Share Posted January 30, 2007 Quote: I have an OS 10.4 mac at home that runs v 6.1. That hard drive has 8 times as much data and multiple users with more permissions and so forth than my work computer. Yet snapshots are created much faster - usually in a couple of minutes or so. Again, this is with a new backup set without baggage from system upgrades etc. This is obviously not consistent with the previous post. Yes it is consistent with the previous post, if you are speaking of my post. On MacOS X "non-Server", ACLs are disabled by default. On Mac OS X Server, they are enabled by default. Watch out if your Mac at home is an Intel Mac - Retrospect crashes if ACLs are turned on unless you have the "special Intel build" of Retrospect that ignores ACLs altogether. Russ Quote Link to comment Share on other sites More sharing options...
bvaandr Posted January 30, 2007 Report Share Posted January 30, 2007 Russ: Thanks for the heads up - so far all I have is a G5 Bas V Quote Link to comment Share on other sites More sharing options...
mauricev Posted February 3, 2007 Report Share Posted February 3, 2007 Quote: Yes it is consistent with the previous post, if you are speaking of my post. On MacOS X "non-Server", ACLs are disabled by default. So if they are disabled, then it should be very fast, but your post was explaining why are they are not fast... Quote: That might be because of the ACL support that had to be added for Mac OS 10.4 I'm not sure why this user is seeing it so fast. For me, it's very slow for all clients, Mac desktop, Linux server, and Windows desktop. And it shouldn't be permission-copying notwithstanding. That I call a bug. Quote Link to comment Share on other sites More sharing options...
lunadesign Posted June 5, 2007 Author Report Share Posted June 5, 2007 As the original poster, I thought I'd report back on a breakthrough after lots of debugging and help from the Retrospect team. For whatever reason a combination of some software that appears to be present on all of my computers *and* Windows' TCP/IP stack's use of delayed TCP/IP ACK's (also known as the Nagle Algorithm) was causing the problem. To read more about delayed TCP/IP ACK's, see http://smallvoid.com/tweak/winnt/network.html and go to section 22. Essentially, the sequence of socket writes and reads that the Retrospect Client does during the "Building Snapshot" phase seems to tickle the worst case scenario of delayed TCP/IP ACK's effectively inserting 250ms delays for each folder (this explains why I was regularly seeing about 4-5 folders processed per second). I unfortunately was not able to identify what piece(s) of software on my PC's was part of the scenario. I did verify that using two systems with fresh Windows XP installs (one as server, one as client) did not exhibit the problem. The workaround is to enable a new registry setting (TcpAckFrequency) on the Retrospect server machine and setting it to 1 (see http://support.microsoft.com/kb/Q328890) for details. With this setting, Retrospect flies through the list of folders it displays in the progress dialog during the "Building Snapshot" phase whereas before it was so slow I could actually read each folder's name. On one test partiion, my "Building Snapshot" phase went from 45 minutes to 7.5 minutes. Woo hoo! Just thought I'd share this discovery with anyone else who's encountering this. I have no idea if this affects non-Windows clients so I can't respond to the Mac issues that were listed in this thread. Quote Link to comment Share on other sites More sharing options...
CurtBrunk Posted June 6, 2007 Report Share Posted June 6, 2007 The link for the microsoft support registry entry doesn't connect. Please repost it. Also, is my laptop the "Retrospect Server machine?" Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.