51ae86ee-d35a-4c46-b879-c3b8399377a3 Posted December 30, 2011 Report Share Posted December 30, 2011 I am trying to set up a monthly backup of a file server (about 11TB of data, and 1.1 million files). The file server is an Xserve Quad-core 2.6GHz with 6GB RAM, while the Retrospect Server machine is an Xserve Quad-core 28GHz machine with 10GB RAM. The initial scan appeared to take too long, so I cancelled and wanted to try using the share as a source instead. Unfortunately the Console locked up when trying to connect to the server-have not had a chance to go in and troubleshoot that aspect yet. But some notes: the initial scan had gotten to about the 6TB point and was scanning about 20 files per second at that point. But on a subsequent scan it got to the 5TB/400k files mark in about 5 minutes (and then slowed down dramatically, though not to the level seen initially (around 60-150 files per second here). The Server seems to hang around at 300-350% CPU usage, and relatively small amounts of RAM used (I see the two Retrospect Processes using about 280MB real/470MB virtual). The network usage is low (about 1Mb/s usage). So it appears that the CPU on the server is the most limiting factor. Some hypothetical questions: - Could the Client software handle the scanning portion so that the server wasn't bogged down, particularly when more than one event was running on a machine? - Could the Client software run a scan after installation to get a list of the files and their state for the initial backup so that list could be sent to the server rather than the server doing it? - Could the Client software use fsevents on the Mac side to maintain a running list of updated files to pass off to the server when a new backup was initiated? This would practically eliminate the need for the scan. Or does some of this happen already? A little insight into how this is working might give us some ideas on how to optimize performance. Quote Link to comment Share on other sites More sharing options...
Guest Steve Maser Posted December 31, 2011 Report Share Posted December 31, 2011 Your hypothetical questions would (I believe) require a rewrite of the software. The current client software does no client-side processing like you would like it to. What OS is on file server were you backing up? Quote Link to comment Share on other sites More sharing options...
779667a8-72b4-4497-8b3a-b920e4a2b806 Posted January 1, 2012 Report Share Posted January 1, 2012 (edited) A little insight into how this is working might give us some ideas on how to optimize performance. Here's a thought that may work. If you're happy for the designated Client computer (Computer A) to be bogged down instead of the Server computer (Computer B ), why not install Retrospect Server on Computer A for the initial seeding backup. Use this to backup to your external tape/hard/other drive, then uninstall Retrospect Server from Computer A. Once this has been done, move the backup drive to Computer B, along with the catalog files, then install Retrospect Server on Computer B. Subsequent backups will still need to scan Computer A fully, so there's no shortcutting the scanning process unfortunately. Edited January 1, 2012 by bcarpenter Quote Link to comment Share on other sites More sharing options...
51ae86ee-d35a-4c46-b879-c3b8399377a3 Posted January 4, 2012 Author Report Share Posted January 4, 2012 Your hypothetical questions would (I believe) require a rewrite of the software. The current client software does no client-side processing like you would like it to. What OS is on file server were you backing up? All machines are 10.6.8. I started on Friday, and so far we are 121 hours in, 8.2TB backed up (2.4TB remaining, and 443k thousand files backed up with 651K files remaining. Then we start the compare... Quote Link to comment Share on other sites More sharing options...
51ae86ee-d35a-4c46-b879-c3b8399377a3 Posted January 4, 2012 Author Report Share Posted January 4, 2012 Here's a thought that may work. If you're happy for the designated Client computer (Computer A) to be bogged down instead of the Server computer (Computer B ), why not install Retrospect Server on Computer A for the initial seeding backup. Use this to backup to your external tape/hard/other drive, then uninstall Retrospect Server from Computer A. Once this has been done, move the backup drive to Computer B, along with the catalog files, then install Retrospect Server on Computer B. Subsequent backups will still need to scan Computer A fully, so there's no shortcutting the scanning process unfortunately. Unfortunately with the amount of data to copy that isn't an option as the tape library can't be connected (just don't have free Fiber Channel ports. So I'll probably just update this set monthly, and start a new set next year. But having to do a full scan every time is just such a waste of time when there is an OS level function that can track when a file has been touched. Given that 8 was a rewrite, and 9 a further upgrade, this type of speedup would be greatly appreciated. It would also let us backup far more machines in a given time slot since that list would greatly reduce the need for a scan- for some clients that can be the vast majority of the backup time (by 10-100 to 1). Maybe for Retrospect X (chuckle, chuckle). Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.