Jump to content

Recommended Posts



We have three nightly backups running in sequence whose source directories are 1.5TB, 620GB, and 280GB respectively in size. Each directory contains lots of files and there's about 12 million files (not my fault!) in total over the three sets. All directories are stored on our local file server and backed up via the backup server onto one of our offsite file servers (with a very good connection). The network traffic looks like this:

local file server <--> backup server <--> remote file server

As you may expect based on the source size and number of files, the backups take a long time to run and they are now starting to run into the working day which is not ideal.


The backup script uses the following options:

  • media verification
  • data compression
  • match source files against the media set
  • don't add duplicate files to the media set
  • use status modification date when matching (Linux)

I was wondering where Retrospect is likely to slow down more so I can figure out how best to admin the file server - is it the sheer number of files or the size of the files? Are MD5 hashing over the network or the checking for duplicate files the costly operations? Incidentally, adding more detailed profile information to the logs would be a great feature for Retrospect 9 unless it's already been implemented.

Link to comment
Share on other sites

  • 3 weeks later...
  • 4 weeks later...

In case anyone reads this, I did convince one user to move a large amount of files to a weekly backup which brought our daily backups down from about 11 hours to three and a half hours. The files did not consume a large amount of space, relatively, so my guess would be that just reducing the number of files made a big difference in speed.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Create New...