Jump to content

Backup: Initial file scanning very slow.


Recommended Posts

I backup 5,000,000+ files from a volume across to my fileserver.

 

Up to and including 7.5.324 the initial scan of those files would only take 3/4 minutes, and then it would proceed to "Matching" and "Preparing to execute" etc.

 

Newer patch levels and now the new 7.6 release take much *MUCH* longer to perform that initial scan of the volume being backed up - so much so that I have stayed on 7.5.324 instead of installing the patches as and when they've appeared.

 

I'm talking in the order of 20/25 minutes just to perform that initial filename scan. The rest of the backup proceeds as expected and nothing is slower here. When I returned to build 324 the scan was fast again.

 

Now that 7.6 is out (making the entire 7.5 release a legacy), and this problem is still there I've dragged myself here to ask if this is deliberate (like you're doing more work during that scan since 7.5.324) or a bug?

 

It's a pain because an incremental backup of the volume in question takes double the time it does with build 324.

 

Thanks.

Link to comment
Share on other sites

Retrospect has only been tested up to 4 million files per volume. When you start going way above 4 million then you are more likely to see major performance problems. Make sure you have at least 2 GB of RAM and 15 GB of free disk space on the C: drive of the backup server. I would also not run any other execution units during this scan.

 

Are you saying that the initial scan of 5 million files would only take 4 minutes? I am surprised about this. I would expect it to be between 30 minutes to 45 minutes with all versions of Retrospect.

Link to comment
Share on other sites

I take your point about the 4,000,000 test limit, however Retrospect does work fine with 5,000,000+ files. It's just that initial filename scan that is now slow, while build 324 and earlier it was not.

 

I also have another volume with only about 800,000 files and the file scan is slow as well with later patches (and now with 7.6)

 

So if EMC have tested with 4,000,000 files I'm surprised they haven't seen the difference.

 

When I mention that it takes only 3 or 4 minutes to scan the 5 million files. I don't mean it's begun backing up after that time, just that it's finished scanning the files and entering the next phase. After that it spends plenty of time in the "Matching" and "Preparing to execute" phases etc.

 

So when you say you're surprised, I'm surprised that you're surpised :-)

 

 

Link to comment
Share on other sites

  • 2 weeks later...

There really is a *huge* difference in speed of the initial filename scan between 7.5.324 (along with earlier builds), compared to later builds. And it can't just be explained by the fact that I'm scanning a million more files than you've tested with, for two already stated reasons: 1)A folder structure containing < a million files is also very slow to scan in comparison, and 2) build 7.5.324 (and earlier) is *fine* - even with this humungous number of files.

 

Well, can I ask you (or a customer reading this) to compare the latest versions with 7.5.324.

 

Many thanks.

Link to comment
Share on other sites

OK, I've independently tested this on a volume with just over 300000 files with the following results:

 

7.6.111 took 4 minutes and 44 seconds.

 

7.5.324 took 2 minutes and 7 seconds.

 

This timing is the scanning only, meaning only while the word "scanning" is visible.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...