Jump to content

backup performance


pronto

Recommended Posts

Hi Community,

 

I installed the lates version of Retrospect 7.7 on a brand new server with Windows Server 2008 R2 with a brand new SAS attached storage with 12 SATA HDs in a Raid 6, but we have approximately the same speed as we had with the old system. Round about 1000MB/min (+/- 200MB/s). I thought that the network performance is the bottleneck but with using GBit adapters I have round about a load of 25%. The server who become backuped is brand new too and have also the same SAS attached 12 bay storage.

 

Are there some tweaks to increase the performance?

 

Thx & Bye Tom

Link to comment
Share on other sites

We can get between 1000-3000 MB/min using iSCSI storage. It depends a lot on what one has to backup. Matching phase can be a big slowdown.

 

Bottlenecks are/can be your CPU. Retrospect doesn't really utilise multi-core technology that well. So generally speaking higher frequency CPU's will be faster. Using software encryption is a performance hit, as is software encryption.

 

To speed up you can try using 'Media verification' instead of 'Thorough verification'. Remember to set 'Generate MD5 digests during backup operations' in Preferences to use this feature.

Link to comment
Share on other sites

For testing purposes we turned verification totaly off. The CPU of the server is indeed a MultiCore CPU but at this time nothing else is available, isn't it? The processor load during a backup run is moderate at 60 to 70%. If we performe verification we get higher results up to 2000 MB/min but of course this is explainable because the transmission rate is accumulated with the native backup rate and the verfification rate. And of course the verification rate is much higher than the native backup rate. Do you have a native backup rate around 2 to 3 GB/min? You can see the single values in the logfile...

 

Thx & Bye Tom

Link to comment
Share on other sites

I probably should have said something like this: less cores with higher frequency is preferred over more cores at lower frequency. ;) Those CPU's are often cheaper as well.

 

We get highly variable results, depending a lot on how many and how large or small the client's files are. MD5 verification however is always fast. Easily doing even 2000-5000MB/min, sometimes even faster. To get a more stable figure I would need to do a recycle backup.

 

Our OSX clients are faster than our Win7 clients as well. My own machine (OSX) is reliably pushing more than 2000MB/min and it's not even directly connected to our core switch. This is an AES-128 encrypted backup, mind you. As are all my figures. :)

 

This specific Retrospect server uses two hexacore Xeons, but we will move Retrospect to a different machine with only one socket and a Xeon X3470 next week. Because we use encryption for most of our backups we're sure it will be faster at a lower tco. The dual hexacore machine will be used as another HyperV platform.

 

To test your throughput I would create/use a (fast) client and create a backup folder with a few, but large files (ISO's for example). Switch off encryption and compression and see what you get.

 

Btw, the real time backup performance indicator is notoriously inaccurate.

 

There are also other factors, like subspec network switches, network cards and drivers. In other words, your mileage may (and will) vary. :teeth:

Link to comment
Share on other sites

I can exclude the involved hardware (like switches, Raid subsystems, connectivity etc) as the responsible components, because I transfered an ISO image with round about 3GB in size and indeed I never saw a faster transfer between two computers before. Sensed less than a minute and by using SMB as transport protocol which isn't the fastest. Both servers are brand new with big SAS attached storages with plenty hard drives. Okay the Raid Level (6) isn't known as the fastest but modern controllers can easily deal with high bandwith transmissions.

 

But it is really disappointing that the new backup system isn't faster than the old one. We are in a graphical industrie and have to deal with a large amount of data. We consolidate a few file server to avoid parallel running backups but we only defang the current situation. We will run into the same problem pretty soon. If we do not find a way to increase the backup bandwith significantly, we have only two options. Either we setup a second backup server, and if the backup client is the bottleneck, of course a second file server too, or we test another product. It depend on what is cheaper.

 

For now we are safe to perform a full backup in a useable time slot but now we deal online only with almost 2 TB on data but we expect 4 TB in 12 month. 2 TB are okay but 4 TB isn't possible with this bandwith. And I don't think that our plan to scale our coreswitch up to 10 GBit support will help us in the backup question because now we use only round about 30 % of a GBit connection. The things look bleak for our backup...

 

Bye Tom

Link to comment
Share on other sites

I think we are in the same boat. We too have to back up a very large volume each day. Therefor we utilise multiple backup servers and multiple storage targets (iSCSI HDD RAID5 storage). We even go as far as running our backup targets offsite using fiber optic leased lines. And we also utilise a secondary backup platform (non-Retrospect) that creates restorable incremental images of all our servers and workstations each day. To top that off we transfer our most important data offline to LTO4 tape each day as well. We might be paranoid about this all, but our data means a lot to us and our clients.

 

RAID6 would be only 20% slower than RAID5, so I don't think it will be the bottleneck. However client performance surely can be! Especially when backing up large amounts of small files (developer machines).

 

How good is your storage at concurrency? You could try to break up your clients into groups and different storage sets and do two or more simultaneous backups. It sounds like your storage should be up to that task, so why don't you try that? Seems like a waste of time to do everything in a serial fashion.

 

Do you need to do a full backup each day? We use incremental backup each day and groom out everything older than 15 backups. We also use two or even three sets, so we can go back in time about six or more weeks if need be. Using our tapes, we can go back even further if need be.

 

When coping with bandwidth problems you need to adjust you backup strategy. Divide and conquer! :teeth:

Link to comment
Share on other sites

How good is your storage at concurrency? You could try to break up your clients into groups and different storage sets and do two or more simultaneous backups. It sounds like your storage should be up to that task, so why don't you try that? Seems like a waste of time to do everything in a serial fashion.

 

Maybe I missunderstood your intention but do you mean I should break our fileserver backup into more backup sets and run several backup tasks simultaneously? I would never come to this idea but it sounds easy to test and I can imagine that this could tune up the backup bandwidth a bit. Indeed...

 

Do you need to do a full backup each day? We use incremental backup each day and groom out everything older than 15 backups. We also use two or even three sets, so we can go back in time about six or more weeks if need be. Using our tapes, we can go back even further if need be.

 

No we don't run a full backup each day. We run a full backup (or a recycling backup) only on Sunday and from Monday to Friday we run a incremental backup (or a normal backup in Retrospects terminology). On Saturday we copy the complete backup set (including the full backup and all incrementals) on tape which is rotating in a six week cycle except server with databases, they only have a three week cycle because nobody really needs a six week old database. This is only a desaster recovery scenario but the fileserver backup become tested by my Users each week more than one time. Apple+S at the wrong time in the wrong file or corrupted Quark files are prominent examples...

 

Thx & Bye Tom

 

Link to comment
Share on other sites

Indeed, breaking up into parallel backups could be your solution. Your enemy will be fragmentation, but you can avoid that by creating separate storage targets on your SAN. And if you don't groom, fragmentation will be a lesser issue I suppose.

 

Our (main) backup server also has an aggregated link to the core switch. This can improve performance a bit too. Remember the bottleneck will be the clients.

 

For all our OSX clients we also have a separate Time Machine storage target. Actually that is located offsite through the fiber optic line. That way we have a complete redundant storage solution apart form Retrospect. Very nice for the sysadmin as well, because users can restore their own files if need be, and a backup interval of 1 hour. Quite cheap to set up. We like our eggs in many baskets! :teeth:

Link to comment
Share on other sites

Indeed, breaking up into parallel backups could be your solution. Your enemy will be fragmentation, but you can avoid that by creating separate storage targets on your SAN. And if you don't groom, fragmentation will be a lesser issue I suppose[...]

 

Okay I will check this out, thanks for this idea. I will report the results as soon as they are available...

 

Bye Tom

 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...