Jump to content

How do I speed up Retrospect backups?


crisw
 Share

Recommended Posts

We are a private school. We had been running Retrospect 6 for years. We migrated to 8 this winter. We are running it on the same type of machine we run 6 on- an original, non-Intel 1.33 GHz Xserve with 2 GB of RAM. The network is exactly the same; we didn't change anything.

 

Restrospect 8 runs so slowly that it is unusable. A client with 1 GB of data takes hours to back up. Looking over the console, backup speeds range from to 409 KB to 16.4 MB per minute. In contrast, our still-running Retrospect 6 server averages 300-600 MB/min- with an identical server, running on the same network.

 

Any ideas on why it runs so slowly, and what we can do to speed it up?

Link to comment
Share on other sites

It will never run fast on a PPC machine, and you've got a very slow PPC xServe at that.

 

The design decision was made to have the backup file format be "little endian" (the byte order for Intel machines) rather than "big endian" (the byte order for PPC machines) so that Retrospect for Windows would be able to have a common backup file format as the Mac version, for exchanging backup sets between the two, and to exchange data over the network in "little endian" format, too.

 

As a result, the PPC version of Retrospect spends all of its time doing byte-swapping of the data structures.

 

To speed it up, you need to get a faster computer running on the Intel architecture.

 

Sorry, but that's just the way it is.

 

Russ

Link to comment
Share on other sites

Running on high end Intel boxes won't be much faster either I'm afraid. Until there is an update to Retrospect, it won't matter what hardware you use. Network backups are just nowhere near the speed that they were under Retrospect 6. Hopefully they figure out how to fix this and it's not just "the way it is." Unfortunately it's been this slow for quite a long time now and there's been no indication that Retrospect considers this to be an issue. I do hope I'm very wrong about that though.

Link to comment
Share on other sites

I'd be very interested in getting more opinions on this. We can move to an Intel box, but if it doesn't give a significant speed upgrade, it won't be worth it for us.

 

Are any users getting decent backup speeds, at least comparable to version 6? If so, what is your setup?

Link to comment
Share on other sites

I'd be very interested in getting more opinions on this. We can move to an Intel box, but if it doesn't give a significant speed upgrade, it won't be worth it for us.

See the EMC Retrospect 8.1 Read Me about this:

 

Known issues with this release

 

Retrospect 8.1 performs slowly on PowerPC-based Macs. For the best performance, EMC strongly recommends that users upgrade their Retrospect servers to Intel-based Macs. For those who cannot upgrade at this time, EMC software engineers are continuing to improve Retrospect 8’s performance on all processor types, and performance boosts are likely in a future release.

Are any users getting decent backup speeds, at least comparable to version 6? If so, what is your setup?

Ah, that's a very different question. Retrospect 8 on an Intel Mac, when it doesn't crash, can use multiple CPU execution units (cores) to achieve better results, and also runs native Intel code on the Cocoa API (rather than emulated PPC code running under Rosetta using the Carbon API).

 

Retrospect 6 will always outperform Retrospect 8 on a PPC, but Retrospect 6 is unsupported for use with Snow Leopard (10.6.x).

 

And yes, there seems to be a performance issue with Retrospect networked clients under Retrospect 8, regardless of the platform.

 

Russ

Link to comment
Share on other sites

No matter what the hardware, Retrospect 8 is painfully slow for networked clients.

 

We have top of the range latest xServe with gigabit network, 12GB RAM, LTO-4 tape and latest Mac Pro clients all running latest software.

 

Backup speeds are about a tenth of Retrospect 6 - so much so R8 is useless for client backups for us, so we're continuing using R6 for this.

 

We're about to upgrade our xServe to 10.6 server, so currently looking at other backup solutions as R8 won't cut it for us anymore with the slow client backups.

 

Christiaan

Link to comment
Share on other sites

Thanks, I've just tried this, but it made no difference to speed - but as I said, these are the latest xServe / Mac Pro clients. May help on slower ones...?

 

Any idea when this is likely to be fixed? Last lot of dialogue was hinting around October last year for an update to the speed issues...?

 

http://forums.dantz.com/showtopic.php?tid/31016/

 

Christiaan

 

Link to comment
Share on other sites

I'm running the Retrospect server on a 2.66 GHz quad core Mac Pro with 6 GB of RAM. In my experience, it's the performance of the Retrospect client software that severely constrains network backup performance, not the Retrospect server engine.

 

As an alternative to using the client software on my Macs, I instead created AFP share points on them to be backed up (which can be done on any Mac OS X machine). Then I made those share points my backup sources on the Retrospect server. Performance went from about 450 MB/min. over a gigabit network to an average of 2.5 GB/min. Turning on deduplication (the "don't add duplicate files to media set" option) reduced the performance to around 2.2 GB/min, but that's acceptable to me. My backups are now significantly faster than they were using an LTO-3 library with BRU Server 1.2.

 

Note that my backup source drives are fiber channel RAID 5 volumes and my destination media sets are made up of individual non-RAID eSATA hard drive volumes. I think the performance bottleneck now is actually the eSATA drives, not Retrospect.

 

I just can't understand why the Intel-native Retrospect client software is so horribly slow.

Link to comment
Share on other sites

As an alternative to using the client software on my Macs, I instead created AFP share points on them to be backed up

 

The trade-off for your speed increase is a loss of inode and ACL information preservation.

 

I don't know what permissions files are given when retrospectengine (running as root) mounts a volume (using some specific non-root login/password credentials) and then writes files to the Media Set's Member, but they're not guaranteed to be what they were on the Source volume. Using the client software _does_ (or at least is supposed to; it's another thing I haven't tested) preserve accurate permission information for files.

 

Dave

Link to comment
Share on other sites

I don't know what permissions files are given when retrospectengine (running as root) mounts a volume (using some specific non-root login/password credentials) and then writes files to the Media Set's Member, but they're not guaranteed to be what they were on the Source volume. Using the client software _does_ (or at least is supposed to; it's another thing I haven't tested) preserve accurate permission information for files.

Backup Bouncer

 

Russ

Link to comment
Share on other sites

I willing to live with the lack of ACLs for now, as my backups are actually D2D2D. The first D2D backup is a nightly clone onto a another volume on the server using Data Backup 3.0, which works quite well. The Retrospect backup then pulls data off that clone, not the actual production volume. So, I always have a full copy of my server volume with ACLs intact that's less than 24 hours old. If I have to restore anything from the Retrospect backups, I can rebuild the permissions.

Link to comment
Share on other sites

Running on high end Intel boxes won't be much faster either I'm afraid. Until there is an update to Retrospect, it won't matter what hardware you use.

 

I respectfully disagree. I do have my issues with Retrospect 8, but actual file transfer performance is not one of them. With my iMac clients on 100baseT I am getting performance in the 350-450MB/m range, with clients on gigabit, I am seeing upwards of 1.2-1.4GB/m.

I am using an intel xserve with overland lto-4 autoloader.

 

2a56hd.png

Link to comment
Share on other sites

really? Is this using the client software? Or mounting the clients as shares? Compression or not?

 

(There's a big speed difference in the two -- and if you have software compression turned on...)

 

 

I just looked through my past activities sorted by "performance" -- while these are all incremental backups at this point (and verification is off), *none* of my clients (10.6 or Windows7) go above 273M/min (that's the max).

 

And this is with everybody on Gig Ethernet...

Link to comment
Share on other sites

  • 4 weeks later...

I too would love to know how scsc tech is getting those speeds with network backups. This is driving me crazy ( well one of many issues with Retrospect) and is something I need to get fixed. Backing up TB's of data with Retrospect 8 just doesn't work unless I'm OK with it taking an insanely long time. Is there anyone else who is actually getting good performance from network clients?

Link to comment
Share on other sites

Just a few more little 'tricks' to speed up your backup process. In these cases you are sacrificing some things for speed.

 

1. Make verification none. This will remove the step of checking the backup all over again.

2. Turn off MD5 digest in the admin preferences. This MUST be done prior to making media sets. Any existing media sets will need to be re-created. (This info is directly from tech support)

3. Turn off link encryption and WOL.

 

This will pretty much give you a barebones 'backup only' system and this will be about as fast as you can get it.

Link to comment
Share on other sites

 

2. Turn off MD5 digest in the admin preferences. This MUST be done prior to making media sets. Any existing media sets will need to be re-created. (This info is directly from tech support)

 

.

 

 

Really.

 

I'll have to test that one out. First I've heard of that one...

 

 

(Turning off software compression makes things go faster, too.)

Link to comment
Share on other sites

I might rerun this because the results seemed odd to me...

 

So I did the following with the following results.

 

This was backing up 65K files/14G of data from my "users" directory as a client over Gigabit Ethernet. Client was not "at rest" -- I was doing normal work stuff while the backups below ran...

 

Made a media set with MD5 on and ran backup with MD5 on (the default): 327M/m

 

Made a media set with MD5 off and ran backup with MD5 off:

355M/m (which would seem to validate the suggestion)

 

Recycled the MD5-on media set and ran with MD5 *off*: 403 M/m (even faster -- but which would imply that tech supports suggestion isn't quite correct as this was faster...)

 

Recycled the MD5-off media set and ran with MD5 *on* -- 350M/min

 

So, having MD5 off (at various points) seems to have a positive effect, but it's only around a 10% difference.

 

Link to comment
Share on other sites

Are you turning off MD5 in the admin console preferences? If so, according to the tech, you cant just toggle it on & off. If its on when you create the media set, that media set will ALWAYS create MD5 checksums even if turn it off afterward. You would have to delete the media set, turn off MD5 (restart the engine just to be safe) then recreate the media set.

 

You may try this procedure if you are in testing mode.

Edited by Guest
Link to comment
Share on other sites

So for what I did above, then I had two sets -- one created with MD5 on and the other off.

 

So the *fastest* speed I got -- 403M/m -- was with the one with MD5 turned *on* when I created the set.

 

 

Go figure.

 

 

(Admittedly, I did *not* restart the engine between any of these tests. No idea if that would make a difference or not.

Link to comment
Share on other sites

Go figure.

It might depend on what the true set of preferences was. Retrospect 8 has some problems saving preferences, so it might be a coin toss what the true MD5 setting was unless you created new media sets for each test.

 

There's also the issue of how fragmented your media set drive is, because the media set might be scattered about.

 

Russ

Link to comment
Share on other sites

Retrospect 8 has some problems saving preferences, so it might be a coin toss what the true MD5 setting was unless you created new media sets for each test.

 

There's also the issue of how fragmented your media set drive is, because the media set might be scattered about.

 

Russ

 

 

True -- the media set drive had a lot of free space on it, so it shouldn't have been that fragmented.

 

But, yes, a true test would have been to stop the engine and reboot the engine computer between backups, etc. Maybe I'll do that tomorrow if I can sneak this in between other stuff.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

×
×
  • Create New...