Very slow backup over ethernet

[DuncanCMiller]

Just switched my Retrospect backup machine to a new Intel iMac, using DVD-R. Backing up client machines via ethernet, I'm seeing very low copy speeds (like 6 MB/min). Using the same setup with an iMac G5, I usually saw 10-20 MB/min. Details: System 10.5.5, Retrospect 6.1.230, driver 6.1.15.101, clients ranging from 6.0.109 to 6.2.229. Any thoughts about why so slow? Does the client version have any impact on copy speed, as long as it's compatible and working? Any suggestions for speeding this up? I've been running 36 hours now and it's just finishing the backup of the first 27GB volume.

 

I know it's comparing apples to oranges, but when I backed up one machine to an external drive using SuperDuper, I was seeing 10 MB/sec (that's per second, not per minute -- a 60:1 difference). Why is Retrospect so much slower? Is it the ethernet connection? Buffering data for the DVD?

 

 

[rhwalker]

How much memory on the new Intel iMac? You could be thrashing.

 

Also, no information on your networking infrastructure, or the speed of the negotiated connection on your iMac (10/100/1000 Mbps?)

 

Check in those directions (RAM, network speed).

 

Russ

[DuncanCMiller]

2 GB RAM.

 

I checked the various hubs in the network. I did find one 10BaseT, which I'll replace with a 100mbps hub and see if that makes a difference. Thanks.

[DuncanCMiller]

... one more question. How would I check on the negotiated speed of the net?

 

[rhwalker]

there are at least two ways:

 

(1) in terminal,

ifconfig en0

 

(see man ifconfig for details)

 

(2) Network Utility

(/Applications/Utilities/Network Utility)

Info tab, select interface for en0

 

Russ

[rhwalker]

2 GB RAM.

Ok, that's a little low for Retrospect on an Intel Mac. Currently, Retrospect on Intel Mac runs emulated under Rosetta, so there's overhead there, and Retrospect itself needs lots of memory, too. I'd suggest at least 3 GB for what you are trying to do.

 

Russ

[CallMeDave]

Just switched my Retrospect backup machine to a new Intel iMac, using DVD-R.

Backing up client machines via ethernet, I'm seeing very low copy speeds (like 6 MB/min).

Using the same setup with an iMac G5, I usually saw 10-20 MB/min.

 

I know it's comparing apples to oranges, but when I backed up one machine to an external drive using SuperDuper, I was seeing 10 MB/sec (that's per second, not per minute -- a 60:1 difference). Why is Retrospect so much slower?

 

So far, you are reporting three entirely different hardware configurations, with two completely different hardware technologies. So it's not at all surprising that you would have three different speed results.

 

- What happens when you use the same make/model/driver/firmware ComboDrive or SuperDrive that's in the iMac G5 in the new iMac Core2Duo?

- What is the make/model/driver/firmware of the drive in the Core2Duo iMac that is working slowly?

- What type of blank CD/DVD media are you using?

- What happens when you use a File Backup Set stored on the same "external drive" as you used with SuperDuper (which does not do backups in the same sense that Retrospect does)?

 

In general, it would be expected for a CD/DVD Backup Set to perform more slowly then other Types of Backup Sets. And different optical burners behave differently. More information about the new hardware you are using would help others to help you.

 

Dave

[rhwalker]

I know it's comparing apples to oranges, but when I backed up one machine to an external drive using SuperDuper, I was seeing 10 MB/sec (that's per second, not per minute -- a 60:1 difference). Why is Retrospect so much slower?

Note also that Retrospect's optical media backups, so that they can be appended, use a different set of commands (packet writing) vs. the commands used by other backup programs that create a "closed session" CD that cannot be appended.

 

See:

http://kb.dantz.com/article.asp?article=5245&p=2

and

http://kb.dantz.com/article.asp?article=5496&p=2

 

Russ

[DuncanCMiller]

Appreciate the help, guys. The Intel iMac was reporting a 1Gbps link, but the machine being backed up so slowly reported only 10Mbps. So I just swapped out the old hub and replaced it with

a 10/100BaseT. Retrospect just started backing up the second drive on the list -- we'll see if this goes faster.

 

The DVD burner is a Matshita DVD-R UJ-875. I'm using Verbatim 16x DVD-R disks.

 

I certainly don't expect Retrospect to copy to DVD at anywhere near the speed that SuperDuper can copy to an external disk via USB 2.0. But one could hope for a 20:1 difference, rather than 60:1.

[rhwalker]

The DVD burner is a Matshita DVD-R UJ-875. I'm using Verbatim 16x DVD-R disks.

(1) note that this burner is supported as of RDU 6.1.14.101

Mac RDU version history

If you ever did a "custom configure" of the drive, you will have a .rdi file in /Library/Preferences/Retrospect that will give poorer results than the hand-crafted preferences in the RDU. If you have such a .rdi file, try moving it to the Desktop (so Retrospect won't find it, and so you can put it back if things go badly) with Retrospect quit, then relaunch Retrospect and retry using the provided parameters in the RDU.

 

(2) Have you tried the same test with +R media?

 

Russ

[CallMeDave]

Backing up client machines via ethernet, I'm seeing very low copy speeds (like 6 MB/min).

 

the machine being backed up so slowly reported only 10Mbps

 

So your speed issue is with only a single client, where other clients used as Sources in the same Backup Set run faster?

 

Seems like a pretty critical bit of data to omit from the original post...

[DuncanCMiller]

I've never attempted a custom configure, so it's very unlikely that I have any random .rdi files.

 

This is my first backup with the new setup. I reordered the list of clients being backed up in such a way that the slow connection happened to come first. Perhaps I didn't notice this problem before because most of the common files had already been backed up before I got to that client.

 

I will try DVD+R after I finish this backup and see how it compares, with all other things being equal. Is there any problem with mixing DVD +R and -R members within the same backup set?

 

I have one desktop machine in a room with no ethernet. I've been backing that up via 802.11g. Perhaps I'll carry it down near my new gigabit ethernet hub and plug it in there, at least for the first backup, which is of course the one that takes lots of time.

[DuncanCMiller]

Update: just backed up an iBook connected to the same Time Capsule the Intel iMac is connected to (both via ethernet). Network Utility on the iMac reports a 1Gbps connection, and the iBook reports 100Mbps. Backup throughput was only slightly higher -- a little over 7 MB/min. So it doesn't look like the network is the limiting factor here. Activity Monitor shows that the CPU spends 99% of its cycles on Retrospect, falling back a bit during or after each DVD write. I have no other applications running.

 

I'm very disappointed by the performance which, as I said, is actually worse than it was on my old iMac G5. How sure are you that increasing RAM above 2 Gig will make a meaningful difference? This would mean pulling out one or both of the brand new 1 Gig SO-DIMMs to do an upgrade.

[rhwalker]

I'm very disappointed by the performance which, as I said, is actually worse than it was on my old iMac G5.

Ok, one way to narrow things down and see whether it is the optical medial drive or something else would be to put the iMac G5's burner into firewire target disk mode, use the Intel Mac to back up to the iMac G5's burner.

 

How sure are you that increasing RAM above 2 Gig will make a meaningful difference? This would mean pulling out one or both of the brand new 1 Gig SO-DIMMs to do an upgrade.

You could test that yourself by using Activity Monitor to show the memory usage.

 

Russ

[DuncanCMiller]

Yes, I've been watching Activity Monitor. It's been showing about 1.1G of memory active, 350M inactive, 200 wired (whatever that means) and 350M free. "Swap used" is minimal, as is "page outs." This suggests to me that the problem isn't RAM, nor (per previous post) is it the local net. I'm backing up the final client now. Then I plan to run a backup from the hard drive on the local host to see how much faster that goes. Previously, I would see 30-60 MB/min for this. The results should indicate whether there's a problem with the DVD burner. I will also try burning to DVD+R, as you suggested, to see what difference that makes.

 

[rhwalker]

wired (whatever that means)

Um, the Help menu for Activity Monitor is instructive (see the "Checking system memory usage" topic).

 

See also:

Memory Management in OS X

(scroll down to "Wired Memory").

 

Russ

[DuncanCMiller]

OK, here are some serious test results, using three different machines.

 

(1) iMac G5, 1.8GHz, 1.25G RAM, System 10.4.11, Matshita UJ-825

(2) MacBook Pro, 1.83GHz, 2G RAM, System 10.4.11, Matshita UJ-857

(3) iMac Intel, 2.66GHz, 2G RAM, System 10.5.5 Matshita UJ-875

 

All are running Retrospect 6.1.230, Driver 6.1.15.101, Device Access 1.0.107. All are conducting immediate backup from local Macintosh HD, verification off. All machines show ample free memory.

 

Results for the first full DVD-R:

(1) 81.8 MB/min

(2) 115.7 MB/min

(3) 10.5 MB/m

 

Besides the speed differences, the main thing I observe is that (1) and (2) typically cycle between reading and writing, spending 10-20 seconds "Copying Macintosh HD" and about 30 seconds "Writing to disk ...". (3) seems to spend all of its time "Copying Macintosh HD" and never reports that it's writing to disk. However, the Completed Files total goes up every few seconds, and the names of the files being read come in bursts of a few seconds each, followed by pauses of a few seconds.

 

It sure seems like Leopard is doing something quite different than the two machines running Tiger. Any further thoughts on what to do or test?

[CallMeDave]

It sure seems like Leopard is doing something quite different than the two machines running Tiger.

The tests you have performed are not adequate to make such a pronouncement.

 

IF you were to run the same tests with all machines on the same OS, or IF you were to run the same tests with all machines using the same OEM CD/DVD burner, THEN you could extrapolate the results into something meaningful in regards to the potential cause of the speed differences.

 

As it stands, there are too many variables to know what's going on.

 

Dave

[DuncanCMiller]

It appears that whatever tests I do are "inadequate." I'm looking for helpful suggestions.

 

I've eliminated as many variables as I can. I'm using the same version of Retrospect and its drivers, and the stock Apple SuperDrive in all three machines, from the same manufacturer. I've removed the network as an issue. The remaining variables are the Mac OS version and the processor type. The results are

 

(1) PPC processor, Tiger OS: OK

(2) Dual Intel processor, Tiger OS: a bit faster

(3) Dual Intel processor, Leopard OS: really slow

 

So it seems to me that this is either something related to how Leopard interacts with Retrospect, or else I have a bad SuperDrive. But the SuperDrive seems OK otherwise -- I can burn a DVD full of files in about 10 minutes. So that leaves me with the question of whether the Retrospect version I'm using is really fully compatible with Leopard or not. I'm not about to upgrade one of my other machines to Leopard without knowing the answer to this.

 

 

 

[DuncanCMiller]

Just to wrap this up ... I finally did the test with DVD+R media instead of DVD-R disks. It made a big difference -- 150 to 200MB/min over ethernet, about 15MB/min from an 802.11b client, and about 30MB/min from an 802.11g client.

 

The EMC database says that DVD-R is not supported for the Matshita (Panasonic) SuperDrives. Thanks to dhwalker and twickland for pointing this out.

[rhwalker]

The EMC database says that DVD-R is not supported for the Matshita (Panasonic) SuperDrives.

I really think that this non-support of DVD-R may be due to a bug introduced in RDU 6.1.10.100 and not fully fixed in RDU 6.1.14.101. See the release notes:

Macintosh RDU version history

 

We only use tape and autoloader, so it's no big deal for us.

 

Russ

[ttigner]

Painful backups

 

I am very discouraged with the speed of Retrospect as well.

 

Our whole family has moved to Mac's in the past year and we need a backup solution. I was going to get a Time Capsule, but I already have a router, and thought that I could get better performance from a highly rated NAS like Netgear ReadyNAS Duo. I wasn't stoked with the bundled NTI Shadow software and researched to find Retrospect a professional, fully featured tool.

 

I have a brand new Macbook Air w 2Gig of memory and a DLink DIR655 Megabit router. (The MBA's USB/Ethernet adapter tops out at 100mpbs.)

 

But in actual backups to the NAS I am getting only 26MB/min. With 38GB of data material to backup that will take 24 hours.

 

Is this speed a function of Retrospect or the limitation of the network. When I backup to a Backup Set without specifying a location (which I assume is then somewhere on the local drive) it goes up to 288MB/min.

 

Questions:

 

Should I dedicate the 24 hours to the backup and then do incrementals? (Will have to figure out how to do that.) Is this reliable on a non windows NAS with Retrospect?

 

How do I avoid the file time stamping issue I've seen talked about in the forum? Does this cross OS time stamping issue render using Retrospect with a non windows OS NAS basically useless?

 

This shouldn't be this hard. Perhaps I should have just gotten the Time Capsule - but now I'm $700 into the NAS and $120 for Retrospect. Yikes!

 

 

 

[DuncanCMiller]

Responding to Terry ...

 

One by-product of my troubleshooting over the past week is that I've gotten a lot of throughput measurements. 200-250 MB/min seems typical for backup over an internal bus, firewire, etc. Retrospect does almost as well over 100 Mbps ethernet. For wireless, I've found that my wife's old 802.11b iBook can only manage about 15 MB/min, while a newer machine can sustain about 30 MB/min using 802.11g. Your reported throughput of 26MB/min is consistent with 802.11g.

 

What you need to remember is that Retrospect only needs to handle your full 38G the first time. After that, it should be doing incremental backups of only those files that have changed. What I'm doing now is making sure my various Macs are all hooked up via ethernet for their initial backup. After that, I can do incremental backups wirelessly with acceptable elapsed times.

 

If you can connect a Cat 5/6 ethernet cable from your MacBook Air to your router for the initial backup, you should see an almost tenfold speed increase and be done in two or three hours. After that, incremental backups at 26 MB/min should take only a few minutes each.

 

I also got a Time Capsule for our newest machine (again connected via ethernet). But the Time Capsule has no capability to avoid file duplications if you're backing up multiple machines -- and you're going to fill up your drive very quickly if you try to do that. This is why I've been using Retrospect for Workgroups (or whatever they call it now) for years. I burn all the backups onto DVDs, and I can easily recreate the state of any of our family's computers as they were on a given Saturday for at least the last five years.

 

Just remember to backup your Retrospect catalog file to at least two disks, and install Retrospect on at least two computers. If you do so, you can find a lost file in minutes, and recover the contents of deceased hard drive in no more than a few hours. If you don't, you may have to go through the process of rebuilding your catalog file from a big stack of DVDs, which can be extremely time-consuming (many hours, or even days). Been there, done that.

[rhwalker]

Is this speed a function of Retrospect or the limitation of the network. When I backup to a Backup Set without specifying a location (which I assume is then somewhere on the local drive) it goes up to 288MB/min.

You haven't provided enough test data and information to answer that, but it looks like the problem might be the NAS, not the network.

 

How do I avoid the file time stamping issue I've seen talked about in the forum? Does this cross OS time stamping issue render using Retrospect with a non windows OS NAS basically useless?

Unclear what you are asking here. There is no "time stamping issue" that I have seen in over 15 years of using Retrospect. I have seen instances posted in these forums where people were using Retrospect's "duplicate" function (i.e., copy), and where the destination filesystem did not support the time stamp granularity or metadata of the source, causing incrementals to be more than expected because, rightfully, the destination was not the same as the source before the duplicate. But you seem to be asking about "backup", which is not duplicate.

 

In order to support backups, the destination needs to have certain networking and OS support for the transfers. You have not indicated how the destination is being accessed/mounted (SMB? version? AFP? version?) on the machine running Retrospect. Your problem seems to have little to do with the original poster's problem, which was an issue with DVD media speed.

 

Russ

 

[CallMeDave]

Is this speed a function of Retrospect or the limitation of the network.

 

Sounds to be the former.

 

When I backup to a Backup Set without specifying a location (which I assume is then somewhere on the local drive) it goes up to 288MB/min.

Impossible to do. When you make a New Backup Set, you are presented with a standard Macintosh file dialog, allowing you to specify the location at which the Backup Set will be saved. This is no different from writing a new Word document; when you name it, you also decide where to put it.

 

Should I dedicate the 24 hours to the backup and then do incrementals?

Yes.

 

(Will have to figure out how to do that.)

It's the default way Retrospect works. Files are Matched against what's already in the Backup Set. If a file is new or changed, it is added in; if a file is already there and has not been changed, it is not copied again. Easy peasy.

 

Is this reliable on a non windows NAS with Retrospect?

 

Yes, as long as the file system on the NAS does not have any single-file size limitations (some do). You also don't list the version of Retrospect you are using, and/or the version of the RDU that is loading with the program. That information is written to the Operations Log each time Retrospect launches.