Jump to content
Sign in to follow this  
dittberner@dbr3.de

How fast are your network backups?

Recommended Posts

Hello,

we are testing the new Retrospect 14.5 on mac as new server backup solution. 
Our plan is to use Retrospect Multiserver on MacPro with 10.12.6 and 10GBit network connection to backup data from Hyper-V Windows 2012r2 virtual machines which are also connected via 10GBit ethernet.
Data and backup sets are on FC Raids (8GBit). If we connect to vm from MacPro via Windows smb share, we get a speed up to 700MByte per second read and write.
 
Our first tests with Retrospect results in network speeds of max 4GB/min (backup and copy) which seems to be much slower than 700MB/s.
 
Do you reach faster network speeds? Did we something wrong?
 
Have a nice day
Kai
 

Share this post


Link to post
Share on other sites

2.5 years of experience using Retrospect Mac on my home LAN shows that the main factor limiting speed of networks is the traversal of multiple files in the file system by the client computer.  This morning I got a speed of 400MB/min doing a Normal backup of my MacBook Pro client, but yesterday I got a speed of 600MB/min doing the same Normal backup—but of somewhat different files.  Last Saturday I did a Recycle backup of the same MacBook Pro, and got a speed (for 36GB over 3 hours) of 200MB/min; my Recycle backups are of everything including cache files.  All these speeds are for backup only; I've omitted the speeds of compares.

 

About 6 months ago I had to replace the hard disk drive on my MacBook Pro with an SSD; the backup speeds did not appreciably increase.  I therefore conclude that the controlling factor in Retrospect traversal of multiple files is the speed of the client computer's processor(s).  It's possible that the speed-limiting file traversal is on the "backup server" side (but see the P.S. below for why I think that unlikely), but my "backup server" for the last 2.5 years has been a 6-processor 2.5 GHz 2010 Mac Pro—with the same USB3 portable drives for my Media Sets.

 

As for LAN speed, my limiting factor is the built-into-the-wall inter-room RG-59  cable between my MacBook Pro and my Mac Pro, which has Actiontec MoCA adapters on each end.  The MoCA adapters I have are limited to 175Mbps "effective raw" = about 1050MB/min "effective cooked" (I assume 10 bits/byte to allow for 20% TCP/IP overhead).  That's obviously a lot faster than the backup speeds I'm getting, which never seem to go much above 600MB/min on an instantaneous basis.  The Cat5 cable between my bedroom MoCA adapter and my Mac Pro might in fact be limited to 100Mbps by the fact that it was attached around 3 sides of the bedroom wall in 1998 by a local contractor using—fairly carefully—metal brads he had brought with him (he didn't know he was going to have to string Cat5 cable along walls), but I'm not about to buy and string new Cat5e cable (especially since on one occasion I did a bypass by running a Cat5 cable directly from the MoCA adapter to the Mac Pro, and it didn't speed up the backup).  I could buy replacement gigabit MoCA adapters for a total of US$150, but I'm reluctant to spend the money since I'm not sure they would make much of a difference.

 

dittberner may want to read this 2013 thread, in which post #2 by the very-knowledgeable Lennart Thelander shows speed results rather similar to mine.

 

P.S.: I knew there was a further reason that I ruled out file traversal on the "backup server" as the speed-limiting factor, and I have now remembered what it was.  Back in June 2015, before I spent a substantial amount of money upgrading my (inherited from a dead friend) Mac Pro's hardware and software to be my "backup server", I ran a feasibility test using the hardware and software I already owned.  I ran a Recycle backup of my MacBook Pro using my ex-wife's 733MHz Digital Audio G4 as the "backup server", SCSI-cabled to my old DAT tape drive for a Backup Set (the old term for Media Set), and running my old copy of Retrospect Mac 6.1.  The backup part of the test ran at 170MB/min, which is almost as fast as the 200MB/min in last Saturday's Recycle backup of my MacBook Pro—which used a 2.5GHz Mac Pro as the "backup server" hardware and Retrospect Mac 14.0 as the "backup server" software.  The test was run over the same LAN I have now, except that my Ethernet switches were then only 100Mbps instead of 1gigabit/sec.

Edited by DavidHertzberg
Added parenthetical ref. to P.S. in 2nd prgf., added current switch speed to that P.S.; changed third prgf. speeds to "effective"

Share this post


Link to post
Share on other sites

Rereading dittberner's OP just now, I realized I had misread what he/she is trying to do.  His/her virtual machines are not Retrospect clients connected to the "backup server" over TCP/IP, but are instead connected to the "backup server" as SMB shares.

Thus his/her situation is closer to what I did today when I ran my Recycle backups of all 6 drives, including 2 drives that are local to my Mac Pro "backup server"—they are inside the "cheesegrater" Mac Pro' s case connected by SATA.  For those two drives, the combined backup-compare speeds were 2.423GB/minute and 1.425GB/minute respectively.  Considering that according to this Wikipedia article SMB evidently has more overhead than SATA, 4GB/minute seems quite reasonable for Retrospect combined backup-compare speeds on what is almost certainly a faster machine than my 2010 Mac Pro.

Note that my first local drive has more data, but fewer files and folders, than my second local drive.  Therefore what I said in the first paragraph of the second post in this thread should still apply in a slightly-revised formulation; traversal of multiple files (and folders) on the Source drive—on whatever computer the Source drive is attached to—seems to be the factor that limits the speed of backups.

 

Share this post


Link to post
Share on other sites

Hello David, 

sorry for the late reply.

We use the Retrospect Client on the backup client machine. The smb speed is only for a comparison.

We have similar speeds with 1GBit ethernet like you. We need to get more speed and upgrade to 10GBit ethernet, not only for the backup of course. But it seems, that Retrospect can't benefit from the faster network as we like. Our latest test comes near to 6GB per Minute which are 100MB per second. And I think it should be more possible with all this hardware. Maybe there is a software limit in Retrospect.

Kai

 

 

 

Share this post


Link to post
Share on other sites

What late reply?  You replied 13 hours after my last post; that's not late for a Forums thread.

Frankly, Kai,  I think we have a couple of Computer Science problems here, and possibly an English problem.

The English problem is that I forgot English is not your native language, and replied with my usual complicated sentences.  Sorry, I'll try to do better.

One Computer Science problem is mine; it turns out SMB runs on top of TCP/IP, so it wouldn't be much slower than TCP/IP alone.

The second Computer Science problem may be either yours or mine.  If you can connect to these virtual machines from the Retrospect "backup server" machine using SMB, why are you bothering with using the Retrospect Client on the virtual machines?  Just follow the procedure on pages 72 through 73 of the Retrospect Mac 14 User's Guide (although the example shows using AFP instead of SMB, so maybe you should use AFP if you don't know what you're doing as far as SMB addresses are concerned—but see the P.S. and P.P.S.below), and mount the virtual machines' drives as Retrospect SMB (or AFP, if that's what you chose) network shares.  Then you can treat them as drives that are local to the "backup server", and eliminate the Retrospect Client software as a speed bottleneck.  Or am I missing something here, you folks with virtual machines?

P.S.: See this post on a Wake Forest University website for how to set up an SMB address under Mac OS X.

P.P.S.: See this "Adding NAS Source Volumes with Retrospect 10" video.  Mayoff only mentions SMB for one second, after AFP, but he shows you how to add a volume.

Edited by DavidHertzberg
Added P.S. on how to set up an SMB address; added P.P.S. with video of how to do it

Share this post


Link to post
Share on other sites

Hi David,

oh, its much easier to read your english answers than write my own. So don't worry please.

Your idea to use smb shares to backup files from the virtual machines are ok but you miss two things. First we have data on the server to backup without possible shares like active directory data, system files and open files like databases. And second you have to use the Retrospect client to read file metadata like acls and file flags correctly.

Kai

 

Share this post


Link to post
Share on other sites

Today, in order to confirm the shocking news in this post in the Windows Professional Forum, I phoned Retrospect Sales and got the chief salesperson.  Werner confirmed that the quoted reply from Retrospect Support was accurate, which rules out a suggestion—that Kai run Retrospect Virtual on his Windows machine and also use it to backup his presumed Macintosh computer(s)—I was going to make to Kai in this thread.

However Werner also told me how he backs up his own Macintosh at home, which may be of some assistance to Kai.  Werner runs VMWare Fusion, which "allows Intel-based Macs to run operating systems such as Microsoft Windows, Linux, NetWare, or Solaris on virtual machines, along with their Mac OS X operating system."  If I understood Werner correctly, he runs the Retrospect Windows client on his Fusion VM, which allows him to backup his files using Retrospect Macintosh.  He then periodically runs some kind of Windows imaging program on his Fusion VM, imaging his VMDK to a CD or DVD.  If his VM (I assume it has to be on a separate partition) has to be restored, Werner first restores the VMDK from the CD or DVD and then uses Retrospect Mac—in combination with a re-installed Retrospect Windows Client—to restore the files on his VM.

Hope that helps. 

Share this post


Link to post
Share on other sites

On 9 December I ran an unplanned test that verified my hypothesis in this post in the thread.

On the night of 5 December my main computer, an Early 2011 15-inch MacBook Pro, wouldn't boot.   I brought it in the next day to Mike's Tech Shop; they said the logic board was dead, and that they couldn't replace it because that model was declared "obsolete" by Apple on 1 January.  I decided to buy a 2016 15-inch MacBook Pro as an "open box" special, doing so because I could take immediate delivery and because it came  with macOS 10.12.6  installed (my rule is never to install an Apple OS whose last digit is not .3 or greater, and macOS 10.13.2 has just been released).  The 4-core processors in the new machine are 2.7GHz vs. 2.0 in the old machine, and RAM on the new machine is 16GB vs. 8GB in the old machine.  Both the new and old machines have 500GB SSDs; the SSD in the old machine was installed in early May, so—within the limitations of the old machine's drive interface—its drive speed ought to be about the same.

Before bringing the old machine in to Mike's I had run a Restore of the old machine's 5 December  Retrospect incremental backup onto a spare portable HDD.  Overnight starting 6 December I used Migration Assistant to copy the files from that HDD onto the new machine.  Despite a couple of missteps and an adapter problem (the Apple Thunderbolt-3-to-Thunderbolt-2 adapter Mike's sold me turns out not to work with DisplayPort monitors, meaning I can't connect the new machine to my 27-inch Apple LED Cinema Display until I receive a  StarTech adapter that will work from NewEgg), I had the new machine more or less operational with its built-in 15-inch monitor on 7 December.

Every Saturday I run Recycle backups of all my 6 drives onto a portable 500GB USB HDD.  The run on 9 December did a LAN backup of my new MacBook Pro in 2.5 hours, vs. the 3.5 hours the backup of essentially the same files from my old MacBook Pro had taken on 2 December.  The MD5 validation of the backup of the new machine was also correspondingly shorter than that of the old machine.   IMHO that proves "the main factor limiting speed of [backups over] networks is the traversal of multiple files in the file system by the client computer."

 

Share this post


Link to post
Share on other sites

Two weeks ago I put an SSD into a spare bay in my 2010 Mac Pro "backup server" machine.  After doing a Restore of the 12-year-old boot HDD to the SSD and re-creating my Media Sets (with some problems) so their Catalog Files are on that SSD, I ran my weekly Recycle backup of all my 6 drives yesterday—booting from and running Retrospect on the SSD.

The run on 6 January did a LAN backup of my new MacBook Pro in 2.25 hours, vs. the 2.5 hours the backup of essentially the same files from my new MacBook Pro had taken on 9 December.  So speeding up the drive on the "backup server" machine makes a slight difference, but not nearly as much as speeding up the processor chip on a "client" machine.  It would be interesting to see what effect a faster processor chip on the "backup server" machine would have; since the "backup server" is a "cheese grater" Mac Pro it would be fairly easy technically to do, but I don't have the budget for an upgrade—which based on past experience wouldn't have that dramatic an effect (see the P.S. here).

Edited by DavidHertzberg
Added link to previous post in thread, where going from old G4 to newer Mac Pro as "backup server" only increased speed of backup of same MacBook Pro by 18%

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this  

×