Jump to content

How fast are your network backups?


Recommended Posts

Hello,

we are testing the new Retrospect 14.5 on mac as new server backup solution. 
Our plan is to use Retrospect Multiserver on MacPro with 10.12.6 and 10GBit network connection to backup data from Hyper-V Windows 2012r2 virtual machines which are also connected via 10GBit ethernet.
Data and backup sets are on FC Raids (8GBit). If we connect to vm from MacPro via Windows smb share, we get a speed up to 700MByte per second read and write.
 
Our first tests with Retrospect results in network speeds of max 4GB/min (backup and copy) which seems to be much slower than 700MB/s.
 
Do you reach faster network speeds? Did we something wrong?
 
Have a nice day
Kai
 
Link to comment
Share on other sites

2.5 years of experience using Retrospect Mac on my home LAN shows that the main factor limiting speed of networks is the traversal of multiple files in the file system by the client computer.  This morning I got a speed of 400MB/min doing a Normal backup of my MacBook Pro client, but yesterday I got a speed of 600MB/min doing the same Normal backup—but of somewhat different files.  Last Saturday I did a Recycle backup of the same MacBook Pro, and got a speed (for 36GB over 3 hours) of 200MB/min; my Recycle backups are of everything including cache files.  All these speeds are for backup only; I've omitted the speeds of compares.

 

About 6 months ago I had to replace the hard disk drive on my MacBook Pro with an SSD; the backup speeds did not appreciably increase.  I therefore conclude that the controlling factor in Retrospect traversal of multiple files is the speed of the client computer's processor(s).  It's possible that the speed-limiting file traversal is on the "backup server" side (but see the P.S. below for why I think that unlikely), but my "backup server" for the last 2.5 years has been a 6-processor 2.5 GHz 2010 Mac Pro—with the same USB3 portable drives for my Media Sets.

 

As for LAN speed, my limiting factor is the built-into-the-wall inter-room RG-59  cable between my MacBook Pro and my Mac Pro, which has Actiontec MoCA adapters on each end.  The MoCA adapters I have are limited to 175Mbps "effective raw" = about 1050MB/min "effective cooked" (I assume 10 bits/byte to allow for 20% TCP/IP overhead).  That's obviously a lot faster than the backup speeds I'm getting, which never seem to go much above 600MB/min on an instantaneous basis.  The Cat5 cable between my bedroom MoCA adapter and my Mac Pro might in fact be limited to 100Mbps by the fact that it was attached around 3 sides of the bedroom wall in 1998 by a local contractor using—fairly carefully—metal brads he had brought with him (he didn't know he was going to have to string Cat5 cable along walls), but I'm not about to buy and string new Cat5e cable (especially since on one occasion I did a bypass by running a Cat5 cable directly from the MoCA adapter to the Mac Pro, and it didn't speed up the backup).  I could buy replacement gigabit MoCA adapters for a total of US$150, but I'm reluctant to spend the money since I'm not sure they would make much of a difference.

 

dittberner may want to read this 2013 thread, in which post #2 by the very-knowledgeable Lennart Thelander shows speed results rather similar to mine.

 

P.S.: I knew there was a further reason that I ruled out file traversal on the "backup server" as the speed-limiting factor, and I have now remembered what it was.  Back in June 2015, before I spent a substantial amount of money upgrading my (inherited from a dead friend) Mac Pro's hardware and software to be my "backup server", I ran a feasibility test using the hardware and software I already owned.  I ran a Recycle backup of my MacBook Pro using my ex-wife's 733MHz Digital Audio G4 as the "backup server", SCSI-cabled to my old DAT tape drive for a Backup Set (the old term for Media Set), and running my old copy of Retrospect Mac 6.1.  The backup part of the test ran at 170MB/min, which is almost as fast as the 200MB/min in last Saturday's Recycle backup of my MacBook Pro—which used a 2.5GHz Mac Pro as the "backup server" hardware and Retrospect Mac 14.0 as the "backup server" software.  The test was run over the same LAN I have now, except that my Ethernet switches were then only 100Mbps instead of 1gigabit/sec.

Edited by DavidHertzberg
Added parenthetical ref. to P.S. in 2nd prgf., added current switch speed to that P.S.; changed third prgf. speeds to "effective"
Link to comment
Share on other sites

  • 2 weeks later...

Rereading dittberner's OP just now, I realized I had misread what he/she is trying to do.  His/her virtual machines are not Retrospect clients connected to the "backup server" over TCP/IP, but are instead connected to the "backup server" as SMB shares.

Thus his/her situation is closer to what I did today when I ran my Recycle backups of all 6 drives, including 2 drives that are local to my Mac Pro "backup server"—they are inside the "cheesegrater" Mac Pro' s case connected by SATA.  For those two drives, the combined backup-compare speeds were 2.423GB/minute and 1.425GB/minute respectively.  Considering that according to this Wikipedia article SMB evidently has more overhead than SATA, 4GB/minute seems quite reasonable for Retrospect combined backup-compare speeds on what is almost certainly a faster machine than my 2010 Mac Pro.

Note that my first local drive has more data, but fewer files and folders, than my second local drive.  Therefore what I said in the first paragraph of the second post in this thread should still apply in a slightly-revised formulation; traversal of multiple files (and folders) on the Source drive—on whatever computer the Source drive is attached to—seems to be the factor that limits the speed of backups.

 

Link to comment
Share on other sites

Hello David, 

sorry for the late reply.

We use the Retrospect Client on the backup client machine. The smb speed is only for a comparison.

We have similar speeds with 1GBit ethernet like you. We need to get more speed and upgrade to 10GBit ethernet, not only for the backup of course. But it seems, that Retrospect can't benefit from the faster network as we like. Our latest test comes near to 6GB per Minute which are 100MB per second. And I think it should be more possible with all this hardware. Maybe there is a software limit in Retrospect.

Kai

 

 

 

Link to comment
Share on other sites

What late reply?  You replied 13 hours after my last post; that's not late for a Forums thread.

Frankly, Kai,  I think we have a couple of Computer Science problems here, and possibly an English problem.

The English problem is that I forgot English is not your native language, and replied with my usual complicated sentences.  Sorry, I'll try to do better.

One Computer Science problem is mine; it turns out SMB runs on top of TCP/IP, so it wouldn't be much slower than TCP/IP alone.

The second Computer Science problem may be either yours or mine.  If you can connect to these virtual machines from the Retrospect "backup server" machine using SMB, why are you bothering with using the Retrospect Client on the virtual machines?  Just follow the procedure on pages 72 through 73 of the Retrospect Mac 14 User's Guide (although the example shows using AFP instead of SMB, so maybe you should use AFP if you don't know what you're doing as far as SMB addresses are concerned—but see the P.S. and P.P.S.below), and mount the virtual machines' drives as Retrospect SMB (or AFP, if that's what you chose) network shares.  Then you can treat them as drives that are local to the "backup server", and eliminate the Retrospect Client software as a speed bottleneck.  Or am I missing something here, you folks with virtual machines?

P.S.: See this post on a Wake Forest University website for how to set up an SMB address under Mac OS X, and this Apple Mac Help page.

P.P.S.: See this "Adding NAS Source Volumes with Retrospect 10" video.  Mayoff only mentions SMB for one second, after AFP, but he shows you how to add a volume.

Edited by DavidHertzberg
Added P.S. on how to set up an SMB address, expanded it to also link to Apple Help page; added P.P.S. with video of how to set up NAS share as Retrospect source
Link to comment
Share on other sites

Hi David,

oh, its much easier to read your english answers than write my own. So don't worry please.

Your idea to use smb shares to backup files from the virtual machines are ok but you miss two things. First we have data on the server to backup without possible shares like active directory data, system files and open files like databases. And second you have to use the Retrospect client to read file metadata like acls and file flags correctly.

Kai

 

Link to comment
Share on other sites

  • 2 weeks later...

Today, in order to confirm the shocking news in this post in the Windows Professional Forum, I phoned Retrospect Sales and got the chief salesperson.  Werner confirmed that the quoted reply from Retrospect Support was accurate, which rules out a suggestion—that Kai run Retrospect Virtual on his Windows machine and also use it to backup his presumed Macintosh computer(s)—I was going to make to Kai in this thread.

However Werner also told me how he backs up his own Macintosh at home, which may be of some assistance to Kai.  Werner runs VMWare Fusion, which "allows Intel-based Macs to run operating systems such as Microsoft Windows, Linux, NetWare, or Solaris on virtual machines, along with their Mac OS X operating system."  If I understood Werner correctly, he runs the Retrospect Windows client on his Fusion VM, which allows him to backup his files using Retrospect Macintosh.  He then periodically runs some kind of Windows imaging program on his Fusion VM, imaging his VMDK to a CD or DVD.  If his VM (I assume it has to be on a separate partition) has to be restored, Werner first restores the VMDK from the CD or DVD and then uses Retrospect Mac—in combination with a re-installed Retrospect Windows Client—to restore the files on his VM.

Hope that helps. 

Link to comment
Share on other sites

  • 1 month later...

On 9 December I ran an unplanned test that verified my hypothesis in this post in the thread.

On the night of 5 December my main computer, an Early 2011 15-inch MacBook Pro, wouldn't boot.   I brought it in the next day to Mike's Tech Shop; they said the logic board was dead, and that they couldn't replace it because that model was declared "obsolete" by Apple on 1 January.  I decided to buy a 2016 15-inch MacBook Pro as an "open box" special, doing so because I could take immediate delivery and because it came  with macOS 10.12.6  installed (my rule is never to install an Apple OS whose last digit is not .3 or greater, and macOS 10.13.2 has just been released).  The 4-core processors in the new machine are 2.7GHz vs. 2.0 in the old machine, and RAM on the new machine is 16GB vs. 8GB in the old machine.  Both the new and old machines have 500GB SSDs; the SSD in the old machine was installed in early May, so—within the limitations of the old machine's drive interface—its drive speed ought to be about the same.

Before bringing the old machine in to Mike's I had run a Restore of the old machine's 5 December  Retrospect incremental backup onto a spare portable HDD.  Overnight starting 6 December I used Migration Assistant to copy the files from that HDD onto the new machine.  Despite a couple of missteps and an adapter problem (the Apple Thunderbolt-3-to-Thunderbolt-2 adapter Mike's sold me turns out not to work with DisplayPort monitors, meaning I can't connect the new machine to my 27-inch Apple LED Cinema Display until I receive a  StarTech adapter that will work from NewEgg), I had the new machine more or less operational with its built-in 15-inch monitor on 7 December.

Every Saturday I run Recycle backups of all my 6 drives onto a portable 500GB USB HDD.  The run on 9 December did a LAN backup of my new MacBook Pro in 2.5 hours, vs. the 3.5 hours the backup of essentially the same files from my old MacBook Pro had taken on 2 December.  The MD5 validation of the backup of the new machine was also correspondingly shorter than that of the old machine.   IMHO that proves "the main factor limiting speed of [backups over] networks is the traversal of multiple files in the file system by the client computer."

 

Link to comment
Share on other sites

  • 4 weeks later...

Two weeks ago I put an SSD into a spare bay in my 2010 Mac Pro "backup server" machine.  After doing a Restore of the 12-year-old boot HDD to the SSD and re-creating my Media Sets (with some problems) so their Catalog Files are on that SSD, I ran my weekly Recycle backup of all my 6 drives yesterday—booting from and running Retrospect on the SSD.

The run on 6 January did a LAN backup of my new MacBook Pro in 2.25 hours, vs. the 2.5 hours the backup of essentially the same files from my new MacBook Pro had taken on 9 December.  So speeding up the drive on the "backup server" machine makes a slight difference, but not nearly as much as speeding up the processor chip on a "client" machine.  It would be interesting to see what effect a faster processor chip on the "backup server" machine would have; since the "backup server" is a "cheese grater" Mac Pro it would be fairly easy technically to do, but I don't have the budget for an upgrade—which based on past experience wouldn't have that dramatic an effect (see the P.S. here).

Edited by DavidHertzberg
Added link to previous post in thread, where going from old G4 to newer Mac Pro as "backup server" only increased speed of backup of same MacBook Pro by 18%
Link to comment
Share on other sites

  • 3 weeks later...

In researching a newer topic in the Retrospect Windows Server forum, I came across this thread from 2010.  It's interesting that the test results are very similar to mine, even though the OP was using Retrospect Mac 8.  It's also interesting that a possible method of speedup, from the OP in that thread,  was similar to what I was implicitly suggesting in this post above.

Edited by DavidHertzberg
Added third sentence
Link to comment
Share on other sites

  • 4 weeks later...

Hi David, 

I just saw that this thread goes on. And I should clarify some things because your answers have nothing to do with my question. Sorry.

We try to use a Mac server to backup files on windows servers. The windows servers are virtual machines but this is not the point. And Retrospect Virtual is a completely different thing.

And we don't use cat5 cables (which are pretty old). We use a very fast fibre channel san with 100 or more TB of data and a 10GBit fibre ethernet network. 

And we needed a fast backup system for 10 or more servers over network. But our test results with Retrospect are too slow, so I asked here whether we've missed something and get some comparative values.

In the meantime we use another product for our backups, which is much faster. 

Thank you for your help. 

Kai

Link to comment
Share on other sites

Hi Kai,

First, the title of this thread—which you created— is "How fast are your network backups?", not "What is the answer to Kai's specific problem?"  I took the opportunity to answer the thread title's question in a general way; judging by the number of views, the question seems to be of interest to many Retrospect administrators.  I've since linked to posts I made in it as recently as yesterday.

I'd very much like to ask you what product you ended up using for your backups.  The problem is that I can't ask you to reply on this forum, because I've got past evidence that the head of Retrospect Technical Support doesn't like such discussions here.  If you want to tell me, please send me a private message via the Forums facilities.

However I'll venture a guess that the product is one specifically designed to backup files on Windows virtual machines, taking advantage of facilities built into virtual machine hypervisors.  Thus the only way you could use a Mac in connection with such a backup product is to—maybe—run an administrative Console to control it.  The Retrospect Inc. product you refer to above, which I can only refer to here as R. V. (otherwise the head of Retrospect Tech Support wouldn't be happy), is—in my limited understanding—such a virtual machine backup product.   In fact in a recent phone conversation with the head of Retrospect Inc.'s North America sales about another user's unrelated problem, he told me that R. V. has recently taken sales away from another virtual machine backup product I'll refer to here as V.—and has done so because R. V. is significantly cheaper than V..

Sorry my answers had nothing to do with what turned out to be your question—which wasn't what you asked in your OP.

David H.

Link to comment
Share on other sites

For the overall purposes of this thread (not what dittberner@dbr3.de really wanted to know), I want to propose an explanation for the phenomenon mentioned in this 2010 thread—which fits in with my more-recent observations.  The phenomenon is that network backups of client machines using Retrospect Client software are much slower than backups of the same client machines when those machines are mounted as shared volumes on the "backup server".  Moreover, network backups of a client machine using Retrospect Client software seem to be greatly sped up if the processor speed of the client machine is increased.

My proposed explanation is that, when the Engine backs up a disk via a Retrospect Client, it sends a "send me the next file" message over the LAN to the client machine.  The Client software receives that message, traverses the directory structure, and then sends the file block-by-block over the LAN to the Engine.  When it has finished, it sends a "that's the end of that file" message over the LAN to the Engine, after which the Engine begins the process again for the next file.  The processing of messages by the Client software seems to require significant amounts of CPU work by the client machine.

By contrast, when the Engine is backing up a shared disk no exchange of messages is necessary—since the Engine software automatically keeps track of what it is doing.  The directory traversal and reading of the file blocks still has to be done, but that seems to require less CPU work.

The foregoing is only a hypothesis, although it fits the available evidence.  The hypothesis implies that message processing requires more CPU time than input/output operations, which is not surprising considering all the work that has been done over the last 60 years on offloading I/O operations to simultaneously-running I/O processors (which IBM originally called "data channels" but are now called DMA controllers).  It furthermore implies that that CPU time is largely consumed by the OS of the client machine, which means that client-server backup speed is not going to be greatly increased if you switch from Retrospect to Backup Application X—so long as Backup Application X is still running in client-server fashion.  The only way to significantly speed up backup of a client machine is to have that machine do its own backup, which is what I think dittberner@dbr3.de's installation has done by using an application that uses backup capabilities built into virtual machine manager hypervisors.

Edited by DavidHertzberg
modern term for simultaneously-running I/O processor is DMA controller
Link to comment
Share on other sites

Hi David,

of course, I will not tell the name of the other product, because it's really not fair to Retrospect. The other product is not comparable (much more expensive and other functions).

My intention was to get some data from other users with the same hardware level to compare.

But the virtualisation is not the bottleneck, we got the same speed with hardware machines.The use of shared volumes is not always possible, because of open files or security reasons.

May be Retrospect 15 is the answer. I hope I have some time to test the software.

Kai

 

 

Link to comment
Share on other sites

  • 7 months later...

If what Retrospect Tech Support has told insont as recounted in this post gets done on schedule, it looks as if Retrospect 16 will go a long way towards solving the speed-of-backups problem.:)  The 21 March Knowledge Base article that post links to talks about adding an option "FileSystemApi=2" to /Library/Application Support/Retrospect/retro.ini; the name of that option sounds to me—a retired applications programmer—as if "the client scan APIs will be completely overhauled" in the post means more than just for Instant Scan.

Now if Retrospect Mac 16 doesn't make the -530 bugs worse, which is what apparently happened with the Retrospect Mac 14.6 Client (explaining why I'm still running the Retrospect Mac 14.6 Engine and Console with the Retrospect Mac 14.1 Cient), I'll be resigned to backing up my secondary Digital Audio G4 (running OS X 10.3 ) separately with Retrospect Mac 6 onto DAT tape after I upgrade to Retrospect Mac 16 for my primary "backup server".  I should report here that I have twice run test versions of Retrospect—and sent them log files—at Tech Support's request, but they haven't sent me a test version that actually fixes any of the -530 bugs in backing up my primary MacBook Pro.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...