Optimizations for 10 Gbps networking? Or is this just the limit of LTO-5?

[Xenomorph]

Server: Windows 2008 R2 w/ 32 GB RAM, running Retrospect Multi Server version 15.6.0.135.

Client: CentOS 7 Linux w/ 64 GB RAM, running Retrospect Client version 15.1.2.101.

The LTO-5 Tape drive is connect to the server via 8 Gbps fiber.

Client and Server are connected to each other with 10 Gbps Ethernet.

Doing direct client to server file copies over SMB easily hits 450-650 MB/sec. The client's disks are set up in a RAID60 configuration, and are able to deliver data quickly.

Yet, when doing backups with Retrospect, I'm getting about 60-80 MB/sec ("4358.7 MB/min" on the last backup job). The CPUs on both the client and server are mostly idle, the memory is mostly free, and the NICs are barely utilized. 

Isn't LTO-5 supposed to be able to get 140 MB/sec (or 280 MB/sec compressed)?

Is this to be expected? Or where should I start troubleshooting something like this?

 

 

[Lennart_T]

1 hour ago, Xenomorph said:

Doing direct client to server file copies over SMB easily hits 450-650 MB/sec

For how long (in terms of megabytes)? What happens when the cache in the RAID is full? What speed will you get after that happens?

What speed do you get copying SMB to client FROM server?

[DavidHertzberg]

Xenomorph,

A thread on the topic of the speed of Retrospect backups from clients was started over a year ago on the Retrospect Macintosh 9+ Forum.  As a result of a later unplanned experiment involving my own MacBook Pro client, I verified a hypothesis I had made in an earlier post in that thread.  That hypothesis is that "the main factor limiting speed of networks [for client-server backup] is the traversal of multiple files in the file system by the client computer."

This later post in the same thread explains my hypothesis in more detail in its second paragraph.  My guess at the time was that, if the OP switched to a client-server backup application other than Retrospect, it wouldn't greatly speed up backups of clients.  However the OP reported in a later post in that same thread that my guess was wrong, showing that there is something in the code for Retrospect Clients that slows down the backup.

A reply by Retrospect Tech Support quoted in a different thread (which I linked to in the most-recent post in the thread I've been discussing in the preceding two paragraphs) says "Client API changes are being worked on and preliminary testing shows them being upwards of 10 times faster than they are currently."  I expect those in Retrospect 16 in March 2019; I hope they'll help in Retrospect Windows.

[Nigel Smith]

To add to the above -- is this test using lots of smaller files or fewer larger ones?

I'd benchmark using something like:

1. Current data set using disk-to-disk backup
2. Current data set using disk-to-tape backup (which you've already done)
3. 1TB tarball, disk-to-disk
4. 1TB tarball, disk-to-tape

...and I wouldn't be surprised if you end up using some form of disk-to-disk-to-tape if you want to maximise tape-write speed -- at least until the API changes David mentioned come down the pipe.

But -- do you want to, if it's at the expense of added complexity? While it's nice to go as fast as you can go, if you're completing during your backup window and not wasting too much tape (i.e. the drive's variable-speed operation copes well enough to reduce "shoe-shining" either completely or to an acceptable amount), then having another step to manage may not be worth the hassle.

Nige