Backing up clients to internet destinations

[erostratus]

One other question I had was how Retrospect handles backups when the Retrospect server doesn't have attached storage. I set up my backup through Minio S3 on my Synology, connected by Ethernet to the local network. If my Mac mini is the Retrospect server, when machines using the Retrospect client back up to S3, will they be sending their data through the Mac mini to S3, effectively doubling the amount of network data required for backup? Or will the clients back up directly to the S3 server? If they have to go through the Mac mini, does that mean the only way to have client machines back up directly to S3 is to install a Solo copy on each machine?

[DavidHertzberg]

erostratus,

They'll be sending their their data through the Mac mini to S3.  (MinIO "is a cloud storage server compatible with Amazon S3, ... Apache License v2.")

However I don't think you've thought through the consequences of having your "clients" back up directly to the S3 server.  Unless you scheduled the backup of each "client" at a different time, all your "clients" would be sending data over the network at once—possibly overloading the network and/or MinIO—because there would be no way of making each un-coordinated "client" "take its turn".  That's why enterprise client-server backup has thrived even though most backups are no longer directly to tape; it makes client machines "take their turn" as dictated by their place in a Backup or Proactive schedule.

If after reading the preceding paragraph you still want multiple "clients" to back up directly to the S3 server, I'd suggest installing Arq on them instead of Retrospect—it'd be cheaper because Arq is per-user-licensed.  Let Stefan Reitshamer deal with your problems, instead of us or Retrospect Tech Support.

P.S.: An important sub-question  in the "back up directly" case is how many "clients" you'll have backing up to S3.  In August 2017 I posted in an Ars Technica Mac thread whose OP had been trying to back up 25 Macs to one OS X Server instance.  He/she found that "when 25+ users are backing up over the network to our Time Machine backup server, there are a lot of backups not being completed. The Apple rep I spoke to actually confirmed that this number of users causes a Time Machine server a lot of issues."  Up to 15 "clients"  may work for you, even if MinIO's  efficiency equals OS X Server's.

Edited October 23, 2019 by DavidHertzberg
P.S.: important sub-question in the "back up directly" case is how many "clients" you'll have backing up to S3; second sentence in first paragraph explaining what "Minio S3" is; Arq is per-_user_-licensed

[Nigel Smith]

On 10/17/2019 at 5:44 PM, erostratus said:

One other question I had was how Retrospect handles backups when the Retrospect server doesn't have attached storage. I set up my backup through Minio S3 on my Synology, connected by Ethernet to the local network.

That sounds as if you are using your Synology as your backup target, connecting your Retrospect server to it via Minio (running on the Synology). In which case you might be making things more complicated than you need to!

Mount the Synology on the Retrospect server machine as an AFP or SMB share. Set that share as your backup destination. No need for Minio at all.

Either way, you'll have a potential bottleneck in that the server's network connection handles roughly twice the load of the other parts of the system -- both incoming from the client and outgoing to the Synology -- just as you surmised. That may not be a problem -- client read/server write speeds may mean your network never saturates -- but, if it is and it means you exceed your backup window, it's easily solved with a second network interface on the Mini (eg Thunderbolt Ethernet adapter) and using one for the clients and the other for the server connection, or even an upgrade to 10Gb.

If you explain what you have (client numbers, amount of data) and what you're trying to achieve, we might be able to suggest ways of doing it.