Replicate Onsite Backup to the Cloud

[kbisignani]

Hello all,

So I'm curious if someone can point me in the direction of "best practice" when it comes to backing up to a NAS and then replicating that backup off-site to the cloud (say something like Amazon Glacier or the like).

Is it as simple and straightforward as Step 1: Backup to NAS.  Step 2: Copy Backup Sets to Cloud

Or is there a more streamlined approach or something that helps to ensure I'm not using excess space in cloud?  I'd love not to have any of this operate during normal business hours and so I think incremental is the best approach here, but not sure what my exact configuration would be.

Any thoughts or pointers are appreciated.  Thank you!

(If helpful, I'll be using a Buffalo NAS)

~ Kyle

[DavidHertzberg]

kbisignani,

Here is the Mac-oriented post I wrote two years ago on the subject.  As you will see, I revised it per the head of Retrospect Tech Support and others, and it has not had any revisions by anyone else since.  I wrote it to deal with the needs of an administrator who wanted a safety net for his local backups, but evidently could not afford to store more than the most recent year's worth of backups on Amazon.  If you don't have that monetary restriction, or do not wish to backup the existing contents of your NAS, you can disregard the steps involving Grooming.  If you do not wish to copy to the cloud your existing NAS backups, you can disregard the steps involving making a copy and shipping it to your cloud provider; if you do wish make that copy, Amazon will now rent you an Amazon Snowball appliance that you must substitute for a shippable disk.

You don't say what under version of what OS you are running the "backup server", but I assume from the fact that you posted in this Forum that it is Windows.  You should therefore—in reading my post—substitute Transfer Snapshots for Copy Media Set, and Transfer Backup Sets for Copy Backup.  Alternatively you may want to read this very detailed Windows-oriented post, which has been updated for Amazon Infrequent Access, and the further-updating post below it.  This Knowledge Base article has a section on Amazon Glacier.

[kbisignani]

Thank you David for the reply - and the links!  I did a cursory review of them last night but there's a lot of good information and follow up conversation that I want to really parse through.  SO thank you for all of that.

You are correct - this is a Windows server running the backups of other Windows (and one Mac) servers.

I'm going to start from scratch with a new NAS, I'll keep old NAS's offline for previous backups - they only have about 6-8 months on them anyway before they ran out of space and Retrospect started to complain.  The pure intention here is to replicate what I have being stored on the local NAS to a remote location.  The Buffalo's actually can do this natively, but I'm not sure if that's a good idea since I don't know what kind of "transactions" will be occuring and don't want to incur extra charges because of my ineptness.  I need to do more research on the topic and have a meeting on Tuesday with one of my vendors who I'll be going through to help me as well.

Again, thank you David.  I'll follow up with what we did, and how we did it, for the good of the order.

 

[DavidHertzberg]

kbisignani,

I remembered later that I had done another version of this post. That version was a partial rewrite for administrator WKTimes, who—unlike administrator jethro for whom the first post was written, I assumed didn't want to do "seeding" of an existing on-site backup to his/her new cloud backup.  I made that assumption because, as of September 2016, only Amazon S3—via its Snowball appliance—and Google Cloud—via high-speed Internet upload of disks/tapes sent to Iron Mountain—offered "seeding".

Amazon S3 and Google Cloud were, and I believe still are, expensive cloud service providers.  As of June 2018, there are a number of substantially cheaper cloud service providers certified for Retrospect. 15.  If you are in the U.S., they include Backblaze B2 and Wasabi and DreamHost.  Wasabi and DreamHost are S3-compatible, but don't provide "seeding".  Backblaze B2 is not S3-compatible, but offers rental of a B2 Fireball "seeding" appliance that sounds similar to an Amazon Snowball.

[kbisignani]

So just to post an update, and follow up on some points you made David, I work for an entity that has some unique (translation: archaic) purchasing limitations.  Unfortunately for me, it ends up eliminating the availability of many cloud storage providers like Backblaze (which I personally use and love).  As such, I'm kind of limited.   Amazon Glacier (while not necessarily the best option) has similar pricing to Backblaze B2 and is something my employer can figure out how to pay for (via a third party, of course).

Question though specific to Glacier - the Retrospect KB Article here that you showed me talks about the data "Lifecycle" options - and setting the option to Archive to Glacier after 1 day.  Does that mean that I wouldn't be storing directly from Retrospect directly into Glacier?  And, if so, subsequently incurring high data storage costs for one day each time I back up to the cloud?

(Again, I apologize for my ignorance - just want to learn as much as possible before I jump into the deep end!)

 

[DavidHertzberg]

1 hour ago, kbisignani said:

....

Question though specific to Glacier - the Retrospect KB Article here that you showed me talks about the data "Lifecycle" options - and setting the option to Archive to Glacier after 1 day.  Does that mean that I wouldn't be storing directly from Retrospect directly into Glacier?  And, if so, subsequently incurring high data storage costs for one day each time I back up to the cloud?

....

kbisignani,

You will probably want to look at this Knowledge Base article, for what it's worth.  I did a Forums search, but there's nothing worth looking at except this answer I gave to an administrator who was having trouble restoring from Google Cloud Nearline—and I pointed him to the same KB article.

P.S.: Ah, but there's always Wikipedia—specifically the last paragraph of  this article section.

[kbisignani]

Spoke to support and yes, that is in fact the case - store to S3, then Lifecycle moves it to Glacier.  I'll have to see what the pricing implications are of this - for all I know this is standard operating procedure for storing things in the iceberg.  I have a phone call slated for next week to start the formalization of this with a vendor, so I'm sure I learn more from that as well.

Thank you again David!

[kbisignani]

A follow up question for the group -

I have Amazon S3 set up so that files uploaded to my bucket are "migrated" to glacier storage automatically as per the documentation on how all this works.  And as per this Knowledge Base article from David and this documentation, I understand I would need to move stuff back into S3 from Glacier in order to restore files.  But what about grooming?  If I only want, say the last 50 backups, I know I can use the following setting in Retrospect to accomplish this task, but will it be able to delete files out of Glacier?