Jump to content

Date/Time specific Incremental Backup for Cloud


Recommended Posts

Hello,

 

I've done a few backups to the cloud and it looks like it's running smoothly. I'd like to create new backup set and have a couple questions. 

 

First, here's what I want to do. I'd like to create a cloud backup that will go back to all the new files in the last 2 weeks. So at the start, I'd have 2 weeks of files and then each day thereafter it would backup any new or altered files for the last 2 weeks. 

 

At some point this would become a large backup so I'd also like to set it so every 2 months it starts over with a fresh set of "the past 2 weeks".

 

This is for emergency backup only. I do have a backup that has all of our files but it is very large and in an emergency it wouldn't be prudent to retrieve the entire backup as about 80% of it, while still in use, I have offsite that I can easily get.

 

Mac OS 10.7.5

Retrospect v.13

 

Any help would be appreciated. 

Randy

Link to comment
Share on other sites

It sounds to me as if you have some variant of the same requirements that jethro had in this thread.  I would suggest that you consider modifying the procedures in part C) of this post in that thread.

 

The first modification you would have to make to the linked-to procedures is in step C1), where you would have to change Months to keep = 1.  It would be nice if there were a Weeks to keep alternative you could set to 2, but there isn't.  Also, in step C1) choose Performance-optimized grooming if you have an option; it will AFAIK be the only option if you are doing "seeding".

 

Whether you would have to make a second modification depends on whether your cloud provider has a "seeding" service, such as I assumed in steps C1) through C5).  Retrospect Mac 13 has provided a neat feature for "seeding" that I assumed back in March 2016 would be used, but I have since learned that only two cloud providers offer any version of "seeding".  Amazon AWS offers "seeding" via a beefed-up large-capacity Snowball drive that they lend to you, but that costs $200 for a 50TB drive plus $30 per TB "seeded".  Iron Mountain offers a "seeding" service for Google Cloud, but it appears that their service involves their doing a high-speed Internet upload to Google of the data from disks/tapes you send to Iron Mountain—and it almost certainly ain't cheap.  Other than that, it appears that no other cloud provider has a "seeding" service.  Of course you could setup Basho Riak S2 on a dedicated server per D) and do your own "seeding", but if you were doing that I assume you would have mentioned it.  If you can live without "seeding", you might consider using DreamHost with Retrospect Inc.'s special deal, under which Cloud storage would only cost you $20/month/TB instead of $30/month/TB with Amazon or Google.

 

Let's assume that you can't do "seeding", so that you would have to modify steps C1) through C6) in the linked-to procedures.  First, do the modification to step C1) in the second paragraph of this post.  Second, in steps C1) and C2), replace "Cloud Media Set" with "new local Disk Media Set".  This would have to be as big as your regular backup Media Set, unless and until (as I have newly officially suggested) Retrospect Inc. implements an age subset of the "Groom to Retrospect defined policy" option in Copy Media Set with a Disk (not just Cloud) Media Set as destination for Retrospect Mac Ver. 13.5—or else you use a Rule in the Copy Media Set to do the equivalent of grooming..  Third, in step C2) (unless and until you are using a forthcoming Retrospect Mac 13.5 with my suggested feature) you will definitely have to Groom the new Media Set afterwards—or else use a Rule in the Copy Media Set to do the equivalent of grooming.  Fourth, in step C3) omit the drive-shipping operation.  Fifth, in step C4) replace everything prior to typing in the account parameters with "Create a new Cloud Media Set with a Cloud member, specifying 'Groom to Retrospect defined policy' with Months to keep = 1 and specifying Performance-optimized grooming".  Sixth, replace step C5) with "Run a Copy Media Set script to copy your new groomed local Disk Media Set to the new Cloud Media Set, with the option Match Source Media Set to destination Media Set unchecked and the option Copy Backups checked.  Check No Verification."   Actually running this script may take hours or days—which is why "seeding" would have been preferable, but thanks to the glories of using the "backup server" as a "second-level client" you can run it over the weekend—or in a separate thread unless you have Retrospect Desktop Edition.  Seventh, in step C6) replace weekly with daily.  

 

Finally, whether or not you can do "seeding", add a new step C7) that says to schedule a Groom script for the new Cloud Media Set to run once every two months.

 

Disclaimer: I've never done any of the above stuff; all I did was to read the Mac User's Guide and look at a couple of Tutorials.

 

P.S.: Lowered pricing for Amazon AWS Snowball "seeding".  Moved step C7) to separate paragraph, indicating it must be done whether or not you do "seeding". 

 

P.P.S.: Corrected step C7); you can schedule a Groom script to run less often than once a month.

 

P.P.P.S: Added options and No Verification to step C5); changed step C6) to run daily.

P.P.P.P.S: Changed my answer to step C2 to say that Copy Media Set can use a Rule as an alternative to grooming per this post.

Link to comment
Share on other sites

This is a rewrite of paragraphs 2, 4, and 5 of the preceding post for the "if you can't do 'seeding'" case, based on merging them into a copy of part C) of this post.  You would want to do the following things:

 

1)  Create a new local Disk Media Set, specifying "Groom to Retrospect defined policy" with  Months to keep = 1.   Choose Performance-optimized grooming.  This would have to be as big as your regular backup Media Set, unless and until—as I have newly officially suggestedRetrospect Inc. implements an age subset of the "Groom to Retrospect defined policy" option in Copy Media Set with a Disk (not just Cloud) Media Set as destination for Retrospect Mac Ver. 13.5.  It would be nice if there were a Weeks to keep alternative you could set to 2, but there isn't.

 

2)  Run a Copy Media Set script to copy your regular Media Set to the new local Disk Media Set, with the option Match Source Media Set to destination Media Set unchecked and the option Copy Backups checked  The resultant size—after grooming—should give you a fairly good estimate of the amount of storage you would need in the cloud; get/put requests I don't know about.  If the size of the new Cloud Media Set local disk member is the same as your regular Media set—which it will be except in the unlikely case that Retrospect does the multi-cycling copying-grooming I suggested it might do in sentences 3 through 5 of the first paragraph of this post, you will have to Groom the new Media Set afterwards—or else use a Rule in the Copy Media Set to do the equivalent of grooming.

 

3)  Meanwhile make the arrangements to setup your cloud account, and keep the account  parameters handy.

 

4) Create a new Cloud Media Set with a Cloud member, specifying 'Groom to Retrospect defined policy' with Months to keep = 1 and specifying Performance-optimized grooming, and type in the account parameters you have kept handy.

 

5) Run a Copy Media Set script to copy your new groomed local Disk Media Set to the new Cloud Media Set, with the option Match Source Media Set to destination Media Set unchecked and the option Copy Backups checked.  Check No Verification.  Actually running this script may take hours or days—which is why "seeding" would have been preferable, but thanks to the glories of using the "backup server" as a "second-level client" you can run it over the weekend—or in a separate thread unless you have Retrospect Desktop Edition.  

 

6) Setup a new Copy Backup script to copy from your regular Media Set to the new Cloud media set, this time with the option Match Source Media Set to destination Media Set checked.  Check No Verification.  Schedule the new Copy Backup daily, at whatever time you want.

 

7) Schedule a Groom of the new Cloud Media Set to run once every 2 months.

P.S: Changed my answer to step 2 to say that Copy Media Set can use a Rule as an alternative to grooming per this post.

Link to comment
Share on other sites

  • 1 year later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...