Encrypting Disk Media Sets

[Andrew Kinnear]

I have a set of 5 'working day' disk media set backups, Mon, Tues, etc.. Each media set has a single member i.e. one physical disk per day backup. The media being backed up is a single Thunderbolt volume.

All 5 media sets are uncompressed and unencrypted. As they are off-site portable backups I would like to convert them to compressed and encrypted. My understanding is this can't be applied directly retrospectively.

My plan is to:

1. Clone each disk to a similar sized new disk using Carbon Copy Cloner
2. Erase the original member disk and create a new media set on it with a slightly different name and compression and encryption enabled.
3. Use a "Copy Media Set" script to copy the cloned set back to the original member disk applying compression and encryption.

I plan to use the two step "clone and copy back" rather than just a single step copy script because I want to keep the oldest disk as the first member of the set. 

My questions are:

• Is it OK to drag-copy or clone Retrospect data files between disks or do all copy operations have to be done using the Retrospect engine as part of a managed database/filesystem?
• When a "Copy Media Set" script is used is the ability to restore from a snapshot point in time preserved? Is any functionality of the media set lost?
• Will software compression and encryption be applied using this method.

Thank you in advance for any assistance with this, as you can probably tell I am a relatively inexperienced user.

Regards,

Andrew

Retrospect 15.0.0

LaCie Rugged Raid 4&5TB Thunderbolt connected disks

[DavidHertzberg]

Andrew Kinnear,

I haven't done this myself, just read the User's Guides and half-experimented using my own Retrospect Mac 14.6 "backup server".  However, for a media set named "Media Set Example",  I think the answers are:

[1] It should be fine to use CCC, provided you copy the parent "Retrospect" folder and the "Media Set Example" folder within it as well as the "1-Media Set Example" data file within the "Media Set Example" folder.  After you've done the cloning, click the Media Sets category on the left-hand side of the Console, click "Media Set Example" in the list pane, and click the Members button above the detail pane below the list pane.  You can then click the the only line in the detail pane (you said you have only one member for each media set), which should be "1-Media Set Example", and after that click the pencil icon at the bottom left of the detail pane; doing so should allow you to redefine the location of "1-Media Set Example"' as being the cloned file instead of the original.  After that you shouldn't have to do a Rebuild of "Media Set Example",  but if you do the instructions are under the heading "Rebuilding a Media Set"—which is on page 220 of the Retrospect Mac 13 User's Guide (the P.P.P.P.S. of this post explains why I'm looking at that UG instead of the Retrospect Mac 15 UG; the only attempt at an explanation of the functionality I've just described is at the bottom of page 15 of the old version).

[2] AFAIK no functionality should be lost for "Media Set ExampleSmall&Cryptic".  The reason it shouldn't be is that the snapshot—the basis for most of the functionality—is part of the  "Media Set ExampleSmall&Cryptic" Catalog File, which your Copy Media Set script will have created as essentially a duplicate of the  "Media Set Example" Catalog File.  If there is loss of functionality, IMHO there should be mass developer suicides 🙄 in Walnut Creek CA, considering that the Copy Media Set type of script was introduced in Retrospect Windows 7.5 by 2008.

[3] I don't use software compression or encryption in my humble home installation of Retrospect Mac (I keep last week's rotated USB3 backup disk in my bank safe deposit box), but AFAIK they should be applied if you've specified them when defining "Media Set Example—Small&Cryptic".

Edited May 16, 2020 by DavidHertzberg
Inserted a second sentence into the [2] prgf., explaining why no functionality should be lost; in [2] prgf., changed new Media Set example name to be shorter and have OK special char. and added smiley, also changed in [3] prgf.

[Andrew Kinnear]

Thank you for your reply David, that all makes sense. My concerns about the "Copy Media set" script came from a call to Retrospect support. He told me all snapshot information was lost when one of these was executed. I must have misinformed him or misunderstood.

Thanks,

Andrew

[DavidHertzberg]

Andrew Kinnear,

Hold on a minute;  [2] in my post above starts out with "AFAIK". 

First listen to the audio in this 3-minute Tutorial.  If the voice sounds like the "he" you were talking to at Retrospect Tech Support, then you were talking to the head of R. T. S. (whom I am no longer naming on these Forums, because—although he no longer has time to post on them—it appears that he scans them for his name and appears like a bolt from the blue if he feels he has been misquoted or belittled).  If indeed you were talking to him, then his 24 years of Retrospect experience means he knows what he is talking about.

If you were talking to him, then I suggest you phone him again and ask about the Copy Backups option for Copy Media Set.  Page 139 of the Retrospect Mac 14 User's Guide says "This copies the point-in-time file and folder listings and information about those files along with any metadata required to provide point-in-time restores from the destination Media Set. Deselecting this option will only copy the files contained in the source Media Set, and the destination Media Set will lack the necessary file/folder listings and metadata to perform complete point-in-time restores."  If he says this doesn't work as described, ask if using instead a Copy Backup script with the Copy Backups option would preserve the information you want; however I don't think it will because the bottom of page 139 in the UG says "Copy Backup scripts are different from Copy Media Sets scripts in a number of ways: They copy only active backups; Copy Media Sets scripts copy all backups."

I'll try to run a test of Copy Media Set followed by an attempted Restore over the weekend.  I did an Advanced Search of the Forums a few minutes ago and found this thread listed on the second page returned.  On the surface it seems to bear out what "he" was saying, but a careful reading indicates that Don Lee probably was using a Copy Backup script with the Copy Backups option instead of using a Copy Media Set script.

[DavidHertzberg]

Andrew Kinnear,

I got worried about whether the head of Retrospect Tech Support might have been correct in what he told you, as reported in your post above.  I therefore ran a Copy Media Set of my this week's "Media Set Blue" to a recycled "Media Set Pink" late this afternoon while I took a post-gym nap and then went to dinner.  "Media Set Blue" had Recycle backups of 6 drives from Saturday, plus No Media Action backups of one drive on each day from Sunday through this (Thursday) morning.  That made a total of 11 backups, which is what a subsequently-initiated (and canceled) Console Restore of selected files and folders said were available on "Media Set Pink".  The Past Backups category on the Console also showed that each drive backup from Saturday onward was on "Media Set Pink" as well as "Media Set Blue".  So I think that I've adequately proved that Copy Media Set in fact "copies the point-in-time file and folder listings and information about those files along with any metadata required to provide point-in-time restores from the destination Media Set", so long as the Copy Backups option is checked.

One slightly-worrisome fact stems from my having left the Media Verification option checked for the Copy Media Set script, which I would not have done if I had realized it would cause a byte-by-byte compare (I didn't intend to actually Restore any files or folders, so I would rather have kept the test run as short as possible).  The log from the Verifying step said 19 files totaling 20.4GB out of the 262.5GB copied were "remaining".  I don't know what "remaining" means, but one possibility is that these files are the ones (eliminating duplicates) that failed verification during the original Backup runs during the week.  An Advanced Search of the Forums shows that administrators—especially Don Lee—have been struggling with the "remaining" problem for many years, prompting one series of response posts by the head of Retrospect T. S. in 2009.  I'm not going to worry too much about "remaining" files.  Note, however, that the copying phase of Copy Media Set took 3.0 hours, and the Verifying step 2.3 hours—which results from the verification being byte-by-byte because both "Media Set Blue" and "Media Set Pink" are on drives (USB3 for the first, USB2 for the second) locally attached to my "backup server".

P.S.: I've submitted a Support Case for the "remaining" question raised in the second substantive paragraph.  I said that for Copy Media Set either this is a long-standing bug, or a long-standing documentation deficiency.

Edited April 7, 2018 by DavidHertzberg
Added P.S.

[Andrew Kinnear]

David, I must clarify that I'm in Europe and I believe that we have a separate support centre to you in US. The gentleman I spoke to was a European multilingual and I felt that English wasn't his first language. He was extremely helpful, but our conversation was quite difficult and combined with my lack of knowledge it's very possible that I've misinterpreted what he was telling me, hence my question here. The subject of the "media set copy" script was also peripheral to our main conversation which concerned catalogue rebuilding. He seemed a little unsure of himself and was keen to escalate the issue to an engineer. That's not a criticism, but what I would consider normal practise for a European software support call centre and I assumed from this that he wasn't a senior support engineer. That was just my impression, I may be incorrect. 

A learning experience for me not to post unreliable information on the forum and I apologise if it's has caused you stress and wasted time. I hope from your tests that you've allayed your anxiety. I'm working through the documentation and anything else I can find before performing any operations and will test throughly, as you have, before going ahead. Apologies again and thank you for your help with this.

[DavidHertzberg]

No problem, Andrew Kinnear; dealing with unreliable Retrospect information is what we Forums volunteers are here for.  Setting up my Copy Media Set test only took me about 15 minutes, and it ran while I was either asleep or out to dinner.  The "remaining" log message interested me; it now appears from a 2014 Forums thread to be a legitimate warning that sometimes appears when what it is warning about is obscure.  I've added an Additional Note to my Support Case both briefly explaining your experience and linking to the 2014 thread.

The root of the European T.S.  person's problem is a situation with the Retrospect User's Guides that is alluded to (it used to be explained more fully, before other Wikipedia editors made DovidBenAvraham pare down the article) in this section of the Wikipedia article.  The circumstances following the introduction of Retrospect Mac 8 in early 2009, including both the impossibility of duplicating the Console in Retrospect Windows and the evident confusion over the change in terminology in Retrospect Mac, left Roxio/Retrospect Inc. selling "two variants of backup server software that, while having nearly identical non-GUI code, operate differently".  Retrospect Inc. decided to leave the old terminology in Retrospect Windows and, when Retrospect Windows 8  (with underlying code corresponding to Retrospect Mac 10) was released in late 2012, to more than double the size of the Retrospect Windows User's Guide—which IMHO was a response to their discovery that nobody ever went broke underestimating the intelligence of some Retrospect administrators (apologies to H. L. Mencken for adapting his phrase that originally referred to the American public).  Retrospect Inc. has since dealt with the problem of being saddled with a 250-page plus a 650-page User's Guide by not properly updating either UG variant, plus making the UGs worse in later versions.  I pity any T.S. person who has to deal with this situation.  According to the head of Retrospect T. S., the management of Retrospect Inc. has further dealt with the situation by omitting any representative of his department from the august Documentation Committee!