Media Sets and Members

[moomin68]

Hi, thanks for taking the time to read my post.

 

I have already read through recent posts extensively regarding my question, so I apologize if it sounds like I'm repeating something.

 

Can somebody please explain what "Media Sets" and "Members" actually are?

 

Feel free to make it sound like you are talking to a 5 year old -- it would really help.

 

I've been using Macs for 20 years. I'm mildly-to-moderately knowledgeable when it comes to systems and software. But I just don't get it.

 

I've read the documentation over and over again. I still don't get it. As part of my job, I write user manuals professionally -- so I am used to translating what clients (usually manufacturers or techies, or worse, both) THINK mades sense to the general public... into something that actually does make sense. But this is a mindbender.

 

Here's what I do understand -- my needs. I want to do two things:

 

1) Backup 2 desktops, and 2 laptops, in a way that can be easily restored in the event of system failure.

 

2) Backup all the files I keep on an external, onto a master-backup external.

 

The "Master-Backup" external is permanently hooked up to one of the desktops (firewire) and stores all the data.

 

I have item #2 running great -- a script copies files from the external, onto the 2TB master-external, once a day.

 

As I understand it -- my #1 requirement means I will not be saving direct copies of files, but instead a series of files that Retrospect will use to restore data if needed -- right? I don't know, the idiot-guide doesn't explain this.

 

Here's what I'm doing to backup, for example, my desktop computer. I hit "backup" and select the computer, selecting "all files". When I'm asked where I want the backup to be stored, I add the entire 2TB drive (no other option). I am then asked to "add a member" whatever the heck that means -- so I navigate to the folder I have set up on the 2TB for Retrospect Backups.

 

Am I doing this correctly? Will it keep copying more and more data to the 2TB until it's full?

 

Essentially what I'm trying to do is create a "clone", an exact snapshot of each computer, once a day.

 

Thanks for your patience reading this. Sorry if some of it sounds dumb, but like a lot of computer users, I don't know much, but just enough to be dangerous.

 

 

[rhwalker]

Can somebody please explain what "Media Sets" and "Members" actually are?

Have you seen page 8 of the Retrospect 8 "Getting Started Guide"?

Retrospect 8 Getting Started Guide

 

Media Sets are backup destinations made up of one or more pieces of media (disk, tape, optical, etc.).

Think of the Media Set as a "bucket of bits" (a database) into which Retrospect throws your backups. It contains the files and metadata (permissions, dates, etc.).

 

The "Members" of the Media Set are the "pieces" - the tapes, disks, etc., that make up the Media Set.

 

Here's what I do understand -- my needs. I want to do two things:

 

1) Backup 2 desktops, and 2 laptops, in a way that can be easily restored in the event of system failure.

 

2) Backup all the files I keep on an external, onto a master-backup external.

 

The "Master-Backup" external is permanently hooked up to one of the desktops (firewire) and stores all the data.

 

I have item #2 running great -- a script copies files from the external, onto the 2TB master-external, once a day.

No, you don't. A copy is not a backup. Backups are a history, and let you go back in time, etc., and have additional metadata. A copy is just a single-instance copy.

 

For example, when you copy (whether by Retrospect or the Finder, whatever) onto a volume, you rely on that volume's filesystem to preserve your metadata (timestamps, permissions, Mac OS Classic's "resource forks", etc.). If the destination filesystem is not of the same type as the source filesystem (e.g., Windows, NFS, SMB, Mac HFS, Mac HFS+, etc.), then the metadata might not be preserved correctly. Additionally, a copy doesn't preserve history.

 

Retrospect Media Sets (f/k/a/ Backup Sets, f/k/a/ Storage Sets) are a database, and preserve metadata regardless of the underlying filesystem capabilities.

 

Although this KnowledgeBase article is a bit dated (uses the term "Backup Set" rather than the newspeak term "Media Set", and "Duplicate" rather than the newspeak term "Copy"), it does explain things a bit:

What is the difference between Backup and Duplicate?

 

As I understand it -- my #1 requirement means I will not be saving direct copies of files, but instead a series of files that Retrospect will use to restore data if needed -- right? I don't know, the idiot-guide doesn't explain this.

No, not a series of files. Think of it as a database, a bucket of bits. metadata and files, whatever is needed to get back the file and its metadata, at a point from the past.

 

Here's what I'm doing to backup, for example, my desktop computer. I hit "backup" and select the computer, selecting "all files". When I'm asked where I want the backup to be stored, I add the entire 2TB drive (no other option). I am then asked to "add a member" whatever the heck that means -- so I navigate to the folder I have set up on the 2TB for Retrospect Backups.

 

Am I doing this correctly? Will it keep copying more and more data to the 2TB until it's full?

Yes and no. Retrospect will copy what it needs to restore what you had at the time that the backup session occurred.

 

You need to get your head around the Retrospect paradigm, which is different from the model presented to the user from other backup programs, and the terminology is important if you want to understand the program and the posts in these forums. Sadly, EMC marketing types (or whoever) keep changing the terminology, which is confusing.

 

Under the hood, Retrospect behaves more or less like other backup programs. It does an initial "full backup" in the first session, followed by "incremental" backups in successive sessions, only backing up things that changed. All of this data goes into the backup set (um, Media Set, to use the current politically-correct names). For simplicity, and taking advantage of the Retrospect "snapshot" paradigm, Retrospect calls all of these "Normal" backups, and "does the right thing" to preserve the "snapshot" illusion.

 

With other programs, in order to restore, you first restore the initial "full backup", followed by all incrementals up to the point where you want your data, leaving you with the data at that point.

 

Different from other backup strategies, Retrospect maintains a "catalog" into its database (um, Media Set) for each "snapshot" (backup session), and the catalog has pointers into the database that tell Retrospect where each file that it saw (when it started the backup session) is within the database. So, files that have never changed from the beginning will have those entries in the recent snapshots point into the initial full backup, whereas files that have changed will have those entries point into a recent incremental.

 

This allows Retrospect to present the illusion, when viewing a "snapshot", that you are looking at the files that were on the source at the time that the backup was made. Instead, you are looking at bits and pieces of the entire backup history.

 

More precisely, Retrospect will, in fact, let you view the files that were backed up in each "session", but it's more useful simply to use the Retrospect paradigm and look at "snapshots" of the source as they were taken in the various backup sessions.

 

Retrospect's win is on the Restore, where you can view the data in a natural manner (as it existed in an instant in time). But that's the illusion presented by Retrospect's "snapshots".

 

New with Retrospect 8 (on the Mac side) is "grooming", where you can set a policy for how many "snapshots" are preserved. Retrospect will then figure out what files it needs from the database (Media Set) to support the retained snapshots, mark the other files for deletion, and then compact the database to free up the files (backup history) that you don't want to preserve.

 

Again, think of Retrospect as a database program.

 

Clearer?

 

Essentially what I'm trying to do is create a "clone", an exact snapshot of each computer, once a day.

Two comments. First, a clone is not a backup. Reread the discussion above.

 

If what you want to do is to create a clone, there are better tools for that job, such as rsync (which has metadata issues), SuperDuper! (my preference), or Carbon Copy Cloner. Google is your friend, search the internet for CCC or SuperDuper.

 

Second, "snapshot" has a special meaning in Retrospect terminology. Don't use that word in connection with Retrospect unless you use it as Retrospect uses that term. See above.

 

There are no dumb questions. No one starts out knowing everything.

 

Hope that helps.

 

Russ

[moomin68]

Russ,

 

First -- thank you so much for taking the time to explain all this stuff to me.

 

Now I understand. And it leaves me wondering if Retrospect is actually the right software for what I'm trying to do.

 

I'm trying to do several things.

 

1) My computer has two separate 500gig internal drives. When I used Retrospect 6, I had it set to copy one drive to the other exactly, once a day (I used the term "clone" to myself, as it best describes what I'm trying to achieve... but it's hard to keep up with Retrospect's changeable language!) I can't tell you the number of times this saved me -- I simply boot from the other internal drive, fix whatever, and I'm good to go. Using Retrospect 8 however -- I notice it's no longer replacing every single file -- I guess it's only replacing the files that have changed.

 

2) I use a 2TB external drive for all my files. I have an identical 2TB next to it, that again used to "clone" itself every day (using Retrospect 6). So if the first one dies, an identical copy is ready to go. (I also have a 3rd 2TB, which rotates out once a week with the 2nd -- maybe overkill, but I had one "lost data" incident too many).

 

Now you have defined "backup" for me -- the only instance where I would use this, is to "backup" my computer onto the 2TB. I already have a direct copy on an internal hard drive, and this is just a second option, should that 2nd internal fail.

 

I am thinking that I need to use different software to make my daily "clones" of whatever items, and only use Retrospect 8 to "backup" our various computers. Seems like it's not so great for just making direct copies of files.

[rhwalker]

Russ,

 

First -- thank you so much for taking the time to explain all this stuff to me.

You are welcome.

 

I'm trying to do several things.

 

1) My computer has two separate 500gig internal drives. When I used Retrospect 6, I had it set to copy one drive to the other exactly, once a day (I used the term "clone" to myself, as it best describes what I'm trying to achieve... but it's hard to keep up with Retrospect's changeable language!) I can't tell you the number of times this saved me -- I simply boot from the other internal drive, fix whatever, and I'm good to go. Using Retrospect 8 however -- I notice it's no longer replacing every single file -- I guess it's only replacing the files that have changed.

Retrospect 8 does this like Retrospect 6 did. The terminology has changed, but it's still the same functionality. What happens depends on the various settings for the duplicate (um, copy), and you might not have had things set up right for Retrospect 6.

 

For this function, though, I believe that SuperDuper! (or Carbon Copy Cloner, just not what I prefer) would be better, or, as discussed below, a RAID 1 mirror split, which happens instantly.

 

Retrospect's strength is in preserving the historical record. It's a backup program, not a cloning program, although, by "feature creep", cloning and copying have been added over the years.

 

As you might imagine, preservation of the historical record is very important if someone accidentally deletes a file, or if a "mystery garbage block from space" damages a file and the damage is not discovered until the file is accessed 10 years later, as can happen in a situation like ours (law firm, where patent cases last 20 years and trademark cases can last forever), or if you need to see the version history on a file.

 

For example, consider the case where your accounting data becomes suspect (been there, done that, when our former bookkeeper embezzled). You need to be able to examine the state of files at various instants in the past, perhaps going back over several years with thousands of revisions.

 

Your cloning technique won't solve that problem, but Retrospect can.

 

2) I use a 2TB external drive for all my files. I have an identical 2TB next to it, that again used to "clone" itself every day (using Retrospect 6). So if the first one dies, an identical copy is ready to go. (I also have a 3rd 2TB, which rotates out once a week with the 2nd -- maybe overkill, but I had one "lost data" incident too many).

For that problem, consider instead using a RAID 1 mirror. If you use SoftRAID, which is a much better product than Apple's RAID on the Disk Utility program, you can add and remove volumes from the mirror at will, and the mirror can rebuild in the background when new volumes are added to the mirror. Strongly recommended, excellent support, very reasonable price.

 

In fact, for years, a RAID 1 mirror split has been the only (and standard) way to clone a running server in order to take a bootable clone offsite (or immediately before a software update, in case things go horribly bad, so you can be back up with a reboot), and the "clone" process happens in an instant when you split the mirror. There are minor "gotchas" that you need to watch out for (turn off some services, such as email, that use databases that need a consistent on-disk database, etc.), but that's the way it is done.

 

But it's not backup, because any deleted file on one member of the RAID 1 mirror becomes deleted on all.

 

Now you have defined "backup" for me -- the only instance where I would use this, is to "backup" my computer onto the 2TB. I already have a direct copy on an internal hard drive, and this is just a second option, should that 2nd internal fail.

No, backup is not just a "second option" because it preserves the historical record, and files that are accidentally deleted are still preserved for restoration.

 

Think of "backup" as preserving individual files and their history, in case you have to go back. It's not how you come back up quickly from a catastrophe - that's a different problem, with a different solution.

 

I suggest, now that you are thinking about the problem, that you take what should have been your very first step, namely, to decide what your backup policy should be, based on your needs. The backup policy for an ISP, where much stuff is transient, will be very different than the backup policy for a law firm or a credit card transaction processing company, etc. There is no such thing as a "one size fits all" backup policy.

 

Here is a good set of issues for you to consider: What should a good backup policy address?

 

I am thinking that I need to use different software to make my daily "clones" of whatever items, and only use Retrospect 8 to "backup" our various computers. Seems like it's not so great for just making direct copies of files.

It's just a matter of using the right tool for the job. Retrospect is a good backup program, but you seem to have needs that are better addressed by other tools. This is not a criticism of Retrospect.

 

Russ

[moomin68]

Thanks again for all the tips Russ, very helpful. We are going to check out Super Duper, it sounds like possibly a better solution for most of our needs. I could not get the "good backup policy" link to work BTW.

 

Could I ask one more question -- and I will use Retrospect-Speak properly this time, and try to be clear :-)

 

I have a script set to "copy" one external to the other. My intent is to make a direct copy, exactly the same as the source drive.

 

With Retrospect 6, the script worked great. Now I am using 8, it is "leaving behind" files on the destination drive. If I create a folder on the source drive, and do a "copy" it appears on the destination drive. But if I delete the folder... and create another folder with a different name... and do another copy... both folders show up on the destination drive.

 

I am guessing it's just a box I have not checked somewhere as to the copying preferences, is this the case?

 

Thanks. Incidentally, we do Branding and Packaging Design, and sometimes work with with TM attorneys. I can see how in your line of work "backups" could be essential -- for us, the need to have snapshots back in time has never arisen, only direct copies of all our stuff on at least 3 big drives so one gets taken off site, or on the road, etc.

 

 

[CallMeDave]

I am guessing it's just a box I have not checked somewhere as to the copying preferences, is this the case?

 

 

Yes.

 

 

 

[rhwalker]

I could not get the "good backup policy" link to work BTW.

Hmmm... tinyurl.com seems to be up and down. You might try again.

 

The link is an odd one, and some browsers don't like it, which is why I provide the tinyurl link.

 

Here is the actual link:

http://iwiring.net/#[[What%20Should%20a%20Good%20Backup%20Policy%20Address%3F]]

 

As to your other question, Dave's answer is correct (of course).

 

Russ

[moomin68]

I am guessing it's just a box I have not checked somewhere as to the copying preferences, is this the case?

 

 

Yes.

 

 

 

Thanks... can I ask which box? I looked and cannot figure it out. Amateurs... tsk.

[rhwalker]

Thanks... can I ask which box?

Perhaps you can.

 

I looked and cannot figure it out. Amateurs... tsk.

Oh, perhaps you want the solution, not just the answer to your question.

 

See page 147 of the Retrospect 8 Users Guide:

Retrospect 8 Users Guide

 

Russ

[moomin68]

Yes, the solution is very helpful, I did not know where to look in the manual -- sorry if I was not clear.

 

I am looking for a preference, or something, I don't know what -- that will make Retrospect override a scheduled Shutdown.

 

With R6 -- if Retrospect was still copying the computer would not shut down. With R8 -- it shuts down, and does not resume the copy on startup.

 

I noticed SuperDuper has the option to tell the computer what to do after backup -- I like that.

 

Sorry if I am posting in wrong thread now and should start another, and sorry if I am violating any other faux pas as regards terminology, or incorrect question/solution request format. Hopefully that global apology covers it all in advance :-)

Edited April 6, 2010 by Guest

[rhwalker]

Yes, the solution is very helpful, I did not know where to look in the manual -- sorry if I was not clear.

You missed my humor, sorry that it was too subtle.

 

I am looking for a preference, or something, I don't know what -- that will make Retrospect override a scheduled Shutdown.

 

With R6 -- if Retrospect was still copying the computer would not shut down. With R8 -- it shuts down, and does not resume the copy on startup.

Even worse, if the shutdown happens during some critical operations (grooming, for example, or during the small window where the preferences file is updated), your preferences can get corrupted and you will lose all of your schedules, sources, destinations, license information, etc.

 

It's a bug, in my opinion, and your suggestion sounds like a reasonable feature to add. You might make that suggestion in the

Mac OS X - Product Suggestions forum. Good luck, I've never had a feature suggestion of mine adopted. I believe that the developers are focusing on bug fixes and stability rather than feature enhancements right now.

 

Sorry if I am posting in wrong thread now and should start another, and sorry if I am violating any other faux pas as regards terminology, or incorrect question/solution request format. Hopefully that global apology covers it all in advance :-)

I would suggest that you start a new thread with questions unrelated to this thread. It's a bit confusing when threads wander off topic, and makes it harder for others to research solutions to their problems.

 

Remember, these user-to-user forums are not just for you, but are to help others with the same problem in the future. It's really not an issue of "correct question/solution format", or faux pas, etc. Put yourself in the position of someone trying to research the solution to their same problem, months in the future. That's one of the reasons that we request full configuration information, so that others in the future can tell if the solution is applicable to them (i.e., you might have found a bug or missing feature that is solved in a future release).

 

Regards,

 

Russ

[moomin68]

Aha. I did consider that I might very well be dealing with humor, but decided to err on the side of politeness and caution. I'm from England. We do our humor deadpan, straightfaced, and subtle. Here in the USA I'm often misunderstood as serious when I'm not. So I'm careful to include the drum roll/cymbal crash often required by the American audience to announce humor, and to apologize even when unnecessary. Brits say "sorry" and "thank you" a lot anyway, for everything, it gets tiresome.

 

You missed my humor in the "correct question/answer... faux pas"... comment. The smiley is my equivalent of the drum roll. Ah well. 10 years in the USA and I'm still learning the language.

 

I appreciate the tip about using the forum, I'm kinda new to this and will post a new topic if any more unrelated questions. Thanks for all your help Russ! (American exclaimation point, denotes excitement and emphasis on point, not widely used as such in the UK). Seriously... you've been awesome... and helped me get most of the way toward figuring out a decent backup strategy that's right for me -- I'm grateful. Cheers mate.

[CallMeDave]

sorry that it was too subtle

 

Well, there's always the _other_ explanation:

 

 

 

 

 

 

that it was not funny...

[moomin68]

Well, there's always the _other_ explanation:

that it was not funny...

 

I thought this forum was for people to help each other out, not poke holes in different attitudes toward humor?

[CallMeDave]

Gotcha.