Is a Duplicate and exact duplicate?

[Norm]

I ask that question for several reasons:

 

1. I'm about to upgrade to Tiger and I may use the Erase and Install option. If so, I was going to use the Migration Assistant to transfer my files from my external FW drive which has a Retrospect Duplicate of my internal drive.

 

2. If I ever have a major crash of my internal hard drive, my plan was to install a new hard drive and the Duplicate the Duplicate back to the new internal hard drive and then Restore the User folder which I backup with Retrospect daily.

 

3. When I have booted from the Retrospect Duplicate on my external FW partition, it does some things that makes me wonder if it is a duplicate. For example, the last time I did this my new version of Now Up to Date applications did not show in the Application folder of the Duplicate nor on my internal hard drive and the new menu icons for Now did not appear in the Apple menu bar. Also, I could not find some newer desktop items on my internal hard drive when booted from the Duplicate and opening my internal hard drive.

 

Appreciate any input. Thank you.

[Lennart_T]

The answer is "Yes".

 

Did you get any error during the duplicate? Checked the log?

[Norm]

No problem in log but it was a Duplicate that had been updated several times.

 

So I ran a fresh Duplicate and again some problems. All the problems I've found so far have been Now Software related.

 

Here is the latest issue after booting from a Duplicate on an external FW drive:

 

All seemed fine except one of the first things I noticed when testing a couple of apps was that Now Up To Date's views had all changed the font selected. And when I went to look at the font that it changed to, American Typewriter, I noticed the font I had it set at, Palatino, was not on the list of fonts. As a matter of fact quite a few fonts were not available in Now Up To Date. So something had changed in the Duplicate. Or is it a issue in running the Duplicate from the external FW drive?

 

Thanks for any tips.

[natew]

Hi

 

I'm curious:

If you use carbon copy cloner to do the copy do you still have the same problem? That would rule out a problem in Retrospect.

 

Thanks

Nate

[Norm]

I too am curious how a Retrospect Duplicate compares to a CCC clone or one by SuperDuper!.

 

If someone knows of any postings of such analysis, I would be very interested.

 

It will be a long time before I get around to doing that.

 

Thanks.

[rhwalker]

Sigh. This is really a much more complex problem that everyone seems to realize. I'm not going to write a treatise here, but I will at least point out the issues, and leave contemplation and investigation for the reader. It's really not a problem that Retrospect or CCC or any other solution can solve on their own.

 

(1) Unix is a modern operating system, and there's a lot going on in the background, especially on a server.

(2) Solutions like Retrospect and CCC, while they operate differently, have the similar design that they "scan" the drive(s) to decide which files to back up, then do the backup/cloning. Files will independently disappear and be created in the background as this scanning and backing up is going on, so there is no way that the "copy" can or will match the original.

(3) Some files MUST NOT or should not be duplicated - the backing store (swap), lots of stuff in /tmp, etc. Mac OS X has gotten better about deleting some of these files if it finds them on reboot, because a similar situation happens in a crash - the droppings are left that must be cleaned up on reboot. There's a discussion of what files CCC **DOES NOT** backup (for this reason) on Mike Bombich's web site, and Retrospect may have similar rules.

(4) Particularly on a server, there are some databases that track certain data files, and database corruption will (not merely might) result unless the database and the files are a "snapshot" taken at a single consistent moment when such a snapshot is possible. An example is the Cyrus mail database in Mac OS X server 10.4.x; I guarantee you that you will get a corrupted mail database using either CCC or Retrospect to clone/backup your server if you boot from the clone/backup, if the clone/backup was made while the mail system is running.

(5) The ONLY way I know to make an "exact" copy of a Mac OS X system (or any system) is to use RAID software that permits you to create a RAID 1 mirror, adding a mirror to the RAID 1 set, allowing the mirror to sync, then to "break" the mirror such that the split-off mirror copy is a duplicate of the original at the instant of the split. Apple's RAID software does not permit this. SoftRAID 3.x does.

(6) Even using a "split" of a RAID 1 mirror is not "perfect", even though it is a perfect instantaneous copy snapshot of a running system, because of the issue of programs running in the background - there is no communication between those programs and SoftRAID (if that's what you use) as to when the "right" moment is to make the mirror split.

 

So what can you do? One option, the safest but most inconvenient, is to shut down your system, boot from another disk so that the disk to be cloned/backed up is not in use, clone/backup that disk, then reboot to bring your system back online. Not an attractive option.

 

Another option, not mutually exclusive with the above, is to separate your data onto two "volumes"/"disks"/"partitions" (they actually mean different things, and I'm not going to digress on that right now), with the OS on one and all changeable user data on another, so that you can freely back up the user data, home directories, etc., as files (whether incremental or clones), and separately, on a lower priority, back up the OS stuff, which is not as changeable. This also makes it easier if you decide to blow away the OS install and do a clean install (you do keep a log of all changes to your system, don't you?). AFP548.com (which is a useful site for Mac OS X Server admins) has scripts to be run by cron that will, for example, shut down your mail system, back it up to another directory, then restart the mail system. You could then use Retrospect to back up the backed up and consistent snapshot of your mail system, etc. There are also decisions to be made as to the "right" place to put your network home directories, your mail system storage, etc., and these decisions are beyond the scope of this discussion.

 

Another option, not mutually exclusive with the above, is to set up Retrospect scripts and selectors on your server to, in sequence:

(a) walk the network, back up all networked clients and shut them down. This makes your mounted filesystems on your server quiet at the time that Retrospect backs them up.

( use a "trigger script" in the Retrospect event handler (I suggest using the Python suite so as to isolate you from "updates" to the various mail clients) to shut down the mail service on your server, duplicate the mail subsystem (or run the appropriate AFP548.com mail backup script), then restart the mail subsystem.

© backup your server, with appropriate selectors so you don't back up the live mail system, but instead back up the cloned/duplicated mail backup from step (, above.

 

As with the mail subsystem, there are similar concerns with your LDAP / AD / OD subsystems. I leave that as an exercise for the reader, but there are similar backup scripts and strategies given on AFP548.com for these subsystems.

 

Another option, using the RAID 1 "split mirror" approach above, is to bring a RAID 1 mirror online, allow the mirrors to sync, shut down the mail service (and I would suggest also shutting off the Retrospect schedule, for reasons discussed below), split the mirror (so that you've got a perfect clone snapshot on a quiet system), bring the mail system back online and turn back on the Retrospect schedule. The reason that the Retrospect schedule needs to be shut off is so that you don't get a surprise when booting the cloned drive weeks later during an emergency, because all scripts will now be runnable due to the passage of time, and you wil get an immediate launch of Retrospect right at your moment of crisis boot from your backup clone whle you are trying to figure out what messed up your system and caused you to fall back on your emergency clone.

 

The point of all this is that Retrospect and CCC and mirror splitting are all complementary and solve different problems; no one solution can solve it all. You need to develop a backup policy to evaluate your needs before you start implementing. Retrospect can do a great job backing up files and allowing you to go back to data that is long gone. CCC can do a great job on a perfectly quiet system, making a copy before a major OS update, etc. And a RAID 1 "mirror split" can be used to keep a server up 24/7 and to get a perfect instantaneous snapshot of a running system, which, if done carefully (see above), can be valuable in the event of crash.

 

Hope this helps.

 

Russ

[Norm]

Quote:

---

Sigh. This is really a much more complex problem that everyone seems to realize. I'm not going to write a treatise here, but I will at least point out the issues, and leave contemplation and investigation for the reader. It's really not a problem that Retrospect or CCC or any other solution can solve on their own.

 

<snip>

 

This also makes it easier if you decide to blow away the OS install and do a clean install (you do keep a log of all changes to your system, don't you?).

 

<snip>

 

The point of all this is that Retrospect and CCC and mirror splitting are all complementary and solve different problems; no one solution can solve it all. You need to develop a backup policy to evaluate your needs before you start implementing. Retrospect can do a great job backing up files and allowing you to go back to data that is long gone. CCC can do a great job on a perfectly quiet system, making a copy before a major OS update, etc. And a RAID 1 "mirror split" can be used to keep a server up 24/7 and to get a perfect instantaneous snapshot of a running system, which, if done carefully (see above), can be valuable in the event of crash.

 

Hope this helps.

 

Russ

 

---

 

 

Thank you very, very much. I really appreciate.

 

I will quickly admit that you lost me during some/much of your explanation. But importantly for me, I got an education on the challenges plus some very helpful tips. Also, I now know the answer.... most duplicates aren't "exact" duplicates.

 

Given what you've said I'm going to try to change the way I duplicate my current system before proceeding with an Erase and Install of Tiger since I will use the duplicate to transfer files. I have two external FW drives so, if I understand you correctly, I will now boot from one and Duplicate my OS to the other using Retrospect Duplicate. I gather that will be better than Duplicating while booting from my OS.

 

BTW, what do you mean by "(you do keep a log of all changes to your system, don't you?)"? Since I'm asking, I think the answer is "no." :-(

 

Going forward your info will help me reevaluate my backup strategy. I backup my OS (whole drive incrementally) daily and I periodically do a Duplicate but from my OS. I think I need to change that if I'm going to prepare for a total crash of my OS drive and I want to be able to reconstruct my OS as best as practically can.

 

Thanks again very much.

[rhwalker]

Quote:

---

BTW, what do you mean by "(you do keep a log of all changes to your system, don't you?)"?

 

---

Just that. You could drop dead unexpectedly tonight (my wife did that a few years back). Who would be able to figure out what you have done on your server to carry things forward? And I don't know about you, but I'm beginning to have "senior moments". So we've got a log book for our server and for each of our networked clients, and each time a change is made (software installed, configurations changed, etc.), it's logged. That way, when something fails, you can go back and figure out what the cause was, or how to come forward from a given backup snapshot. Sorry that some of what I wrote wasn't clear; it wasn't intentional on my part. I was just trying to briefly show some of the issues that I have discovered, to save you some pain on one of those crisis days when you try to do a "bare metal" recovery (as from a fire or catastrophic hardware failure). The stress is high enough at that moment, so it's better have planned things in advance so that you can be the rock in the center of the storm. Regards, Russ

[Norm]

I would agree, more "senior moments" each day.

 

Now I understand about the logging. Thanks. I thought that you might have been referring to some report/log by OS X system that one could find if they knew how.

 

No, what you wrote was in fact clear. I just don't have some of the computer knowledge to understand all of it.

 

And your tips will help me evaluate my backup plan. Trying to have the backups/duplicates that make those moments as stress free as possible.

 

Thanks for the help.

[Norm]

Russ:

 

Do any of your reasons that duplicates are not exact copies provide you a clue as to why I would get the following Now software font issue regardless of how I create the duplicate. Granted all of my duplicates are with Retrospect.

 

The symptoms I reported above are:

 

One of the first things I noticed when testing a couple of apps was that Now Up To Date's views had all changed the font selected. And when I went to look at the font that it changed to, American Typewriter, I noticed the font I had it set at, Palatino, was not on the list of fonts. As a matter of fact quite a few fonts were not available in Now Up To Date. So something had changed in the Duplicate. Or is it a issue in running the Duplicate from the external FW drive?

 

Thanks for any ideas.

[rhwalker]

I'll have to confess I have never run Now Up To Date, and I've never seen this problem. A couple of questions I will ask, though, that might point you in the correct direction:

 

(1) Where is the font that was "forgotten"? There are quite a few places that fonts can be: The System font folder (/Library/Fonts); Your (or some other user's) home directory private fonts (~/Library/Fonts); The OS 9 fonts (/System\ Folder/Fonts). Using Terminal, try "locate Palatino". You might have more than one Palatino.

 

(2) Since you used Retrospect Duplicate, it may be a selector issue (for some odd reason, you may not have selected the Palatino font).

 

(3) I seem to recall some font issues with ancient fonts that depended on the order in which they were added to the Font folder in OS 9, because the fonts were assigned a "font number" that some older programs would use rather than the name. The way I handled this back in OS 7/8/9 was to lock all fonts in the Fonts folder so that the system couldn't change their font number. Is your Now Up To Date an old program, perhaps a Classic program? Doubtful that this is your issue unless you are dealing with an old (non- OS X) program.

 

The closest thing that I have seen to this (and it's not a Retrospect problem, but was a Microsoft problem) was that. a few years back on OS 7/8/9 and Microsoft Word 5.1a (which I haven't run in a while), a "bare metal restore" by Retrospect would cause Word 5.1a to forget the locations of the gazillion documents that I had placed in the "Work" menu because Word 5.1a remembered them not by path, but by location on the disk, which changed when the bare metal restore was done. (the Apple FS equivalent of Unix inode #). How helpful of Microsoft. You may be seeing a similar thing here. I can't imagine how it would be related to running a Duplicate from an external FW drive.

 

However, when I re-read your original first post, I see that you planned to use this duplicate with Migration Assistant on a Tiger upgrade to move files up from the duplicate. If that is what has happened (Tiger upgrade, then Migration Assistant of some files or manual moving of files/fonts), you might have ended up with a different "Palatino" as a result of the Tiger upgrade, which might somehow be related. Are you using some sort of an add-on font management utility that could be complicating things here?

 

To be candid, the only ways that I have "duplicated" a disk (the only ways I would feel comfortable doing so) are asr, Disk Utility (which uses asr as a back end), CCC, and a RAID 1 mirror split. So I've never used the approach that you are using.

 

Got me stumped, or else I'm just having another senior moment.

 

Regards, wish I could be more help on this. Russ

[CallMeDave]

Quote:

---

the only ways that I have "duplicated" a disk (the only ways I would feel comfortable doing so) are asr, Disk Utility (which uses asr as a back end), CCC, and a RAID 1 mirror split.

 

---

 

If you're willing/able to boot from a different volume, a Retrospect Duplicate from one non-boot volume to another non-boot volume works fine.

 

As you note above, working with a live disk will always make things more complicated.

 

Dave

[Norm]

Russ, thanks for those thoughts.

 

The good for me since my last post is that I have upgraded to Tiger using Migration Assistant and Now transferred with all the fonts that were missing and with the proper one selected. Go figure as they say.

 

I'm not sure where the fonts are located. I've never installed any fonts so I assume Now was using Apple fonts but I'm certainly no expert as my posts indicate.

 

BTW, why do you say those utilities are the only way you would duplicate/clone a drive? In other words, you would not duplicate using Retrospect's Duplicate function as I've been doing in these cases?

 

Appreciate the help.

[rhwalker]

Quote:

---

BTW, why do you say those utilities are the only way you would duplicate/clone a drive? In other words, you would not duplicate using Retrospect's Duplicate function as I've been doing in these cases?

 

---

Simply for the reasons Dave gave above. If I'm going to duplicate (clone) a drive prior to some big update event, I take the drive offline during scheduled maintenance and use the tools I listed above. I do use Retrospect's Duplicate function to make scripted copies of subvolumes as a part of normal operation. I guess it's simply a question of using the right tool for the specific job; Retrospect is good for file recovery and archive. Glad you are now up and running after the migration, hoped this opened your eyes a bit about the difficulty of backup. Russ

[Norm]

Thanks for the additional help.

 

It has opened my eyes. But they are somewhat cloudy as to whether maintaining Retrospect duplicates is a valid way of replacing a crashed drive. Especially if those duplicates have been added to/modified over several Retrospect sessions. I'll ponder that a bit more.

 

I appreciate the help and education.

[rhwalker]

One other quick comment on the subject of "bare metal restore", related to the above. If you are not going to boot the "bare metal restore" disk on the same computer on which it was made, additional complexities are introduced. For example, perhaps you are bringing up a different model computer after retirement / destruction of the old one on which the backup was made. Your backup OS might not support the new computer; also, Mac OS X creates cache files to enable the OS to boot faster, and the cached files might not be appropriate for the new "bare metal restore" computer. A similar thing is seen when doing an install in TDM (target disk mode), when the install TDM disk is on a different model than the installing computer (e.g., a TDM install of Mac OS X Server for an Xserve, if done on a non-Xserve, doesn't enable the necessary monitors for the fans, etc.). It's for this reason that we have the OS LUN (volume) separate from the user data so that we can do the bare metal restore from install disks and then use Retrospect to update the configurable data. Dantz/EMC require the restore target disk to be of the same OS level for similar reasons, but this still doesn't address the cache issue. Mac OS X has gotten better about handling these situations, but one of the "cache cleaner" programs can be useful in such a situation if a bare metal restore doesn't work. Just another complexity to make your head hurt.

 

Regards, Russ

[Norm]

Russ:

 

I'm not even sure if this senior knows what a "bare metal restore" is but I like to learn and am curious so thank you for your posted response. I'm learning more.

 

I think along the lines of your post.....

 

I do know that I "duplicated" my wife's PB (one drive not partitioned with OS X 10.3.9) with Retrospect over our LAN to a FW drive connected to my computer. Then when I upgraded her computer to Tiger using Erase and Install and Migration Assistant, I connected my external drive to her PB. I quickly realized, or maybe I got a dialog warning (past my memory horizon), that when connected to my wife's PB that drive's ownership selection had changed to Ignore Ownership. Not really understanding those things, I un-checked that, tested booting and that was OK, and then proceeded to use that drive in the Upgrade process. All went well. At least after two days I can say that.

 

Thanks again for all this education. But with more comes more questions and more head swimming.

[rhwalker]

"Bare metal restore" is the most difficult bootstrap situation - you've got bare metal (computer, disk) but no software whatsoever, no way to boot. How to go from there to having what you had before. The Retrospect manual describes how to do that (perhaps not using that well-known term), essentially a double restore - first install an OS, configure enough to run Retrospect and access your devices, then restore on top of that. Contrast that to the situation where you've already got a running system, and you can use that to do the restore of parts.

 

In the case of your wife's PB, if you look at the steps that you did (Retrospect duplicate, then Erase and Install of Tiger and Migration Assistant), the "Erase and Install" of Tiger finessed any issues by completely replacing the OS. That's why you didn't see any issues; you ended up with a running bootable system, and then merged files into it. BTW, you can't "ignore ownership" on your live boot drive; unix won't work that way. I haven't used Migration Assistant - we migrated forward from a fire that destroyed all our computers, and we did have to do bare metal restores from off-site backups onto new computers. But I suspect that Migration Assistant does have to ignore ownership on the connected "migrate from" external drive so that it can read everything to do the migration, and also so that it can finesse the different unix UIDs that might get assigned for different systems in a migration from an old computer to a new one (e.g., "Norm" might not be the same UID - "User ID" - on the old computer as on a new one; unix ownership is by UID, not by name).

 

Glad everything is working well for you now.

 

Russ

[Guest]

Wow, great explanation / analysis! Thanks!

 

You mention another solution to getting a 'perfect' copy of a filesystem that involves taking the system down, booting from some alternative volume, then mounting the filesystem read-only and cloning it. That would seem to get you a good clone of the filesystem in a quiescent, ready-to-boot state. Of course, it doesn't interact well with 24x7 availability, and I'm not sure how I'd script it. Any idea how to make this less painful?

 

 

My sympathies and condolences on the sudden death of your wife, which you mention in a subsequent posting.

[rhwalker]

Quote:

---

You mention another solution to getting a 'perfect' copy of a filesystem that involves taking the system down, booting from some alternative volume, then mounting the filesystem read-only and cloning it. That would seem to get you a good clone of the filesystem in a quiescent, ready-to-boot state. Of course, it doesn't interact well with 24x7 availability, and I'm not sure how I'd script it. Any idea how to make this less painful?

 

---

Not that I know of; that's why I only do it during scheduled maintenance before an OS upgrade. But the RAID 1 mirror split with active services off is a pretty neat trick that is just as good with only momentary unavailability when you can't take the server down. If it's not clear, I think that SoftRAID briefly discusses this trick on their web site (SoftRAID Scenarios) - see Offsite Backup and Server Archive). SoftRAID is in the process of adding scriptability to their product so that this mirror split can be automated with services stop/start. But it's a different and complementary scheme from what Retrospect's tape backup provides, and I wouldn't forego one for the other.

 

Thanks for your condolences about my wife. She was a fine person.

 

Russ