Using Retrospect with Catalina volume structures

[Nigel Smith]

19 hours ago, amkassir said:

How is it that Retrospect costs so much more than these but can't manage a streamlined restore?

Because it works in a different way to the others you list, so is much more impacted by Apple's ever-moving security goal posts. That "different way" also gives it many of the advantages David lists above.

Horses for courses. RS can quite easily do a "bare metal restore" if you can mount the device to be recovered to in target disk mode, or recover to an external drive then use Migration Assistant -- obviously not options for everyone, which is why I'd recommend a combination of RS plus something else for people in that situation.

[amkassir]

On 5/3/2020 at 5:18 PM, DavidHertzberg said:

As for your "How is it that Retrospect costs so much more than these but can't manage a streamlined restore?" question, Retrospect "Professional" has been targeted since 1990 at customers with the following requirements:

• Must back up multiple machines to a destination more compact than the same number and capacity of disk drives, which in many cases still means to tape.
• Must store at least copies of the backups off-site, because of the danger of fire or flood (a favorite location for organizations' on-site server rooms has been in the basement underneath the cafeteria—so use your knowledge of cafeterias to imagine what can happen) or strikes at the worksite.
• Must allow an employee to easily restore an obsolete copy of one or more individual files, not just the entire contents of a machine's disk drive.
• Must allow recovery from a ransomware attack that may not reveal itself until several days after the drives of one or more machines have been poisoned.
•  Must prevent an individual employee disabling backup of his/her machine to speed processing—not easy to do with a "push" backup application.

Thank you to David and Nigel for your posts.

I don't think I mentioned this before, but I'm running Retrospect Desktop, not Professional, and I'm only backing up one machine although I do several different types of backups with different sources and destinations on that machine.

Looking at your list of user requirements, I believe CCC allows users to restore obsolete copies of individual files, as long as your source and destination are formatted as APFS volumes and you have snapshots enabled. In this case, CCC does a kind of incremental backup, saving snapshots of the source, as long as destination space permits. You can peek inside of earlier snapshots to pick out individual files for restore. Similarly, you can restore the entire volume from an earlier snapshot in case of a ransomware attack. SuperDuper also uses snapshots to store earlier backups but (a) it's not as easy to access the earlier snapshots and (b) I don't know if you can browse them and pick out individual files like you can with CCC. (I used to use SuperDuper, but because Shirt Pocket Software took a long time to make it compatible with Catalina, I began using CCC and stuck with it because IMHO CCC is superior to SuperDuper.

I make these observations in case a reader is considering options for cloning software to use in addition to Retrospect. I have been using Retrospect since 1990 (was that Retrospect 6?) and have never stopped using it, and don't expect to. In fact, I have nearly 20 years of medical records stored in Retrospect's proprietary file backup format, so if it ever comes up, I can show that there is no possibility of being able to illegally modify medical records after the fact. I continue to rely on Retrospect for this perhaps unusual application. Off the top of my head, I'm not aware of any other backup software that employs similarly unmodifiable dated backups.

While not truly off-site, I backup to both a local hard drive and a removable SSD, and carry the SSD with me in my briefcase when I leave the office.

Thanks again for your posts. I find them illuminating! 👍

Edited May 5, 2020 by amkassir

[DavidHertzberg]

amkassir and Nigel Smith and others,

I had phoned the head of North America Sales, and he just phoned me back.  He checked within Retrospect "Inc.", and was told that there was ""a ton of coding" done in 17.0.1 to support the additions to pages 154-156 in the latest revision of the Retrospect Mac 17 User's Guide.  So what I wrote in the third paragraph of this up-thread post was wrong—I've now corrected it, even though my bare-metal Restore of a High Sierra MacBook Pro slightly more than a year ago seemingly was a klunkier version of the same procedure.  Maybe Apple made a bare-metal Restore substantially more difficult with Catalina and Mojave, and the engineers had to revise Retrospect to compensate for that.

In regard to CCC, amkassir, here's "Leveraging Snapshots on APFS Volumes", and here's "Everything you need to know about Carbon Copy Cloner and APFS".  My understanding of these two documents is that CCC simply creates a snapshot on an APFS-formatted source volume, and uses the contents of that snapshot as the source.  If you're doing a bare-metal Restore, that snapshot itself is not available—only the copy of the snapshotted volume that CCC made on the destination volume.  If the source volume was poisoned by ransomware before the snapshot was made, then you can only restore a good copy of the source volume if you had CCC also save a the contents of a previous snapshot.  That would require a destination volume with—if the source volume is nearly full—twice the capacity of the source volume.  So my "Must back up multiple machines to a destination more compact than the same number and capacity [my emphasis] of disk drives" requirement still argues in favor of also using Retrospect.

[Nigel Smith]

12 hours ago, DavidHertzberg said:

If you're doing a bare-metal Restore, that snapshot itself is not available—only the copy of the snapshotted volume that CCC made on the destination volume.  If the source volume was poisoned by ransomware before the snapshot was made, then you can only restore a good copy of the source volume if you had CCC also save a the contents of a previous snapshot.

Not quite right, IMO. I believe the first sentence is true, but the second is only correct if you have to you have to erase and restore that volume -- and the point of snapshots is that you can roll back the volume to the earlier state without erasing it.

Bear in mind that eg WannaCry can (and will) delete volume shadow copies on Windows when user access to VSSadmin.exe hasn't been disabled -- it was, and I believe still is, enabled by default -- which is why they aren't considered a strong defence against ransomware "out of the box". You'd need admin privs to delete CCC/Time Machine/etc snapshots, and they can't be otherwise modified, so CCC et al are a much better defence by comparison.

Part of the problem here is that "backup" is such a woolly concept, meaning different things to different people. So whether CCC is good for backing up depends primarily on what you require from your backup... But any backup is better than none and, IMO, the extra redundancy and flexibility gained by using both RS and CCC is well worth the extra expense.

[DavidHertzberg]

amkassir and Nigel Smith,

Reading this article and this article reinforced my understanding of snapshotting of APFS-formatted volumes under High Sierra, Mojave, and Catalina.  The key points are (a) snapshots are a capability of the APFS filesystem—but a capability that backup applications must be programmed to initiate, and (b) APFS snapshots are made on a source volume and consist of links to files (links taking up essentially no space, which I knew)—so a backup application that initiates a snapshot must then copy the linked-to files.  Time Machine and CCC do both these things, but Retrospect doesn't do them yet.    The second article says:

Quote

Seems pretty good, but snapshots have a few downsides. First up, snapshots are part of the startup drive. They don’t exist anywhere but on the same startup drive as the data you’re trying to protect. That makes snapshots vulnerable to any issues the volume may encounter. Accidently erase the startup drive and the snapshot is gone as well. Drive having some type of errors? The snapshot is likely experiencing the same problem. The point being, don’t rely on snapshots as a backup. You still need a dedicated backup that keeps data stored on a physically separate device.

and, because "a snapshot essentially marks all the currently used data blocks on a volume to be preserved; that is, no changes can be made to them":

Quote

Editing a file can significantly magnify the amount of space being used, as the old versions of the file are retained in the snapshot, while outright deleting a file frees up no space at all since the snapshot will retain all the preserved data

Both those articles describe how to use commands in the Terminal to create and to restore from snapshots, assuming the source volume containing them is still available.  This article describes how CCC makes copies of hourly, daily, and weekly snapshots that it initiates on the source volume; it also describes CCC's Time-Machine-like policy for retaining its copies of those snapshots.  The procedure this article describes for Restoring from a CCC-retained copy of an APFS snapshot is simpler than the procedure in the Retrospect Mac 17 User's Guide, but it only works for a startup drive if you've still got one that's functioning.  That's why Bombich Software says:

Quote

Support for snapshots at the filesystem level is an important and integral component of a backup strategy, but snapshots are not a complete replacement for a true backup on physically separate hardware. If your startup disk fails, all the snapshots in the world aren't going to help you restore your startup disk and data. Having a bootable backup on an external disk will get you back to work immediately.

Also note that AFAICT the CCC snapshotting capability only works if the destination volume is also formatted with APFS.  AFAICT that rules out USB drives that that don't have the capacity of the source drive,  NAS drives, and also the cloud as destinations. 

It is also of interest to everyone that the Retrospect Knowledge Base articles on "macOS Catalina Support", "macOS Catalina – Application Data Privacy", and "macOS Mojave – Application Data Privacy" were updated on 5 May.