Duration 14:47:43 for Immediate Duplicate 60G

[alanackerman]

I followed Joe Kissel's advice and purchased Retrospect Desktop. Now I am sorry. I don't want to tie up my machine for 14 hours straight.

 

Just for comparison, I downloaded trial copies of Intego Personal Backup X4 and SuperDuper.

 

Times to make a bootable copy of the same volume, to the same external hard drive:

 

Intego 2:33

Super 2:53

 

Drive was erased before I started.

 

Same 60G (on an 80G volume). Same external volume. Results are all bootable.

 

The operations log contains no time stamps, other than start and end. Dantz doesn't even make it possible to cut & paste, or I would. Any suggestions?

[alanackerman]

Figured out how to copy the log:

? Retrospect version 6.1.126

launched at 1/15/2007 7:59 PM

+ Retrospect Driver Update, version 6.1.5.102

 

+ Executing Immediate Duplicate at 1/15/2007 8:25 PM

 

- 1/15/2007 8:25:10 PM: Copying Gauss…

Can't read file

<etc.>

1/16/2007 8:00:00 AM: Comparing Descartes 1…

File “com.apple.LaunchServices-014504.csstore”: different creation date/time

<etc.>

1/16/2007 11:12:42 AM: 244 execution errors.

Completed: 534845 files, 56.8 GB

Performance: 159.0 MB/minute (108.0 copy, 301.6 compare)

Duration: 14:47:32 (02:36:52 idle/loading/preparing)

Quit at 1/16/2007 6:54 PM

[CallMeDave]

If someone advised you to purchase Retrospect specifically to make duplicates/clones of drives, they gave you bad advice. While Retrospect can successfully and reliably Duplicate volumes, the program is most powerful as a Backup tool. There are lots and lots of other clone solutions for OS X, such as the ones you tried as well as Carbon Copy Cloner and others.

 

Also note that other programs do not compare their copies; so if you're willing to do without that extra step of reliability you can configure Retrospect not to Compare the Source to the Destination after it's done writing files (although according to your Log this would only have saved you about 3.25 hours from the total).

 

108.0 MB/minute is very slow for modern hardware, although you omit any specific configuration information from your post. We see copy and compare rates many orders of magnitude higher then what you're reporting here.

 

Retrospect will likely always be slower then some of the other options. Still, it's matching capabilities are strong, and depending on how you intend to use the program (information also omitted from your post) it might be worth the extra time required.

[alanackerman]

Joe Kissell wrote a book called Take Control of Mac OS X Backups.

 

I do intend to perform both archives and clones, but naturally I started with a clone. Better a bootable backup to start with. The refreshes will presumably be faster, but I still face at least four 14-hour nights.

 

This is neither a fast Mac nor is the external disk very fast. But it still seems that 4x the time to do a backup is out of kilter. I set Energy Saver to never & never, and unchecked sleep hard drive -- but the screen saver did activate, and it looks like Retrospect might have hung for a while. Impossible to tell without time stamps in the log.

 

This is a flat-panel (G4) iMac, 4.5 years old,

 

Machine Name: iMac

Machine Model: PowerMac4,2

CPU Type: PowerPC G4 (2.1)

Number Of CPUs: 1

CPU Speed: 800 MHz

L2 Cache (per CPU): 256 KB

Memory: 768 MB

Bus Speed: 100 MHz

Boot ROM Version: 4.3.5f1

 

Internal drive:

 

ATA Bus:

 

ST380011A:

 

Capacity: 74.53 GB

Model: ST380011A

Revision: 3.06

Serial Number: 3JVDB0XX

Removable Media: No

Detachable Drive: No

BSD Name: disk0

Protocol: ATA

Unit Number: 0

Socket Type: Internal

OS9 Drivers: Yes

S.M.A.R.T. status: Verified

Volumes:

xxxxx:

Capacity: 74.4 GB

Available: 15.26 GB

Writable: Yes

File System: Journaled HFS+

BSD Name: disk0s10

Mount Point: /

 

External drive:

 

FireWire Bus:

 

Maximum Speed: Up to 400 Mb/sec

 

ROC PORT:

 

Manufacturer: ROCSTOR

Model: 0x0

GUID: 0x30E003E0454649

Maximum Speed: Up to 400 Mb/sec

Connection Speed: Up to 400 Mb/sec

Sub-units:

ROC PORT Unit:

Unit Software Version: 0x10483

Unit Spec ID: 0x609E

Firmware Revision: 0x101

Product Revision Level: YL10

Sub-units:

ROC PORT SBP-LUN:

Capacity: 111.79 GB

Removable Media: Yes

BSD Name: disk1

OS9 Drivers: Yes

S.M.A.R.T. status: Not Supported

Volumes:

yyyyy:

Capacity: 111.66 GB

Available: 53.35 GB

Writable: Yes

File System: Journaled HFS+

BSD Name: disk1s10

Mount Point: /Volumes/Descartes 1

 

Similar times for my external La Cie drive.

[CallMeDave]

> Joe Kissell wrote a book called Take Control of Mac OS X Backups.

 

Like I always say, if someone wrote it and posted it on the internet, it must be true.

 

> I do intend to perform both archives and clones,

 

"Archive" is a term with specific meaning in Retrospect, one you are not likely trying to do, and "clone" is not a term the program uses at all. Retrospect performs "Backup" and "Duplicate" operations.

 

> but naturally I started with a clone. Better a bootable backup to start with.

 

Naturally.

 

You're assuming that readers know what your data security and disaster recovery strategies are here, but we don't. Do you have 4 physical or logical volumes that you intend to use as Destinations for Duplicates? Do you have additional storage allocated for your Normal Backups?

 

> The refreshes will presumably be faster, but I still face at least four 14-hour nights.

 

You should test your setup to see if your hardware is capable of faster throughput, which might lead you to discover why your initial experience was so slow. Setup an Immediate Duplicate of some moderate sized Source (define a folder as a subvolume), to an empty Destination (an empty folder also defined as a subvolume). The Retrospect Users Guide will probably be a better place to look for specific directions then a third party eBook.

 

You should also familiarize yourself with the practical and operational differences between a Backup (such as to a File Backup Set) and a Duplicate, and use them to your best advantage depending on your specific needs. Backups will generally be faster to complete, and they maintain a historical record of changed/deleted files. Duplicates will generally take longer, due to the need for Retrospect to scan both the Source and Destination volumes before beginning to copy files, but they result in Finder readable copies that can be used immediately in the case of disaster.

 

Version 2.0 of Mr. Kessell's Take Control of Mac OS X Backups was released on January 25th, 2007, so if you like his writing you might want to check out the new text.

[alanackerman]

>"Archive" is a term with specific meaning in Retrospect, one you are not likely trying to do, and "clone" is not a term the

>program uses at all. Retrospect performs "Backup" and "Duplicate" operations.

 

Foo on Retrospect's silly terminology.

 

>You're assuming that readers know what your data security and disaster recovery strategies are here, but we don't. Do

>you have 4 physical or logical volumes that you intend to use as Destinations for Duplicates? Do you have additional

>storage allocated for your Normal Backups?

 

I fail to see how that makes any difference to why Retrospect is so slow, but here goes:

 

I intend to make both Backup and Duplicates. Two of each, one to take offsite. I have two external Rocstor 4Cx 120G portable hard drives that I purchased for this purpose. I intend to partition them each into 2 partitions, one for backup and one for duplicate. Daily backups, weekly duplicates, swap disks every week. (All of which comes straight out of Joe Kissell's book.)

 

We also have 3 laptops around. If I get this working, I planned on doing client/server backups for them. (If client/server is Retrospect terminology.)

 

>You should test your setup to see if your hardware is capable of faster throughput, which might lead you to discover

>why your initial experience was so slow.

 

I did. It is capable of much faster throughput, with SuperDuper or Intego Personal Backup. Why is Retrospect so slow, and can it be fixed?

 

>Setup an Immediate Duplicate of some moderate sized Source (define a folder as a subvolume), to an empty

>Destination (an empty folder also defined as a subvolume).

 

OK, I will try that.

 

>Version 2.0 of Mr. Kessell's Take Control of Mac OS X Backups was released on January 25th, 2007, so if you like his

>writing you might want to check out the new text.

 

I did. I read the whole thing.

 

I find the tone of your entire response VERY condescending. Is that necessary? You haven't answered my questions at all, you've merely attacked me for using the wrong terminology and for not providing information that I don't see the use for (but am happy to provide).

[alanackerman]

My "data security and disaster recovery strategies" are completely irrelevant.

 

I simply want to perform a Duplicate of my 80G internal hard drive (60G full) to my external fire-wire Rocstor 4Cx 120G hard drive in under 8 hours.

 

Is that possible or not?

 

If not, I will throw Retrospect Desktop in the trash and warn the world.

 

>Setup an Immediate Duplicate of some moderate sized Source (define a folder as a subvolume), to an empty

>Destination (an empty folder also defined as a subvolume).

 

OK, I will try that.

[CallMeDave]

Quote:

---

Foo on Retrospect's silly terminology.

 

---

 

???

 

If you are going to post questions about a specific application, and you want to get accurate information regarding that program, it behoves you to use the correct terms. For example, if you ask a question about Archiving in Retrospect, a Forum community member might end up wasting your time and his/her describing behavior that you really aren't interested in.

 

> I fail to see how that makes any difference to why Retrospect is so slow,

 

It doesn't. But it does make a difference in addressing the issue of your stated regret for purchasing the product.

 

> I planned on doing client/server backups for them. (If client/server is Retrospect terminology.)

 

Sort of. "Server" is a hard one, as it's used to describe a specific sort of Script in Retrospect. Sadly, there is no better term to describe the machine running Retrospect, connecting to Clients, other then "the machine running Retrospect."

 

> Why is Retrospect so slow, and can it be fixed?

 

The suggesting was for you to test your hardware capability _with Retrospect_ by performing some Immediate tasks and observing the real-time throughput. If you see 100 Mb/sec then you may have a hardware issue. If you see faster rates then perhaps there was something slowing things down when you ran overnight.

 

> I find the tone of your entire response VERY condescending. Is that necessary?

 

Yes. That's the way I write.

 

> You haven't answered my questions at all...

 

For true? Well, that hurts.

 

Many times initial posts require additional information before answers can be revealed. So initial replies are often requests for additional information or suggestions of tests that can be taken.

 

Still, the only question you asked in your initial posts was "any suggestions?" which I took to be open-ended enough to engage in the necessary discussion to find why you were dissatisfied.

 

> I simply want to perform a Duplicate of my 80G internal hard drive (60G full) to my external

> fire-wire Rocstor 4Cx 120G hard drive in under 8 hours.

 

> Is that possible or not?

 

> If not, I will throw Retrospect Desktop in the trash and warn the world.

 

If Retrospect does not provide the disk-to-disk duplication that you want you should switch to another application that does. But you may still want to use Retrospect for incremental Backups to disk, as well as for proactive "Backup Server" scripts to address portable machines that come and go from the network.

 

There's lots of discussion online regarding Retrospect's strengths and weaknesses. Macintouch has an ongoing Special Report on Backups that you could read and contribute to. But the whole world reads it, so you may want to make sure you've done your due diligence before posting.

 

 

Dave

[alanackerman]

OK, so I asked a vague question. You've help me make it clearer.

 

I read the Macintouch discussion. I didn't see anything about Retrospect being slow -- only that it has a lousy user interface.

 

So I think there is a good chance that it is something about my hardware or software. I hope so, so then I can maybe fix it.

 

Others only seem to see Retrospect as 10% slower than SuperDuper -- not 300%.

 

I did try turning off the screen saver and performing a Safe Boot last night. That did help. Only 11 hours instead of 14.

 

? Retrospect version 6.1.126

launched at 2/2/2007 10:38 PM

+ Retrospect Driver Update, version 6.1.5.102

 

+ Executing Immediate Duplicate at 2/2/2007 11:01 PM

 

- 2/2/2007 11:01:35 PM: Copying Gauss…

Can't read file “J:dische Feiertage.vcs”, error -43 (file/folder not found), path: “Gauss/Applications/Palm/Holiday Files/Deutsch/J:dische Feiertage.vcs”.

(etc)

2/3/2007 7:01:29 AM: Comparing Descartes 1…

File “console.log”: different data size (src: 280, dest: 120), path: “Gauss/Library/Logs/Console/508/console.log”.

(etc)

2/3/2007 10:08:45 AM: 83 execution errors.

Completed: 538638 files, 57.0 GB

Performance: 174.7 MB/minute (121.4 copy, 311.3 compare)

Duration: 11:07:10 (00:00:05 idle/loading/preparing)

Quit at 2/3/2007 10:17 AM

[alanackerman]

OK, I tried to Duplicate my user folder (not Safe Boot):

 

? Retrospect version 6.1.126

launched at 2/3/2007 5:29 PM

+ Retrospect Driver Update, version 6.1.5.102

 

+ Executing Immediate Duplicate at 2/3/2007 5:32 PM

 

- 2/3/2007 5:32:43 PM: Copying alanacke on Gauss…

2/3/2007 5:44:31 PM: Comparing Duplicate on Descartes Backup 1…

File “.database.lockN”: different modification date/time (src: 2/3/2007 5:37:46 PM, dest: 2/3/2007 5:11:20 PM), path: “alanacke/Library/Application Support/AddressBook/.database.lockN”.

File “Envelope Index”: different modification date/time (src: 2/3/2007 5:38:18 PM, dest: 2/3/2007 5:11:13 PM), path: “alanacke/Library/Mail/Envelope Index”.

File “com.apple.iCal.AlarmScheduler.plist”: different creation date/time (src: 2/3/2007 5:41:34 PM, dest: 2/3/2007 4:41:32 PM), path: “alanacke/Library/Preferences/com.apple.iCal.AlarmScheduler.plist”.

2/3/2007 5:56:27 PM: 3 execution errors.

Completed: 27567 files, 3.3 GB

Performance: 281.6 MB/minute (283.6 copy, 279.6 compare)

Duration: 00:23:44 (00:00:02 idle/loading/preparing)

 

At this rate, 57G would take 57/283.6*1024 = 3:26 to copy and 57/279.6*1024 = 3:28 to compare. If you are correct that the other products do not perform a compare, then that is what I would expect. But that is not what I get for the entire hard drive.

 

Is there something that is known to copy more slowly?

[Lennart_T]

Quote:

---

Is there something that is known to copy more slowly?

 

---

5 GB worth of many small files (such as the Mac OS X system files/folders) compared to a single 5GB file. Especially if your destination is tape and the file's contents is easily compressed (using the tape drive's hardware compression).

 

Also, Retrospect makes SURE it sets all the file permissions correct after the Duplicate, which many "clone" apps simply doesn't do.

[CallMeDave]

Retrospect does more then just blindly copy files from one place to another; it needs to scan the Source, and then Compare it to the Destination, before even starting to copy files. This Matching is what allows the program to ignore unchanged files, but it adds time to the operation.

 

So in addition to the time necessary to read/write files, you'll find that Scanning the Source takes time (the more files, the longer the Scan time (size of files doesn't matter)), Scanning the Destination takes a similar amount of time (not for the first run to an empty volume, but subsequent Duplicates will requre full scanning), and then the mysterious Closing phase (where Retro checks/sets permissions) all add to the total time.

 

If you set up another full volume Duplicate, I'd suggesting checking it in real-time to observe the transfer rate. If you're getting 300 Mb/sec or more, then any additional time above your calculations would be from the other steps the program takes.

[alanackerman]

Since the Destination, in my case, has been newly erased, I fail to see how the scanning is any advantage to me -- just more time. Spending 14-15 hours "checking it in real-time" isn't going to happen -- I'm not that obsessive.

 

Actually, by doing a Safe Boot, and excluding cache files from the backup, I managed to get it down to "just" 9.5 hours. (I still don't want to watch it.) On the subsequent Duplicate run to the same (unerased) Destination it took only 3:15. I can live with that, since I already own a copy of Retrospect. But the next time they ask me for more money for an upgrade, I am going to look at the other products out there.

[alanackerman]

>Also, Retrospect makes SURE it sets all the file permissions correct after the Duplicate, which many "clone" apps simply doesn't do.

 

Both Intego Personal Backup and SuperDuper create bootable copies. I have tested them. Further, "Repair Disk Permissions" doesn't find any errors.

 

I ALWAYS test my Duplicates to make sure they are bootable. (Got in the habit when using Carbon Copy Cloner, which sometimes failed to do this.)

[CallMeDave]

There are many, many good programs for OS X that will duplicate one volume to another. And while Retrospect has a Duplicate feature, it may not perform as well as a newer Cocoa built application.

 

If you are not going to take advantage of Retrospect's greatest strengths, such as it's ability to do Incramental Plus backups or its Client/Server model, you should certainly consider other options.

[alanackerman]

I am doing Incremental backups daily. Those take about 40 minutes. (Is that reasonable?) I haven't used the Clients yet, but I intend to. So far I cannot get the scheduled backups to work, so they are all manual.

 

But that's not my point. I spent money for this package. Now you are telling me to spend money again for some other package. How do I get my money's worth?

[Lennart_T]

Quote:

---

Both Intego Personal Backup and SuperDuper create bootable copies. I have tested them. Further, "Repair Disk Permissions" doesn't find any errors.

 

---

The fact that a disk is bootabale doesn't guarantee it's correctly copied/cloned. The files/folders in the "Users" foldermay not be correctly copied, especially if many users share the computer or it is a server.

"Repair Disk Permissions" just checks a few system files/folders. It does not check anything in the "Users" folder, for instance.

 

But since you are using Retrospects incremental (or rather "progressive") backup, this "cloning" discussion seems off topic.

[rhwalker]

Quote:

---

I am doing Incremental backups daily.

 

---

No you are not. Everything you have said so far is that you are doing DUPLICATES. Those are not backups, they are copies. Duplicate is an appendage grafted onto Retrospect. It's not backup. That is what Dave is trying to say. If what you want is to do duplication, you have selected the wrong tool.

 

We are trying to help you. You don't seem to want to be helped, you just want to gripe. Good luck.

 

Russ

[alanackerman]

Yes I am. I have figured out the difference between Duplicate and Backup, and I am doing BOTH. Weekly duplicate and nightly backup. (Only the scheduling doesn't work...)

 

>Duplicate is an appendage grafted onto Retrospect.

 

That might explain why it is so slow. But I think Duplicate is essential, while Backup is only nice to have. When the hard drive fails, I need to be able to boot from a fairly recent Duplicate. Backup is better for rescuing single files I accidentally deleted. I'd hate to have to reinstall and then restore from a Backup.

 

But Backup is off topic for this thread. Someone just said I wasn't using Retrospect for anything it was good for and why don't I go away and try some other product, so I said what I am using Retrospect for.

 

>We are trying to help you.

 

What I asked for was some way to speed it up. So far the only answer seems to be, "go find some other product".

 

I did figure out, on my own, to do a Safe Boot and to exclude cache files for Duplicate. But I cannot automate doing a Safe Boot.

[CallMeDave]

Quote:

---

What I asked for was some way to speed it up

 

---

 

That was part of your original post, yes.

 

And in that original post, you included from the Log:

 

Performance: 159.0 MB/minute (108.0 copy, 301.6 compare)

 

This is dismally slow. Perhaps it's due to running on a G4 with less then a Gig of RAM, or perhaps it's a problem with your external FW device (cable, host, chipset, etc).

 

I suggested you peek at the execution a couple of times in real-time to see if you're achieving higher throughput (since the log only gives average times, not peak times), but you declined.

 

Other users get performance many times greater then you're seeing. The software is capable of it, but your hardware seems not to be.

 

Dave

[rhwalker]

Alan,

 

I'm not going to get in the middle of this debate, especially because we really don't use Retrospect's duplicate function; our use of Retrospect has always just been tape backup using an autoloader, and Retrospect does a pretty good job there (well, there are two bugs about which I could get on the soapbox, but won't because they aren't relevant to your issues).

 

But I will make a "thinking different" suggestion that might help your effort to duplicate/clone your disk in a very short time (well, an apparent short time). It's the solution we use on our Xserve before we make any software changes, as a fallback in case things go wrong, and it enables us to keep our server up 24/7/365 except for a brief moment to reboot to the new software (when necessary) or reboot to the prior clone (if disaster). It's a rather standard approach for servers, and is the only way I know to make an instantaneous clone/duplicate (not going to use the word "snapshot" here for terminology confusion reasons, but that's really the appropriate word).

 

The method is to bring a disk online as a RAID 1 mirror secondary, allow the mirror to rebuild, then split the (just added) mirror secondary off from the mirror. The split mirror secondary is then an instantaneous duplicate/clone at the instant of the mirror split, and can be unmounted, removed, whatever. It's a little more complex than this for a server, because there are some databases (for example, the Cyrus email database) that must be self-consistent, so you have to shut down the mail service the instant before the mirror split and then restart the mail service after the mirror split, etc., so that you don't end up with a corrupt email database. On a non-server machine, I would still make the computer quiescent at the time of split, quitting programs that have specialized databases (email clients, etc.).

 

We use SoftRAID for this because of its ease in adding/splitting mirrors, and there's even a white paper on their web site

SoftRAID web site

explaining the technique.

 

A possible downside for your purpose is that SoftRAID isn't scriptable right now, so you have to do the mirror secondary add, mirror split, etc., from the GUI, but scriptability is coming "real soon now", perhaps with Leopard. It's a nice product, support is excellent, and extremely reliable.

 

It's not a replacement for Retrospect backups, but it might do what you need using a different approach. The mirror would rebuild in the background, and your effective loss of use of the machine would be very short while the mirror was split. On our Xserve, we see a 50 GB mirror (our OS volume) rebuild in less than an hour, just to give you a comparison. Remember, this is not a duplicate of files, but a mirror rebuild of every sector on the volume, and the computer stays up and usable the entire time.

 

Hope this might help,

 

Russ

[alanackerman]

OK, I watched it for an hour.

5:59 - 6:34 Scnning 524,299 files

6:34 - 6:37 Spinning beach-ball

6:38 Duplicatiing

 

Speeds:

 

6:38 624MB/min

6:39 517

6:40 357

6:41 295

6:42 294

6:43 208

6:44 207

6:45 204

6:46 203

6:47 203

6:48 188

6:49 199

6:50 222

6:51 237

6:52 255

6:53 236

6:54 224

6:55 219

6:56 207

6:57 191

6:58 186

6:59 183

...

7:09 152

...

7:12 143

 

I cannot really tell what it is backing up -- the file names change too fast and mostly mean nothing to me. (Lots of Classes.nib!) Why doesn't Retrospect give the folder being backed up, instead of the file?

 

Are these averages since it started, or for a shorter time span?

These numbers don't tell me anything -- what do they tell you?

[CallMeDave]

Discarding the high and low values in your sample above, the resulting average is about 235Mb/Min, which is in line with the 283.6 you report in message #93961 above.

 

What it tells me is that Retrospect can copy files from your internal HD to your external FW HD in the 200-300 MB/Min range. Not a crawl, but not a speed monster, either.

 

Given the fact that Retrospect requires extra time in order to provide extra features, your total backup time is limited by your hardware. Note that Retrospect uses the exact same programming APIs that the Finder uses to copy files (although Apple might change/improve those interfaces over time, while Retrospect is unlikely to get any updates to take advantage of new Apple technologies going forward). Maybe the unix ditto or cp programs used in newer OS X duplication packages are faster then what Retrospect uses; I don't know. As I said earlier, speed is a feature, and if you're not getting the features you want you should consider other options. There's nothing wrong with using one program for disk duplication and another for ongoing incremental backups.

 

You don't describe much about your external drive; the Rocstor website is pretty, but we don't know the make/model of the hard drive inside your case, cache amount, rotation speed, etc. But if it's a few years old, you might be amazed at what a new Seagate Barracuda drive will do (after you jaw drops at the lack of noise you'd become accustomed to).

 

Dave