Speed

[kaikow]

I've seen a product suggestion requestion inproved speed for a particular DVD drive.

 

A more general question may be the following.

 

Does Dantz have a list of the expected speeds for various types of devices?

 

For example, what is the expected throughput for a Maxtor 5000LE USB drived connected to an Adaptec USB 2 controller when the hard drives are all connected to an Adaptec 2940UW SCSI card? I've seen spees of 62.0 to 225.7 MB per minute, with typical speeds around 100MB per minute.

 

Since a backup is I/O intensive, how much is the speed affected by CPU speed, by amount of memory, by pagefile size, by disk fragmentation?

 

Source are currently all FAT 32. Destination is FAT 32.

Would it be faster if source were FAT 32?

 

Obviously, there can only be guidelines, not exact estimates, but I/we would like to know what we should expect.

 

I am assuming that auntie virus software is not "on" during the backup, e.g., AutoProtect in Norton Auntie virus would be Off.

 

[natew]

Hi

 

I think we would all like to see that list - unfortunately there are just too many variables in play to make it practical or anywhere near realistic.

 

Your best bet is to make an estimate from what the manufacturer/industry says the bus and device will allow and then cut that by 10 to 50%. The more CPU and ram you have, the better the chances are that the device will run close to manufacturer spec. Same goes for disks that are less fragmented.

 

Aside from some options with CD/DVD drives, there is no way to control the speed of Retrospect. It simply goes as fast as the system will allow it to.

 

Ithink file system has a pretty minor affect on speed. This gets debated a lot with people doing digital recording - to date I have not seen a defineative answer yet.

 

Nate

[kaikow]

Quote:

---

natew said:

Hi

 

I think we would all like to see that list

- unfortunately there are just too many variables in play to make it practical or anywhere near realistic.

 

Your best bet is to make an estimate from what the manufacturer/industry says the bus and device will allow and then cut that by 10 to 50%. The more CPU and ram you have, the better the chances are that the device will run close to manufacturer spec. Same goes for disks that are less fragmented.

 

Aside from some options with CD/DVD drives, there is no way to control the speed of Retrospect. It simply goes as fast as the system will allow it to.

 

I think file system has a pretty minor affect on speed. This gets debated a lot with people doing digital recording - to date I have not seen a defineative answer yet.

 

Nate

 

---

 

One factor that we should be able to control is the number of sessions wriiten to a backup srt before doing a recyle backup.

 

I expect that the longer one goes bewteen recylces, the longer it takes to do the "matching" and catalog update.

 

Is there a recommended period at which a recycle nbackup should be run?

 

My system is already at maxumum memory and the paging file is not fragmented, i.e, OS is on G drive, 576MB paging file on C as a fixed size and 1152MB on F as a fixed size/

 

So the only variables I could control are:

 

1. Fragmentation.

2. When to do a Recycle backup.

 

However, guidelines would be useful for future systems.

[natew]

Hi

 

Agreed, lots of people want to run a recycle backup automatically when the set gets to be a certain size. It has been submitted as a feature request.

 

Matching doesn't take all that much time as it is all done in memory. I would gess 30 seconds to a minute max. What takes the most time is scanning the source volume in the first place. There isn't a way to get around that no matter what you do.

 

Nate

[kaikow]

Quote:

---

natew said:

Hi

 

Agreed, lots of people want to run a recycle backup automatically when the set gets to be a certain size. It has been submitted as a feature request.

 

Matching doesn't take all that much time as it is all done in memory. I would gess 30 seconds to a minute max. What takes the most time is scanning the source volume in the first place. There isn't a way to get around that no matter what you do.

 

Nate

 

---

 

I would never choose to run a Recycle backup automatically.

 

With 10 drives, the matching takes a large portion of the backup time.

[natew]

Hi

 

Are you sure you mean matching and not scanning? If the hard disk is working hard chances are it is scanning. Once that stops the matching should begin and end quite quickly even with many volumes. How many sessions do you have in your set?

 

Nate

[kaikow]

Quote:

---

natew said:

Hi

 

Are you sure you mean matching and not scanning? If the hard disk is working hard chances are it is scanning. Once that stops the matching should begin and end quite quickly even with many volumes. How many sessions do you have in your set?

 

Nate

 

---

 

Below is the History from the last backup.

The overhead is high:

 

Total duration: 00:17:38 (00:15:51 idle/loading/preparing)

 

OS is on G.

There are no files on the L and M drives.

C, F and J have other instances of Win 2000 and rarely change.

H and K are fairly stable and rarely change.

 

Bulk of changes are usually on D, G, and I drives.

 

Recyle backup was done on 13 Sept.

Normal backup was run on 20, 21, 23 and 24 Sept.

Another backup set was used between 13 and 20 Sept.

So, for each of the 10 logical drives there are no more than 5 sessions.

 

AutoProtect in Norton Auntie Virus 2003 is disabled.

Scanner software seems to like to change files even when the scanner is not being used, or even plugged in.

 

+ Normal backup using DiskMicron-2 at 9/24/2003 18:54

To Backup Set DiskAllMicron-2...

Retrospect Professional only supports Open File Backup on Windows XP.

 

- 9/24/2003 18:54:14: Copying Micronc (C:)

9/24/2003 18:55:09: Snapshot stored, 3,839 KB

9/24/2003 18:55:11: Comparing Micronc (C:)

9/24/2003 18:55:13: Execution completed successfully

Completed: 2 files, 2 KB, with 0% compression

Performance: 0.1 MB/minute (0.1 copy, 0.1 compare)

Duration: 00:00:59 (00:00:55 idle/loading/preparing)

 

Retrospect Professional only supports Open File Backup on Windows XP.

 

- 9/24/2003 18:55:13: Copying Micrond (D:)

9/24/2003 18:59:50: Snapshot stored, 17.0 MB

9/24/2003 18:59:53: Comparing Micrond (D:)

9/24/2003 18:59:56: Execution completed successfully

Completed: 20 files, 85 KB, with 43% compression

Performance: 0.8 MB/minute (0.4 copy, 4.9 compare)

Duration: 00:04:42 (00:04:29 idle/loading/preparing)

 

Retrospect Professional only supports Open File Backup on Windows XP.

 

- 9/24/2003 18:59:56: Copying Micronf (F:)

9/24/2003 19:01:06: Snapshot stored, 7,296 KB

9/24/2003 19:01:08: Comparing Micronf (F:)

9/24/2003 19:01:10: Execution completed successfully

Completed: 2 files, 2 KB, with 0% compression

Performance: 0.1 MB/minute (0.1 copy, 0.1 compare)

Duration: 00:01:13 (00:01:08 idle/loading/preparing)

 

Retrospect Professional only supports Open File Backup on Windows XP.

 

- 9/24/2003 19:01:10: Copying Microng (G:)

9/24/2003 19:06:04: Snapshot stored, 42.1 MB

9/24/2003 19:06:08: Comparing Microng (G:)

File "G:\Program Files\EPSON\EPSON SMART PANEL for Scanner\SmaPanel.ini": different modify date/time (set: 9/24/2003 19:02:42, vol: 9/24/2003 19:06:12)

9/24/2003 19:06:17: 1 execution errors

Completed: 63 files, 13.3 MB, with 79% compression

Performance: 93.2 MB/minute (88.0 copy, 99.0 compare)

Duration: 00:05:07 (00:04:48 idle/loading/preparing)

 

Retrospect Professional only supports Open File Backup on Windows XP.

 

- 9/24/2003 19:06:18: Copying Micronh (H:)

9/24/2003 19:06:44: Snapshot stored, 846 KB

9/24/2003 19:06:46: Comparing Micronh (H:)

9/24/2003 19:06:48: Execution completed successfully

Completed: 2 files, 2 KB, with 0% compression

Performance: 0.2 MB/minute (0.1 copy, 0.1 compare)

Duration: 00:00:29 (00:00:28 idle/loading/preparing)

 

Retrospect Professional only supports Open File Backup on Windows XP.

 

- 9/24/2003 19:06:48: Copying Microni (I:)

9/24/2003 19:07:59: Snapshot stored, 2,501 KB

9/24/2003 19:08:00: Comparing Microni (I:)

9/24/2003 19:08:30: Execution completed successfully

Completed: 325 files, 73.1 MB, with 65% compression

Performance: 148.5 MB/minute (136.9 copy, 162.2 compare)

Duration: 00:01:41 (00:00:41 idle/loading/preparing)

 

Retrospect Professional only supports Open File Backup on Windows XP.

 

- 9/24/2003 19:08:30: Copying Micronj (J:)

9/24/2003 19:09:13: Snapshot stored, 2,549 KB

9/24/2003 19:09:15: Comparing Micronj (J:)

9/24/2003 19:09:17: Execution completed successfully

Completed: 2 files, 2 KB, with 0% compression

Performance: 0.2 MB/minute (0.1 copy, 0.1 compare)

Duration: 00:00:46 (00:00:44 idle/loading/preparing)

 

Retrospect Professional only supports Open File Backup on Windows XP.

 

- 9/24/2003 19:09:17: Copying Micronk (K:)

9/24/2003 19:11:04: Snapshot stored, 6,132 KB

9/24/2003 19:11:06: Comparing Micronk (K:)

9/24/2003 19:11:09: Execution completed successfully

Completed: 2 files, 2 KB, with 0% compression

Performance: 0.1 MB/minute (0.1 copy, 0.1 compare)

Duration: 00:01:51 (00:01:46 idle/loading/preparing)

 

Retrospect Professional only supports Open File Backup on Windows XP.

 

- 9/24/2003 19:11:09: Copying Micronl (L:)

9/24/2003 19:11:27: Snapshot stored, 3 KB

9/24/2003 19:11:29: Comparing Micronl (L:)

9/24/2003 19:11:31: Execution completed successfully

Completed: 2 files, 2 KB, with 0% compression

Performance: 0.2 MB/minute (0.1 copy, 0.1 compare)

Duration: 00:00:22 (00:00:20 idle/loading/preparing)

 

Retrospect Professional only supports Open File Backup on Windows XP.

 

- 9/24/2003 19:11:31: Copying Micronm (M:)

9/24/2003 19:11:49: Snapshot stored, 3 KB

9/24/2003 19:11:51: Comparing Micronm (M:)

9/24/2003 19:11:53: Execution completed successfully

Completed: 2 files, 2 KB, with 0% compression

Performance: 0.2 MB/minute (0.1 copy, 0.1 compare)

Duration: 00:00:21 (00:00:20 idle/loading/preparing)

 

9/24/2003 19:11:53: 1 execution errors

Total performance: 96.8 MB/minute with 67% compression

Total duration: 00:17:38 (00:15:51 idle/loading/preparing)

 

[kaikow]

In order to get a better idea of the overhead, compression, etc. of Retrospect, I did the following:

 

1. Ran a recycle backup for all 10 logical drivesd on the 3 internal SCSI drives. Backup was to a disk backup set on a USB external hard drive. Accoring to the log, 254269 files of about 15.17GB were backed up and compared in a time of 08:43:41 (00:14:43 idle//loading/preparing).

 

Compression was a disappointing 13% and performance was a disappointing 99.6MB/minute.

 

Based on that information, I would expect that most/many Normal back ups to the same backup set would have idle/loading/preparing time of 14+ minutes.

 

I was also wondering why Retrospect did not make more use of the system's memory?

The highwater memory mark was, according tp Task Manager, only 298260MB. System has 768MB of memory.

 

2. For comparison, I used Sonic's BackUp MyPC version 4.85 to create a File backup set on the same USB drive. According to the log, 254032 files, consisting of 26376141022 bytes, were backed up and compared in 04:29:00. That's quite a bit faster than Retrospect. And compression was a more respectible 28%.

 

Note that BUMP also did not take advantage of the added memory.

 

I ran a similar comparison back in June 2003 and found similar results. At that time the total fiiles as about 19GB.