Possible performance reporting bug?

[Ramon88]

Can anybody confirm this:

 

Two runs of the same (first) test backup:

 

1) No encryption - No Compression - Media Verification

7.3 GB in 00:11:13 (679,5 MB/min)

 

2) No encryption - No Compression - Thorough Verification

7.3 GB in 00:13:59 (1086,2 MB/min)

 

The first one reports correctly, 7.3 GB divided by 00:11:13 indeed is (near) the indicated 679,5 MB/min.

 

The second one takes longer to backup the same data, so that 1086,2MB/min is not correct.

 

I get the impression Retrospect, when doing Thorough Verification, inserts the last momentary result of the (during backup) performance in the History log, not reflecting overall performance.

 

Note: Retrospect used was Multi Server, version 7.6.123, client used was (OSX) 6.2.234. Storage device was iSCSI based.

Edited August 14, 2009 by Guest
Added Note.

[rhwalker]

I think a bit of marketing speak is being used with the reported throughput. Just because the second backup takes longer doesn't mean (from Retrospect's view) that the throughput is slower - it all depends on what the throughput is measuring.

 

It's my impression, at least from Retrospect 2.0 (Mac) to the present, that the reported throughput is I/O throughput. The clock starts when each destination read/write issues and stops when each read/write completes, so that things like tape eject and replace do not affect the time. I haven't done tests on this in many years, but it could be done. It's why Retrospect (at least on the Mac version) used to report time spent waiting, etc., which did not affect the reported I/O throughput.

 

The byte-by-byte comparison, if it causes the issuance of reads/writes to be delayed, can cause the elapsed time to increase without affecting the I/O throughput. If there was enough CPU horsepower, and if the program is properly buffered and multi-threaded, then the added computation load shouldn't affect the elapsed time either, and the elapsed time should only be a function of I/O latency.

 

The second test, during the write phase, requires reading the full source and writing the full destination and, during the verification phase, requires reading both the full source and the full destination.

 

The first test, during the write phase, requires reading the full source and writing the full destination and, during the verification phase, requires reading only the destination but not the source.

 

So, the first test does approximately 3X I/O and the second test does approximately 4X I/O.

 

Depending on things like rotational latency and seek and data transfer overlaps, drive buffering, read-ahead, etc., if the reading of the destination can be completely overlapped with the reading of the source, if they both complete in about the same elapsed time (and they do, approximately), the I/O throughput of the second test should be about 4/3 that of the first test, which is roughly what was seen.

 

I think that the results are about right.

 

Russ

[Ramon88]

Hi Russ,

 

Actually I don't really care the performance measured in time is different. You are correct the second method takes longer, no problem with that. I also don't mind momentary throughput readouts during the backup phase (actually that's quite interesting information). What I do mind is the History not reporting the overall throughput for that particulary run correctly. It's just amount of data divided by the time it took. That shouldn't be hard at all to implement. That way you can compare runs, performance-wise, more easily.

 

It's not a big deal, but I'm a bit of a completist

[rhwalker]

What I do mind is the History not reporting the overall throughput for that particulary run correctly. It's just amount of data divided by the time it took. That shouldn't be hard at all to implement.

Oh, I agree completely with you; hope it didn't appear otherwise.

 

I have always chalked it up to "marketing speak", with a bit of amusement, every time I have seen these figures over the past 16 years that we have been using Retrospect. I've occasionally seen throughput in the umpteen gazillions of GB/Min for odd backups that only have a single file.

 

I always look skeptically at reported specs when I see them, and sometimes the underlying test conditions are oddly contrived to produce specs to make a product appear in a more favorable light to a prospective customer. That seems pretty common in the tech industry. The specs are rarely false, just measured under um, unusual, test conditions. Understanding the metrics is key.

 

Russ