AES-256 Encryption — how much of a performance hit/capacity loss is there?

[theboyk]

Hello,

 

Since neither the Retrospect 8 User's Guide that comes with Retrospect 9 (?!?!) nor the Retrospect User's Guide Addendum for Retrospect 9 touch on AES-256 encryption with regards to Media Set Security, I'm wondering if anyone has any information on:

 

- What kind of performance hit do you take by including AES-256 encryption in a backup set?

- What kind of capacity hit do you take by including AES-256 encryption in a backup set?

 

All I've been able to find via a web search is a Retrospect White Paper which basically says there WILL be both loss in performance (speed) and reduction in capacity (storage space) when using AES-256 encryption, but it doesn't go into any details.

 

Thanks,

Kristin.

[prl]

Encryption will essentially prevent any hardware compression of the data going to tape. I'm not sure what order software compression is done with respect to encryption. If it's Encrypt(Compress(data), key), encryption shouldn't affect compression; if it's Compress(Encrypt(data, key)), then that will defeat any compression. Any storage hit you take will depend on how well the underlying data compresses, so it's not possible to give any number in general, except that if encryption is done before compression, you'll get 0% (or even negative) compression.

 

Strong encryption should make the data appear random; random data isn't compressible.

 

Sounds like an experiment is in order to see what performance hit you take. AES was designed for efficient software implementation (according to Wikipedia, about 18 clock cycles/byte on a Pentium Pro, for about 11MiB/s (660MiB/min) for a 200MHz machine).

 

Edit: added performance numbers.

[theboyk]

Interesting. It's too bad Retrospect doesn't document this anywhere (ie. that using AES encryption basically eliminates the ability to use hardware compression) and it's even worse that they couldn't program the GUI to grey-out the option to enable hardware compression when you've selected encryption. I'll run a test to see if software compression works with encryption, but even if it does, using software compression isn't really an option since it's so slow (brutally takes a 2.5+ GB/min backup down to ~800 MB/min). None-the-less, I'll give it a test just for the "fun" of it (and I'll post the results).

 

That said, if I want to use hardware compression — it looks like my only "security" option would be password-based. Do you know if password-based security would allow for hardware compression?

[twickland]

Do you know if password-based security would allow for hardware compression?

Yes it does. That's what we use with our tape sets.

[theboyk]

Thank Tim — do you know if there are any details available regarding Retrospect's password protection?

Thanks again,

Kristin.

[theboyk]

OK, just an update. Started the "software compression + AES" test, but quickly noticed the Retrospect 9 activity details showing "Compression - NO", so aborted that test. But, I was curious as to just what kind of performance hit AES encryption would cause so ran a test with and without AES encryption, and these were my results:

 

WITH AES ENCRYPTION: 15GB folder, regular backup, 1.2 GB/min, 28 minutes (includes verification, idle/rewind time, etc.)

WITHOUT AES ENCRYPTION: same 15 GB folder, regular backup, 2.7 GB/min, 14 minutes (includes verification, idle/rewind time, etc.)

 

Now, obviously not looking at definitive results here, but in my environment, I'd be looking at a rough doubling of backup times as a result of incorporating AES encryption. So, considering the capacity loss (as a result of not being able to use compression) and performance hit, I think I might go with a hardware compressed, password protected workflow (for "some" security, and optimal performance).

 

k.

[prl]

... the Retrospect 9 activity details showing "Compression - NO", so aborted that test. ...

Like some other Retrospect GUI messages (like the ones surrounding Copy - Overwrite entire volume), that appears to be a misleading indicator. I think it actually means "Hardware Compression - NO.

 

For one of my backups (to optical media where hardware compression is not available, data transfer is slow anyway and the compression cost is alleviated by the reduction in the amount of data to be transferred), I have Compression OFF showing in the report, but when I actually do the backup, the log shows:

Completed: 689 files, 310.2 MB, with 29% compression

 

So it may be worth an experiment, but it'll probably take a fairly hefty performance hit. OTOH, compressing will reduce your encryption cost, provided that it's done the right way Encrypt(Compress(data), key).

 

...I think I might go with a hardware compressed, password protected workflow (for "some" security, and optimal performance).

Does the password protection encrypt, or does it just stop the drive from accessing the data?

 

The last time I was responsible for backing other people's data, we kept the backups in a safe, with only the operator and sysadmin having the keys. That was so long ago that we thought that the 2.4GB we could get on an Exabyte 8mm tape was huge! Which it sort-of was, since we were mostly backing up 600MB Fujitsu Eagles.

[theboyk]

Like some other Retrospect GUI messages (like the ones surrounding Copy - Overwrite entire volume), that appears to be a misleading indicator. I think it actually means "Hardware Compression - NO.

 

Actually, it was my mistake. I must have incorrectly set the software compression to OFF, not ON. After further testing, when I enable Software Compression, it reports correctly. But, with regards to Hardware Compression, I don't think Retrospect is reporting Hardware Compression at all. Hardware Compression is enabled by default on my drives and no matter what I do with Retrospect, I get zero indication as to whether or not Hardware Compression is enabled/disabled (even though I can enable/disable it when creating my Media Sets) and zero indication as to how much the data was compressed (via the logs). So, I have absolutely no idea whether or not Hardware Compression is actually taking place?

 

Anyway, that said, I ran the test with Software Compression + AES (Retrospect correctly showed "Compression: ON"), but unfortunately, the performance hit was pretty severe — results go something like this:

 

WITHOUT AES ENCRYPTION & NO COMPRESSION: 15 GB folder, regular backup, 2.7 GB/min, 14 minutes (includes verification, idle/rewind time, etc.)

WITH AES ENCRYPTION & NO COMPRESSION: same 15GB folder, regular backup, 1.2 GB/min, 28 minutes (includes verification, idle/rewind time, etc.)

WITH AES ENCRYPTION & SOFTWARE COMPRESSION: same 15GB folder, regular backup, 740 MB/min, 43 minutes (includes verification, idle/rewind time, etc.)

WITH PASSWORD PROTECTION & HARDWARE COMPRESSION (?): same 15GB folder, regular backup, 2.8 GB/min, 13 minutes (includes verification, idle/rewind time, etc.)

 

Does the password protection encrypt, or does it just stop the drive from accessing the data?

 

It's just a basic password-protection scheme where the drive is unable to access the data without a password (ie. not encrypted). The GUI itself reports "Password only (no encryption)".

[prl]

...

WITH AES ENCRYPTION & SOFTWARE COMPRESSION: same 15GB folder, regular backup, 740 MB/min, 43 minutes (includes verification, idle/rewind time, etc.)

...

Just out of curiosity: did this report a sensible amount of compression in the log? I.e. is software compression done before encryption?

 

The rest doesn't sound very promising for getting your data compressed and encrypted to tape without taking a significant performance hit.

 

It's strange to me that you're seeing Compression ON in the report when you have software compression enabled. I've never seen a report with Compression ON for any of my software compressed backups, even though the log says that there was a reasonable amount of compression. Maybe the problem is confined to optical drive backups.

[theboyk]

Yea, compression is done first, then encryption. Retrospect states this in the "documentation". As for my level of compression - with software compression I see roughly 30% on 15 GB of data (in design files, raw video footage, various image formats, FileMaker databases, HTML files, word docs, etc. - I'm testing with pretty much the full range of files we find on a day-to-day basis. Unfortunately, the bottleneck in backup speed in general, is Retrospect - adding software compression makes it worse, and adding AES on top of that, even worse. So, I'll be sticking with uncompressed (since hardware compression is still broken in Retrospect), password-protected backups for the time being (basically, until I find find new backup software to replace Retrospect).

 

Regarding compression details in the summary - the ONLY time I have it see "Compression: ON" is when I enable software compression. Enabling/disabling hardware compression has no effect on what the summary reports for data compression. The only time I see any report of hardware compression being on/off is when I look at Media Set details. Outside of that, and the useless "use hardware compression" checkbox, hardware compression is non-existent in the insanity of the Retrospect GUI.

[theboyk]

Just an update — I've been going back and forth with Retrospect Support regarding Hardware Compression support and this is what I can confirm, for anyone interested:

 

- Hardware Compression is enabled by default on supported devices (in my case, LTO-4 tape drives).

- You can disabled Hardware Compression when creating the Media Set (by unchecking the "Allow Hardware Data Compression" checkbox).

- Retrospect in unable to get any data from the drive relating to Hardware Compression.

- In both Script Summary and Activities Details, any information relating to Compression being on/off is based on Software Compression.

- The only indication as to whether Hardware Compression is on/off is by viewing the Media Set Overview.

- When using Hardware Compression, when viewing Media Set details, any information relating to the USED, FREE and CAPACITY values are incorrect; FREE and CAPACITY values are completely inaccurate (so you can ignore them) and the USED value only reports the amount of data that was SENT to the hardware; it does not reflect how much was actually compressed/written since that is unknown (as Retrospect can't get this information from the drive).

- The drive determines when a tape is full and reports this to Retrospect (ie. Retrospect does not determine this itself based on USED, FREE and CAPACITY values).

- You should not use both Hardware Compression AND Software Compression as this will, most likely, result in the bloating of files beyond the original size (thus, doing the opposite of what you're trying to do).

 

Hope that helps a little.

k.