Jump to content
emulator

Hardware-based encryption options

Recommended Posts

Does anyone here know of any options that are compatible with Retrospect that provide for hardware based encryption and compression? Right now, the software encryption/compression in Retrospect have two problems as it pertains to our organization:

 

1) The compression and encryption process is too slow

2) The compressed/encrypted data is too big when compared to a tape drive's hardware compression

 

Any info would be greatly appreciated.

Share this post


Link to post
Share on other sites

EMC has not tested with these cards in several years. See if they will give you an evaluation unit to test performance and compatibility before purchase.

Share this post


Link to post
Share on other sites

Not directly. If it is hidden to Retrospect and managed by the hardware, then it would probably work. I don't have anything more specific though.

Share this post


Link to post
Share on other sites

It looks like Indra has been doing some testing with the current versions of Retrospect. That is good for card compatibility.

 

LTO Encryption is on our list of features, but isn't planned for the immediate future.

Share this post


Link to post
Share on other sites

Yes! On a side note, I did hear from them, and they tell me the same thing: that Retrospect works fine with their card. I think that we may be doing a trial run.

 

I'll report my findings back here.

Share this post


Link to post
Share on other sites

Hi All...

 

I installed the compression card on Friday, and have been running it through some paces over the weekend and today. The compression speed is quite good, but the big speed up is in the encryption+compression. Our encryption and compression rates are 3 to 4 times that of Retrospect's software based methods running on our dual 3GHz Xeon.

 

I will post more data here later, as more information becomes available.

Share this post


Link to post
Share on other sites

Hi!

 

We have been putting the card through its paces, and I do believe that it was a good buy (it cost around $1995). Here are some points that I thought worth mentioning:

 

1. After the proper Windows drivers are installed, Retrospect immediately recognizes the card as a "hardware/encryption accelerator". Retrospect even emailed this information to me. This is pretty nice.

 

2. To use the card's hardware compression, you must tell Retrospect to use "software compression". Retrospect then off-loads its "software compression" to the hardware accelerator, and indeed, I'm seeing several items in the Retrospect operations log stating the compression percentage followed by a "using hardware accelerator" notation.

 

3. To use the hardware encryption, you must create a new backup set with either AES 128 or AES 256 bit encryption. The card does not support DES or SimpleCrypt (and really, who cares...they are not nearly as secure as AES).

 

4. To enable both hardware encryption and compression, make sure that you create a backup set that supports AES encryption, and then in your backup scripts, enable software compression.

 

NOTE: When creating tape backup sets, make sure that you UNcheck the "use hardware compression" option. Yes, Retrospect will warn you that if you use encryption that it will have to disable hardware compression, but from my experiences, it appears that the hardware accelerated compression is not being used at all, even when you check "use software compression". It seems that if you do not initially disable the drive's hardware compression, the card's built-in compression doesn't work.

 

5. You may not get as good compression using this card (or Retrospect's software-based encryption, for that matter) vs. your tape drive's built-in hardware compression. Also, no matter how much compression you get, each tape member of a backup set will still read as the maximum that the tape can hold. For example, if you are using LTO3, your tape members will report a capacity of around 400GB, the tape's maximum capacity. This is in direct contrast to using the drive's built-in compression, in which each member appears to report 500-600GB (in the case of the LTO3). Indra networks explained these discrepancies to me as follows:

 

When you use tape drive's compression, Retrospect never sees the compressed data. It is writing uncompressed data to the tape and that is what it counts. Depending on how well the data compresses, it writes more or less before it fills up. When you use our hardware compression, Retrospect itself does the compression. The portion of Retrospect that actually writes to the tape and keeps track of the data written is now seeing compressed data. Hence, the size reported in that case is the compressed size.

 

AND

 

Now, why does the tape drive's hardware compression do a better job of compression? I think the answer lies in the way Retrospect does compression. When using our hardware, I believe they compress each file individually and then write it to tape. When using the tape drive's compression, Retrospect is just writing a continuous stream of bytes. The tape drive doesn't know where the file boundaries are and it just does compression in one continuous stream. That is what enables the better compression ratio. I think this difference is unavoidable.

 

That being said, I do believe that the card is working very nicely for us. We have seen that Retrospect is MUCH more responsive, as the compression and encryption are now being handled by additional hardware, freeing the CPUs to do other tasks. I also feel very good that our tapes are being encrypted at the highest possible level at a very fast rate.

 

I hope that this helps. I will try and post some numbers on compression ratios with and without encryption.

Share this post


Link to post
Share on other sites

Thanks for taking the time to write all that up! We're already doing all of those things, and in fact our hard disk backup sets are encrypted as well as the tapes, so when doing tape transfers we are CPU bound.

 

I want to get our transfer rates up above the 32MB/s LTO-3 minimum streaming threshold so the drive isn't continuously backhitching / shoeshining as it is now. We're using a 3.2GHz P4 and on this sort of transfer (where there's decompression / decryption and compression / encryption happening simultaneously) we're seeing a max of around 16MB/s, so we need to double the throughput.

 

Right now, I'm looking at either buying an Indra card or upgrading the backup server hardware to a system with a faster CPU. Does anyone know if Retrospect threads so that it would benefit from multicore in a case like ours (with simultaneous decompress / decrypt and compress / encrypt in a single execution unit)? Are there any benchmarks about compression / encryption performance with different CPUs?

Share this post


Link to post
Share on other sites

Here's our results. We backed up one of our Windows Server 2003 boxes over a gigabit network connection to a directly attached SCSI drive (a rather old drive, so the performance isn't the most spectacular; but you should get a general idea of what the card can do).

 

Total data backed up: 52.2GB

Encryption Used: AES 256-bit

 

Non-compressed, non-encrypted

Total time: 1 hour: 52 minutes

Compession: 0%

 

Compressed, non-encrypted

Total time: 2 hours: 1 minute

Compression: 12%

 

Compressed and encrypted

Total time: 2 hours: 2 minutes

Compression: 12%

 

From the results above, we can see that the compression adds only 9 minutes to the backup time versus no compression. Compression and encryption add only an additional minute on top of compression alone. That's not too bad, considering that the drive being backed up to is our "spillover" drive, old and creaky as it is.

 

Another very interesting and pleasing result is that the encrypted/compressed data is identical in size to the non-encrypted, compressed data. This eased some of my fears that the encrypted data might grow too large, even though it was being pre-compressed.

 

I'll post more of our general results in the next week or so.

Edited by Guest
minor grammatical correction

Share this post


Link to post
Share on other sites

On our LTO4 drives, we are getting a copy speed of over 1.2 gigabytes per minute, or about 20 megabytes per second. This is with both compression and encryption (AES256) enabled. It's not as good as the 1.8 gigabytes per minute with native drive compression, but it's still very acceptable for our environment.

 

I hope that this helps others looking into hardware assisted compression and encryption.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×