No Data Compression on Tape Backup - Multi-Server Win 7.7

[gnoelken]

I recently upgraded the tape drive I use with Retrospect 7.7 to an HP Ultrium 1760 SAS Autoloader. This tape drive is identical to one on another Retrospect backup server (version 7.6) I administer (except the new one is SAS, the older one is SCSI).

 

After running the first 'Transfer Snapshots' script on the new tape drive, I discovered that no compression on the tapes was performed. These are 800Gb/1.6Tb, LTO-4 tapes (note the 'Used' portion of the 1st 3 tapes on the image below):

 

When I check the drive status with HP's Online web interface, I see that 'Data Compression' is turned off:

 

 

I can turn drive compression on with HP's Tape & Library tools, but as soon as I start Retrospect, the drive compression becomes disabled.

 

When I view the properties in Retrospect of the new tape drive, compared to my 'working' system, I see that there is no Data compression under Media ==> Format. My 'working' system's tape drive with compression has 'Format: U-416, DC' (I assume the DC stands for Data Compression), while my new drive only lists 'Format: U-416':

 

New Drive:

 

'Working' Drive:

 

I also used HP's Library and Tape Tools to test the drive compression. It tested positive with read and write results of 2.61:1 compression. But the test had a warning that the 'hardware algorithm is switched off. This was probably set by your backup software...':

 

 

Also, all my backup sets are set to use 'Hardware Compression' and none use encryption (which would disable the hardware compression).

 

In addition, I called HP tech support. We insured that the firmware was up-to-date on the new system, confirmed that the drive's data compression worked properly and did a complete drive assessment. All tests passed.

 

System Details:

OS: Windows Server 2008 w/Service Pack 2

Retrospect: 7.7.325, + Driver Update and Hot Fix, version 7.7.3.102 (64-bit),

Tape Library: HP Library: 1x8 G2 AUTOLDR

Tape Drive: HP Ultrium Gen 4 DC

Server: Dell Precision T3400 64-bit, 4Gb Memory, 3.16 GHz Core 2 Duo CPU

 

I did purchase a service agreement and opened a ticket, but thought I would check in the forum for any help/thoughts/suggestions.

 

Greg Noelken

 

 

[Mayoff]

http://kb.dantz.com/article.asp?article=5658&p=2

 

[gnoelken]

Robin,

 

Thank you for your quick response. I've read the FAQ, but please let me explain further.

 

I have 2 Retrospect servers (Multi-Server). On one, call it 'Server A', I have Retrospect 7.6. I have an HP Ultrium 1760 (HP StorageWorks 1/8 G2 LTO-4 Ultrium 1760 SCSI Tape Autoloader) attached to it. On 'Server B' I have Retrospect 7.7 with an HP Ultrium 1760 (HP StorageWorks 1/8 G2 LTO-4 Ultrium 1760 SAS Tape Autoloader) attached to it.

 

Server A has worked flawlessly since being installed in '08. Without any setup or configuration it uses hardware compression to compress the data onto the tapes. On the image below, from Server A, I have highlighted; the drive information, the 'DC' (indicating that the media is utilizing Data Compression), and a Backup Set that does indicate that I am indeed utilizing data compression (1001.1 G) on a 800 Gb/ 1.6 Tb tape:

 

Server A info:

 

 

 

I've copied the same info on 'Server B':

 

Server B info:

 

 

Note the drive info is the same (the revision is different because Server A is SCSI and Server B is SAS). Note the differences; the media is not using data compression (missing the 'DC' flag), and all the tapes fill up at slightly less than 800 Gb.

 

And, just to be sure, the backup sets (all of them) are set to use hardware compression:

 

Hardware compression:

 

 

So, I am fortunate that I do have Server A. I *know* that an HP Ultrium 1760 does provide hardware data compression *with* retrospect multi-server backup. I also know, after going through HP's tech support, that the new tape drive is working properly. I also observe that, when I manually set data compression 'On' with HP's Library & Tape Tools on Server B, then start Retrospect, data compression is immediately switched 'Off'. I also have the most recent version of Retrospect and Driver Update. The backup file types are very similar on Server A and Server B; Windows OS, Linux, and some Mac.

 

Questions:

 

Why is data compression immediately turned off when Retrospect starts?

 

To the forum members: Is anyone else using the HP Ultrium 1760? Getting hardware data compression?

 

Am I missing a setting somewhere that is disabling the hardware compression?

 

Can you provide technical assistance, perhaps a driver hotfix, that would resolve this issue?

 

I may take the time to attach the drive from Server B onto Server A and test it. Would that help support the compression issue if it did indeed work on Server A?

 

Thank you for the help and regards,

 

Greg

[Dbitter]

Retrospect cannot "upgrade or downgrade" datacompression on the fly. Meaning you must create an entire new backup set to change the datacompression to on or off. I am guessing that when you ran that initial backup the tape drive wasn't set to hardware compression, thus creating a backupset that isn't compressed, and since Retrospect cannot change the compression, it is turing of DC whenever you load. I would attempt creating a new backupset making sure that all hardware compression is turned on and then running a new backup with the new backup set. Hopefully once it has DC on it will continue with the DC in the future.

[gnoelken]

With many more hours of experimenting I have found what causes the tapes to either have data compression (DC) or not. While creating an experimental backup set, I wound up adding a tape to the backup set that had DC. But I wasn't sure how I did it. After many more trials, I found how I can create a backup set with (or without) DC.

 

All of the backup sets as described in my previous posts were created or recycled in the same manner. Since I archive the backup sets every 6 months, it was time to do so on July 1. I just purchased the HP Ultrium 1760 and new tapes to coincide with the archive. I put new tapes in the library and then selected 'New Backup Set' from the 'Action' menu on my backup sets. I added the new tapes in the manner I always do: click the 'Members' tab, then 'Add'. When the 'Add tapes to backup' dialog appears, I click on the member I want to add then click add. I do that for all members, then click 'Done'. After running my 'transfer snapshots' from my disk backups to the tapes I discovered that the tapes did not have DC and posted my first post above.

 

Apparently it was the way I added the members to the backup set that determined whether DC was implemented or not. Here are my tests:

 

I created a new backup set 'A'. I made sure 'hardware compression' was checked and continued until the 'Add new members' dialog box appeared:

 

 

I'll add members 5, 6, and 7.

 

To add member 5, I click on member 5, then click on the 'Add' button:

 

 

Once it is added I view the properties. Note that there is no 'DC' flag under the 'Format:' section:

 

 

I repeat for mediums 6 and 7; click on the medium, then on 'Add'. Again no DC flag appears for the properties for either one. And when I look at the HP remote web interface, it confirms that DC is off:

 

 

I click 'Done' after adding the media, then look at the properties of 'Backup Set A':

 

 

2344.2 Gb for 3 media, or 781.4 Gb/tape. So once again confirming no data compression. (I do realize the available space is theoretical.)

 

But now I create another backup set and this time it has full compression. I go through the 'New Backup Set' dialog boxes until I get to the 'Add new members' dialog box. *This time*, before I click on the member and then on 'Add', I *move* the member into the drive, then click 'Add'.

 

 

Once it is added, I look at the properties:

 

 

Wow! DC is now a property of the 'Format:'.

 

I *move* the next media into the drive and click 'Add'.

 

Again, DC!:

 

 

And the third!:

 

 

And the properties:

 

 

Wow, 1562.8 Tb/tape!

 

And the HP remote web interface!:

 

 

So, the only difference is; when adding members to Backup Set A, I clicked on the medium, then clicked 'Add', for Backup Set B, I moved the medium into the drive, then clicked 'Add'. This action is repeatable...

 

Questions:

 

Is this a bug, or isolated to my environment? I think it is a bug...

 

What will happen when, during a backup, new media is required and Retrospect uses a blank one to automatically add it to the backup set? Will it be flagged for Data Compression?

 

I am running my backup set transfer now. It took 3+ tapes with no compression to complete. I'll post my results of this test after it completes.

 

Greg

[gnoelken]

I've completed my backup set transfer. The data is compressed after adding media to the backup set as described in my previous post.

 

Before (same backup sets):

 

 

After:

 

 

From 3+ tapes down to 1+ tapes. This may not be a bug and just be unique to my system. At least I can get hardware data compression.

 

Any explanations would be appreciated.

 

Regards,

 

Greg

[johnnymacgo]

Hey Greg,

 

It seems like you found a bug to me. I'm using Retrospect V7.7.325 (64-bit) with driver update and hotifix V7.7.3.203 connected to a Dell PowerVault TL4000 library that also has a LT04 SAS tape drive. I was not getting any compression at all on all of my tape backup sets (created with compression enabled) and the DC was not displayed under "Format: U-416". I recycled 4 of my tape backup sets and added the tapes as members just like you said, put them into the drive first and then add them. Now they all show us as "Format: U-416, DC". I will be running backups this weekend that normally take 8 tapes, lets see how many tapes it takes now.

 

Good find,

 

Johnny Mac

Edited July 22, 2010 by Guest

[rhwalker]

This may or may not be related to a bug (feature) of VXA tapes that I've seen for years. VXA drives read a header at the beginning of the tape to determine whether the tapes are VXA-2 or VXA-3 format.

 

If blank tapes are "pre-erased" and pre-named by Retrospect as members of a particular backup set (or erased by the Exabyte/Tandberg utility with the proper settings for compression), a header will be written to indicate that the tape is compressed for VXA-3. If a tape is used out of the wrapper without this procedure, the tape will be used as VXA-2 (with the associated loss of VXA-3 compression). When the drive goes through its BOT load sequence for tapes loaded out of the autoloader, the drive looks at this header and sets an appropriate status bit as to the format of the tape (then rewinds to BOT), and Retrospect seems to respect that format bit.

 

It's not possible to change the VXA-2 / VXA-3 format of a tape unless that initial header is overwritten.

 

I've always treated this as a "feature" and have always pre-erased our tapes with the correct member name at the same time I barcode the tapes. The LTO drives may be doing a similar thing.

 

Good catch, though.

 

Russ

[gnoelken]

Hey Greg,

 

It seems like you found a bug to me. I'm using Retrospect V7.7.325 (64-bit) with driver update and hotifix V7.7.3.203 connected to a Dell PowerVault TL4000 library that also has a LT04 SAS tape drive. I was not getting any compression at all on all of my tape backup sets (created with compression enabled) and the DC was not displayed under "Format: U-416". I recycled 4 of my tape backup sets and added the tapes as members just like you said, put them into the drive first and then add them. Now they all show us as "Format: U-416, DC". I will be running backups this weekend that normally take 8 tapes, lets see how many tapes it takes now.

 

Good find,

 

Johnny Mac

 

Johnny Mac,

 

Thanks for replying. At least I know I'm not alone with this problem. From what I see, I think you're backup sets will have data compression.

 

Please post the results.

 

Regards,

 

Greg

[johnnymacgo]

Hey Greg,

 

Yes, I finally did get compression on my tapes so there is defiantly a bug within Retrospect V7.7 that you found! This is the first time I have seen my tapes go over 800GB of capacity. I got 1100GB on some of them. We were looking into buying new tapes because we were running out of capacity on the ones we have and you saved us from doing that!

 

Were you going to log a call with them so they fix this issue?

 

Thanks for posting this,

 

Johnny Mac

[gnoelken]

Hello Johnny Mac,

 

I'm glad that you were able to get compression too.

 

I did open a ticket. I was told that "we have an escalation open" on it, which I assume means they have elevated it to their upper level support. Your posting on the forum helps. It means my problem is not unique to me.

 

I'll certainly post any input/updates I receive from them.

 

Regards,

 

Greg

[AndrewA]

I was able to reproduce this with an LIB-D81 library with and SDX-500c tape drive so we have a Hardware Engineer looking at it right now.

[posadzka]

Hi!

 

I have this problem too - i've found that some of my tapes have capacity over 1,5GB but some only 750MB. I didn't know why and when it happened. Using your simple solution (manually moving tape to drive before adding member) I've managed to get compression!

BTW - great problem description and solution, big respect from me. Hope The Retrospect programmers will fix this bug soon.

 

I have ActiLib 1U Autoloader LTO-4 HH SAS and Windows 2008 RC2 Standard 64bit.

 

Best Regards!

AP

 

Edited August 16, 2010 by Guest