Using less than half Tape capacity!

[sarm]

We are running a stand-alone LTO3 backup system, not connected to a network. You bring your drive, plug it in to the firewire port, run a backup, and that's it - simple!

 

Over the past couple of days, I have run a couple of test backups from a drive containing 310GB of data, to a *new* media set and tape. For reasons unknown, it copies 160-200GB of data and then requests the next tape.I gave it a new tape and it eventually completed the copy/verify process.

 

The members indicate a capacity of 390GB so why isn't it being used?

 

Thanks

 

Specs:

G5 Dual 2.3GHz

4GB RAM

OS X 10.5.8

 

Retrospect 8.2.0 (399)

Fast catalog rebuild = off

 

Atto UL4S s/n UL4S112490

Firmware April 10 2009

Atto Driver 4.4.1

Disable tagged command queueing

Sync rate 160DT

 

Quantum LTO-3 HH SCSI drive (Firmware 2181)

Edited September 30, 2010 by Guest

[twickland]

The members indicate a capacity of 390GB so why isn't it being used?

Any estimate of tape capacity is just that--an estimate. The amount of data you can actually get on a tape depends on a lot of factors, including the compressibility of the data (assuming you're using compression), how close the rate of data you're supplying matches what's optimum for the drive (too little data results in the drive having to constantly backhitch), etc.

 

LTO-3 tapes have a native capacity of 400 GB. In our setup with network backups, we typically get 500-600 GB on a tape with our data mix, so your numbers do seem a bit low. You aren't by chance using LTO-2 tapes in your drive, are you?

[sarm]

Compression is off. Fast catalog rebuild is off. Tapes are definitely LTO3 (says 800GB on the tape itself!).

I've run 2 test backups in the past day or so, and each time it didn't fill the first tape.

 

The mind boggles.....

 

[rhwalker]

Note that Retrospect writes until it gets an error (EOT, read-after-write error, etc.).

 

In a sense, that's probably good - if you have old tapes that are starting to fail, you need to move on to the next tape and stop using the current tape member.

 

Perhaps the drive needs cleaning?

 

Russ

[sarm]

Thanks Russ, just picking up on this one again.

 

So, in effect, the drive could report a non EOT error but that could still cause Retrospect to signal a request for the next blank tape?

 

Do all tape-based systems operate in a similar manner or is this a Retrospect specific way of handling errors?

 

Thanks

Steve

 

[rhwalker]

Thanks Russ, just picking up on this one again.

 

So, in effect, the drive could report a non EOT error but that could still cause Retrospect to signal a request for the next blank tape?

Yep. Been there, done that, and it's probably the right design decision.

 

As an aside, note that the physical EOT strip is not an "error", but is really a warning indicator that the driver better finish up transfers soon before the tape unwinds off the reel. When the physical EOT strip is detected, there is still a bunch of tape past that on the reel (amount is determined by the spec for the particular type of tape, and is related to how long a CPU is allowed to react to the physical EOT sensing, considering the tape speed, etc.), and the driver is supposed to finish the current set of blocks "real soon" and then put down two file marks (EOF), which signify the "logical" EOT, past which the tape driver should not attempt to read or write. Hitting physical EOT does NOT stop the transfer in progress - it simply sets a status bit that the driver checks when it gets the "transfer complete" interrupt for the current tape block being written. In the absence of other errors (read-after-write error, etc.), the last block written has been correctly and fully written, even though EOT has been passed. But the driver better not keep writing or grave disorder may result. Some tape applications, if they want to ensure that each tape contains a complete set of files and no partial files, may do a "Space Reverse File" to move back over the previous file mark that terminated the previous file, then "Space Forward File" to move past that same previous file mark, then write a second file mark to indicate logical EOT (two file marks in a row) after the previous complete file. That's a detail to be handled by the application. End of trivia.

 

Do all tape-based systems operate in a similar manner or is this a Retrospect specific way of handling errors?

Many tape systems just give up on error and require operator action as to what should be done.

 

The operator is supposed to monitor the console output that spews forth on the teletype. Perhaps you are too young to remember.

 

Russ

[adrianlambert]

We've had a similar problem occur this week that wasn't previously happening.

Our tapes would usually fill to about 15GB from the stated capacity (No compression is used) but this week I recycled a set and found it used around the amount stated by the OP. We're using LTO4s as well. The one thing that differed to our usual method was that previously we'd supply the drive a tape manually when it wanted it rather than have a set of tapes already available for the set to use.

[rhwalker]

Well, we don't have LTO4 (probably will migrate to that when/if Retrospect 8 stabilizes/matures and becomes ready for production use in our environment).

 

However, here's a possibly-related bug that you might want to use to inspire your investigations.

 

An earlier release of Retrospect had a similar bug for VXA-3 tapes (physically identical to VXA-2, but recorded at twice the density). The first time that the tape is written, a small burst pattern header is written by the drive at BOT, depending on the drive's density setting, to signal that this tape is a VXA-3 tape. When a tape is inserted, the drive reads this header as it positions at BOT, and reports back the density of the tape. The tape density can't be changed once writing has commenced on the tape, but the tape can be erased and the tape density can be changed by that process.

 

Out of the wrapper, for backwards compatibility, blank VXA tapes would appear to be VXA-2. Retrospect had a bug (perhaps now fixed) whereby, in a VXA-3 drive, it would always use those tapes as VXA-2 (so that half VXA-3 density would be seen). The workaround was to use the Exabyte/Tandberg command line diagnostics to erase the blank tape into VXA-3 density before it was used the first time.

 

You may be seeing a similar thing with LTO-3 / LTO-4. Just a thought for you to investigate.

 

Russ

[sarm]

Russ,

Thanks for your replies, informative and educational as ever!

 

At the risk of making this thread OT, I have moved the card, cables and drive to a windows machine so I can run Quantum's xtalk diagnostics tool.

I've run 2 tests on different tape brands, each resulted in the following entry in the logfile:

LTO Error Rate Test -------- : Failed

Problem Detected

 

So, over to you Quantum support.....

 

 

Steve

[rhwalker]

Thanks for following up and letting us know.

 

Retrospect has always had a problem that it expects the tape drive subsystem to work perfectly. Over time, the Retrospect classic (6.x and earlier) matured to the point that somewhat-useful diagnostic messages would be put in the log if a tape error occurred, so that those with detailed error manuals for the particular drive could figure out what the problem was.

 

It's very hard to debug and test tape error routines - you have to be able to reproduce the error (some are transient) and then figure out how to handle it. Been there, done that in my youth, when I wrote some Unix tape driver routines as part of a 4.2BSD port. What fun.

 

Good luck on getting the drive fixed.

 

Russ

[offthewall]

We run LTO4 with a Quantum SAS drive. Had a similar issue when we switched to Retrospect 8. We switched from Quantum tapes to Sony tapes and now get 2-3 times the capacity. Go figure...

[sarm]

Just a quick update on this.

After enduring Quantum's tech support and RMA process, we have a replacement drive.

I'm glad to say that we are now back to filling each tape with a consistent ~390GB uncompressed data, using the default NVRAM settings on the Atto SCSI card.

 

From day one of using the original drive, I was constantly fiddling with the Atto settings, trying to get it backup without some kind of error. This progressed onto tapes not being filled to capacity.

 

Hope this is of some help!

[mbooker2]

I'm having similar problems!

 

I recently upgraded an XServe with a 48 tape LTO4 autoloader from R6 to R8 (8.20 399). The backup was stopped when it finished tape #22 (this was brand new Sony media). The tapes hold 800Gb uncompressed. The backup was approx 12Tb. Software compression was off. Erase all the tapes, revert to R6 and the capacity goes back to normal.

 

I have a similar story with a new MacPro (10.6.6) 6Gb RAM, new ATTO H680 and new SAS Tandberg LTO5. These tapes should hold 1.5Tb uncompressed and I am getting anything between 200Gb and 1.6Tb per tape.

 

 

[Lennart_T]

I recently upgraded an XServe...

Is that a PowerPC XServe or an Intel XServe?

 

What is the source for the backups? Clients on a slow network? Fast local hard drives? Fast server on a fast network.