Error -205 (Lost Access To Storage Medium)

[ff319750-c948-480f-aca1-faa163601d06]

We're using Retrospect v8.2.0 (399) on a 10.6 MacPro connected via SCSI to a Tandberg Data Autoloader 7803 (8 bay, HP Ultrium LTO4 drive). We have shared storage that is mounted locally to this machine. We have always been able to back up individual projects or sets to a LTO media set with a single member (less than 800 GB) from the same storage (source) without issues. However we're now trying to backup to LTO Media Set with >1 member and failing with the following error in the log (Execution Incomplete in the status column for Activities and elsewhere):

 

+ Normal backup using Backup Assistant - 5/2/11 7:59 AM at 5/2/11 (Activity Thread 1)

To Media Set Dubs Ongoing Archive...

****** Warning: volume "ED2_ReadOnly" has the "Ignore ownership" setting enabled. ******

- 5/2/11 8:17:59 AM: Copying ED2_ReadOnly

5/2/11 4:48:54 PM: Snapshot stored, 10.9 MB

5/2/11 4:48:58 PM: Comparing ED2_ReadOnly

> !Trouble reading: "1-Dubs Ongoing Archive" (704012), error -205 ( lost access to storage medium)

5/2/11 5:03:06 PM: Execution incomplete

Remaining: 42249 files, 1,953.7 GB

Completed: 208 files, 43.6 GB

Performance: 4,030.5 MB/minute (4,031 copy, 4,005.7 compare)

Duration: 08:45:07 (00:06:34 idle/loading/preparing)

 

5/2/11 5:03:06 PM: Execution incomplete

Total performance: 4,030.5 MB/minute

Total duration: 08:45:07 (00:06:34 idle/loading/preparing)

 

We have selected a volume that (obviously) contains more data than a single tape can store, we've got hardware compression disabled. There are 8 members in the Media Set. The backup is a single event (not scripted) and appears to run fine throughout, however when it appears to have written the last files to the last LTO that was needed (in this case member #3), the autoloader loads the first LTO that was used as a part of that backup and then this error appears. When we look at the Media Sets and the status of the members after this happens, they report they have the correct amount of data (spanned).

 

Has anyone seen this or have any thoughts? We CAN do this selectively as we've done on a project basis, however we'd like to move to archiving EVERYTHING on these volumes which are huge. This is just one of a lot.

 

Thanks in advance.

 

-Troy.

[f30e6e60-a263-4d69-8f60-c92703669308]

a 10.6 MacPro connected via SCSI to a Tandberg Data Autoloader 7803

 

What's the SCSI host bus adapter make/model/firmware?

[ff319750-c948-480f-aca1-faa163601d06]

Thanks for replying. Here's the info for the card:

 

ATTO ExpressPCI UL5D LP

Kext version: 4.4.1

 

I've seen references to 'random' loss of SCSI communications, but we've literally never seen this until this use case when we started using a Media Set with multiple members that are tapes. It's happened several times (we've attempted the same backup twice and another similar sized backup from different volumes of the same storage set. All have failed in the exact same way, at the end (seemingly) the original tape (member) that the backup was started on is loaded back into the drive and then almost immediately we get that error.

 

Thanks again for taking time to think about this and reply.

 

-Troy.

[twickland]

When we look at the Media Sets and the status of the members after this happens, they report they have the correct amount of data (spanned).

What does Retrospect show for the tape member that's actually loaded when you go to Storage Devices and click the disclosure triangle for the tape drive?

 

Have you confirmed that the autoloader is qualified for use under Retrospect? (I didn't see anything with the exact model number you stated.) What about the tape drive itself? Do both devices have the latest firmware updates?

[ff319750-c948-480f-aca1-faa163601d06]

Hi Tim,

 

Thank you for replying. Here's what we see:

 

What does Retrospect show for the tape member that's actually loaded when you go to Storage Devices and click the disclosure triangle for the tape drive?

 

Have you confirmed that the autoloader is qualified for use under Retrospect? (I didn't see anything with the exact model number you stated.) What about the tape drive itself? Do both devices have the latest firmware updates?

 

Nothing unusual when looking at the tape drive. It appears as any other member does when loaded in the drive. It reports (as do others) it's format as U-416. It reports roughly 23 GB free space as do the other members that are 'full'.

 

I thought I saw the autoloader on the approved list, I'll double check. The tape drive is confirmed. The HP Ultrium 4 firmware reports as W51U, the storage loader reports as 0343. I'll double check each to see if there are any newer revs available, thanks for the suggestion.

 

We tried restoring some files from this media set and that process sometimes fails with the same error (-205) but sometimes (2 of 3 attempts so far) completes successfully. The one time it failed, member 3 was in the drive (that was the member of the media set this archive started on and also ended with loaded into the drive) and didn't unload or move, it seemed to try to run and then failed. We tried other files and the behaviour went like this: member 3 in drive to start, progress happened for a while, then member 3 unloaded and member 6 was loaded and the restore completed. Next attempt, member 6 unloaded, member 3 loaded, progress bar happened for a bit, then member 3 unloaded and member 6 loaded and restore completed successfully. This pattern seems odd as the files we were choosing to restore all came from the same folder of the original directory structure. Is this normal? We usually only deal with one tape member at a time and aren't sure what to expect I guess, but regardless, we've never run across the -205 errors.

 

-t.

[ff319750-c948-480f-aca1-faa163601d06]

Hi everyone,

 

As a update, worked with Tandberg to get the latest firmware for drive and loader installed. We're running a diagnostics test on the HP drive now to see if we can get the system to generate any SCSI bus errors. Will report back with results tomorrow.

 

-t.

[twickland]

Nothing unusual when looking at the tape drive. It appears as any other member does when loaded in the drive. It reports (as do others) it's format as U-416. It reports roughly 23 GB free space as do the other members that are 'full'.

My point in asking what tape was in the drive was that you indicated in your first post that the error occurred at the beginning of the Compare phase. The log indicates that the backup proceeded properly; that the autoloader correctly loaded members 1, 2, and 3 of "Dubs Ongoing Archive." When beginning the compare phase, the autoloader should have unloaded member #3 and loaded member #1. Sometime during this process, the drive stopped communicating with the host computer or with Retrospect. I was trying to find out whether member #1 was actually physically loaded in the drive, and if so, did Retrospect recognize which member was in the drive.

 

We tried restoring some files from this media set and that process sometimes fails with the same error (-205) but sometimes (2 of 3 attempts so far) completes successfully. The one time it failed, member 3 was in the drive (that was the member of the media set this archive started on and also ended with loaded into the drive) and didn't unload or move, it seemed to try to run and then failed.

That is certainly suggestive of a hardware problem. Running Tandberg Data diagnostic software and HP's Library and Tape Tools to check the operation of the autoloader and the tape drive might reveal your problem.

 

We tried other files and the behaviour went like this: member 3 in drive to start, progress happened for a while, then member 3 unloaded and member 6 was loaded and the restore completed. Next attempt, member 6 unloaded, member 3 loaded, progress bar happened for a bit, then member 3 unloaded and member 6 loaded and restore completed successfully. This pattern seems odd as the files we were choosing to restore all came from the same folder of the original directory structure. Is this normal?

Yes, this is normal. It simply means that some of the current files for this folder are on member #3 and some are on member #6.

 

We usually only deal with one tape member at a time and aren't sure what to expect I guess, but regardless, we've never run across the -205 errors.

The -205 errors are being generated because something is going wrong during the process of swapping tape members, if that swap occurs during some Retrospect activity (in your case, a backup or restore operation).

[ff319750-c948-480f-aca1-faa163601d06]

Thanks again for the reply and the information, we're learning more and more about this and now we know what's 'normal'.

 

After running the diagnostic software as you suggested, it generated errors. So we have some sort of SCSI hardware/communication problem. We're troubleshooting that now, but seem to be on our way to a solve.

 

Thanks again to everyone who took time to weigh in. I'll post the final (true) problem and resolution when we get it.

 

-t.

[ff319750-c948-480f-aca1-faa163601d06]

Thanks again for the reply and the information, we're learning more and more about this and now we know what's 'normal'.

 

After running the diagnostic software as you suggested, it generated errors. So we have some sort of SCSI hardware/communication problem. We're troubleshooting that now, but seem to be on our way to a solve.

 

Thanks again to everyone who took time to weigh in. I'll post the final (true) problem and resolution when we get it.

 

-t.

 

As a update: after looking over the error logs and running more tests it was determined that the SCSI path was indeed fine, it appears to be a problem with the HP Ultrium drive but only on playback/read operations. The unit is going in for warranty repair.

 

-t.