Jump to content
Frank N Berry

Retrospect 6.1.138 and VXA-320 Firewire loader - out of sync errors

Recommended Posts

I am uncertain, but it is the very latest and correct update as verified by tech support, several times.

Well, then, without that information, I cannot help you. Your problem statement indicates that you have been working on this for "the past few weeks". The corrected driver was only released a few days ago, and EMC has some version issues with dates, etc., on their web site that makes it impossible to tell what you have unless you provide the RDU version listed in the Retrospect log.

 

FYI, here is the link to the RDU version that is supposed to fix precisely this issue and none other:

RDU 6.1.15.101 - fix for VXA-320 out of sync

 

Russ

Share this post


Link to post
Share on other sites

Yes, that version number looks familiar.

 

Normally, I'd have contacted support immediately with such an issue, but the thing DID work for a short time, leading me to believe it was *my* fault. I chased this issue for some time before contacting EMC and finding out it was not my fault after all. My last successful backup was June 26.

 

Evidently, there are people with the Tandberg/Intel xserve combo that are having success, as I am getting the impression I'm the exception.

Share this post


Link to post
Share on other sites

Robin,

 

we are now using the latest German Retrospect 6.1.230 with the new RDU 6.1.15.101. Backup was working now without error 205 with Systemversions 10.5.3 and 10.5.4 for two backupsessions of about 60 gigs each and will hopefully work for more.

 

The issue seems to be fixed by now. Thanks!

 

The communication with the vxa320 via Exabytes libtool worked all the time.

 

regards,

Markus

Share this post


Link to post
Share on other sites

"An alternative to try would be to revert to RDU 6.1.11.101, which is the last RDU before this bug was introduced. "

 

 

Tried this, with much optimism I might add, but no change. I am finding an awful lot on search engines about other folks having the same issue. Have they all fixed the problem, and I am the only one that cannot seem to install an update?

 

Every time retrospect hangs, and I must force-quit, it is a matter of time before my xserve gives me a message that I must restart immediately. Might be 2 minutes, might be 10, but it's coming. Of course, if I sit there too long in disbelief, it just restarts itself (insert sounds of office personnel freaking out as the server drops out of sight).

 

Saw an Apple update for firmware today, downloaded and installed it, thinking this might be part of the puzzle, but no.

 

So I'm down to rsync. Yeah, I know, good grief. However. at the moment it is better than what I can get from Retrospect. I have hard drives and I have a nice internet connection, so I wont be totally up the creek here, but...I have this lovely Tandberg 10-tape backup device, and I have a retrospect license, and I sure would like them to work nicely together.

Share this post


Link to post
Share on other sites

would it be possible for you to provide exact information about your hardware setup, firmware versions, software versions, etc? That's never been provided by you, just "don't work, I'm frustrated" posts. We feel your pain and are trying to help.

 

(1) Which Tandberg (Exabyte) VXA-320 do you have?

a standalone drive? the version with the 10-slot 1u rackmounted autoloader?

 

(2) What is the hardware connection between your Tandberg (Exabyte) and the Xserve? Firewire? if so, 800 or 400? SCSI?

 

(3) if SCSI, what HBA controller do you have? only the ATTO SCSI HBAs work. Been there, done that.

 

(4) if SCSI, what firmware version and driver for the HBA?

 

(5) what firmware version for the VXA-320? There have been recent updates.

 

(6) If you have the 10-slot autoloader, what firmware version?

 

(7) What MacOS version on your Xserve?

 

It's also possible that all of your force quits have corrupted Retrospect's preferences. Have you tried deleting (or moving to the desktop) the contents of

/Library/Preferences/Retrospect

? (You will have to re-enter license info, etc., but if the test gives same results, then you can move the old stuff back to get your scripts, etc., back)

 

I don't have the same configuration you have (we have an Exabyte VXA-2 1x10 1u Packetloader (SCSI) attached to an ATTO UL4D in our Xserve, running 10.4.x Server), so I can't test / report your issues.

 

Russ

Share this post


Link to post
Share on other sites

"(1) Which Tandberg (Exabyte) VXA-320 do you have?

a standalone drive? the version with the 10-slot 1u rackmounted autoloader?"

 

the one I have is the 10 slot carousel.

 

 

"(2) What is the hardware connection between your Tandberg (Exabyte) and the Xserve? Firewire? if so, 800 or 400? SCSI?"

 

The xserve is brand new, came with fw800 only. Thats also the only type of connection on the back of the Tandberg.

 

 

"(3) if SCSI, what HBA controller do you have? only the ATTO SCSI HBAs work. Been there, done that.

 

(4) if SCSI, what firmware version and driver for the HBA?"

 

"(5) what firmware version for the VXA-320? There have been recent updates.

 

(6) If you have the 10-slot autoloader, what firmware version?"

 

Hmmm, going to have to get that. The machine is brand new, installed in June along with the xserve. How recent are the updates? Evidently I have to use VXAtool to get this information (you'd think they would make that available via the LCD on the front), and I'm not at the location right now. Besides, I promised everyone in the office I would not mess with the backup stuff during work hours anymore. One too many server crashes. I suppose this version information is going to be critical, so Ill get it as soon as I can.

 

"(7) What MacOS version on your Xserve?"

 

Leopard, latest updates. Here's output from sw_vers:

ProductName: Mac OS X Server

ProductVersion: 10.5.4

BuildVersion: 9E17

 

 

"It's also possible that all of your force quits have corrupted Retrospect's preferences. Have you tried deleting (or moving to the desktop) the contents of

/Library/Preferences/Retrospect

? (You will have to re-enter license info, etc., but if the test gives same results, then you can move the old stuff back to get your scripts, etc., back)"

 

Yes, went through that procedure. I kept both sets of preferences, swapping back and forth as I tried this and that. More than a couple things have been corrupted. I have not seen an application able to guarantee to take down a mac like this since OS9. I am absolutely amazed that this xserve cannot pull itself out of the hole upon a force quit of Retrospect. I have a laptop on which I run Mac OS, XP and FreeBSD, and while it does twist itself up once in a while with all that going on, it always manages to pull itself out. Maybe I should install the Retrospect license on my laptop instead.

 

"I don't have the same configuration you have (we have an Exabyte VXA-2 1x10 1u Packetloader (SCSI) attached to an ATTO UL4D in our Xserve, running 10.4.x Server), so I can't test / report your issues."

 

I figured as much. From what I see on Google, people having the issue are using an intel mac with 10.5 server.

Share this post


Link to post
Share on other sites

Ok, that's helpful. The Exabyte (Tandberg) updates are very recent for the drive.

 

Current drive firmware is here, with release notes:

Exabyte firmware downloads

 

If you have the Ethernet admin version of the autoloader (we do, helps in debugging these things), the Ethernet firmware is:

October 20, 2006 V10E0B, V10G0B, V90G0B

The ethernet firmware version will not affect what you are seeing.

 

The VXA-320 drive firmware is:

June 2, 2008 V13222

and the link is here:

VXA-320 firmware

 

Our VXA-2 drive and autoloader firmware was out of date when we purchased our unit.

 

Autoloader firmware is here:

Autoloader firmware

 

See if that helps.

 

Russ

Share this post


Link to post
Share on other sites

macinrack,

 

We're using the same tape loaders as you (Tandberg VXA-320 1x10 1U Firewire 800). The below might help you, it's mainly from our experience with Exabyte/Tandberg VXA products and Retrospect 6 for OS X. I'm not sure if the below is sanctioned by any of the parties involved, it's just what's worked for us...YMMV.

 

Previous to the latest Retrospect for OS X update, we were using 6.1.138 with an older Retrospect Driver Update (6.1.11.101) which worked fine from OS X.5 Server to OS X.5.2 Server (these are 2008 Mac Pros). After updating to X.5.3 and then X.5.4, we've seen major hangs in Retrospect. If you wait it out long enough, Retrospect will give up on the drive and you can Force Quit without bringing down the server. If you don't wait long enough (depends on the configuration and what the drive is reporting, but generally 4-6 hours), and you Force Quit, the Mac will kernel panic trying to communicate over Firewire (panic log will attest to this). It will also kernel panic if you Force Quit and then start up Retrospect soon thereafter. There was no solution to the eventual hangs in Retrospect until after the latest 6.1.230 version with the RDU 6.1.15.101. It's all working fine now, but you should note the following 'gotchas' from our experience:

 

- When installing the update to Retrospect 6.1.230, make sure you restart afterwards. Can be right away or later, just don't try to run a backup until then. We've found that even though you've installed a new Retrospect and it's RDU, sometimes the old one stays in memory until you reboot.

 

- The loader and tape drive in the device above have separate firmware. Make sure you are up to date on both. Russ lists the links above.

 

- Erase your tapes with VXA-3 headers before using them with Retrospect. You do this with the vxaTool and libTool from Tandberg. Load the tapes with libTool and erase with vxaTool. Then start up Retrospect and erase your tapes with it, labeled correctly with your backup set (So if you had a backup set called 'A', then you should erase your tapes in preparation, eg. 1-A, 2-A, etc.). This is so that Retrospect doesn't label the tapes arbitrarily (since they will be unlabeled after vxaTool gets done with them).

 

- If you have old backup sets/tapes, you should recycle the backup so the tapes and backup set are used like 'new'. The above is the preferred method. We've run into hangs when half of the backup was from 6.1.138 RDU 6.1.11.101 and the other was from 6.1.230 RDU 6.1.15.101. Another symptom is that Retrospect will display a Catalog out of sync error and doing rebuilds of it will not help. Starting clean is best.

 

- Go into Preferences and click on OS X. Uncheck Backup ACLs for Intel Machines. I'm not sure if this is fixed by now, but it was problematic in the past.

 

Hope that helps.

 

Share this post


Link to post
Share on other sites
- Erase your tapes with VXA-3 headers before using them with Retrospect. You do this with the vxaTool and libTool from Tandberg. Load the tapes with libTool and erase with vxaTool. Then start up Retrospect and erase your tapes with it, labeled correctly with your backup set (So if you had a backup set called 'A', then you should erase your tapes in preparation, eg. 1-A, 2-A, etc.). This is so that Retrospect doesn't label the tapes arbitrarily (since they will be unlabeled after vxaTool gets done with them).

Yes, you should put down VXA-3 headers first because the tapes ship with VXA-2 headers, which will give you 1/2 capacity.

 

No, it does no good to put Retrospect erased tape headers on the tapes beforehand. Sorry, but the following is a rant, and it one of my great irritations with Retrospect.

 

There is a long-standing bug in Retrospect Mac that I reported years ago under our service contract and which was fixed in Retrospect Windows years ago, but which EMC has refused to fix in the Mac version, and has even refused to log as a bug because, arguably (and I disagree), this "feature" (bug) is documented in the user manual. See the top right-hand paragraph on page 43 of the User Guide, "Tape Library Media Requests", which EMC support cited when I filed the bug report years ago. Compare with the release notes for Windows Retrospect 6.5, build 319, released in 2003, listing the following bug fix:

Retrospect searches through all the tapes in a library looking for an erased tape with the correct name, rather than using the first erased tape.

EMC Support has suggested that I make a "feature request" for this behavior to be changed, which, of course, I did years ago when EMC Support refused to log this as a bug and which I have done several times since then.

 

The Macintosh version of Retrospect completely ignores the headers on erased, pre-named, barcoded tapes in an autoloader. When it needs new media, it will choose, in an apparently random and non-deterministic fashion, from all of the "erased" tapes in the autoloader, regardless of whether some erased pre-named barcoded tape in the autoloader has the correct name of the now-needed member, and start using whatever erased tape it decides to use.

 

This makes management of barcoded tape inventory impossible. In combination with the above-discussed issue (tape error at BOT makes Retrospect believe that tape is erased), it creates the situation where existing members of any backup set in an autoloader can be overwritten unexpectedly by mistake when new media is needed.

 

End of rant.

 

I have been told that this bug ("feature"?) is expected to be fixed in Retrospect X, which is based upon the Windows code base. We shall see.

 

Russ

Share this post


Link to post
Share on other sites

Russ,

 

Couldn't agree with you more. This is an annoyance of ours as well.

 

Macinrack,

 

One more thing: If you've downloaded the Retrospect 6.1.230 installer posted before 7/14, you will get RDU 6.1.15.100 instead of the current installer which has 6.1.15.101. 6.1.15.101 is the one that fixes VXA-320 catalog sync errors so check your RDU just in case. The latest RDU is posted separately as well.

Share this post


Link to post
Share on other sites

Well, I have installed all the updates. Downloaded installed and ran vxatool, libtool, and then the firmware updates. Made sure I had the latest latest retrospect and the driver. Shut down everything, restarted, ran Retrospect, crashed. What a surprise.

 

Wiped out preferences (again), tried a bunch of different operations in Retrospect, all of which crashed the machine. So, half a dozen force-quits/crashes/restarts later, and I have given up again.

 

Erased tapes, more fun with vxatool and libtool.

 

It has now been over a month since my last backup. Honestly I expect this sort of aggravation when compiling Apache with PHP and MySQL on Solaris. I'm in record-breaking territory here. Never in 23 years of Mac use have I fought so hard to get something to work. I've got EMC saying their software works and is up to date. I have Tandberg saying their firmware works and is up to date. Apple says everything with their OS is up to date and works. Yet, I have a backup system that is presently worthless. Come on, somebody has to have an answer. This whole situation is absolutely ridiculous. If someone can point out a hoop through which I have not yet jumped, I'd be grateful. Otherwise, suggestions for hardware/software products that work would be welcome.

Share this post


Link to post
Share on other sites

Ok, macinrack, let's see if we can summarize, because your problem is a bit different from the original poster's, and you don't specifically say whether you have done the same troubleshooting problems that he has.

 

You seem to have:

Intel Xserve of unknown model and configuration (despite requests for info) running MacOS Server 10.5.4

 

No indication that you have run Xserve memory diagnostics or CPU diagnostics. You could have RAM issues or hardware issues because you say you have a "new" Xserve. You don't report whether Server Admin is reporting any ECC errors. You don't provide any information from the console log regarding crashing, only that Retrospect hangs:

Every time retrospect hangs, and I must force-quit, it is a matter of time before my xserve gives me a message that I must restart immediately. Might be 2 minutes, might be 10, but it's coming. Of course, if I sit there too long in disbelief, it just restarts itself (insert sounds of office personnel freaking out as the server drops out of sight).

No information about this message from the Xserve. No console log info. Sounds like either uncorrectable ECC errors or else the Firewire driver has scribbled all over memory. Retrospect can't help either of these issues. Retrospect doesn't talk directly to the Firewire port.

 

Maybe I should install the Retrospect license on my laptop instead.

well, that would provide useful information as to whether it might be a hardware problem on your Xserve.

 

Tandberg/Exabyte VXA-320 1x10 1u PacketLoader (Firewire 800). Still no info from you on firmware version, but you say you have "the latest" firmware. No information on whether you have run the Exabyte diagnostics in vxatool.

 

No information on whether you have tried backing up to a file backup set. That could provide useful information whether your problem is isolated to the Firewire subsystem.

 

Sounds like a hardware issue on the new Xserve. Testing on another machine (your laptop) would provide useful information as to where the problem might lie.

 

Russ

 

Share this post


Link to post
Share on other sites

Update. I managed to get my hands on a firewire 800 external HD. Had to be 800, only thing available on the back of my xserve. Started a new backup set, and my backup is flying along.

 

So it seems the xserve and Retrospect work well together, so long as the Tandberg is out of the mix.

 

It's also true that the xserve and the Tandberg are okay with each other, hooked up and powered on, so long as Retrospect is not launched. Wish I had an option to Retrospect, even just for testing purposes, to see if that crashes too. Actually, now that i think about it, the Tandberg utilities communicated with the device just fine, in various operations, for what that's worth.

 

Here are some version numbers:

Retrospect 6.1.230

RDU 6.1.15.101

Drive Firmware V13222

Autoloader Firmware V1A110

 

Right now, and until I am armed with new things to try, I am working remotely, with no physical access to the machine. I am unable to run some of the utilities you mention. Does not matter much anyway, as I promised I would no longer launch the Retrospect/Tandberg combination during business hours. However, I have been using BSD for a long time, and have run some of my favorite memory-sucking unix apps to see if I can trip up the server. All I can say about this Apple is that it is a very impressive little machine. Everything we throw at this server, aside from Retrospect/Tandberg backups, the server hardly notices.

 

So, based on my successful 318 gig Retrospect backup to an external firewire HD, plus my overwhelming success with the xserve in every other area, I have to bet the xserve is fine. I'm open to anything I can run via ssh, if you have suggestions.

 

I would like to try a different mac just for yucks, but I'd need to "borrow" one of the intel mac pros in the office, and that presents its own set of issues. Seems to me the test would not be of much value unless I tested with intel architecture- right? Even then, it might be a bit different than an xserve, or correct me if I'm wrong.

Share this post


Link to post
Share on other sites
I would like to try a different mac just for yucks, but I'd need to "borrow" one of the intel mac pros in the office, and that presents its own set of issues. Seems to me the test would not be of much value unless I tested with intel architecture- right? Even then, it might be a bit different than an xserve, or correct me if I'm wrong.

No, testing with another Mac of any vintage would confirm that the Tandberg Firewire works well with Retrospect.

 

Retrospect runs under Rosetta and is not Universal Binary, so it would be the same code running either way. Biggest variable would be the OS version.

 

I'm open to anything I can run via ssh, if you have suggestions.

vxatool diagnostics. That would exercise the VXA-320 via firewire without the Retrospect variable. If you run vxatool without any flags, it gives you the options. I would suggest:

./vxaTool -S

(to show the devices, e.g., TAPE1)

./vxaTool TAPE1 -t -m 100

(read/write tape test, 100 MB - use a scratch tape)

 

Russ

 

 

 

Share this post


Link to post
Share on other sites

Update.

 

I have not yet had an opportunity to connect a different mac to the tandberg device. However, I do have other info.

 

vxatool and vxalib seem to work well. At least, they do not take down the xserve. First, I tried:

./vxaTool TAPE1 -t -m 100

and got a medium error:

"Error occurred, Sense=03H [Medium error]"

It seemed to me that could mean something with formatting, so I removed all tapes with vxalib, put them back in, and formatted the tapes with VXA-3 format:

./vxaTool Tape1 -e 3

Then I ran that test command again:

./vxaTool TAPE1 -t -m 100

This time I got a different error:

"Error occurred, Sense data not valid"

I then ran diagnostics, output to a file. I looked over the file, but it is largely garbley-gook, evidently needing something special to read it. After running the diagnostic, I tried that test again:

./vxaTool TAPE1 -t -m 100

and this time I got the medium error again.

 

As I mentioned, I have not yet tried retrospect with another mac. However, retrospect works just fine on this server, so long as it is connected to an external drive (firewire). Also, this retrospect/tandberg/xserve combination was working for a while, until updates were installed. It is hard to say if it was the 10.5.4 apple update, or the retrospect update, or the tandberg updates, or a combination of all three that caused this thing to puke, but it *was* working once upon a time.

Share this post


Link to post
Share on other sites

Found something on these errors, but I dont see how this narrows the search at all:

 

Medium Error 3h Indicates that the command terminated with a non-recoverable error condition that may have been caused by a flaw in the tape or an error in the recorded data. The tape drive may also return this sense key if it is unable to distinguish between a flaw in the tape and a specific hardware failure (sense key 4h).

 

Hardware Error 4h Indicates that the tape drive detected a non-recoverable hardware failure (for example a device failure or parity error) while performing the command or during a self-test.

 

Of the ten tapes in the carousel, only 3 or 4 have been used, and very lightly. I don't think there are flaws in the tape. The formatting went well, anyway.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×