Jump to content

File "xxx" appears incomplete and Can't read file "xxx", error -43 problems


Recommended Posts

OK - I posted a question about this issue some time ago, but that died, and I had put it under a thread about an iPhoto library as the symptoms were similar.

 

Host computer: MacPro running 10.4.9 Server, 1GB Ram, about 4-6 AFP users.

Backup Drive: Exabyte/Tandberg/whoever this week VXA2a, Firmware 2122 (which seems to be the latest)

Connection medium: Firewire 800

Retrospect Version: 6.1.126, RDU 6.1.11.101

 

Symptom:

When Retrospect runs backup (either scheduled or manually triggered), it will fill the logs with lines like the following:

File “AI_Collect_Read_Me.rtf” appears incomplete, path: “RAIDBoot/Shared Items/MacServer/Roche2/SCRIPTS/Illustrator_Collect_for_Output/AI_Collect_Read_Me.rtf”.

 

It will do this for virtually every single file it comes across, which I base on:

 

10/2/2007 8:23:14 PM: 7059 execution errors.

Completed: 7127 files, 13.3 GB

 

I just noticed also, that this is always under the "comparing" log entry:

 

The other message that it will present, though never during the same backup as the above messages.

 

Can't read file “pairing_pin.tif”, error -43 (file/folder not found), path: “RAIDBoot/Shared Items/MacServer/Roche1/Rod/DM_Layout/Links/pairing_pin.tif”.

 

This one is on the "copying" section, but again, I don't see the two different entries during the same script run. Also, the second entry doesn't seem to be a frequent one, but the first is present for nearly every run.

 

We migrated the data over the network (I think) from a G5 to the MacPro, which had everything installed fresh and we started new backup sets (or recycled, I've lost those notes).

 

Now, I suppose that I could just disable the "verify" option, but we REALLY want to be sure that the files were backed up and so on, so that's not an attractive option.

 

I will not have easy access to the machine for further info gathering, and it will take time to try anything suggested, so please be kind. I'll grab the logs and such for offline reference at least.

 

TIA, guys.

 

Doug

Link to comment
Share on other sites

Doug,

 

I seem to recall seeing this on our Xserve G5 a couple of times (we've got the SCSI version of the tape drive and autoloader that you have). As I recall, it was related to the fact that Retrospect's launch was confused, and it may have been related to the "two copies of Retrospect running" issue or the "Retrospect fails to autolaunch" issue. I always saw it after a failure to do a scheduled autolaunch.

 

The fix for me was to cause the "retrorunfile" to be rebuilt so that the scheduled autolaunch happened correctly:

quit Retrospect,

trash /Library/Preferences/Retrospect/retrorunfile

launch Retrospect,

quit Retrospect

 

That causes the retrorunfile to be rebuilt for the next scheduled autolaunch.

 

See if that helps.

 

Quote:

MacPro running 10.4.9 Server, 1GB Ram, about 4-6 AFP users.

 


by the way, that is way too little RAM to run Retrospect on a server doing AFP. Retrospect, being ancient and not updated for the Intel Macs (which have only been out for almost three years), is emulated under Rosetta on an Intel Mac, and you've got the overhead of AFP, Mac OS X Server, and Retrospect. I'd suggest at least 2 GB RAM on an Xserve G5 to run Retrospect, at least 3 GB RAM to run Rosetta, Retrospect, and AFP on an Intel Mac. Even with as few users as you have. It's possible that the tight cramp on RAM is contributing.

 

Russ

Link to comment
Share on other sites

> Symptom:

> When Retrospect runs backup (either scheduled or manually triggered),

 

Assuming that you confirm that Retrospect is authenticating properly (check Activity Monitor to see that it's running as root) then this doesn't seem to be related to the auto-launch bug.

 

 

> it will fill the logs with lines like the following:

> File “foo_file.txt” appears incomplete, path: “Source_Volume/foo/foo_file.txt”.

 

> I just noticed also, that this is always under the "comparing" log entry:

 

The "appears incomplete" log entry does not make clear if the problem is with the Source or with the Destination (maybe Robin can help us here?). They only similar entries I have in any of my logs come after "Trouble Communicating" type errors, which suggest that the log entry is referring to files written to the backup media.

 

 

The first test I'd do is confirm if this problem occurs with all/any Backup Set Types.

 

Create a new File Backup Set and define /foo/ as a Subvolume (using, of course, your actual path).

 

- Do the same errors show up?

 

 

> The other message that it will present, though never during the same backup as the above messages.

> Can't read file “fluff.tff”, error -43 (file/folder not found), path: “Source_Volume/fluff/fluff.tiff”.

 

This is an entirely different symptom, which may be unrelated.

 

The only reason that Retrospect knows the name of this file is because it saw it during the scanning phase, and noted it in its file tree (which is, I believe, kept in memory). When the scanning is complete, Retrospect begins to copy the files it knows about. If a file is added to the Source, Retrospect won't copy it (since it doesn't know about it). If a file is removed from the Source, Retrospect will notice that it's missing and present this error.

 

So the first question would be, is it possible that the user deleted the file while the backup was in progress?

 

 

> Now, I suppose that I could just disable the "verify" option, but we

> REALLY want to be sure that the files were backed up and so on, so

> that's not an attractive option.

 

No, no no. You're getting Compare errors; now would not be the time to disable that!

 

> I will not have easy access to the machine for further info

> gathering, and it will take time to try anything suggested, so please

> be kind. I'll grab the logs and such for offline reference at least.

 

 

I think the first thing you should do is test to see if this is in some way device related, or if it is Source related. A test File Backup Set could prove the null of the later (if there is no error, it is _not_ Source related). Knowing for sure what the log entry means (Source vs. Destination) would be quite helpful here, too.

 

 

 

Dave

Link to comment
Share on other sites

Quote:

> Symptom:

> When Retrospect runs backup (either scheduled or manually triggered),

 

Assuming that you confirm that Retrospect is authenticating properly (check Activity Monitor to see that it's running as root) then this doesn't seem to be related to the auto-launch bug.

 


Dave, while I agree with your analysis, in our case it did appear to be authentication related, and was caused by a corrupt retrorunfile. All of the several times that I have seen this (except one, and that one time was the single "two Retrospect copies running" instance that I saw). No, I don't know why waving the rubber chicken fixes it.

 

Russ

Link to comment
Share on other sites

appears incomplete means that the file on the backup is corrupted.

 

Retrospect is doing a byte by byte verification of all of the files. The file on the backup does not appear identical to the original file.

 

Basically the data on the backup is not readable and probably can not be restored. You will probably get the same error doing a tools>Verify

 

I would do the following:

 

1) Run Disk utilities on the source drive just to make sure the drive isn't having any issues

 

2) Try different backup media or a different backup device. The data written to the destination is broken, probably due to a backup device problem. Change tapes, clean the tape heads, change FW cables, etc.

Link to comment
Share on other sites

OK - I'm going back tomorrow morning to check on another issue (file that was lost between recycles.... eek.

 

So, I had planned to just do a full restore to a test destination to see what happens.

 

What I plan to do (as of the moment) is to remove that retrorunfile and do a new backup to a fileset on the machine and then test restore.

 

We've been cleaning the tape drive when prompted (24 hours, I think).

Short of doing a test backup to the internal drive, there's not much of an option for another backup volume yet.

 

Seeing as I transferred the preferences from the G4 server to th MacPro, I expect the problem is related to the retrorunfile.

 

Memory - is it really that beneficial to have 2-3gig in a server doing only those tasks? Wow, I know Rosetta is a pig, but sheesh. Not arguing, just shaking my head.

 

At any rate, thank you for the directing replies and I'll try to keep you updated as I find out more going forward.

 

Doug

Link to comment
Share on other sites

Quote:

Seeing as I transferred the preferences from the G4 server to th MacPro, I expect the problem is related to the retrorunfile.

 


 

Although I don't like to push back against Russ on much of anything, I see this as a big, fat red herring.

 

Russ provided scant details about his experiences, but the fact that you see this condition when you _manually_ launch Retrospect makes clear that the retrorun file (who's only job is to _automatically_ launch Retrospect) is unlikely to be the issue. Go ahead and delete it (it will be rebuilt the next time you quit/launch Retrospect). But if everything you have told us so far is true (and I never forget Rules #1 and #1a), it probably won't make any difference.

 

Take Robins' advice and be suspicious of the complex device to which you are writing your valuable data files; it very likely is not serving you well.

 

 

Dave

Link to comment
Share on other sites

I agree about the "appears incomplete" error - I wasn't addressing that error; I was only talking about the 7000+ errors in a single setting. I agree that the retrorunfile issue is a red herring unrelated to the "appears incomplete" error.

 

The VXA-2 drive is supposed to do read-after-write verification on the fly, and the error recovery is pretty good, too. If you've got the latest firmware, which the 2122 is, you can do a diagnostics dump of the error buffer from the drive using the latest vxatool utility:

Code:


mail:~/Documents/Exabyte/VXA_firmware admin$ ./vxaTool

vxaTool V4.62 -- Copyright © 1996-2006, Exabyte Corp.

 

Supported drives:

VXA1 VXA1T VXA2 VXA2(OEM) VXA172 VXA320 VXA320(OEM)

 

Usage: vxaTool <device> [-deimtCFIP]

-d <filename>

Retrieve diagnostics dump to <filename>

-e [1/2/3] (optional)

Erase tape (set VXA-1, 2 or 3 format, on VXA-2/VXA-3 only)

-i

Identify the device.

-m <x> (used in conjuction with '-t')

Write and read <x> Megabytes (100-99999)

-t [W/R/F] (optional)

Test write/read functionality of the device.

 

Optional [R]ead-only, [W]rite-only and [F]ull-tape flags

*** WARNING *** Any data on tape will be overwritten!

Use a SCRATCH tape when testing the tape drive.

-u

Unload tape (also in combination with -t)

-C [1/0]

Enable (1) or Disable (0) hardware compression.

NOTE: Compression state returns to default after a reset or power cycle

-F <filename>

Flash Firmware from file <filename> to the device

NOTE: Do NOT interrupt a firmware update in progress!

Your device may become inoperable if you ignore this warning.

 

-S

Scan the system for available devices

 

-I [0-13]

Change the device Inquiry to mimic a different vendor.

Specify 0 to return to factory default.

 

<<< WARNING! '-P' options should only be used when instructed by >>>

<<< Exabyte Technical Support >>>

-P S/C

Prefer (S)peed or ©apacity optimisation.

-P B/Q

Prefer (B)usy or (Q)ueue status while loading a tape.

-P I[012]

Show (I) or set (Ix) response after receiving an immediate SCSI command

Where x means: 0 = Return Busy, 1 = return Check Condition, 2 = Queue

-P L[01]

Show (L) or set (Lx) Write Density lock on VXA-172 and VXA-320

Where x means: 0 = Unlocked, 1 = Locked in VXA-3 format

 

Example : vxaTool Tape1 -t -m 100

 

NOTE: Please use 'vxaTool -S' to list available devices on this computer

 

Done


That would give some insight and would be helpful when you call Exabyte (now Tandberg Data) support. I've found Exabyte's support to be excellent.

 

Russ

Link to comment
Share on other sites

OK - So I returned briefly this morning to the customer site.

 

I did not remove the "retrorunfile", but tried to restore a folder from the backup that had run last night. After it requested a previous tape and then the current one, it failed with "error 212" media is erased. *sigh*

 

So, I have left it running a verify of the said backup set. They will swap out tapes and/or let me know of the status and I'll return in the morning (again) to see what's going on. I expect I'll be on the phone with Exabyte shortly after getting there.

 

I did notice that the "file appears incomplete" was only coming up on one backup set, which is off-site at the moment. In case that means anything.

 

Thanks again!

Doug

Link to comment
Share on other sites

Quote:

it failed with "error 212" media is erased. *sigh*

 


Ok, now this is a different issue that we may be able to solve. Been there, done that.

 

Retrospect has an odd algorithm for deciding whether a tape is "erased". The decision seems to be based on whether there is an error at BOT.

 

Last summer we had a batch of bad VXA-2 tapes from Exabyte. Well, not totally bad, just marginal. This was combined with an Exabyte firmware release that had issues with some of the error recovery routines (see the release notes). It was hard for me to figure out the cause, but eventually I tossed the batch of tapes, got a new batch. That cured the problem on new backups. For old backups, the firmware update this past January (you have 2122, which is even more recent) seemed to help, and I found that the new firmware (including 2122), coupled with doing a cleaning cycle just before doing the backup run, allowed the tapes to be used. Once Retrospect accepted the tape (no error at BOT), the normal error recovery routines seemed to kick in, and things were fine. At least that's what we found with our Exabyte VXA-2 1x10 1u PacketLoader (SCSI).

 

So, here's the drill:

put in a cleaning tape, do a manual cleaning cycle from the front panel (there's a still-different Retrospect bug if you do a cleaning cycle from Retrospect, the program never comes back after the tape is moved to the drive, you have to quit and then relaunch Retrospect to recover; but that's a different issue. Sigh). Then, fire up Retrospect's Configure > Devices, move the problematic tape to the drive from the autoloader. If Retrospect sees it as not erased, you are good to go, retrieve files. If, during your restore, Retrospect gives this error and asks for the correct tape, use Configure > Devices to unload the tape and reload it, or unload / clean / reload, and Retrospect will move on with the restore because it now sees the tape with the right header.

 

There's another bug in Retrospect that this interacts with (Retrospect ignores pre-named, barcoded, pre-erased tapes of the correct name when looking for an erased tape, and whimsically uses whatever "erased" tapes it finds in the autoloader to scribble on), but I'm not going off on that rant right now. I've tried for years to get that bug fixed under our service contract, and I've given up hoping that Retrospect will ever see any bug fix maintenance programming. The bug was fixed a long time ago in the Windows version, but not in the Mac version. Grumble. Write protect your tape with this backup on it before you put it in the autoloader so that you don't get bitten by this bug in the process.

 

Let us know if this works for that particular tape. You might also be working from a bad batch of tapes, like I was last summer.

 

Russ

Link to comment
Share on other sites

Cleaning tape before restore (backup too?) might do it, eh? Otherwise, I'm suspicious of the tapes now, so I'll give that a shot tomorrow as well.

 

Just to be sure, this is a single drive unit connected via FW800. But the instructions make sense still.

 

Sorry for changing the thread, but there is obviously more than one issue going. Thanks again guys! I really appreciate it.

 

Doug

Link to comment
Share on other sites

Quote:

Cleaning tape before restore (backup too?) might do it, eh?

 


Yep. See details in my post. But do it from the front panel, manually, not using Retrospect, because Retrospect gets confused (at least on the SCSI version attached to ATTO UL4D). It may be an Exabyte issue, I'm not in the mood to point fingers - I just want to get it to work.

 

Quote:

Just to be sure, this is a single drive unit connected via FW800.

 


Ok, we've got the same drive in the 1u rackmount Exabyte autoloader (SCSI). But the firmware is the same in the drive as what you have, so the issues are still the same.

 

Quote:

there is obviously more than one issue going.

 


Yep. Took me a long time to figure out whether our issue was tapes, firmware, or both. Turned out to be both for us. Takes a long time to do thorough testing with tapes when you've got hundreds of gigabytes. The 2122 firmware seems to be OK.

 

Have fun.

 

Russ

Link to comment
Share on other sites

OK, I'm back onsite. The verify failed after they fed the third tape in (which was reported as "erased" when I tried to restore data from it).

 

Gave "Trouble in Retrospect: Internal consistency check failed: Assertion check at "elem.c-918"

 

*sigh* Only reference I find to that particular code is that it is "fixed" with RDU 6.1.5.something. Theirs is running 6.1.11.something. And the backup set is a new one from July.

 

So, I've done the diag dump with vxaTool, but it's useless to me. I've marked the suspect tape as "missing" as it's not usable anyway and I'll run a vxaToom test of about 5 gig (it has 4.4gig supposedly) and we'll see what the tool says about it.

 

Thanks again,

Doug

Link to comment
Share on other sites

OK - Exabyte/Tandberg is looking at media as the problem. I didn't realize that the problem tapes are over 2 years old and were used in different drives with different firmware.

 

So, I'm going to see if I can get them to buy a new set of tapes and we'll manually erase them in the drive and see how they work.

 

Thanks again guys, and you may not hear from me for a bit - we'll see how things go.

 

Doug

Link to comment
Share on other sites

You might ask them how valuable their data is. A VXA-2 / VXA-3 tape (X23, nominal 160GB/320GB uncompressed/compressed for VXA-3, nominal 80GB/160GB uncompressed/compressed for VXA-2) is about $70. We never reuse tapes, but instead archive them when they fill. Data is valuable, tapes are cheap (relatively), and the IT budget should reflect that.

 

You might want to look at the number of passes spec'd for these tapes in their lifetime, realizing that you start from BOT on each backup, whip out to the place where the backup is to be done, make one write pass, rewind, verify read pass, rewind. That head rubs on the tape a lot faster than one might think.

 

Russ

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...