Chunk Checksum error (-641) on first backup

[khasel]

I installed Retrospect, configured scripts, sets, and such. Everything looked correct.

 

I ran a backup of all of our servers, including the Exchange server information store, but wanted to leave the brick-level backups of individual mailboxes out until I had a complete, solid backup in place. Everything ran well.

 

I added the individual mailboxes to the backup, and it was running fine until it had completed about half of the mailboxes. Then every subsequent mailbox failed with (name changed to protect the innocent):

 

 

6/7/2005 2:24:06 AM: Copying Fake Name on mail

 

Trouble matching Fake Name on mail to Server Backup Set A, error -641 (chunk checksum didn't match)

 

 

After these mailbox failures, the rest of the backup completed normally.

 

Any advice or suggestions?

 

-Kurt

[khasel]

As a follow-up, last night's backups saw the problem get worse, as now _all_ of the individual mailbox backups fail with the same error -641.

 

Advice? Suggestions?

 

-Kurt

[natew]

Hi

 

What version of Retrospect do you have and what version of Exchange?

 

It could be that the Retrospect preference files are corrupt. Try renaming the Retrospect preferences folder and setting up a new mailbox backup.

 

The preference folder is c:\documents and settings\all users\application data\Retrospect.

 

Thanks

Nate

[khasel]

We're running Retrospect Multi-Server 7.0.265 on a Windows Server 2003 machine, and the mail server is Exchange 2003 on Windows Server 2003.

 

It strikes me as odd that the Retrospect preference files would become corrupt in the middle of a backup.

 

But I'll try your suggestion about renaming the prefs folder and running a test backup.

 

Thanks for your attention to this.

 

-Kurt

[khasel]

I renamed the prefs folder, re-setup Retrospect, added the mailboxes as volumes and ran a mailbox-only backup which succeeded without generating any of the chunk checksum errors. It seems as though corruption may indeed have occurred, but it still strikes me as odd that it would complete some number of mailboxes successfully before becoming corrupted in such a way that only mailboxes would fail.

 

How would you suggest I proceed from here? Perhaps going back to the original prefs, forgetting the mailboxes as volumes and then re-adding them?

 

-Kurt

[nembot]

Yes!!! this worked as well by renaming the file.

[khasel]

I returned to the original prefs folder, "forgot" the individual mailboxes as volumes, re-added them, and then ran the backup. They all, once again, are reporting the "trouble matching", chunk checksum error (-641).

 

Since all of the mailboxes were "forgotten" and then re-added, shouldn't this have reset them as far as Retrospect is concerned? If so, how can they be generating "trouble matching" problems?

 

Advice and suggestions are welcome.

 

-Kurt

[khasel]

Latest update:

 

Over the weekend, a full (recycle) backup ran and successfully backed up all of the mailboxes. Last night the backup reported correctly backing up some, then failing with chunk checksum -641 errors on 4 mailboxes in a row, then backed up more, then another error, then successfully backing up the rest.

 

I need to be able to consistantly back up these mailboxes as the prospect of restoring the entire data store to access mails from a single user is sub-optimal.

 

I'd appreciate any advice, suggestions, or explanations as to why this is happening.

 

-Kurt

[natew]

Hi

 

 

 

Since you get the error with new Retrospect prefs it may be the mailboxes themselves that are failing to checksum properly.

 

 

 

After you get the 641 error are you able to do a normal file backup to this backup set? Or is the backup set corrupt after that?

 

 

 

One thing to keep in mind:

 

For Ecxchange backup Retrospect uses the built in Microsoft APIs. That means we call the built in back up functions and Exchange/Windows does the grunt work of actually extracting the databases. That also means that errors that occur are often exchange errors that are then reported to Retrospect.

 

 

 

Type control + Alt + P + P at any screen in Retrospect and set Mailbox logging to 7. This should give us some of the errors that exchange is reporting to Retrospect.

 

 

 

Thanks

 

Nate

[khasel]

Thanks for the response Nate.

 

With new prefs I _didn't_ get errors, nor did I get any when I did a full (recycle). But the first "incremental" after the full, a few showed up, and then another the next day. The same ones have been failing for the rest of the week.

 

I'm certainly willing to believe that the mailboxes themselves have issues, but it strikes me as odd that they'd back-up initially, and then fail.

 

Setting the mailbox logging to the highest level is an _extremely_ good idea, I'll do that immediately and should have more info tomorrow.

 

Thanks again.

 

-Kurt

[Gandalf465]

I have an instance of this same error, but in my case, I get the failed checksum match when trying to search for files in my backup set. I backup nightly successfully to this backup set. I can view and restore files by viewing "selected files". I only get this error when using "find files" on my backup sets.

 

I have not tried the recommendation on tossing the preferences as suggested.

 

Any thoughts would be appreciated.

--

Daniel

[natew]

Gandalf465,

 

Can you do us a favor and post this again to the Windows server forum? This forum is for issues related to Microsoft Exchange backup.

 

Thanks

nate

[khasel]

An update:

 

Even with the mailbox logging set to the highest level, I'm not getting any more info in the logs regarding the cases of the chunk checksum error I was still getting last week.

 

This weekend's full ran successfully, and for the first time the ensuing daily also ran successfully, with no chunk checksum errors on any of the mailboxes.

 

While this is obviously a good thing, if the errors recur then the idiosyncratic nature of the problem will make debugging more difficult.

 

-Kurt

[natew]

Hmmmmmm

 

So this is starting to look like a problem with the Retrospect preferences again. Either that or corruption in the catalog.

 

Try this:

Unlicense the exchange server in the Retrospect configure->volumes window. Then close and restart Retrospect.

 

Open up the reports -> Backup report screen and delete all events.

 

Open up the activity monitor and delete all history and events

 

Close and restart again.

 

Then re-license the exchange server.

 

That should reset just about all of the database information that could cause this kind of trouble.

 

What type of backup device are you using? Where is the catalog file stored?

 

Thanks

Nate

[khasel]

Nate-

 

The backups have been running smoothly all week, with nary a chunk checksum error in sight. While this is good, it always worries me when persistant errors just magically vanish without my intervention.

 

Regardless, the situation appears to have remedied itself. If the errors recur, I'll let you know.

 

Thanks for your assistance and attention.

 

-Kurt

[khasel]

And the errors have recurred...

 

After successful, full backups over the weekend, the first daily backup afterwards had two of the -641 chunk checksum errors (names changed, obviously):

 

- 7/5/2005 12:49:40 AM: Copying John Doe on mail

Trouble matching John Doe on mail to Server Backup Set B, error -641 (chunk checksum didn't match)

 

And two brand-spanking new errors like this:

 

- 7/5/2005 12:59:46 AM: Copying Bob Smith on mail

Retrospect error code error -3401 (unknown)

Reported by function kMsg_DirScan

You must login to an Exchange container.

Scanning incomplete, error -3401 (unknown)

 

And also one error like this:

 

- 7/5/2005 1:33:40 AM: Copying Mary Jones on mail

mlvtrDirStore: Bad tree, duplicate dir found, name=Junk E-mail.

Scan skipping multiple subfolders with same name Junk E-mail.

While scanning volume Mary Jones,

Folder Mary Jones\Junk E-mail\,

Duplicate dirid detected: 0x00000000

7/5/2005 1:33:40 AM: No files need to be copied

7/5/2005 1:34:54 AM: 1 execution errors

Duration: 00:01:13 (00:01:16 idle/loading/preparing)

 

I'll check through the site for more info on the new error types, but wanted to get the results up here.

 

-Kurt

[khasel]

Another status update:

 

The two errors of type

 

Retrospect error code error -3401 (unknown)

Reported by function kMsg_DirScan

 

have vanished, so those appear to have been (hopefully) a one-time thing.

 

The duplicate directory error is still there and I haven't had a chance to get with the user to examine his set up.

 

The two chunk checksum errors are still there, and two more have occurred in new mailboxes.

 

-Kurt