Rebuild catalog issue: where to save it?

[gasser]

After several years of working fine, Retrospect came up with this error on a disk backup set:

 

Grooming Media Set Daily 4TB-1 failed, error -2,241 (Catalog File invalid/damaged)

You must recreate the Backup Set's Catalog File.
See the Retrospect User's Guide or online help for details on recreating Catalog Files.

 

So I went into the Media Sets tab, selected my media set, chose the one member, and clicked the Rebuild... button.  Then it gave me a dialog asking where to save the catalog set.  But this dialog does not list my builtin hard drive, only my removable drives.  My current catalog is stored in /Library/Application Support/Retrospect/Catalogs/Daily 4TB-1.rbc.  Shouldn't I be putting the new catalog there?  In any case, I don't want to put the catalog on a removable drive.

 

Can I write it to a removable drive and then just copy it to /Library/Application Support/Retrospect/Catalogs?  If I do that, how will Retrospect know to find it there next time?

[Lennart_T]

It is strange that you can't see the internal drive.

 

I would, cancel the rebuild, restart the Retrospect Server computer and try again.

 

If you are forced to store the catalog file elsewhere, you can do that.

Then, in the Finder, move/copy the file to the correct location. Then double-click the catalog file (in the correct location) and Retrospect will pick up where it now resides.

[DavidHertzberg]

Somehow this reminds me of the situation nearly a year ago when I had two local drives named both "Macintosh HD".  As in your situation, Morrie, Retrospect 12.0 couldn't catalog onto the real boot drive —which was also the drive on which Retrospect was installed.  I recounted the problem in the OP in this thread.  Just sayin', that's all.

[gasser]

Thanks David and Lennart.  It took the advice of rebooting my Mac, and now Retrospect is seeing my internal drive.  I chose the original Catalogs folder that had the old catalog, and it's now rebuilding the catalog.

 

I wonder if I even needed a rebuild in the first place.  If it somehow lost access to my hard drive then it may have lost access to the catalog, too.  This happened right in the middle of a grooming, so maybe it was really complaining it couldn't read the drive.  Strange, since no other application on my machine was affected, and I only have one internal drive.  So maybe a simple reboot would have fixed the problem.  Well, too late...now I have to wait several days for a 4 TB catalog to rebuild.

[Lennart_T]

I remember error -2241 from my former employer, where I ran the Windows version of Retrospect. The catalog files normally resided on the boot C: drive, but some where on a RAID-5 fast NAS server. The error could occur regardless of where the catalog files was stored.

 

The cause of the error was never resolved, it happened for a number of years in a number of versions.

 

Recataloging took about 6 hours per TB, for backup sets stored on the NAS or about 4 hours per TB for a local SATA drive.

How is your 4 TB drive connected to the server? Is it a single drive? A RAID system?

[gasser]

It's a single Western Digital USB drive.

[Lennart_T]

USB 2 or 3?

And does the server have USB 3 ports or just USB 2 ports?

[gasser]

It's USB 3.  My machine is an iMac with 4 USB ports, but my disk is attached to it through a USB 3 hub.

[gasser]

The rebuild seems to have gotten through the whole disk, 3.6 TB's worth (the progress bar is at the end) but it's now just spinning on "Matching Daily 4 TB-1".  I don't know how long it's been doing that.

[Lennart_T]

USB 3 should be fast enough, hub or not.

 

I wonder what it is matching. Is it really still rebuilding? Or has it started to groom or taking a backup?

[gasser]

When I checked it today, the rebuild completed with errors and then two more daily backups were done, both completed successfully.  Does this mean my catalog rebuild worked?

 

The log from the rebuild looks like this.  Should I worry about this?

 

Executing Rebuild at 4/3/16, 7:51:01 PM

To Backup Set Daily 4TB-1...
Scanned /Volumes/Daily 4TB-1/Retrospect/Daily 4TB-1/1-Daily 4TB-1 up to backup set data file index 21,179
4/3/16 7:52:52 PM: Finished scanning backup set data files
!Generated MD5 digest for file "/Volumes/DAS 1TB/Movies/Family, Kids, Parties, Events/Budlong School Reunion.dv" does not match stored MD5 digest.
E-mail notification failed: error -597 (mail server not found)

The Disk Media Set data files "AA000009,824.rdb" to "AA000009,825.rdb" reported missing by user
!Generated MD5 digest for file "/Volumes/DAS 2TB/private/var/run/vmware/morrie_501/1406852652787466_23506/configFile" does not match stored MD5 digest.
!Generated MD5 digest for file "/Volumes/DAS 2TB/private/var/run/vmware/e0f4ff37d19d4c2bf2585715c1e64ed5" does not match stored MD5 digest.
!Generated MD5 digest for file "/Volumes/DAS 2TB/private/var/run/vmware/morrie_501/1407278322683259_58485/configFile" does not match stored MD5 digest.
!Media Set format inconsistency (5 at 50874360)
4/4/16 1:26:10 PM: 6 execution errors
Completed: 5,785,620 files, 3.6 TB
Performance: 4,118.9 MB/minute
Duration: 17:33:50 (02:13:35 idle/loading/preparing)
4/4/16 1:26:10 PM: Script "Rebuild" completed with 6 errors
E-mail notification failed: error -597 (mail server not found)

[Maser]

The "reported missing" file -- is the .rdb file actually gone?  If so, don't worry about that from the rebuild.

 

As far as the MD5 errors go -- I have nothing.  I wouldn't necessarily worry the files in /private/var/run, but you might try restoring the .dv file to make sure the backup copy works.

[gasser]

I tried to do a restore, but no matter how I try it, it says "Backup unavailable".  Yet, it's backing up right now (again, in grooming state since last night).  Is it not possible to do a restore while it's grooming?  I selected multiple backups from several weeks ago, all with the same problem.  I also just tried to browse the backup set with the same results.

 

I'm worried now I won't be able to access anything on the backup disk, even though it's happily completing nightly backups.

[gasser]

It said "needs media" and is asking me to choose a new member of my media set named "2-Daily 4TB-1".  Why would it do that, when I told it to groom?  Shouldn't it remove old files, or do whatever it needs, to fit my one backup drive?  The log looks like this.  

 

Normal backup using Daily Backup at 4/7/16, 10:33:45 PM

Adding to 3.7 TB of backup set data, starting at file index 21,297
To Backup Set Daily 4TB-1...
- 4/7/16 10:33:45 PM: Copying DAS 3TB
Using Instant Scan
4/7/16 10:34:16 PM: Found: 3623927 files, 498796 folders, 2.2 TB
4/7/16 11:24:42 PM: Finished matching
4/7/16 11:34:31 PM: Copying: 983 files (5.5 GB) and 1 hard links
4/7/16 11:37:44 PM: Building Snapshot...
4/7/16 11:37:44 PM: Checking 498,796 folders for ACLs or extended attributes
4/7/16 11:38:01 PM: Finished copying 27,251 folders with ACLs or extended attributes
4/7/16 11:41:34 PM: Copying Snapshot: 2 files (1.1 GB)
4/7/16 11:42:15 PM:
Grooming Backup Set Daily 4TB-1...
4/8/16 8:59:06 PM:
Groomed 10.0 MB from Backup Set Daily 4TB-1.
4/8/16 8:59:06 PM: Groomed 9.9 MB from Backup Set Daily 4TB-1.
E-mail notification failed: error -597 (mail server not found)

[Lennart_T]

You are running a backup. When Retrospect runs out of space, it tries to groom. If grooming (with your current groom settings) does not release enough space to perform the current backup, you are asked for a new member. In this case only 10 MB is groomed out. This indicates you are trying to preserve too many old backups.

 

So, what is your groom settings? (It is part of the "Media set" properties.)

 

A common mistake is to think the number is the number of days to keep the old backups, but it is the number of backups to keep. So, if you run a backup once a week, that makes a lot of difference.

[gasser]

I'm using Retrospect defined policy: last backup of each day for last week, last backup each week for last month, and last backup each month for 12 months.  I'm doing nightly backups.  Since my backup drive is 4 TB and the volume I'm backing up only has 2.13 TB of data on it, it seems unlikely that I've used up the entire backup volume with this policy.  I have no huge files that I'm constantly changing.  I really would like to avoid going to a 2nd member.  If it's really true that the volume is filled up, it must be because it has backed up some huge files I'm not aware of, perhaps ones that I no longer have.  Is there any way for me to find the largest files on the backup and delete them?

[Lennart_T]

If you have set that member to use at most (say) 60 %, then I would see the behavior you see. Check the Media Set properties (for this member).

 

Also, if you have (about) 1.6 TB of non-Retrospect data on the disk, you would see this behavior. Check in the Finder.

 

Maybe you should repair the disk using Apple's Disk Utility. A problem with the disk directory may cause Retrospect to see the disk as full.

[gasser]

I have it set to use 3725 GB or 100%.  In the finder, it says I'm using 3.7 GB and it's all in the Retrospect folder on the disk.  The disk has nothing else.  Disk Utility's verify says the disk is OK.  So, that gets me back to wondering whether something very large is backed up on it that I'm not aware of.

[Lennart_T]

What is in the Retrospect folder (and its subfolders)?

It looks like the disk is (or has been) a member of another Media set. That one uses 1.6 TB of data.

[gasser]

That's a thought, but it has just one subfolder, Daily 4TB-1, which in turn contains one subfolder, 1-Daily 4TB-1, which in turn contains; 9847 .rdb files, 4TB's worth.

[Lennart_T]

Since you have some errors and problems with this media set, I would stop using this media set. 

I would buy a new 4 TB disk and start a new media set with that disk as a member. I would then store the old backup disk off site, as an archive.

 

As an alternative, you could buy a new disk, start a new media set with that disk as a member and "Copy" the current media set to the new media set. But then you have the risk of copying the problem you have to the new media set.

 

Unless, of course, you don't mind losing the backups and just do a "recycle" to empty the disk member you have now.

[gasser]

The other choice I have is to buy a new disk and use that as the 2nd member of this media set.  How would grooming work with 2 members?  Does it need to have both member online for every backup, so it can do grooming across both members?

[Lennart_T]

The other choice I have is to buy a new disk and use that as the 2nd member of this media set.  How would grooming work with 2 members?  Does it need to have both member online for every backup, so it can do grooming across both members?

 

Since you have problems with member one, I would recommend strongly that you don't continue to use it as part of an active media set.

 

As for your specific questions: I don't have a clue. Sorry. I have always used only one member.

[Scillonian]

How would grooming work with 2 members? Does it need to have both member online for every backup, so it can do grooming across both members?

 

From the little testing I have done with multi members disk backup sets (on Windows) all members need to be online. Also it appears that when space is freed on earlier members it is not reused by later backups.