Cannot rebuild catalog

[gasser]

I groomed my 2-disk 5-TB USB-attached media set and it reported "catalog out of sync"'. I rebuilt the catalog (which took 2 days) and tried a backup. It failed again with the same error. I did this 2 more times over the span of more than a week, and still the same problem.

 

I don't know if this matters, but the rebuild always ends with hundreds of lines like this:

 

TString::setBufferSize:65,545 capped at 65,535

TString::setBufferSize:65,548 capped at 65,535

TString::setBufferSize:65,545 capped at 65,535

TString::setBufferSize:65,546 capped at 65,535

TString::setBufferSize:65,546 capped at 65,535

TString::setBufferSize:65,546 capped at 65,535

TString::setBufferSize:65,546 capped at 65,535

TString::setBufferSize:65,547 capped at 65,535

TString::setBufferSize:94,144 capped at 65,535

 

so I guess the rebuild never finished?

 

Since that happened, I upgraded to Retrospect 14 and tried a rebuild again. This time I got these errors:

 

[*] ArcDisk::arcExportSessionFileTo: failed to create "/Volumes/Offsite 3 TB/Retrospect/Offsite 3 TB/1-Offsite 3 TB/AA129964524749290000u.session" (11/3/12 9:41:14 PM) (voldex: 0): error -1,115 (disk full)

[*] ArcDisk::arcExportSessionFileTo: failed to create "/Volumes/Offsite 3 TB/Retrospect/Offsite 3 TB/1-Offsite 3 TB/AA129989553118820000u.session" (12/2/12 7:55:11 PM) (voldex: 0): error -1,115 (disk full)

[*] ArcDisk::arcExportSessionFileTo: failed to create "/Volumes/Offsite 3 TB/Retrospect/Offsite 3 TB/1-Offsite 3 TB/AA130015261015140000u.session" (1/1/13 2:01:41 PM) (voldex: 0): error -1,115 (disk full)

[*] ArcDisk::arcExportSessionFileTo: failed to create "/Volumes/Offsite 3 TB/Retrospect/Offsite 3 TB/1-Offsite 3 TB/AA130068041369500000u.session" (3/3/13 4:08:56 PM) (voldex: 0): error -1,115 (disk full)

[*] ArcDisk::arcExportSessionFileTo: failed to create "/Volumes/Offsite 3 TB/Retrospect/Offsite 3 TB/1-Offsite 3 TB/AA130072390719190000u.session" (3/8/13 4:57:51 PM) (voldex: 0): error -1,115 (disk full)

3/20/17 7:23:34 AM: Execution stopped by operator

Completed: 3,719,023 files, 2.4 TB

Performance: 2,092.4 MB/minute

Duration: 20:33:37 (00:50:40 idle/loading/preparing)

[*] UAllowSystemSleepAgain: IOPMAssertionRelease failed, err = -536,870,200

 

Why am I getting a disk full on the backup volume? Why is it trying to write to the backup volume to rebuild the catalog? My catalog is not being stored there.

[DavidHertzberg]

I have a suggestion, based on gut feeling rather than knowledge.  (If you want knowledge, wait for Lennart Thelander to respond to your thread, or contact Retrospect Support.): 

 

As to your questions about your rebuild with Retrospect Mac 14, did you by any chance used to have Fast Catalog Rebuild specified for this Media Set in Retrospect Mac 12 or earlier?  That feature was eliminated in Retrospect Mac 13, but page 220 of the Retrospect Mac 14 User's Guide still has this paragraph:  "Note: Retrospect has a feature, found in the Options tab of a Media Set, called Fast Catalog Rebuild where, every time it starts a tape after the first in a Media Set, it writes the current Catalog to the beginning of that tape. This speeds rebuilding of the Catalog by only requiring the last piece of media belonging to the Tape Media Set to be scanned by Retrospect. The Fast Catalog Rebuild option can also be used on Disk Media Sets that don’t have grooming turned on."  The messages you are getting sound to me as if somehow this feature is still being activated.

 

Anyway, my suggestion is that you Remove and Add your Media Set.  Write down the Media Set name, and check its spelling twice.  Then, after Removing it, go to root/Library/Application Support/Retrospect/Catalogs/ in the Finder and delete the file with that name—if you don't find the file there, then that mis-cataloging probably explains your problem.  Then Add a Media Set with the exact-same name, and start doing Add Member for all the members in the old version of your Media Set—and you should make sure beforehand you know on what drives these members are.  The process is described starting on page 100 of the Retrospect Mac 14 UG.

 

P.S.: Forgot to say that after Adding back the Media Set, you will have to add it back to all the scripts that use it and re-add the schedules that use it within these scripts and—for any Copy Media Set script that uses another Media Set in addition to the one you Added back—drag it in the Summary detail list to the sequence you want to be processed.

[DavidHertzberg]

If you think this is a bug that should be fixed by Retrospect Inc., you will have to submit it as a Support Case.  For English speakers, that is done by going here, and filling out the form (sorry, I don't know what the equivalent addresses are for non-English speakers, but they can figure it out from their appropriate Retrospect website address).  IMHO this is quite reasonable; obliging you to fill out the form provides Retrospect Inc. with useful details about your Retrospect installation that they would otherwise have to query you for.

 

As a result, Retrospect Inc. will pay no attention to your post in this sub-forum.  On 12 December 2016, in response to a letter I snail-mailed to Mayoff,  I received an e-mail through a Mayoff account that was signed by JG Heithcock, CEO, Retrospect, Inc..  In it he says "From reading your letter, I think the main issue is that you view the forums as a good place to talk to us, Retrospect, Inc. But we view the audience of the forums as restricted to our customers [my emphasis]. The one caveat we have made on that is for feature requests, largely as we would like to see if other customers also agree on the desirability and feature set for these requests."

 

That means that the only audience for "Retrospect bug reports" in this sub-forum will be other administrators of Retrospect Mac.  Nevertheless, by posting in this sub-forum you are providing a useful service to us fellow Mac administrator peasants—one that is denied to administrators of Retrospect Windows who are evidently considered too peasanty (insert appropriate smiley here) to benefit from such bug notifications.  Thank you.

 

Please be aware that the "description of your issue" in the Support Case form is IME limited to about 2000 characters by the Support Case software.  If you go over that limit your "description" will be broken up into a "description" plus one or more "additional notes".  The same is true for any additional notes you may later post yourself.  I suggest that, to avoid the appearance of choppiness in your Support Case, you create your case in a post in this forum and then copy it paragraph-by-paragraph to your Support Case. 

 

If this post sounds formulaic, that's because I intend it to be.  I intend to post it in every new thread that appears in this sub-forum, unless the OP indicates that he/she has or will open a Support Case for the bug that the thread reports.  Of course, Mayoff could take 5 minutes of his time to post a slightly-more-polite version of this post as a  "sticky thread" that will always appear at the top of the forum.  I don't intend to hold my breath until that happens (insert appropriate smiley here).

[gasser]

Thanks, David. I may have had fast catalog rebuild turned on, I don't know. I will try the remove/add option if all else fails. One thing curious about this media set: Retrospect shows it as "Used 4.9 TB" and "Capacity 2.7 TB". Clearly that makes no sense. It consists of two 3-TB drives. It seems that it's only counting one of the drives in its Capacity calculation.

[Maser]

In order to do a full catalog rebuild (which I've done many times) -- make sure you remove all the .session files before you do the rebuild.    It's also possible that after the groom, one of your .rdb files is hosed (I had that happen and had to go through a whole bunch of iterations to figure out which one was messed up -- and it ended up being some 11K .rdb file causing my rebuilds to fail...)

[gasser]

Maser, are  you saying those .session files are disposable?  I see a whole bunch of them, but I'm skeptical that would avoid the "disk full" errors.  Yes, the disk is full--that's why I'm grooming.  FWIW, my last backup attempt seemed to work, but failed grooming:

 

Normal backup using Offsite Backup at 4/3/17, 7:07:35 PM

    4/3/17 7:07:36 PM: Finished scanning backup set data files

    To Backup Set Offsite 3 TB...

    -  4/3/17 7:07:35 PM: Copying DAS 3TB

    4/3/17 7:18:22 PM: Found: 3771325 files, 490230 folders, 2.6 TB

    4/3/17 7:19:27 PM: Finished matching

    4/3/17 7:20:28 PM: Copying: 151683 files (345.8 GB) and 34 hard links

    -  4/3/17 7:20:53 PM: Verifying DAS 3TB

    Grooming Backup Set Offsite 3 TB, optimizing for storage...

    [*] ArcDisk::arcExportSessionFileTo: failed to create "/Volumes/Offsite 3 TB/Retrospect/Offsite 3 TB/1-Offsite 3 TB/AA129666052405140000u.session" (11/24/11 9:47:20 AM) (voldex: 0): error -1,115 (disk full)

    [*] ArcDisk::arcExportSessionFileTo: failed to create "/Volumes/Offsite 3 TB/Retrospect/Offsite 3 TB/1-Offsite 3 TB/AA129964524749290000u.session" (11/3/12 9:41:14 PM) (voldex: 0): error -1,115 (disk full)

    ... and so on for 100 or so lines.

[Maser]

They are disposable if you want to do a *full* catalog rebuild (sorry -- been away from the board for a while while work got in the way).

[DavidHertzberg]

Morrie, FYI there is a Retrospect Mac 14 Release Note that says the Engine was fixed so that "Thorough Catalog Rebuild now correctly deletes previous .session files (#6598)."

[gasser]

That's funny, because I'm using Retrospect 14.

[DavidHertzberg]

Then IMHO it sounds increasingly as if Morrie's only remaining practical choice is to do what I suggested in the second substantial paragraph and P.S. of post #2 in this thread.

[gasser]

I haven't tried David's method in post #2 of this thread yet, but I'm wondering if that will help.  I first tried deleting all the .session files and am trying a full rebuild, but I keep getting disk full errors when it tries to create new .session files.  Of course it can't create those files, because my disk is full.  I don't see why removing and re-adding the media set will change this.  I thought Retrospect was supposed to be able to groom a media set so I can recover space when the disk is full, but it won't groom unless the catalog is in sync, and it won't create a new catalog because it doesn't have space for .session files.  Is this a Catch-22?

[DavidHertzberg]

When you said your disk is full, you were referring to your "2-disk 5-TB USB-attached media set".  However I'm beginning to wonder if you have enough space on your catalog-storage volume, which probably has a folder whose hierarchical name is root-volume/Library/Application Support/Retrospect/Catalogs.  If you don't, implement what I said in paragraph 3 and the P.S. of post #2 in this thread, but put the new catalog on another drive that has enough space.

 

OTOH even if you have enough space on your catalog-storage volume, I would suggest you purchase a new 4TB USB3 portable external hard drive—they're slightly over $100 on NewEgg.com tonight, initialize it with a different name, and copy the "Retrospect" folder and everything underneath it on your Offsite 3 TB drive to the newly-purchased drive.  Then do what I suggested in paragraph 3 in post #2, but when Retrospect asks for your second Member (assuming that second Member was on your Offsite 3 TB drive), give it the newly-purchased drive instead of the original Offsite 3 TB.  Then do what I said in the P.S. of post #2.  Finally, when the catalog has been recreated and everything is OK, wipe your Offsite 3 TB disk and figure out something else to do with it; do you have grandchildren or nephews you could give it to?