Rebuild catalog file difficulties

[lucas_mat]

Hey guru's - can someone help?

I'm running R 8.1 (build 622) on a 2GHz/1GB Mac Mini running OS X 10.5.8 with both console and engine on this machine.

For the minute it's on a trial license since we're evaluating it prior to possibly upgrading from v6 to v8.

I'm trying to groom one of my drives (a USB bus-powered 2.5" drive) which is the only member of this particular media set.

 

The groom fails (it's set to groom to 1 version) reporting

Executing Grooming at 10/12/2009 11:38 AM (Execution unit 1)

Grooming Media Set Test2...

>

Grooming Media Set Test2 failed, error -1964349318 ( unknown)

You must recreate the Media Set's Catalog File.

See the Retrospect User's Guide or online help for details on recreating Catalog Files.

10/12/2009 11:38:15 AM: Execution incomplete"

 

Ok, so I select that Media Set in console and select Repair.

 

That runs fine, with the log output as follows:

+ Executing Recatalog at 10/12/2009 12:04 PM (Execution unit 1)

To Media Set Test2...

10/12/2009 12:04:07 PM: Execution completed successfully

 

So I attempt to groom again, and that gives the first error.

 

So I attempt to rebuild, select Disk as the first option, then I have to Add Member (should it not be in there to start with?), so I add the drive from the list, hit Next and then I'm prompted to "Select Disk Media Set Member" and the list is empty, nor is there the option to add a member.

 

Why is this - the drive is a member of that Media Set (it's visible in the Media Set when you select it)? What am I doing wrong? Have I gone about re-cataloging the media set correctly? And if so, why is it still reporting it as needing re-cataloging?

 

Any help gratefully received.

[Mayoff]

You need to do a Rebuild, not a repair of the catalog.

[CallMeDave]

(should it not be in there to start with?)...

 

No; Retrospect doesn't know which/what Member you might have to work with; sometimes you might need to Rebuild a Media Set when you're missing (a) Member(s).

 

so I add the drive from the list, hit Next and then I'm prompted to "Select Disk Media Set Member" and the list is empty, nor is there the option to add a member.

 

Why is this (?)

 

Because you selected the wrong item from the "Rebuild a Disk Media Set" drop-down sheet's "Select a Disk Member Folder:" drop-down sheet.

 

Either select the Retrospect folder if it's there (Foo/Retrospect/) or select the root of the disk (Foo). If you select "1-Foo" or "2-Foo" the next drop-down sheet will be empty.

[lucas_mat]

Hmm, ok, well I've gone back through this - let me take you through it stage by stage to ensure we all know what I'm talking about.

 

In the console, I select Media set on the left, highlight the relevant media set (in this case Test 2) and hit Rebuild in the top menu bar.

 

This drops down the sheet where I can choose File, Disk or Tape etc.

I choose Disk

 

Then we have the 'Rebuild a Disk Media Set' sheet, which is blank, so we click on "Add Member".

At this point I can either select the root of the drive (Little 75G in this case) or drill down into that drive and select the Retrospect Folder if I've understood the above posts correctly.

 

If I select the root of the drive and hit Next, we then get the sheet entitled "Select Disk Media Set Member" which is empty.

 

If I select the Retrospect folder and hit Next, i get the same result.

 

In fact it doesn't matter what I select from that drive, the following sheet is always blank with the only option being to Cancel.

 

Now there's obviously data on there, because when I drill down into that Little 75G drive there is not only the Retrospect folder, but also 15 Copy jobs, all dated correctly etc.

 

So where am I going wrong? I shouldn't have to wipe the drive, remove it as a backup destination and recreate it all from scratch, surely?

[Maser]

You have to dive down to select the "1-" folder *within* the media set folder to start with.

 

Then it should work. Depending on the size of your media set, it *can* take a while for things to show up after you've selected things before you can actually start the rebuild.

[CallMeDave]

Copy jobs?

 

Disk Media Sets aren't dated, but have the structure and paths of (for example):

Little 75G/Retrospect/1-Test 2/AA000000.rdb

Little 75G/Retrospect/1-Test 2/AA000001.rdb

Little 75G/Retrospect/1-Test 2/AA000002.rdb

 

In the console, I select Media set on the left, highlight the relevant media set (in this case Test 2)

 

Before clicking any button, what does it show in the "Members" tab for the Location of the first Member of the Media Set?

[lucas_mat]

what does it show in the "Members" tab for the Location of the first Member of the Media Set?

 

It shows Name: 1-Test2, Location: /Volumes/Little 75G/

 

Ok, so in the "Select a Disk Member Folder" sheet within the rebuild dialog, I drill down to the following location:

/Little 75G/Retrospect/Test2/1-Test2 and with that folder highlighted, hit Next.

 

Still no joy - still comes up with a blank "Select Disk Media Set Member" dialog which I cannot add anything to, only select Cancel.

 

Just for reference, the actual catalog file location for that Media Set is: /Volumes/OS Drive/Library/Application Support/Retrospect/Test2.rbc, however I've tried selecting that under the File option in the Rebuild Dialog and it just tells me "wrong path" in red, so I'll assume that's also wrong, lol.

 

This really isn't enormously intuitive, is it? I mean, granted, it's not like a core functionality type affair, but really shouldn't be this difficult, should it?

 

Any more thoughts?

 

[rhwalker]

This sounds suspiciously similar to Steve Maser's problem:

Maser - Anyone tried a groom with 8.1.622 and big media sets?

 

Russ

[Maser]

How large is your media set?

 

I've seen it take "a while" after selecting the "1-" file to then actually populate the box so you can proceed with the rebuild.

 

if you have a lot of .rdb files in the media set, it can take some time for things to continue on from that point.

[lucas_mat]

How large is your media set?

 

Well, not huge - the drive's only 75GB, about 50GB used. I noticed you put that earlier in a post above, so tried leaving it for a while to see if it would populate, but nothing after 15mins, so binned it.

 

[CallMeDave]

...(this) really shouldn't be this difficult, should it?

 

No. For me, I can select either [color:purple]Foo/[/color] (which shows all the Media Sets in Foo/Retrospect/) or [color:purple]Foo/Retrospect/[/color] (which shows the same thing) or [color:purple]/Foo/Retrospect/MyBackup/1-MyBackup/[/color] (which shows only the MyBackup Media Set).

Only selecting Foo/Retrospect/MyBackup/ gives me an empty drop-down sheet (my post above was in error).

 

Can you confirm the actual contents of

[color:blue]Little 75G/Retrospect/Test2/1-Test2/[/color]

? What's in that folder?

[lucas_mat]

Hah, bugger all. Nothing. Not a sausage. What should be in it?

 

[Maser]

That sounds like your problem.

 

It *should* look something like this attached screen shot...

 

 

In my case, when I want to rebuild this catalog for this media set, I select the "1-Servers 8" directory and it goes from there. Because there are over 3000 .rdb files in it, it takes a minute or two to continue on after I select this.

 

 

But if you don't have any .rdb files in your media set, then there's nothing to rebuild a catalog for!

Edited October 13, 2009 by Guest

[lucas_mat]

Hmm, that's interesting. Those files are all in the /Little 75G/Copy of Test1 12 Oct 09/ and other similarly named folders.

 

Is this perhaps cos they're all copy jobs on that media set and not backup jobs?

 

[Maser]

.rdb files are the individual files that make up a disk "media set" -- regardless of how they were created (backup vs. copy, etc.)

 

If you have a disk media set that has no .rdb files in it, then any of the following might be possibilities:

 

1 -- it was set up and never used

 

2 -- it was used, but "recycled"

 

3 -- the .rdb files were manually moved/deleted

 

4 -- you have some kind of disk problem (where you might at least run "Disk First Aid" to verify the disk for problems...)

 

5 -- based on your description, maybe you had this media set set as a "destination" for a copy job, but the copy really didn't copy anything to it? Does your log indicate that happened?

[CallMeDave]

g. Those files are all in the /Little 75G/Copy of Test1 12 Oct 09/ and other similarly named folders.

 

Is this perhaps cos they're all copy jobs on that media set and not backup jobs?

 

I though I was pretty clear in Post#129480 above when I described the structure of a proper Disk Media Set and commented on your use of the term 'dated.' If you're not doing Backups, then there's no Media Set to Rebuild.

 

- Is "Test2" (as noted in the log of your original post) a Disk Media Set?

 

Retrospect doesn't do contextual naming of folders, the way some utilities such as Hazel can. Are you running something that's manipulating your files in the background?

 

 

David

[lucas_mat]

Ok, let me take the points above one by one:

 

1 -- it was set up and never used

 

2 -- it was used, but "recycled"

 

3 -- the .rdb files were manually moved/deleted

None of the above are true - I set it up, and it has run for 15 days on the trot - the disk was fresh and empty, it now has 50-odd GB in it - so something's happening.

 

It's never been recycled - there is nothing set up in any of the jobs/media sets to recycle. The files haven't been moved - not by me and not by anything else - the only process using that drive is R8.

 

4 -- you have some kind of disk problem (where you might at least run "Disk First Aid" to verify the disk for problems...)

I've run Disk First Aid - it reports there is nothing wrong with the drive

 

5 -- based on your description, maybe you had this media set set as a "destination" for a copy job, but the copy really didn't copy anything to it? Does your log indicate that happened?

As noted above about data volumes - but the log output from one of these copy jobs is as follows:

 

+ Duplicate using Local to External at 10/11/2009 10:00 AM (Execution unit 1)

To volume Little 75G...

****** Warning: volume "Little 75G" has the "Ignore ownership" setting enabled. ******

- 10/11/2009 10:00:00 AM: Copying Test1

10/11/2009 10:05:09 AM: Comparing Little 75G

10/11/2009 10:09:55 AM: Execution completed successfully

Completed: 41 files, 4.2 GB

Performance: 850.1 MB/minute (818.5 copy, 884.3 compare)

Duration: 00:09:55

 

I though I was pretty clear in Post#129480 above when I described the structure of a proper Disk Media Set and commented on your use of the term 'dated.' If you're not doing Backups, then there's no Media Set to Rebuild.

 

Disk Media Sets aren't dated, but have the structure and paths of (for example):

Little 75G/Retrospect/1-Test 2/AA000000.rdb

All I can tell you is the folder structure that's been created, by Retrospect and nothing else (not me, no other programs) is the following:

Little 75G/Copy of Test1 30Sep09/1-Test1/AA000000.rdb

 

There are 15 other dated folders (e.g. Copy of Test1 01Oct09/1-Test1) and all of them have multiple .rdb files in.

 

There is also:

Little 75G/Retrospect/Media Set 1/2-Media Set1 (this one I think is a remnant of a Media Set using this drive that I set up initially and subsequently deleted)

and

Little 75G/Retrospect/Test2/1-Test2

both of which are empty.

There is also the file:

Little 75G/Retrospect/EMC which is down as a Unix executable.

 

Is it perhaps because the copy job is a copy of an existing backup? I know you've all said above that a media set is a media set, regardless of whether it's a backup or a copy job, but the copy is copying a backup (the initial backup media set is called Test1, the copy job media set is called Test2).

 

Is "Test2" (as noted in the log of your original post) a Disk Media Set?

Yes, see above.

 

So then, all things considered, something's going wrong. As far as you are all concerned, the media set catalog file and .rdb files are in the wrong place, or rather in the wrong folder structure.

 

As far as I'm concerned, the folder structure is simply that which has been constructed by Retrospect - I haven't altered it in any fashion, nor do I use any other programs that have any interaction whatsoever with that drive (with the possible exception of Spotlight).

The copy jobs have been running fine, and still are: there is a folder:

Little 75G/Copy of Test1 13Oct09/1-Test1/AA000000.rdb through to AA000044.rdb

a total of 3.84GB with a corresponding log entry associated with a successful copy job. The next one is due to run in 15 minutes (10AM UK local).

 

So what's going wrong?

[CallMeDave]

I know you've all said above that a media set is a media set, regardless of whether it's a backup or a copy job

 

No; no one has said that. It's not true, and is likely the basis for why it's still unclear to us what you have done, what you are doing, what you are seeing, etc.

 

For example, the log above states "Duplicate using Local to External" but you have never before noted what "Local" and "External" might be the names of.

 

 

The copy jobs have been running fine, and still are: there is a folder:

Little 75G/Copy of Test1 13Oct09/1-Test1/AA000000.rdb through to AA000044.rdb

 

It sounds as if you have a Copy script, with "Copy to a new folder" for the Destination. I see that this new folder is created with the current date in the name (a new feature in 8).

 

But why are you copying the files that make up a Disk Media Set from one place to another? If you want secondary or tertiary copies of Media Sets you can use a Copy Media Set Script.

 

And if you want to backup to a Disk Media Set, you have to tell Retrospect what you want to use as a Member of that set.

 

So what's going wrong?

 

You're doing lots of things and getting results that you don't expect. But you haven't yet made clear to Forum readers exactly what/everything you are doing. You have a Retrospect/Foo/1-Foo/ folder that's empty, which tells us nothing, really. Retrospect does what you tell it to do, and if you're confused about what it's doing you need to be clearer about explaining what you are telling it to do.

 

Perhaps some screen grabs of your Scripts list and your Media Sets list would help us understand better.

 

 

Dave

[Maser]

yes -- or at least post some log files from your initial *backups*. That might help explain the locations/names of the original set of data files.

[lucas_mat]

Lol, sorry guys.

 

Ok, I'll run through the setup:

 

Media Sets:

 

Media Set 1 has 1 member: /Volumes/Backup (a secondary partition on the internal drive)

Media Set 2 also has 1 member: /Volumes/Little 75G (an external USB 2.5" drive)

 

Scripts: (names and breakdown)

 

"Backup to Internal"

Type: Backup

Source: /Users folder on my OD server (a separate machine from the backup machine)

Media Set: Media Set 1

Rules: All files except cache files

Schedule: every day at midnight (set up as 'repeat: weekly' with every day selected)

Options: Backup/Matching/Match source files....

Backup/Matching/Don't add duplicate...

Source/Synchronise clock

Source/Macintosh/Use attribute modification date....

Source/Macintosh/Set source volume's...

 

"Local to External"

Type: Copy

Source:Backup/Test1 (this is a Favourite folder, the actual path for which is: /Volumes/Backup/Retrospect/Test1)

Destination: Little 75G (copy to a new folder) - I'm guessing this is the option that creates the dated folders???

Rules:All Files

Schedule: Every day at 10am (set up as repeat: weekly with every day selected)

Options:Copy/Thorough verification

Source/Macintosh/Use attribute modification...

 

There are two Groom jobs set up aswell, one for Media Set Test 1, one for Media Set Test 2. I won't bore you with the details of those unless you want me to. Oh, and a proactive job for my laptop. Same applies. The groom job for Media Set Test 2 fails every time (which is part of the reason for starting this thread in the first place).

 

I'd stick some screenshots up, but can't figure out how to do it - someone attached one earlier, all I can see how to do is give a URL. Do I have to stick them on the web?

 

The effect I'm after is this:

Backup various parts of my LAN every night to an internal drive (or probably, if I roll this out properly, to a large-ish external drive).

Then every day while the actual source data is being used, copy that backup to a portable external drive. The theory being that the main backup drive stays in place always, and constantly groomed to maintain one week's worth of backups, and the external drive gets is wiped clean at the start of the week, builds up a week's worth of backups then is removed off site and replaced with another identically named removable drive which is wiped clean then continues to copy the backups for that week etc etc.

 

As it stands the groom jobs aren't set up correctly yet - still got to do a bit of messing around with them, but the whole point of this thread is to work out how to get the bl**dy grooming working correctly so I can implement it.

 

Someone give me a "how to attach screenshots to posts for dummies" lesson, and everyone chip in with what screenshots they'd like to see to clarify, and I'll stick some on.

 

Thanks for the continued perseverance with this guys. Genuinely much appreciated.

[Maser]

Out of curiousity, why are you using a "copy" script instead of a "copy backup" script?

 

 

And what is "Media Set Test 2"? That's not referenced anywhere in your description above.

Edited October 15, 2009 by Guest

[CallMeDave]

the whole point of this thread is to work out how to get the bl**dy grooming working correctly

 

Ah, my bad then. I thought the whole point of this thread was described by the subject you choose, "REBUILD CATALOG FILE DIFFICULTIES."

 

At this point I can either select the root of the drive (Little 75G in this case) or drill down into that drive and select

 

Media Set 1 has 1 member: /Volumes/Backup

 

So your backup is done to a Member that lives on [color:purple]/Volumes/Backup/[/color] and you've experienced difficulty Rebuilding that Media Set when you point Retrospect to an entirely different volume. That would likely explain the problems you're having with your Rebuild operation.

 

It would also explain why [color:purple]Little 75G/Retrospect/Test2/1-Test2[/color] is empty, since it's not, you know, a Member of any of the Media Sets you've described.

 

I'll leave the Grooming experiences to Steve, whose done it the most around here.

 

 

Dave

[lucas_mat]

Out of curiousity, why are you using a "copy" script instead of a "copy backup" script?

 

Well, that's a very good question. The reason being that I had no idea there was a "copy backup" script so assumed that "copy" was the most appropriate tool for what I was after achieving.

 

And what is "Media Set Test 2"?

 

My apologies - I named the two media sets incorrectly above - thank you for pointing it out. They are named "Test 1" and "Test 2" and have the internal backup drive and the external 75GB drive in them respectively (one in each).

[lucas_mat]

Ah, my bad then. I thought the whole point of this thread was described by the subject you choose, "REBUILD CATALOG FILE DIFFICULTIES."

Alright, no need to be like that. The catalog file rebuild difficulties have come around cos of trying to get the grooming to work as planned.

 

So your backup is done to a Member that lives on /Volumes/Backup/ and you've experienced difficulty Rebuilding that Media Set when you point Retrospect to an entirely different volume. That would likely explain the problems you're having with your Rebuild operation.

 

No! The backup is done to one member (/Volumes/Backup). The copy job is done to another member (/Volumes/Little 75G). I've asked Retrospect to rebuild the catalog for the media set that contains the Little 75G (Media Set "Test 2"). And I point it at that drive. But it doesn't work.

 

It would also explain why Little 75G/Retrospect/Test2/1-Test2 is empty, since it's not, you know, a Member of any of the Media Sets you've described.

Again no! It is a member of Media Set "Test 2". And, as I've explained above, has scripts directed at it.

 

I came on here asking for help, not to be patronised. I'm assuming you know more than me about all this, so please either help me out and leave the sarcy comments for someone else or don't bother.

[lucas_mat]

Ok so it appears that to achieve the objective I'm after, I should be using the "Copy Backup" script. Is that right? Will that give me the functionality I'm after (as I described in my earlier post). Will I be able to Restore directly from that as I would from the original backup destination.

 

If all of that's true, what are "Copy" scripts used for - just for replicating directory structures and contents in another location? And does that not give me the same functionality anyway (I appreciate that if that's what it's designed for then I should be using the script that's intended for what I want to do, but if I'm just replicating what's on the initial backup drive, then does it not achieve the same effect)?

 

And in that case, how will I need to set up the groom jobs to ensure that at the beginning of the week the new drive is effectively wiped ready to start copying afresh?

 

And, more importantly, will this (do you think) get around this catalog rebuild issue? Is there a catalog generated for a simple copy job (this is the point I brought up right near the beginning of the thread), cos if not, then there isn't anything to repair, hence possibly the issue???

 

Obviously I could probably get around the whole thing by wiping, removing the Test 2 media set and starting again, but I'd much rather get an understanding of how it works rather than blindly binning and starting over.