locate cloud media set on new retro server?

[137491D092A58DE1E040000A2A666149 0]

Retrospect 17.5.2 on Mac OS 10.14.6. I had Retro (same version) running on an older server and backing up to Backblaze. Now I want to run Retro from the newer server. I copied the catalog file (it's a disk group) to the new server. But when I try to "Locate" it, it creates the media set but says it's empty. Do I have to rebuild it? That will take ages.

 

[Lennart_T 152]

I guess you do something wrong. Retrospect should not create an empty media set when you use the "Locate" function. Instead it should add the existing media set.

I suggest you remove the new (empty) media set and try again with the "Locate" button. 

I just tried it here and it worked fine, albeit with a disk media set.

 

[137491D092A58DE1E040000A2A666149 0]

I'm not actually sure if it's "empty" or not. It actually shows all the past backups, but the media member itself shows 0 of 4 TB used. I think maybe that's bogus. I started a new backup and will see if it's backing up everything or just the changes. thanks.

[DavidHertzberg 73]

137491D092A58DE1E040000A2A666149 (if I've mis-typed your IMHO-idiotically-long "handle", that's tough noogies),

I think your basic problem is when you say "... the catalog file (it's a disk group)", because I suspect by "disk group" you mean a Storage Group.  As the second paragraph of this post in another thread says, I strongly recommend that you not use Storage Groups with Retrospect Mac unless you have to.  That's because a Storage Group in both variants of Retrospect is implemented as a container folder for individual component Media Sets, and the Retrospect Mac GUI was not enhanced in 2019 to allow access to the component Media Sets—as the second paragraph after the disclaimer in this follow-on post in the same thread explains in terms of assumed StorCentric priorities.  When you did a Locate of the "catalog file", I suspect you did a Locate of your Storage Group.  I'm not surprised you thereby created an empty "Media Set" (you're lucky doing so didn't render you dead or sterile 🤣  ), which means you created a copy of the container folder—which not surprisingly shows an empty Media Set member since the copied Storage Group contains no components (you can check this using the Finder).  I am surprised it "shows all the past backups".

I think what you could do is to copy all the component Catalog Files within the Storage Group container from the Library->Applications Support->Retrospect->Catalogs folder on your older server machine to the same location on your newer server machine.  But it would probably be better to follow Lennart_T's suggestion, which is to Remove the new Media Set and try again—but (I'm saying) using Add instead of Locate.  But do all the copying with the Finder, because I'm pretty sure Retrospect's Copy operation hasn't been upgraded to work properly with Storage Groups.  If doing an Add gives you a message about a Catalog File already existing on the newer server machine, do a Finder delete of that Catalog File—which if I'm correct would consist of the empty container folder .

[jweisbin 2]

Sorry about the "handle" - I previously signed in using a very old account and for some reason it defaulted to that, I have no idea why. 

Seems to me storage groups simply don't work well with cloud storage. I started the backup and sure enough it copied everything up again as if it was all new.

[DavidHertzberg 73]

5 hours ago, jweisbin said:

Sorry about the "handle" - I previously signed in using a very old account and for some reason it defaulted to that, I have no idea why. 

Seems to me storage groups simply don't work well with cloud storage. I started the backup and sure enough it copied everything up again as if it was all new.

Come to think of it, that's not at all surprising.  When one backs up one or more sources to a Storage Group, Retrospect creates a new component Media Set for each designated source combination of machine and volume/Favorite-Folder the Storage Group doesn't already contain—I've tested this.  Then, since that component Media Set has effectively had a Recycle performed on it—because it is newly-created, the entire contents of the designated source machine and volume/Favorite-Folder is backed up into a newly-created-on-destination Member of the component Media Set.

Therefore, since AFAICT your Locate created a Storage Group—which is weird—containing no component Media Sets,  your backup operation did exactly what it is supposed to do.  Whether or not the destination of the operation was in cloud storage didn't make any difference; IMHO there's nothing that could be characterized as  "don't work well" about what Retrospect did during the backup operation.

The only thing that puzzles me is how "It actually shows all the past backups".  Possibly records of past backups are actually maintained at the Storage Group level, and in your case were copied over to the new Storage Group you created with your Locate to a different Library->Applications Support->Retrospect->Catalogs folder on your newer server machine.  Since you're the one who had this experience, why don't you create a new Support Case discussing both these peculiarities with doing a Locate for a Storage Group?  (I don't use Storage Groups, so I'm not a candidate for creating this Support Case.)

[Nigel Smith 25]

17 hours ago, 137491D092A58DE1E040000A2A666149 said:

But when I try to "Locate" it, it creates the media set but says it's empty.

When you used "Locate", what did you select? I haven't used a cloud target, but I suspect it is the same as a mounted-NAS target and you should be selecting the volume/directory that contains the "Retrospect" directory that holds your backups, not the "Retrospect" directory itself.

(I often get that wrong, usually with the results you describe!)

[jweisbin 2]

8 hours ago, Nigel Smith said:

When you used "Locate", what did you select? I haven't used a cloud target, but I suspect it is the same as a mounted-NAS target and you should be selecting the volume/directory that contains the "Retrospect" directory that holds your backups, not the "Retrospect" directory itself.

(I often get that wrong, usually with the results you describe!)

Using the "Locate" command I located the catalog file that I had copied over from the old server. In this case there was one called "retro-misc-ny-B2.rbc", and a folder called "retro-misc-ny-B2" containing files like "FileServer-macsrv001_Users_admin_Desktop.rbc", "FileServer-VMFS03_Helios.rbc", etc. Using the locate command on any of the ones in the folder does not return an error, but does not seem to do anything. Maybe I'm wrong but I don't think the "locate" command is meant to work with the actual backups - for that I would have to do a rebuild, which would take a very long time I think. (Backblaze is slow, but also relatively inexpensive as compared with Amazon for example). I'm not sure what was meant by the suggestion to "add" - in Media sets there is the big Add button which would simply add a new media set - not an existing one as far as I know. Then in the member list there is a plus button which would add another member, but that would also be a new one, not an existing one.

The locate command worked insofar as I can access the full media set of backups and can restore files from it. The problem is that, from the point of view of this new server the sources are different, and therefore all new backups are going into new container folders in the cloud media set, so there is no de-duplication. But at least I have access to the older backups, which is nice.

The listings indicating 0 of 4 TB space used are clearly in error, because all the backups are there and can be accessed. 

[DavidHertzberg 73]

jweisbin (nice to see your current "handle" again),

I did some experimenting last night using Retrospect Mac 16.6 with Locate on my existing Media Sets—I don't mess with Storage Groups for reasons explained up-thread.  In doing so I was partially inspired by this 2012 post.   zedred 's Profile said he was Vice President of Engineering, and he signed his real name—which is JG Heithcock (he only made two posts using the zedred "handle; I wonder what he's doing now 🤣  ).

My conclusion is that Locate for an existing Media Set is supposed to redefine the drive location of its Catalog File.   Its Select the Catalog dialog defaults to bootdrive->Library->Application Support->Retrospect->Catalogs, but you can navigate to a folder on another bootable drive.  My boot drive is "Macintosh HD SSD", but—for ease in possibly running a different release of Retrospect—my production Media Sets now have their Catalog Files on "Macintosh HD New".  Just now I did a Locate of "Media Set Blue" and selected an April 2019 Catalog for it on "Macintosh HD SSD".  That effectively did an Add of a second "Media Set Blue", with nothing shown in its Backups tab; you can bet your bippy I immediately did a Remove of it. 

Last night I attempted to do the same Locate, but somehow effectively got an Add of a second "Media Set White"—my current Media Set this week, with the second "Media Set White" showing Backups from April 2019.  I should mention that last night I first attempted to quickly simulate the effect of a Rebuild of "Media Set Blue" by Finder-copying its Catalog File to the Desktop, Finder-deleting it from its regular drive location, and then copying it back.  That may in some strange way explain why I effectively got an Add of a second "Media Set White"

Edited January 8 by DavidHertzberg
Add additional sentences to last paragraph, giving key detail of my test last night.

[Nigel Smith 25]

14 hours ago, jweisbin said:

Using the "Locate" command I located the catalog file

Sorry -- my stupidity. Of course the "Locate" command is used to point to the catalog file. D'oh!

Can you post the paths of the two Backblaze Storage Group directories for the same backup client? Might be a clue there...

[jweisbin 2]

Not sure what you mean by paths in this context. But the bucket is called "retro-misc-ny-B2" and inside that there is a folder called the same. Inside that folder are the container folders for the individual sources. So there is a folder called "FileServer-VMFS03_Helios" which was the container from the old server. The new container is "QNAP15TB-C_Helios". It's a copy of the same folder on the new server. If I had used a client running on the source this probably would not have been an issue. But I've had too many issues with the client disappearing and having to be restarted, or saying it's busy when it's not, or "backup set reserved" whatever that means. So I rsync the data to the new server and then back that up to the cloud.

So in this context I think it would have been better not to use a cloud storage group, because here I am duplicating everything onto the cloud, which is expensive.

[Nigel Smith 25]

20 hours ago, jweisbin said:

Not sure what you mean by paths in this context.

Your guess was perfect!

IIRC from my testing, each client/volume (volume can also be a Favorite Folder) combo gets its own "container" in the Storage Group. If you'd called the new Server "FileServer" and the volume/Favorite in that "VMFS03_Helios" then RS might have carried on with the same container and not doubled up your data but, as you've seen, there's no dedup across containers.

Storage Groups aren't appropriate for what you are trying to do -- they're really a way to parallelise multiple slow backup streams from different clients to a (pseudo) single set. Use a cloud-stored Disk Media Set instead. You might be able to retain continuity by migrating your Storage Set backups to the new Disk Media Set using "Copy Media Set" scripts, but you'll probably be paying ingress/egress fees on all that data all over again...