Copy Script no longer copies Media Sets

[lstone19]

Just upgraded from 14.x to 15.1.1 (102). For years, I've run copy scripts that copy the media sets to external disks used for offsite storage. Starting with 15.x, these scripts will no longer copy the media sets inside the Retrospect folder. If I rename it Retrospect2, it copies (but that breaks backups since that folder must be named Retrospect). I can create other folders called Retrospect down a few level and they copy. But the special Retrospect folder where it puts the media sets won't copy.

Specifics: Media sets are on an external drive in a folder called "Retrospect Backups" inside of which are the catalogs (.rbc) and a Retrospect folder with the media sets inside. Copy script uses the entire disk as source with a rule that says include "Folder Mac Path contains /Retrospect Backups/". Destination is a Retrospect "favorite folder" (other scripts copy to other folders on the destination volume). Prior to v15, this worked fine but now it ignores the Retrospect folder found inside Retrospect Backups (ignores here means it neither deletes files that have gone away nor copies any new or updated ones). As a test, I deleted that Retrospect folder on the destination and ran the script and while it recreated the folder, it is empty (source folder contains a few hundred GB).

There's other data on the source volume (hence the selection rule) and as mentioned above, other scripts copy other stuff to other folders on the same destination volume so any workaround is complicated.

Any ideas here?

[lstone19]

Replying to myself, I found it in the release notes. I guess copying media sets with another Retrospect script was considered a bug.

Mac 15.0.0.190 – March 6, 2018

Engine

• Fixed Full volume backup and copy now exclude backup set content, which can still be backed up or copied using favorite folders (#7212)

As it says, they can still be copied using a favorite folder but my testing says the favorite has to be the "Retrospect" folder which has inside it the "Name_of_media_set" folder which in turn has the "1-Name_of_media_set" (or other number as appropriate) folder. So reworked my scripts and it now takes two scripts to do what I used to do in one.

Above, I said "ignores the Retrospect folder ... (ignores here means it neither deletes files that have gone away nor copies any new or updated ones)." Now I'm not so sure. In one case, it looks like it left the contents on the destination untouched but in at least two others (with a different script), it had been deleted.

 

[DavidHertzberg]

4 hours ago, lstone19 said:

Replying to myself, I found it in the release notes. I guess copying media sets with another Retrospect script was considered a bug.

Mac 15.0.0.190 – March 6, 2018

Engine

• Fixed Full volume backup and copy now exclude backup set content, which can still be backed up or copied using favorite folders (#7212)

As it says, they can still be copied using a favorite folder but my testing says the favorite has to be the "Retrospect" folder which has inside it the "Name_of_media_set" folder which in turn has the "1-Name_of_media_set" (or other number as appropriate) folder. So reworked my scripts and it now takes two scripts to do what I used to do in one.

....

 

lstone19,

I may be missing something, but I don't understand why it now takes two scripts to do what you used to do in one.  Just adapt what I suggested in the second paragraph of this post, and your "Retrospect" Favorite Folder will be the last thing copied in the script.  The trick is changing the order in which Sources are copied by dragging them in the Summary tab of the Script, which is a technique I got in 2015 from derek500 that the august Documentation Committee evidently decided Retrospect Mac administrators are not sufficiently worthy to know (insert appropriate smiley here).

[lstone19]

7 hours ago, DavidHertzberg said:

lstone19,

I may be missing something, but I don't understand why it now takes two scripts to do what you used to do in one.  Just adapt what I suggested in the second paragraph of this post, and your "Retrospect" Favorite Folder will be the last thing copied in the script.  The trick is changing the order in which Sources are copied by dragging them in the Summary tab of the Script, which is a technique I got in 2015 from derek500 that the august Documentation Committee evidently decided Retrospect Mac administrators are not sufficiently worthy to know (insert appropriate smiley here).

I think what you're missing is that this is a "copy", not a "backup". Unless I'm missing something, copy scripts can only have one source unlike backup scripts which can have multiple sources. The way I've set things up, the catalogs and the Retrospect folder containing the .rdb files are in the same higher level folder. So one script to copy the catalogs, another for the .rdb files.

Edited June 7, 2018 by lstone19
multiple sources, not multiple destinations

[DavidHertzberg]

2 hours ago, lstone19 said:

I think what you're missing is that this is a "copy", not a "backup". Unless I'm missing something, copy scripts can only have one source unlike backup scripts which can have multiple destinations. The way I've set things up, the catalogs and the Retrospect folder containing the .rdb files are in the same higher level folder. So one script to copy the catalogs, another for the .rdb files.

Oops, you're right, lstone19.  Sorry, I don't use Copy very often.

But, come to think of it, fix #7212 is actually conceptually correct.  If you want to Copy or Backup any files that are altered as part of a Retrospect activity, you should want to do it in such a way as to reflect the results of the entire activity. You can't be sure that that is going to happen, unless those files are copied or backed up after everything else for the activity has been done.  So making you do that separately, either as the last source for a Backup or as the only source for a Copy, is IMHO justified.

[lstone19]

On 6/7/2018 at 12:57 AM, DavidHertzberg said:

Oops, you're right, lstone19.  Sorry, I don't use Copy very often.

But, come to think of it, fix #7212 is actually conceptually correct.  If you want to Copy or Backup any files that are altered as part of a Retrospect activity, you should want to do it in such a way as to reflect the results of the entire activity. You can't be sure that that is going to happen, unless those files are copied or backed up after everything else for the activity has been done.  So making you do that separately, either as the last source for a Backup or as the only source for a Copy, is IMHO justified.

I don't think I agree. With how I structured things, it used to be one copy copied both the catalogs and the .rdb files. Now it takes two. For the copy to be valid, both the catalog and rdb files must be static throughout - it doesn't matter which is copied first. The real key to integrity (for my use case) is to insure that no other backup activity is occurring - just the copy. But I'm not sure that can be assured in Retrospect unless you set Retrospect to use just one activity thread (I've only recently started allowing multiple activity threads and while I've noticed that it does block some conflicting activities (I think it will not allow a copy from a favorite folder while an enclosing folder is also being copied), I don't know about a copy of a folder while a media set inside the folder is open).