Duplicate of iPhoto Library re-copies some apparently identical files

[amkassir]

I am duplicating my "iPhoto library" package to another volume by selecting to duplicate my Pictures folder (that contains the "iPhoto library" package). The options are set to Replace Entire Contents with verification on. The duplication completes successfully.

 

However, if I do the same duplication again immediately, Retrospect selects over 3GB of the 20GB package to re-copy. I have made no changes to the iPhoto library (and haven't even opened iPhoto).

 

I can't figure out why this is happening, or why some files are judged to have changed by Retrospect. Previewing some of the files selected, I see that some are jpg files (photos) but while some of the photos are selected, others in the same folders are not. If I manually compare one of the photos in both locations (the original and the duplicate) that Retrospect wants to copy, I see no differences in their Get Info window, including date/time created, modified, size, etc.

 

I'd appreciate any help to understand why this is happening! (Is there some way to have Retrospect report WHY it is thinking certain files are different?)

 

Most importantly, how can I fix this, so the duplicate operation only copies files that have actually changed?

 

I'm using Retrospect 6.1.138 in Leopard 10.5.2 on a Quad PowerMac G5.

 

Thanks.

[rhwalker]

The [color:purple]options are set to Replace Entire Contents[/color] with verification on. The duplication completes successfully.

 

However, if I do the [color:purple]same duplication again[/color] immediately, Retrospect selects over 3GB of the 20GB package to re-copy.

Could you please let us know EXACTLY how you are doing the "same duplication again"?

 

If, in fact, you are doing the "same duplication again", with "Replace Entire Contents" set, I would expect everything to be recopied.

 

If you are doing something different, and your statement about "same duplication again" is in error, then check your "matching options". The manual has a good description of how these are used.

 

Russ

[CallMeDave]

Please see the Retrospect for Macintosh Read Me.html file that is installed in the same folder as Retrospect, where it states:

 

NOTE: Any time you copy a file with extended attributes or ACLs to a volume (for example during a restore or duplicate operation), the volume incorrectly sets the attribute modification date to the current date and time. This means that Retrospect will copy all files with extended attributes or ACLs during the first backup after a restore and during every duplicate operation, since the attribute modification dates will no longer match. Apple has been notified of this issue and is working to resolve it.

Try disabling ACL copying for your Duplicate and see if the situation improves.

 

 

Dave

[amkassir]

One of the nice features of Retrospect is, when doing a duplication, it checks for files that haven't changed and doesn't re-copy those files again, even if Replace Entire Contents is enabled. If you preview the files to be copied, it tells you how many files/GB appear to be already copied (i.e., haven't changed) and how many need to be copied. It also tells you how many need to be deleted.

 

In any case, since I haven't changed anything, I was expected zero files needed to be deleted and replaced.

[amkassir]

Thanks very much for explaining this. I'll try disabling ACL copying and see if that fixes my "problem." It appears the problem is really on Apple's end.

 

One final question on this: Will disabling ACL copying create any adverse effects down the line, should a restore (reverse duplication) be necessary? This question really stems from a lack of understanding of what ACLs are, I suppose.

 

Thanks.

[rhwalker]

One final question on this: Will disabling ACL copying create any adverse effects down the line, should a restore (reverse duplication) be necessary?

It means that ACL metadata won't be preserved. ACLs are a parallel (and different, with greater control) set of permissions, in parallel with the unix permissions. If an ACL exists, it will control.

 

For details, in Terminal, type

man acl

 

Russ

[CallMeDave]

Will disabling ACL copying create any adverse effects down the line, should a restore (reverse duplication) be necessary?

 

No. An iPhoto Library is generally going to be used by a single user, not shared with a matrix of users requiring multiple levels of access. The standard POSIX permissions (that _are_ retained in your ACL-Free backup) will be fine.

[amkassir]

Thanks CallMeDave and rhwalker for your helpful replies.

 

An iPhoto Library is generally going to be used by a single user, not shared with a matrix of users requiring multiple levels of access.

Actually, I _do_ share my iPhoto library with another user on my computer. We each have an alias to the iPhoto library in our pictures folder, which points to the actual iPhoto library which resides in /Users/Shared/Pictures/. Somehow I doubt this would qualify as the "matrix of users requiring multiple different levels of access" so the lack of ACLs will probably not result in any problems (right?).

 

I sometimes use the utility BatChmod to change permissions for the iPhoto library (and everything in it) to be readable, writable, and even executable by everyone in our "group," so no glitches occur.

[rhwalker]

See my previous reply. ACLs allow finer granularity of control than does the standard Unix (POSIX) "user, group, others" permissions model. I don't know what the BatChmod utility does, but I suspect that it might allow batch processing of a list for the "chmod" shell command, which uses the unix permissions model. If I had need for such a tool, I'd probably just whip up a quick shell script myself.

 

In fact, if ACLs were turned on and present for your iPhoto shared library of stuff, you could chmod all day and it might have no effect because ACLs, if present, would control. See "man acl" for details.

 

I have my own suspicions, because ACLs are on by default in MacOS 10.5.x (present in 10.4.x client but off by default and can be turned on and managed by CLI, present and turned on by default in MacOS 10.4.x Server and can be managed by CLI and by GUI tools there), that Apple will be forcing us to "think different" (by leaving no alternative) and only use ACLs at some point, causing incompatibility with standard unix software such as chmod, etc., unless you have a dedicated volume that is ACL free.

 

For now, though, unless you are mucking about with Apple-supplied OS software, some of which uses ACLs, you are probably ok by ignoring ACLs except when you are trying to create a bootable disk using Retrospect for MacOS Server 10.4.x or server/non-server 10.5.x. For that task, I would suggest creating the basic disk using the MacOS install disk, bringing it up to speed with updates, then restoring on top of that, which would preserve the ACL structure (if Retrospect had ACL metadata backup turned off).

 

Russ

[amkassir]

Thank you very much for your detailed, knowledgeable, and informative reply!

[amkassir]

The Retrospect preference option to disable ACL copying specifies "on Intel machines." I have a PowerMac Quad G5 (PowerPC, not Intel). I checked it anyway and tried the duplicate operation.

 

The duplicate operation still re-copies about 3.2 GB of photos (presumably those with ACL data).

 

Any way to disable ACL copying or comparisons on a PowerPC machine? Any other ideas?

 

Thanks.

[rhwalker]

The Retrospect preference option to disable ACL copying specifies "on Intel machines." I have a PowerMac Quad G5 (PowerPC, not Intel). I checked it anyway and tried the duplicate operation.

Then you disabled ACL copying. The preference language is bogus. The preference is there to work around an Apple bug in the ACL API, and the bug happens in any "Universal Binary" version of MacOS, not just on "Intel machines".

 

The duplicate operation still re-copies about 3.2 GB of photos (presumably those with ACL data).

Then ACL data is not your issue. Check your selectors and your matching settings. Something is causing these files to be copied.

 

Any way to disable ACL copying or comparisons on a PowerPC machine?

That's not your problem if you unchecked the ACL preference.

 

Russ

[CallMeDave]

The Retrospect preference option to disable ACL copying specifies "on Intel machines."

 

This is not the preference to be changed.

 

This is partly my fault, as while I quoted the correct text from the Retrospect ReadMe.html file in post #104365 above, I should have also more explicitly noted that the section refers to the "Use attribute modification date when matching" option.

 

Immediate->Duplicate->Options->More Choices->Matching

 

Of course, you could have opened the ReadMe file and studied it a bit, too...

[amkassir]

CallMeDave:

 

Thank you. Deselecting the "Use attribute modification date when matching" option solved this problem. I hope Apple's resolution to this problem arrives before long.

 

I've done more than one restore from a backup, and it is a little disturbing to know that the extended attributes on some of my files may be wrong as a result. This may not be a common occurrence, however, since (my amateurish understanding suggests) the actual extended attributes or ACL data itself is not changed, only the modification dates of those attributes or data. If I'm wrong, please feel free to correct me.

 

:thanks:

[CallMeDave]

I've done more than one restore from a backup...

 

You're not doing Backups, so you're not doing a Restore. You are using Retrospect's Duplicate feature, which is entirely different.

 

Retrospect's strength lies in its Backup abilities. Except for a _different_ Apple bug that prevents it from working on some specific system installs, Retrospect can backup and restore access control lists correctly.

 

Dave

[amkassir]

CallMeDave:

 

Thanks again for the help. Maybe I'm confused. From the Retrospect for Macintosh Read Me.html file, to which you previously referred:

 

NOTE: Any time you copy a file with extended attributes or ACLs to a volume (for example during a restore or duplicate operation), the volume incorrectly sets the attribute modification date to the current date and time.

 

This seems to indicate restores are susceptible to the same ACL-related problem that duplications are.

 

I understand your kind explanation regarding the difference between duplications and restorations (the latter being from a backup). I have done restores from backups, and do the duplication separately as an added level of security for my iPhoto library.

 

Just a few days ago, I duplicated my entire volume to another volume, and then had to duplicate "backwards" to get my original volume back to a previous state (before I did some mucking around trying to rollback the Leopard Graphics Update 1.0 -- a different story!). As a result of this, I expect the ACL modification dates were changed.

 

Your diligence in responding to my posts is very much appreciated.

[CallMeDave]

This seems to indicate restores are susceptible to the same ACL-related problem that duplications are.

 

It's a problem with Duplicates (to populated volumes) because it prevents Retrospect from otherwise matching files that haven't actually changed (causing all files to be copied).

 

It's _not_ a problem with Backups because files are matched to the Backup Set, where the ACL information is accurate.

 

I suppose there might be a usage scenario where Restores from Backup might be impacted by this behavior, but I can't think of one at the moment.

 

Dave