Retrospect 6.5 and Duplicate Replace - why copying existing files?

[Larry Gegel]

Sorry I haven't been back in a few days to provide more info. It's been a hectic week. I should have some time to get back to this over the weekend.

 

Larry

[CallMeDave]

Uh-oh. My testing was faulty, and the problem I noted before remains.

 

 

 

I now have a folder, populated with read-only disk images, wich will not Match on Duplicate. Comparing the files (from both Retrospect and Terminal "ls -la") shows identical attributes. Touching one of the files did not make a difference.

 

 

 

Here are images of the Browser window, and the Get Info windows of the Source (as selected in the Browser) and the Destination (as selected from Configure->Volumes->Browse) (drat; can't seem to attach images to my messages...Hmmm. FAQ says admin only, but I've seen images in messages before...)

 

 

 

I have no idea why an Immediate Duplicate (Replace entire disk) followed by another Immediate Duplicate would copy all files again. The _only_ odd thing about this particular test bed is that the Source is HFS+/Journaled, while the Destination is HFS+/Journaled & Case Sensative. But the majority of files on these two volumes Match as expected, with unchanged folders not being Marked for Duplicate.

[waltr]

hi dave,

 

if you want to test my theory about the "attributeModDate" being part of the problem, go into the 'Options' of your Duplicate, click 'More Choices' and then go to 'Matching'. uncheck the box for 'Use attribute modification date when matching'.

[CallMeDave]

"A-6"

 

BINGO!

 

But how does a read-only file created in 2002 (and never modified again) pick up extended attribute information that Retrospect sees?

 

And what bash command will reveal those attributes in Terminal?

 

Dave

[waltr]

hi dave,

 

 

 

i use 'ls -e' to look at "ACL's", but i don't think that will show the "attibuteModDate". i'll explore & report back if i find out more.

 

 

 

meanwhile, here's some bedtime reading:

 

 

 

http://arstechnica.com/reviews/os/macosx-10.4.ars/8

 

 

 

EDITED to add:

 

i typed too quickly. check the previous page of the above url to find the 'xattr' command line tool. i've set a 'bin' directory in my home directory to be in my '$PATH' so installing it there let's me run a 'xattr -l' on any file. unfortunatley, i don't seem to have any extended attributes that i can find. your milage may vary.

[waltr]

hi dave,

 

 

 

mentions the attribute:

 

 

 

http://developer.apple.com/technotes/tn/tn1150.html

 

 

 

a tool to download, and a page to read:

 

 

 

http://www.kernelthread.com/software/hfsdebug/

 

 

 

install 'hfsdebug' in your path, become 'root' and type: 'hfsdebug /path/to/your/file'

[Larry Gegel]

Sorry it took me so long to get back. I haven't been in front of the computer for a while except a few minutes here and there to check email.

 

I reviewed the discussions over the last few weeks and checked a few things. On the source volume the attributemoddate is the same as the createdate and/or contentmoddate. On the destination volume the attributemoddate has been changed to the date the file was backed up.

 

It seems strange that it only happens to some files, but consistently the same files (at least most of them seem the same I haven't checked all 10K or so files). Here are a few example files.

 

/Applications/RockSim 8 Demo/rocksim_update_8_0_1_12.dmg

/Users/lawrence/Downloads/CHUD_4.3.1.dmg

/Users/Shared/Larry's Stuff/Rocket Stuff/Jimz Rocket Plans/www.dars.org/jimz/catalogs/71cen83.gif

/Users/Shared/Larry's Stuff/Rocket Stuff/Jimz Rocket Plans/www.dars.org/jimz/catalogs/71cen83.jpg

/usr/X11R6/lib/X11/fonts/75dpi/ncenR24-ISO8859-10.pcf.gz

/Library/Application Support/GarageBand/Instrument Library/Sampler/Sampler Files/Grand Piano/065_F3KM56_H.wav

/Developer/SDKs/MacOSX10.1.5.sdk/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/IOKit/firewire/IOFireWireDevice.h

/Library/JBoss/3.2/deploy/welcome.war/chapter_1_section_1.html

 

Many .nib and .bom files

Many of the Resources directories from inside application packages in my Applications directory

 

I don't see any rhyme or reason to which files have the problem and which ones don't. Some of the files are pretty new (only a few months old) others have been on my system a long long time.

 

I tried setting the match attribute modification date option to off. I expected this might work around the problem, but much to my surprise it didn't make any difference.

 

Larry

[CallMeDave]

Quote:

---

On the source volume the attributemoddate is the same as the createdate and/or contentmoddate. On the destination volume the attributemoddate has been changed to the date the file was backed up.

 

---

 

Since you have a test file that exhibits this problem every time when Duplicated with Retrospect, the question becomes, does the same thing happen if you drag copy the file in the Finder?

 

Dave

[rhwalker]

Larry, I've got a wild thought - Mac OS 10.4 has a background "auto defragmenter" that will move files around to automatically defragment your disk to improve performance. There are special conditions for what files it will and won't move, details in the developer notes. But could it be possible that this auto-defrag altered some hidden things even though the file's Get Info stuff didn't change? This might explain the randomness. Just a thought.

[waltr]

russ--interesting thought, i may try to check up on that if i have time.

 

larry--sorry unchecking the box did not work for you. other users have noted that it worked for them. i am reposting some info i posted to another thread in the hopes that it is helpful:

 

Quote:

---

let me take a moment to talk about that, 'Use attribute modification date when matching' checkbox. hopefully this will help someone else in the future.

 

from what i've been able to learn, this modification date is only used when 'attributes' are modified. so what are 'attributes'? here we go:

 

1) they only exist in 10.4 or later--earlier versions did not have these

2) specifically, they are ACL's or Metadata. if you are _not_ using either of these, there is no risk of data loss by unchecking the box. Retrospect will just not backup any ACL's or Metatdata attached to the files. NOTE: there may be another attribute that i'm not aware of, but i've researched this pretty well and don't believe this is so.

 

one last thing: when Retrospect copies the file, it then makes the attribute modification date the same as the original file---however, there is a bug in 10.4 where the OS changes this date after a file that has attributes has been moved. it comes along behind Retrospect and changes the date. Apple is aware of the bug an will hopefully fix it.

 

i don't understand exactly what attributes are attached to your files, or where they are coming from (since you indicate that you are _not_ using them), but based on the empirical evidence this is my theory. i believe that unchecking the box will not cause data loss in your case based on the information you have given me.

 

other theories welcome.

 

 

---

[waltr]

hi larry,

 

new info for you. i took a look at the files you've listed and checked out some of my own files that are similar to yours with the hfsdebug program. of particular interest is this .dmg file, which has a both a different 'AttributeModificationDate' and some kind of Attribute i've never seen before:

 

Quote:

---

root# hfsdebug xattr.dmg

path = Macintosh HD:/Users/walt/Desktop/xattr.dmg

# Catalog File Record

type = file

file ID = 472102

flags = 0000000000000110

. File has a thread record in the catalog.

. File has extended attributes.

reserved1 = 0

createDate = Wed Apr 5 15:07:12 2006

contentModDate = Wed Apr 5 15:07:13 2006

attributeModDate = Wed Apr 5 15:07:34 2006

accessDate = Wed Apr 5 15:54:25 2006

backupDate = Fri Jan 1 00:00:00 1904

# BSD Info

ownerID = 503 (walt)

groupID = 503 (walt)

adminFlags = 00000000

ownerFlags = 00000000

fileMode = -rw-r--r--

linkCount = 0

textEncoding = 0

attrBlocks = 0

# Finder Info

fdType = 0x42494e41 (BINA)

fdCreator = 0x68446d70 (hDmp)

fdFlags = 0000000000000000

fdLocation = (v = 0, h = 0)

opaque = 0

# Data Fork

logicalSize = 62269 bytes

totalBlocks = 16

fork temperature = not a hot file

clumpSize = 32 bytes

extents = startBlock blockCount % of file

0xacb11c 0x10 100.00 %

16 allocation blocks in 1 extents total.

16.00 allocation blocks per extent on an average.

# Resource Fork

logicalSize = 0 bytes

 

# Attributes

# Attribute Key

keyLength = 76

pad = 0

fileID = 472102

startBlock = 0

attrNameLen = 32

attrName = com.apple.diskimages.recentcksum

# Inline Data

recordType = 0x10

reserved[0] = 0

reserved[1] = 0

attrSize = 80 bytes

attrData = 69 3a 34 37 32 31 30 32 20 6f 6e 20 43 34 39 30

i : 4 7 2 1 0 2 o n C 4 9 0

35 46 36 34 2d 39 34 34 42 2d 33 36 32 33 2d 38

5 F 6 4 - 9 4 4 B - 3 6 2 3 - 8

33 30 37 2d 39 34 46 31 41 36 36 39 36 36 45 43

3 0 7 - 9 4 F 1 A 6 6 9 6 6 E C

20 40 20 31 31 34 34 32 37 34 38 33 33 20 2d 20

@ 1 1 4 4 2 7 4 8 3 3 -

43 52 43 33 32 3a 20 24 46 42 35 42 36 36 42 34

C R C 3 2 : $ F B 5 B 6 6 B 4

 

 

---

 

notice the Attribute "attrName = com.apple.diskimages.recentcksum". do your files have anything resembling this?

[Larry Gegel]

I will check on the additional attribute. Further investigation has led me to realize that checking the check box did solve the problem for some files. I'm starting to suspect there are multiple things going on.

 

 

 

Also, I will be on vacation for the next week or so. I won't get a chance to check back until next week some time.

 

 

 

Larry