Backups filling up too quickly

[Christiaan]

I'm using removable hard drives and incremental backup, and they are filling up every couple of backups, using 5 gig or so each backup. But we don't produce 5 gig of data in this time.

 

 

 

Having had a quick look at things it seems as if Retrospect is backing up any file that gets opened by somebody (even if it wasn't edited), essentially duplicating it. (I'm wondering if it might have something to do with the Mac OS X 10.4.5 update?)

 

 

 

Is anyone else having a similar problem?

[CallMeDave]

Quote:

---

it seems as if Retrospect is backing up any file that gets opened by somebody (even if it wasn't edited), essentially duplicating it.

 

---

 

You don't provide any details about your setup, so it's impossible for others to know why the files are getting Marked for Backup. But if the actions of the users is changing any of the attributes that Retrospect uses to Match, the files will be backed up again. In some cases, simply "touching" a file will change its modification date, even if none of the bytes the file contains are modified.

 

- Where are the files that "somebody" opens?

- What sort of files are they?

[Christiaan]

Hi Dave, thanks for your help. This problem has only recently cropped up so either I'm using different settings inadvertently or something has changed with Mac OS X or Retrospect.

 

Quicktime movie files is one file I've noticed that I wouldn't expect to see being duplicated on the backup, just because someone opened it. All files are on a local server running Mac OS X 10.4.5.

[Christiaan]

In fact looking at some of the files it is duplicating I can't imagine someone has even opened them. Too many are old files that are very unlikely to have been opened (unless someone was searching for something and opening everything, but the files are in the thousands).

 

In any case MS Word files are another example. I've cross checked their modfication dates in the Finder with the date they were backed up and they don't correlate. One Word file for instance was backed up on the first full backup (23/3/2006) and then again on the 2nd incremental backup (4/4/2006) and, checking in the Finder, it still has a modification date of only 13/12/1999.

[Lennart_T]

Quote:

---

In fact looking at some of the files it is duplicating I can't imagine someone has even opened them.

 

---

"If one of the criteria has been changed, Retrospect will back up the file again. On a Mac, Retrospect looks at name, size, type, creator, creation date and time, modify date and time, and label."

 

Article 5922 in the Knowledgebase:

http://www.emcinsignia.com/supportupdates/technical/retrospect/

[Christiaan]

Yes but my problem is that my backups have started filling up faster than usual.

 

 

 

In addition, as I implied above, files appear to be getting duplicated even if name, size, type, creator, creation date and time, modify date and time, and label have not changed.

[Lennart_T]

What I meant was that if someone labels a file (or a bunch of files) in the Finder, it will get backed up.

Or if someone changes ANY of the other criteria, the file(s) will get backed up.

[Christiaan]

Yes, and that isn't happening. Even the modification dates aren't being changed (according to Finder). The only thing that appears to be triggering their duplicate backup is that they've being opened, or maybe even simply previewed in Finder?

 

Again, my problem isn't that backups have always filled up to quickly. My problem is that something appears to have changed and now my backups are filling up to quickly because of that change.

[Lennart_T]

Quote:

---

Yes, and that isn't happening. Even the modification dates aren't being changed (according to Finder).

 

---

FYI: The mod date doesn't change when setting a label.

Quote:

---

Again, my problem isn't that backups have

always

filled up to quickly. My problem is that something appears to have changed and

now

my backups are filling up to quickly because of that change.

 

---

Well, SOME of the fields must have changed on the files, otherwise Retrospect hasn't backed them up.

 

You might be interested in this thread which has some interesting info towards the last posts.

[Christiaan]

Quote:

---

FYI: The mod date doesn't change when setting a label.

 

---

The label settings haven't been changed. (you're talking about the Colour Label right?)

 

Quote:

---

Well, SOME of the fields must have changed on the files, otherwise Retrospect hasn't backed them up.

 

---

I'm assuming so, but they're not attrributes I can see in the Finder (via Get Info).

 

Quote:

---

You might be interested in

this thread

which has some interesting info towards the last posts.

 

---

 

Yeah I've been suspecting the "Use attribute modification date when matching" option, which I see mentioned in that thread. I was wondering about switching that off to see what happens (I may have had it switched off originally), but judging by its title if I switch it off it sounds like I may not back up some files that I want backed up.

 

What does "A-6" referred to in that thread by you stand for?

[Lennart_T]

Yes, I'm referring to the color label.

 

I have no clue about what A-6 is. Sorry.

 

About the "Use attribute modification date when matching" option, I suggest you try it out and see if it helps. And make SURE it backs up the files you want backed up.

[waltr]

hi chris,

 

Quote:

---

One Word file for instance was backed up on the first full backup (23/3/2006) and then again on the 2nd incremental backup (4/4/2006) and, checking in the Finder, it still has a modification date of only 13/12/1999.

 

---

 

if you still have that file available, could you give the 'creation date' also?

 

you should try running the backup with the 'Use attribute modification date when matching', unchecked and let us know the results.

 

the 'A-6' comment was not mine, but i think it refers to the game BINGO. possibly a joke of some kind. nothing to do with our subject at hand.

[Christiaan]

Thanks Walt. It's the same as the modification date. I'll try the suggestion with tonight's backup.

[Christiaan]

Unchecking 'Use attribute modification date when matching' would appear to have solved my problem. Thanks for your help. I just hope I'm not missing any files in my backup.

[waltr]

hi christian,

 

let me take a moment to talk about that, 'Use attribute modification date when matching' checkbox. hopefully this will help someone else in the future.

 

from what i've been able to learn, this modification date is only used when 'attributes' are modified. so what are 'attributes'? here we go:

 

1) they only exist in 10.4 or later--earlier versions did not have these

2) specifically, they are ACL's or Metadata. if you are _not_ using either of these, there is no risk of data loss by unchecking the box. Retrospect will just not backup any ACL's or Metatdata attached to the files. NOTE: there may be another attribute that i'm not aware of, but i've researched this pretty well and don't believe this is so.

 

one last thing: when Retrospect copies the file, it then makes the attribute modification date the same as the original file---however, there is a bug in 10.4 where the OS changes this date after a file that has attributes has been moved. it comes along behind Retrospect and changes the date. Apple is aware of the bug an will hopefully fix it.

 

i don't understand exactly what attributes are attached to your files, or where they are coming from (since you indicate that you are _not_ using them), but based on the empirical evidence this is my theory. i believe that unchecking the box will not cause data loss in your case based on the information you have given me.

 

other theories welcome.

[CallMeDave]

Quote:

---

one last thing: when Retrospect copies the file, it then makes the attribute modification date the same as the original file---however, there is a bug in 10.4 where the OS changes this date after a file that has attributes has been moved. it comes along behind Retrospect and changes the date. Apple is aware of the bug an will hopefully fix it.

 

---

 

But if this is the case, wouldn't that cause Retrospect to Mark/Not-Match _all_ the files in a Duplicate session, if the use attribute mod date is set to on?

 

But my (and others') observation is that only some files don't Match, while others do.

 

Now that I've finally installed hfsdebug, I see that any file I drag copy in the Finder does indeed set that date to the current date; but my Retrospect Duplicates still only re-copy a selection of unchanged files, not all of them.

 

Curiouser...

 

Dave

[waltr]

EDITED: sorry for the double post!

[waltr]

Quote:

---

hi dave,

 

Quote:

---

however, there is a bug in 10.4 where the OS changes this date after a file that

has attributes

has been moved

 

---

 

Quote:

---

But if this is the case, wouldn't that cause Retrospect to Mark/Not-Match _all_ the files in a Duplicate session, if the use attribute mod date is set to on?

 

But my (and others') observation is that only some files don't Match, while others do.

 

---

 

have we established that _all_ files have ACL's or Metatdata? i can't find that in the thread. perhaps if the files you are seeing are similar to the ones others note there is a common background process that modifies ACL's or Metadata on these specific types of files. i'm not sure.

 

however, i would definitely caution against assuming that all files have this information. that is not borne out by the evidence as presented.

 

---

[CallMeDave]

Quote:

---

have we established that _all_ files have ACL's or Metatdata?

 

---

 

I know painfully little about this stuff, but the hfsdbug tool provides an attributeModDate value for every file I've checked with it.

 

Here's how it looks for a random old file (formatted for monospaced font):

createDate = Fri Jun 20 23:17:53 1997

contentModDate = Fri Jun 20 23:17:53 1997

attributeModDate = Sun Feb 19 00:16:36 2006

accessDate = Tue Nov 1 17:00:23 2005

backupDate = Fri Jan 1 00:00:00 1904

 

Other old files in the same directory have ModDate entries within seconds of the one above; so it's likely that something system-wide added the attribute. But if this tool is to be believed, then all the files have the attribute.

 

Dave

[waltr]

hi dave,

 

 

 

Quote:

---

I know painfully little about this stuff, but the hfsdbug tool provides an attributeModDate value for every file I've checked with it.

 

---

 

 

 

yes, every file has an Attribute Modification Date, but it won't change unless the file has an Attribute.

 

 

 

it's just like if you create a file, the modification date will be the same as the creation date. that date won't change unless you modify the file. the Attribute Modification Date does not change until an Attribute is added (or changed) to the file.

 

 

 

and just because it has a modification date (or AttrModDate) does not mean it's been modified (or has Attr's).

 

 

 

EDITED to add:

 

this date

Quote:

---

attributeModDate = Sun Feb 19 00:16:36 2006

 

---

 

 

 

could this be the date you did an update to your system? an update that might have added Attributes?

[Lennart_T]

Quote:

---

however, there is a bug in 10.4 where the OS changes this date after a file that has attributes has been moved.

 

---

Then just update to 10.4.1 or later and the bug is fixed? Or you meant to write "10.4 through 10.4.6"?

[waltr]

hi lennart,

 

sorry if i wasn't clear. i meant the whole 10.4.X series so far. which would include 10.4.6.