Jump to content

Incremental backs up too much


Recommended Posts

I’m using a Mac OS X Server running Retrospect Workgroup 5.0.205 and backing up to a DLT 7000 tape. Yesterday, after some useful suggestions from this forum on getting things set up, I did a complete normal backup of 4 computers. Today I did our usual incremental backup, and one computer had a dozen or so files to back up, but for the other three, Retrospect backed up the entire computer once again. The User’s Guide, p. 201, says there are two possible reasons for this:

 

 

 

1. Local time zone was changed. (I know this did not happen on any of the computers.)

 

 

 

2. When the system clocks of the server and the backup Mac are not synchronized within a certain range. (It is unclear just what this “certain range” is, but the clocks on the computers in question were within a couple of minutes of each other.)

 

 

 

I will note that I did not have the Network Time Server enabled on any of these computers, thinking that any little adjustments made to the clocks would throw off the backup. But perhaps I have this just backwards, and the Network Time SHOULD be enabled, keeping the computers, I would guess, within a few seconds of each other. Is this the “certain range” and “acceptable margin” for the clocks mentioned in the User’s Guide?

 

 

 

Or is there perhaps some other reason for this failure of incremental backup? Any suggestions or advice would be valued.

 

 

 

 

Link to comment
Share on other sites

Retrospect uses several matching criteria to compare files that have already been backed up to what is about to be backed up. If one of the following has been changed at all, Retrospect will back up the file again: name, size, type, creator, creation date and time, modify date and time, and label.

 

 

 

If Retrospect is reporting that it is about to back up files you know have not changed since your last backup, take a closer look at a sample file. Choose a file and start a restore by search for that file. When you get to the final window (you don't actually have to do the restore), click Files Chosen and Get Info (type Command-I) on the most recent version of the file on the backup. Print or take a screenshot of this window.

 

 

 

Now, go to the Configure tab from the Retrospect Directory and click Volumes. Choose the volume the file is on and click Browse. Find the file in question and Get Info on it. Put the two windows side by side and look for even the slightest difference in any of the criteria I listed above.

 

 

 

Many things can cause a change in a file, even if the file has not been accessed by the user. The operating system, as well as applications, are constantly making changes to files.

 

 

 

Unfortunately the only known way around this problem is to do a full backup of your data. I know of no way to force file attributes to regress to the way they were before. Either do a full backup or use this time to introduce new media.

Link to comment
Share on other sites

Thank you for your helpful response; I’ve done some investigating as you suggested and here’s what I found:

 

 

 

I started my current backup set on 1 May, and have run it 4 times. While there were some problems getting all of my computers to back up (which problems were solved by other posts on this Forum), one computer backed up fine all 4 times, though each time it backed up completely (i.e., not incrementally), all 7.8 gigs worth. When I did a restore by search for a selected sample file from this computer, and get info on the 4 files that were found, this is what appeared in the Modify area:

 

 

 

Backup of May 1: Modified Wed, Sep 26, 2001, 9:43:51 AM

 

Backup of May 3: Modified Wed, Sep 26, 2001, 9:43:50 AM

 

Backup of May 7: Modified Wed, Sep 26, 2001, 9:43:46 AM

 

Backup of May 8: Modified Wed, Sep 26, 2001, 9:43:45 AM

 

 

 

This file was untouched by anyone, and yet the modify time had changed slightly. I checked a sample file on a couple of other computers that we back up and found a similar “time slippage” in the creation and modify times. Well, it appears that Retrospect is functioning properly, and backing up files that have “changed,” by whatever method, so perhaps this problem is out of your bailiwick, but do you have any idea what might be happening? The computers I back up contain about 20 gigs so it is really not feasible to back them up completely every day, yet if the times are changing like this it will be impossible to do an incremental.

 

 

 

Anyone have any ideas or suggestions?

 

 

 

Thanks for any help you might offer.

Link to comment
Share on other sites

I believe that this is a bug with the released version of Retrospect that was identified in the beta version. I am running into a similar problem with backup/modify dates being recorded incorrectly in the backup set. I did a backup of a Windows2000 AFP volume this afternoon, the created and modified date were immediately off buy 2 seconds, but the Backed Up date was off by over 70 years! If I went to resore the file, it has a backed up date of today, if I go to do an immediate backup of the file, it says it was backed up in 1931.

 

 

 

Also, I have been able to reproduce the problem by doing the following:

 

1 Do a backup on a folder on an AFP volume on a windows 2000 server

 

2 Attempt to backup the same folder - retrospect reports that the folder is already backed up

 

3 Quit Retrospect and dismount the server volumes.

 

4 remount the server volumes and launch Retrosect

 

5 Attempt to backup the same folder - Retrospect reports that the file has now changed, and wants to back it up again.

 

 

 

 

 

Now, obviously something is causing Retrospect to believe that the file is changed, the quuestion is what. I have been using 4.3 for several years without a problem, and nothing has changed on the server, so that leaves OS X or Retrospect 5. Since my backups off of 4.3 still run correctly, that indicates that the OS X is not changing the modify date on the server - otherwise 4.3 would want to backup all the files again, which is not the case. That leaves Retrospect 5.

 

 

 

This problem was first identified back in Feburary, and I assumed that it would have been fixed in the released verision, or I would not have spent the $$$ to upgrade my server version to 5, only to find out that i can't use it because it always tries to backup all the files every night! (well over 150 GB).

 

 

 

I know this can be fixed it the problem is identified and taken seriously.

 

 

 

Thanks,

 

Charlie Rieger

Link to comment
Share on other sites

Let's try and narrow it down a bit. If you look at a sample file in Reports > Contents, and then compare it to the file on the source, are they identical?

 

 

 

If you copy that file from the source to the local volume and then compare Get Info windows, are they identical?

Link to comment
Share on other sites

The following is the results of your request. As I read it, Retrospect is incorrectly identifying the modify date of the file, as both the original on the network and the one copied locally today both have the same modify date, which matches the 1st backup, but the second backup has incremented up by 9 seconds, which does not match the acutal state of the file as reported by the finder or the Win2k server on which it resides.

 

 

 

 

 

(File) (Size) (Created) (Modified) (Backed Up)

 

 

 

(Original on Network) (280,802 bytes) (Thu, Mar21, 2002, 4:07 PM) (Fri, Mar 22, 2002, 9:57 AM)

 

 

 

(Original on local Drive) (280,802 bytes) (Thu, Mar21, 2002, 4:07 PM) (Fri, Mar 22, 2002, 9:57 AM)

 

 

 

(1st Backup) (280,802 bytes) (Thu, Mar21, 2002, 4:07:29 PM) (Fri, Mar 22, 2002, 9:57:51 AM) (Fri, may 10, 2002, 12:47:42 PM)

 

 

 

(2nd Backup) (280,802 bytes) (Thu, Mar21, 2002, 4:07:20 PM) (Fri, Mar 22, 2002, 9:58:00 AM) (Mon, May 13, 2002 9:26:55 AM)

 

 

 

 

 

 

 

 

 

The info from the orginals is as of 5/15/02

Link to comment
Share on other sites

We also have this problem. The incremental backups have become worse with the new X server version of Retrospect.

 

 

 

In previous versions of Retrospect, the AFP mounted volumes from a Xinet Fullpress server (best damn OPI/File Server for Macs http://www.xinet.com) running off of Sun Solaris had no day to day problems recognizing and comparing incremental changes. However, twice a year with the daylight savings change the Retrospect storage set would see all the data as modified. No problem really, as we all should rotate sets/tapes anyway.

 

 

 

Now with OS X and the current version of Retrospect Server for X, day to day incremental backups do not work at all with AFP mounted volumes. From all the time I invested on this, don't bother trying to get the clocks synchronized, it did not matter at all. The problem is not a user caused or correctable problem. AFP mounted volumes are unsupported by Dantz see page 161 of the new manual.

 

 

 

Dantz support denied a problem. I have some sympathy, surely they must recover some develop costs to get to X. We have tried over several years and several versions to bring this discussion to Xinet and Dantz. Neither of them believe this problem can be solved calling it an Apple issue, but Dantz can't explain it anymore than we can and Xinet recommends Flashnet (http://www.sgluk.com/print/non_flash/afset.htm). Dantz, Xinet and Apple don't talk. I will mention on Xinet's behalf they do have Fullpress for OS X Server and that would likely work since the data would be on the same machine as Retrospect (not a mounted network volume). Problem is Mac OS X is so immature related to third party SCSI and SAN related add-ons, no one in there right mind can risk the large mission critical data on another new Apple Mac server (don't get me started on that history: ANS, ASIP, X 1.2).

 

 

 

So to end my rant. There is a world of Unix products for industrial sized datasets. Time to go looking.

Link to comment
Share on other sites

> (2nd Backup) (280,802 bytes) (Thu, Mar21, 2002, 4:07:20 PM) (Fri, Mar 22, 2002, 9:58:00 AM) (Mon, May 13, 2002 9:26:55 AM)

 

 

 

 

 

Where is this report from? Are these numbers reported in Retrospect's report contents browser? If you do a Get info on these files after the backup, what, exactly, shows up in the windows? Is Retrospect trying to back up both the local file and the other file again?

Link to comment
Share on other sites

The local and network info was fron the finder info window. The 1st and 2nd backups were taken from Retrospect's report contents browser. The finder info was taken several days AFTER the backup, and yet shows a modification date/time prior to that of the second backup. If i tried to initiate a 3rd backup the file modification date increments up yet again, but only in retrospect (immediate backup), not the finder info.

 

 

 

Retrospect would attempt to backup both the local file (because the location has changed) and the original file on the network. (eroniously believing that the modification date has changed.

 

 

 

I am now trying to run 5.0.205 in OS9.2 to see if it behaves correectly

Link to comment
Share on other sites

5.0 under OS 9 seems to do the matching correctly, unlike under OSX. Unfortunately I have seen many problems associated with my VXA drives in 5 that are nor evident in 4.3. So, until the issues with 5.0 and OSX get resolved, I'm stuck with using 4.3 under OS9.2 as my primary backup solution. Not a good solution since I had hoped to complete the transition to OSX with the release of Retrospect 5. Has anyone at Dantz been able to confim these issues as reproducible bugs?

 

 

 

Charlie

Link to comment
Share on other sites

I have also had this problem. It's a real headache, since I have many Gigs of files to backup.

 

 

 

My problem occured when backing up files on an ASIP6 server, and Snap Servers that were mounted using Appleshare.

 

 

 

When I tried to figure out what was going on, I created a whole new backup set, and backup script. I placed a file in a folder on a ASIP6 server, and set the script to backup the entire folder. I ran the script and backed up the files. I then immediately ran the script again, and the files were again copied into the backup set even though they had not changed. The same thing happen the third time I ran the script. Examining the details on the files (as suggested by the moderator) reveals that the creation and modifcations time stamps are different by 1 second for each backup. Since the time stamps change, obviously Retrospect will perform an incremental backup.

 

 

 

My question is, is this an Appleshare problem under OS X, or is Retrospect recording the wrong time information?

 

 

 

 

Link to comment
Share on other sites

My vote goes for a retrospect problem, since in my tests the finder shows the modification date to be unchanged, even after retrospect has backed up the file 3 separate times. Also, a backup done under OS9.2.2 using Retrospect 4.3 will not attempt to back up the file again, even after V5 has said that it must. To me that proves that the file hasn't changed. Either Retrospect or OSX is showing some modification to the file that doesn't really exist. And since the finder under OSX doen't show a change, that narrows it down to Retrospect.

 

 

 

Can this be confirmed as a bug yet?

 

 

Link to comment
Share on other sites

I'm backing up an ASIP Server too. I have to mount the volumes because the client crashes the server.

 

 

 

I get the same non-inremental behavior too. The server has over 300 GB. Most of that data is unchanging archives. I backup to a 30 tape DLT library. However, after 3 or 4 non-incremental backups the backup set grows over 1 TB. This is the size limit for a backup set. So after about a week I have to load the library with tapes and start a new backu set.

 

 

 

I have verified the non-incremental behavior the best way I know how. I loaded a CD ROM in the ASIP server and shared it. I mount the shared CD to the Retrospect 5 server and back it up with the other ASIP volumes. Sure enough after a few days Retrospect has determined the data on the CD ROM has changed. So it backs it up again.

 

 

 

I also backup workstation clients. They seem to act up too, but not as fequently as a mounted volume. I have not research them as much as the ASIP server.

 

 

 

Something is broken here.

 

Please fix it.

 

 

 

Eric

Link to comment
Share on other sites

In reply to:

I know this can be fixed it the problem is identified and taken seriously.


 

Unfortunately, so far none of these posts has done a very good job of identifying the problem!

 

 

 

Charlie's first post has some steps, but leaves out critical information such as what operating system Retrospect is running under (while the Macworld Preview was an OS X only release, there was a private beta program that worked for both OS 9 and OS X)!

 

 

 

Here's the test I did:

 

 

 

- On OS X 10.1.4 machine mount network AFP volume (from ASIP 6.3.3 server under Mac OS 9.1)

 

- Launch Retrospect 5.0.205

 

- Configure->Volumes

 

- Select volume name in Volume Database window

 

- In Volume menu, select Configure (command+J)

 

- Enter ASIP password

 

- Unmount ASIP volume (from either Volume menu or Finder)

 

- Immediate->Backup and configure Source using grey icon of AFP volume (click "Subvolume" to see volume mount on Finder Desktop)

 

- Execute Immediate backup of AFP volume (or a defined subvolume) to File Backup Set

 

- Quit Retrospect and relaunch

 

- Immediate backup window should have saved the backup information; click on "Preview" and note what files are selected for incrimental backup

 

 

 

In my test Retrospect behaved as expected with no files needing to be copied (and no files are copied if the backup is executed without first clicking on Preview).

 

 

 

Would someone who's seeing a problem kindly list their STR (steps to reproduce), or at least how their steps differ with the ones above?

 

 

 

Dave

Link to comment
Share on other sites

Dave,

 

The only additional step needed to reproduce this issue in my tests has been to unmount the AFP volume after quiting Retrospect. I followed your steps, and retrospect did not try to add the files to the backup set again. I then quit retrospect, unmounted the AFP volume, launched retrospect and had it mount the volume, and lo and behold, the files needed to be backed up again! In some tests, it has taken 2 or 3 iterations of Launch/Backup/Quit/unmount/Launch/Backup... to get the files to need backing up again. However, it is consistent in that it will occur. Below is the details of my setup:

 

 

 

- On OS X 10.1.4 machine mount network AFP volume (from win2000)

 

- Launch Retrospect 5.0.205

 

- Configure->Volumes

 

- Select volume name in Volume Database window

 

- In Volume menu, select Configure (command+J)

 

- Enter password

 

- Unmount AFP volume from Finder)

 

- Immediate->Backup and configure Source using grey icon of AFP volume (click "Subvolume" to see volume mount on Finder Desktop)

 

- Execute Immediate backup of AFP subvolume to Tape Backup Set

 

- Quit Retrospect unmount voume from Finder, and relaunch

 

- Immediate backup window should have saved the backup information; click on "Preview" and note that it wants to back the files up again. The modification time has changed by 2 seconds, however the date remains the same (Friday March 22, 2002)

 

 

 

Please let me know if you need any additional info, as i am happy to give all I can to get this issue identified and resolved.

 

 

 

Thanks,

 

Charlie Rieger

 

 

Link to comment
Share on other sites

I've done all I can to figure this out through this forum, but the people at Dantz don't seem to be very responsive. I believe I've listed the exact steps needed to reproduce it, as CallMeDave requested, and answered all of IrnenaS questions to the best of by ability, yet several days have gone by without a reply. So am asking again...

 

 

 

Can anyone at Dantz confirm this as a bug and put it at the top of the list (or very near the top) of issues to be resolved? Incremental backups are one of the cornerstones of Retrospects features. To have it not work because the program eroneously increments up the modify time (the actual file is NOT modified of altered in any way) after unmounting the volume is inexcusable.

 

 

 

Charlie Rieger

Link to comment
Share on other sites

pnorton wrote:

 

> My question is, is this an Appleshare problem under OS X, or is

 

> Retrospect recording the wrong time information?

 

 

 

Good question!

 

 

 

CharlieRieger wrote:

 

> And since the finder under OSX doen't show a change, that narrows it down to Retrospect.

 

 

 

My finder does not show seconds for the modification time in the Get Info window, are you certain the Finder does not also believe the file modification date has changed?

 

 

 

Try checking the modification date of a file including the seconds through some means independent of Retrospect, such as using "ls -lT" in Terminal. (I can give more explicit instructions if you need them.) Check the mod time before you even launch Retrospect, then check the mod times reported by Retrospect, and then check the mod time again after you quit Retrospect if you note a difference.

 

 

Link to comment
Share on other sites

I'm not familiar with IS-IT, so some instructions would be helpfull, however, since the modification date and minutes remains the same (several weeks ago) it seems logical that if it were changed on the server, the date would be changed to todays date as well. Also, if I switch back to Retrospect 4.3 running under OS 9.2, and attempt to backup the same file it says that it doen't need to. To me that indicates positivley that the file has not changed, and retrospect 5.0 is identifying the modification time incorrectly.

 

 

 

That being said, please let me know how to verify it indenpndently and I certainly will.

 

 

 

Charlie

Link to comment
Share on other sites

Emulator posted this same problem in a new forum called Incrmental Backup Problems

 

 

 

IrenaS responded with the same Danz line of making sure there where no differences in the file. Even though Emulator was talking about 300 GB worth of files I guess it's possible that every file had changed from the day before.

 

 

 

Again I submit for you approval (from one of my previos posts)

 

I have verified the non-incremental behavior the best way I know how. I loaded a CD ROM in the ASIP server and shared it. I mount the shared CD to the Retrospect 5 server and back it up with the other ASIP volumes. Sure enough after a few days Retrospect has determined the data on the CD ROM has changed. So it backs it up again.

 

 

 

Retrospect V5.0.205 on OS X 10.1.4 backing up mounted ASIP 6.3.3 / 9.1 volumes.

 

 

Link to comment
Share on other sites

> I'm not familiar with IS-IT, so some instructions would be helpfull,

 

 

 

Open Terminal (in the utility folder of the Applications folder).

 

Type into the Terminal window (or copy and paste), and add a space on the end:

 

ls -lT

 

(ls is the List command, the flags are for Long format with complete Time, those ambiguous 1lI letters are L's.)

 

 

 

The trailing space is so we can easily put in the filename or folder to do the listing for. Go to a file you are interested in using the Finder, and drag and drop the file (or folder) onto the Terminal window. Terminal will put the full unix path of the file on the end of the current line. Just switch back to Terminal and hit Return, you'll see something like:

 

 

 

[p188:~] john% ls -lT /Users/john/ia5mac.zip

 

-rw-r--r-- 1 john staff 30536944 May 13 17:08:08 2002 /Users/john/ia5mac.zip

 

 

 

> To me that indicates positivley that the file has not changed,

 

> and retrospect 5.0 is identifying the modification time incorrectly.

 

 

 

Since you are mounting the server using a different operating system, and from other reports I understand the modification date only wanders by seconds, I am still interested in verifying what the file system thinks is happening. (I haven't scoured the forum so apologies if someone already checked this.)

 

 

Link to comment
Share on other sites

In reply to:

The only additional step needed to reproduce this issue in my tests has been to unmount the AFP volume after quiting Retrospect


 

 

 

By George, that does it!

 

 

 

But the problem is _not_ caused by Retrospect. It's apparently yet-another OSX versus AppleFileProtocol problem, which can be seen using the Terminal (the flag "-lT" is a lower case "L").

 

 

 

Steps:

 

 

 

1- Mount AFP volume (ASIP 6.3.2 / Mac OS 9.1 in this test)

 

2- Get info on random file:

 

[localhost:Volumes/Server/Shared_Items] joe% ls -lT Truce.mp3 

 

-rwx------ 1 joe unknown 2526076 Apr 18 08:14:54 2001 Truce.mp3

 

3- Unmount AFP volume (must first exit shell process or volume will be busy)

 

4- Remount AFP volume

 

5- Get info on same file:

 


 

[localhost:Volumes/Server/Shared_Items] joe% ls -lT Truce.mp3

 

-rwx------ 1 joe unknown 2526076 Apr 18 08:14:55 2001 Truce.mp3

 

 

Retrospect probably gets a file's date from the operating system. And since the operating system is apparently capable of getting the dates wrong, Retrospect acts as if the files have changed.

 

 

 

Sucky.

 

 

 

Dave

Link to comment
Share on other sites

Dave,

 

Thanks for confirming it! Now I know I'm not crazy (although this really proves nothing in that respect).

 

 

 

Ho do I go about pointing Terminal at the AFP mounted volume? I'm not as familiar with Terminal as I should be...

 

 

 

Also, is Dantz addressing this issue with Apple? I'd hate to have to wait months for a fix just because Apple was unaware of the problem!

 

 

 

In the mean time, are there any other methods of mounting the share that will presever the resource fork but avoid the modify time problem? SMB, FTP etc.? I'm mounting a Win 2k SFM share, and the Retrospect client doent backup the resource fork, so I don't think that is a viable workaround.

 

 

 

 

 

Thanks again for your help

 

 

 

Charlie

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...