Normal back-up backs up way to much data

[arjenpubliqare.nl]

Hi all,

 

I encounter the following problem;

 

The normal backup which occurs after a recycle backup, thinks it has to back-up almost anything again.

 

Environment:

 

Retrospect 7.5 on W2k3 server

Clients:

W2k3 server (Active directory master, also time server for all other servers)

OS X server 10.4.10 (Connected to a directory system)

 

 

3 week schedules

- Friday = recycle back-up (runs OK)

- Mo, Tu, We, Th = Normal backup to the same backup set.

 

The amount of data is around 1,5 TB

 

I expect Retrospect to back-up everything on friday and to back-up only the files that have changed since the last back-up to the same back-up set. (probably something like a couple of GB's)

 

What happens is that Retrospect thinks that a normal back-up is around 960 GB's of data...

 

I have tried al kind of settings (The ACL thing, NTFS rights etc.

 

This is absolutelyy driving me insane, I thougt I knew Retrospect rather well, but I can't solve this one...

[robvil]

and you do not have any software (AV) running that changes the time stamp on files causing the files the get the A bit set?

 

 

[arjenpubliqare.nl]

No, for sure not on the OS X server.

 

 

[DigiAngel]

I seem to have the same issue. From the User's Guide:

 

"A Normal backup copies only files which are new or newly modified. During a Normal backup, Retrospect compares the list of files selected to be backed up against the list of files in the Backup Set's Catalog File, then copies only those files which are not already present on the media". That doesn't seem to be happening in my case.

 

Here's what I have:

 

- 3/19/2008 3:02:45 AM: Copying Share (V:) on vcfexch

3/19/2008 9:20:33 AM: Snapshot stored, 49.1 MB

3/19/2008 9:21:25 AM: Execution completed successfully

Completed: 127138 files, 184.9 GB

Performance: 518.8 MB/minute

Duration: 06:18:36 (00:13:51 idle/loading/preparing)

 

and the following night:

 

 

- 3/20/2008 3:02:20 AM: Copying Share (V:) on vcfexch

3/20/2008 9:23:13 AM: Snapshot stored, 48.5 MB

3/20/2008 9:24:03 AM: Execution completed successfully

Completed: 127134 files, 184.9 GB

Performance: 516.4 MB/minute

Duration: 06:21:43 (00:15:17 idle/loading/preparing)

 

That share is barely accessed now...any thoughts?

 

 

[Mayoff]

What is the destination in both examples above? Same backup set or different backup sets? Your log example is missing critical info to help us understand what Retrospect is trying to do.

[arjenpubliqare.nl]

Same back-up set.

[DigiAngel]

Exact same backup set

[MRIS]

The obvious question, what MATCHING options do you have set for the job? Everything unchecked? If so then that is the problem. Try pressing the "defaults" button on the Matching section of the job options.

 

If sometime during the day you open the immediate backup window and set the source and targets the same as the job, and then press the preview button so that the scan occurs, what does it suggest is the quantity of data that needs to be backed up?

[arjenpubliqare.nl]

I have started an immediate backup with all the same settings.

It tells me it needs to back-up (normal) 175 GB of 1600 GB data. This could be OK.

It shows me the results (when I hit "Preview") like in the attachment.

Basically the check boxes for all the files are set, but a lot of them are matched (probably OK).

The backup runs right now, but I have the feeling it back-ups everything, I will check back later with the results...

[arjenpubliqare.nl]

OK, the answer is already there...

 

One of the volumes I am backing up is in total 26.x GB large.

It is backing it all up. Which should not occur. I do not see modify 5 people word en excel files in such an amount...

 

So the question is:

 

Is it correct that Retrospect is selecting files that are matched?

Shouldn't they be deselected automatically?

 

 

 

[Mayoff]

Files with a Check mark will get backed up

Files with a check mark and a diamond will be skipped

files with diamond will be skipped.

[arjenpubliqare.nl]

I thougt so, but the behaviour I do see is not what you describe above.

 

What else could be wrong here?

Any thoughts are very much apprieciated.

[MRIS]

If the 26GB volume is on the OSX computer, then what are the backup script options, I am specifically interested in the two sections:

Execution

Matching (top 2 options ticked?)

Macintosh

Client (use attribute modification date ticked?)

defaults or customized on these screens?

[arjenpubliqare.nl]

The 26 GB volume is on a W2k3 machine...

For matching I have the upper 2 options set.

It seems to me that this problem is not platform specific, which I have thought at first.

[arjenpubliqare.nl]

OK, gettin' stranger...

 

Some volumes backup incremental and some back-up everything.

So I guess it isn't any setting in Retrospect.

I am running an scripted backup right now, will check tommorow morning (europe here).

[MRIS]

I'm thinking that there might be an anti virus product involved. For example CA eTrust creates an alternate data stream on each file it has checked, either with the on-access scan or the sceduled scan. Presumably it uses the ADS to tag each file with the definition level it was already scanned with. It presumably can use this to determine whether a file is already cleared, and thus can skip a re-scan and thus gain performance. It also presumably attaches the ADS without affecting the archive bit, something Retrospect doesn't use. I'm guessing Retrospect sees this change in the file and backs it up.

[rhwalker]

It appears that you are doing a duplicate rather than a backup (from the matching JPEG). Correct?

 

You might investigate the underlying filesystems and how the shares are being presented to the computer running Retrospect. It may be that the metadata support is not complete as compared to the source volume, such that some stuff appears changed, or that the time stamp granularity of the source doesn't match that of the destination, such that the timestamps don't match.

 

See if the same thing happens for a backup rather than a duplicate.

 

Russ

[clatus]

Hi,

 

I have had this behavior regularly enough to consider it a Retrospect bug. It has been occurring for over a year. It has occurred on multiple sites (at least 4 that I'm aware of). We don't always catch it happening unless it unexpectedly fills a tape. For us it happens at random. We might run incrementals successfully for a month of more, then Retrospect will backup everything again.

 

We reported it to our Australian rep, who basically said "too bad".

 

Because it happens at random, I don't believe it is AV related. In the Retrospect catalog, it knows that the file has not been modified since the last backup. I suspect it relates to some sort of catalog corruption.

 

Good luck with solving your issue.

 

Sean

Edited April 1, 2008 by Guest

[Mayoff]

If you suspect catalog corruption, just do a catalog rebuild. Easy.

 

We have a known bug with some file types that report "file didn't compare". In all other situations, the file or the file metadata is changing in a way that would force the data to be copied again because it is different.

[jccl2]

I have the similar problem but here we have Windows CSC (Client Side Cache), I ran backup full than several normal but they get almost all data. The CSC change the view date and time but the file still the same, thinked this is the problem, so can bypass this check from retrospect? How can put the retrospect just for check the change date and time not the view time?

 

Becouse the options Match source volumes to catalog files and don't add duplicates to backup set are maked but still backup full and don't normal and the files for sure not changed only few files about 10Gb not 480Gb.

 

Thanks.

[MRIS]

You should probably add the CSC folder to the exclusions list:

 

C:\WINDOWS\CSC\

[jccl2]

On the server side ?

 

[MRIS]

sorry, I meant that you should probably modify your retrospect selector so that this folder is specifically excluded from backup.

 

For example if you are backing up "All files except cache files" then copy that selector, edit the copy to add this folder to the excluded items list, then edit the script to use this edited copy of the selector.

 

I am making a silent assumption that files stored in the CSC also exist on one of your servers elsewhere, and am expecting that these servers are being backed up. Thus the data on the clients in the CSC folder is largely disposable because it is a copy of data held safely elsewhere.

[jccl2]

Ok, lets me show my structure:

 

Storage => All Files

Server => With CSC

Clients => Use My Doc. on storage and CSC at local, but all files stay on storage and can be access from any client station, just logon with they user.

 

So, the server retain all files and when user logon the CSC check the files, but don't change anything, just the access time.

 

If the user logon just for print some .doc when the backup start the retrospect copy all files check by this user, its mean all files from My documents. I sure the user don't change anything but still making all files backup.

 

So, on retrospect whow can I override this action, because with normal operation it's change only some files not the entire folder. On server I cant put the My Documents on exclude list, but I need backup only the changed files not all accessed files.

 

Thanks.

[MRIS]

I am not sure I completely understand the structure however I will make some general comments.

 

Normally file servers host all files and share them via network shares. It is the servers that should be fully backed up with Retrospect. Are you having a problem with this part?

 

Next the client PCs access the files shared by the server, usually via a network drive or UNC path. These files are already being backed up by the server above.

 

Also, any file accessed via a share is also stored by each PC in its own CSC. This is a technique to ensure files are available to the user on the PC should they be away from the server/network. So the CSC itself does not need to be backed up, because these fils are really on the server.

 

Next each user on each PC by default has a My Documents folder locally, not to mention all the applications and the operating system. Backing up each PC with Retrospect is recommended, so that all these files are backed up.

 

Now I understand that the CSC folder on the PC represents a significant quantity of data that appears to change frequently and what you are saying is that this appears to be filling up your backup sets quickly.

 

My suggestion and recommendation is to exclude the CSC folder so that the backups of the PCs don't include it.

 

So:

 

Backup the servers normally.

Backup the PCs normally.

 

For the backup of the PCs, exclude the following folders:

 

C:\RECYCLER\

C:\WINDOWS\CSC\

C:\SYSTEM VOLUME INFORMATION\