incremental backup is backing up 90% of the drive each time

[cummings]

we have a disaster recovery script running that utilizes two backup sets (tapes): offsite A and offsite B. the script specifies to do a recycle backup of offsite A every other friday (and recycle offsite B on the alternate fridays). then an incremental (normal) backup is done mon, tue, wed & thu nights to that friday's full backup.

 

like this:

FRI (offsite A recycle)

MON (offsite A normal)

TUE (offsite A normal)

WED (offsite A normal)

THU (offsite A normal)

FRI (offsite B recycle)

MON (offsite B normal)

TUE (offsite B normal)

WED (offsite B normal)

THU (offsite B normal)

FRI (offsite A recycle)

and so on...

 

so on last friday night the following happened....

 

- 11/4/2006 8:18:13 AM: Copying d on fspp

- 11/4/2006 8:24:59 AM: Verifying d on fspp

- 11/4/2006 11:00:03 AM: Copying d on fspp

- 11/4/2006 3:53:27 PM: Verifying d on fspp

- 11/4/2006 5:33:50 PM: Copying d on fspp

- 11/4/2006 10:30:09 PM: Verifying d on fspp

- 11/5/2006 12:12:25 AM: Copying d on fspp

11/5/2006 2:45:10 AM: Snapshot stored, 46.0 MB

11/5/2006 2:45:17 AM: Execution completed successfully

Completed: 210443 files, 988.9 GB

Performance: 1360.9 MB/minute

Duration: 12:32:58 (00:08:55 idle/loading/preparing)

 

ok, now all 988GB have been backed up to tape.

 

then on monday night, the incremental kicked off....

 

- 11/6/2006 9:55:44 PM: Copying d on fspp

- 11/7/2006 2:43:14 AM: Verifying d on fspp

- 11/7/2006 3:58:20 AM: Copying d on fspp

- 11/7/2006 8:56:38 AM: Verifying d on fspp

- 11/7/2006 10:39:27 AM: Copying d on fspp

11/7/2006 12:40:04 PM: Snapshot stored, 45.3 MB

11/7/2006 12:40:42 PM: Execution completed successfully

Completed: 65464 files, 835.3 GB

Performance: 1222.7 MB/minute

Duration: 11:49:32 (00:09:59 idle/loading/preparing)

 

now i am CERTAIN that the 6 people who access this "d on fspp" are not adding/modifying 65,000 files in an 8 hour day (835GB worth of files).

 

most likely, tomorrow i will get another report like this, saying that another 60,000 files were modified today and had to be backed up when i know that that is not the case. that has been the history for several weeks now.

 

our vendor who sold us retrospect and the computer it runs on seems to be stumped, so i figured i would ask here.

 

what would cause retrospect to think that ~800GB needs to be backed up on this drive every night in an incremental (normal) backup?

 

1. antivirus is not running on either machine

2. nothing else is backing up these files

3. the backup source is the UNC \\fspp\d. up until last week, it used the client as a source but it was changed to this to see if it would fix the problem (and it hasn't).

 

thank you.

[cummings]

here is last night's (tuesday night) log.

 

- 11/7/2006 8:52:07 PM: Copying d on fspp

- 11/8/2006 1:17:31 AM: Verifying d on fspp

- 11/8/2006 2:44:20 AM: Copying d on fspp

- 11/8/2006 8:43:36 AM: Verifying d on fspp

- 11/8/2006 10:26:03 AM: Copying d on fspp

11/8/2006 10:37:48 AM: Snapshot stored, 45.3 MB

11/8/2006 10:38:08 AM: Execution completed successfully

Completed: 55961 files, 645.7 GB

Performance: 1049.2 MB/minute

Duration: 10:39:27 (00:09:16 idle/loading/preparing)

 

again, it is impossible that 6 people manipulated 55,000 files in their 8 hour day. i don't know why retrospect thinks it needs to backup 645GB of the 988GB on there.

 

there is no way that that much data actually was added/modified and needed to be backed up with a normal backup last night.

 

and it only happens on that one drive (the other drives that the script backs up don't have this problem)

 

here are the script settings.

 

[Mayoff]

Maybe it has to do with the way you are connecting to D$ on fspp. What type of volume is this? NTFS? UFS?, HFS? What OS is running on that share?

 

While the user isn't changing the files, they appear to be changing for Retrospect. We look at the name, size, mod date and time, creation date and time. If the filesharing connection is not giving us consistent file dates, then this will happen.

[cummings]

Quote:

---

Maybe it has to do with the way you are connecting to D$ on fspp.

 

---

 

it was originally connected by client and we get the same results. connecting d$ on fspp was an attempt by our vendor to solve the problem.

 

Quote:

---

What type of volume is this? NTFS? UFS?, HFS? What OS is running on that share?

 

---

 

NTFS, 2003

 

Quote:

---

While the user isn't changing the files, they appear to be changing for Retrospect. We look at the name, size, mod date and time, creation date and time. If the filesharing connection is not giving us consistent file dates, then this will happen.

 

---

 

the client gave us the same results.

 

any suggestions for what i could try? should i analyze the 'archive' bits on the files? see if they are changing every day on the source?

[cummings]

last night's log...

 

 

- 11/8/2006 8:39:37 PM: Copying d on fspp

- 11/9/2006 3:48:04 AM: Verifying d on fspp

- 11/9/2006 5:39:57 AM: Copying d on fspp

File "\\fspp\d\mac_production\TrueflowPrintUtilMacE23.dmg": can't read, error -1101 (file/directory not found)

11/9/2006 8:37:50 AM: Snapshot stored, 45.5 MB

11/9/2006 8:38:11 AM: Execution completed successfully

Remaining: 1 files, 2,604 KB

Completed: 50740 files, 627.9 GB

Performance: 1071.7 MB/minute

Duration: 10:08:06 (00:08:16 idle/loading/preparing)

 

 

50,000 files backed up again. and the verify is still happening at 930am this morning. if it is still happening at 10am, it will holdup my AM backups.

 

if nobody on the forum can help, is there a way i can contact retrospect directly? the software is still new, we've only had it since march.

[Mayoff]

http://www.emcinsignia.com/contactsupport

 

Tech Support can help review the individual files you are backing up and hopefully identify the cause for the files being repeatedly copied.

 

I highly recommend returning to the client. Backup will be faster with the client either way.

[nekr0phage]

Cummings,

 

I didn't see any mention of which version of Retrospect you are using. I have heard of people experiencing matching issues with earlier versions of 7.5. I believe this is resolved with the 7.5.8.101 rdu. If you are using 7.5, make sure you are using the most recent release.

[djkerber]

I'm having the same problem. I upgraded from 6.5 to 7.5 (the most current version of driver and program). I only backup Documents and Settings folder. The first backup with a newly created backup set and script, backs up about 10 GB to DVD-RW. If I backup immediately again or the next day, Retrospect wants to back up 9 GB. Somehow Retrospect thinks that I have modified 51K files. I checked the logs and it show an almost identical files list in both sessions. The backup is set to 'Normal'. Any ideas on what else to check?

[cummings]

 

i am running 7.5.8.101

 

but i may have just figured it out. i will know thursday morning.

 

here is my theory....

 

14 volumes are being backed up by retrospect to tape for disaster recovery. 13 of them are being backed up B2D in addition to disaster recovery. however, the one volume that is giving me this "i need to back up almost every single file" also has a D2D script that runs on it twice a day, not B2D like the rest. i think THAT is the factor that may be causing the problem.

 

so...

 

i cancelled tonight's disaster recovery, and cancelled the D2D on the troublesome volume for tonight, tomorrow morning and tomorrow night.

 

then i will look at tomorrow night's DR log and see if it copies 50,000 files or just the few hundred that actually get modified.

 

i will report back.

[djkerber]

I copied the same script that backs up to DVD and changed it to backup to a hard disk. The hard disk script works beautifully, but the back up to DVD requires files that have been backed up to be backed up over and over even though the files have not changed.

 

Anyone have any ideas?

[nekr0phage]

Quote:

---

I'm having the same problem.

 

---

 

You are not backing up to tape, you are not backing up over the network, you are not having the same problem, only the same symptoms. Rather than hijacking someone else's thread, please be polite and start your own.

[djkerber]

If I would have posted a new thread someone would have yelled at me for starting a new thread on the same topic.... I give up - just looking for help.

 

I thought he said DVD it was D2D (not sure what that means). Also, for grins I backed up a volume over the network to a DVD and had the same problem with almost all the files wanting to backup again. This looks very similar to what occurs when there is bad media and/or the source can not be verified - files that are not verified are re-backed up. The difference here is that I see 0 errors in the log. Could cummings and I have bad media or the source is not verified and is not being reported .

[nekr0phage]

Quote:

---

If I would have posted a new thread someone would have yelled at me for starting a new thread on the same topic.... I give up - just looking for help.

 

 

---

 

I have never seen someone chastized for repeating a thread on this forum. I have seen people simply link to an existing thread that covers the issue at hand.

 

I would like you to start a new thread because, from what I know about the problems so far, I believe Cummings' problem is related to matching, while yours is more likely related to media. And there is a seperate host of questions you will need to address which are not related to Cummings' problem and would detract attention from his issue.

[djkerber]

Cool.

 

Let me try new media - get some data and start a new thread if I still have issues.

 

L8r

[cummings]

so i don't have great news. here is the result of my doings.

 

- 11/15/2006 8:05:26 PM: Copying d on fspp

11/16/2006 4:25:37 AM: Snapshot stored, 48.4 MB

11/16/2006 4:26:19 AM: Execution completed successfully

Completed: 34357 files, 460.7 GB

Performance: 955.9 MB/minute

Duration: 08:20:54 (00:07:30 idle/loading/preparing)

 

i looked at old logs of the D2D on this troublesome volume and usually only 300-500 files change daily. not 34,357.

 

however, i did look further into this and i have something else i want to try. i am still looking for differences between the way this troublesome volume is treated compared to the others that work properly.

 

not only is this troublesome volume the only one that uses a D2D script to back it up, but also that D2D script has been the only one that accesses the volume through a network share and not the client (excluding the last couple weeks).

 

so now i've tweaked the D2D script to use the client like the rest. i've also set the DR script to use the client as well like it always used to.

 

so now no backups are being done that aren't through the client anymore, and we'll see if the problem gets solved. i'll report back again.

[djkerber]

I tried new media no difference. I tried again with the same media, but changed the verification setting from the MD5 to the thorough setting and I no longer have the problem of backed up files being re-backed up. I only have this problem with the DVD backup. The disk back work fine with either setting.

 

Out

[cummings]

SUCCESS!

 

- 11/16/2006 7:47:14 PM: Copying SAN_Storage (D:) on FSPP

11/16/2006 8:05:30 PM: Snapshot stored, 48.4 MB

11/16/2006 8:05:38 PM: Execution completed successfully

Completed: 330 files, 10.3 GB

Performance: 750.9 MB/minute

Duration: 00:18:24 (00:04:23 idle/loading/preparing)

 

moral of the story....

 

if you are going to back up a volume with two scripts (an onsite backup & an offsite backup) don't use the network path for either of the backups, use the client for both.