Lost using Retrospect 8.1.626

[alecio]

Hello,

 

Backing up using Retrospect 8.x has been a nightmare for me.

Right now I'm at version 8.1.626.

I used to be an only windows user using NovaBackup and that ran like a charm.

I considered Retrospect as it came with the Iomega External Drive I bought and as it proclaimed the many capabilities that it had, this seemed like the logical choice.

I've started backing up in August last year and of all the backups that I have, I doubt that I have a full or reliable backup.

Each backup has been a Full backup and yesterday I decided to do a full backup without cache files.

I have over 700K files and also two virtual windows machines of 100GB of which one is one file of 99.9GB and the other one is subdivided in chunks of 2GB.

All my backups show the number of files to be very little.

This is a sample of the latest backup info.

 

+ using Backup Assistant - 4/4/10 19:37 at 4/4/2010 7:38 PM

To Media Set Full Backup...

- 4/4/2010 7:38:49 PM: Copying Macintosh HD

4/4/2010 8:16:49 PM: Snapshot stored, 191.7 MB

4/4/2010 8:17:19 PM: Comparing Macintosh HD

 

*File "/Volumes/Macintosh HD/Library/Application Support/Retrospect/Catalogs/Full Backup.rbc": different data size (set: 780549096, vol: 781180128)

 

*File "/Volumes/Macintosh HD/private/var/db/ntp.drift": didn't compare

 

***** In between there were additional messages of the same type …

 

4/4/2010 8:34:36 PM: Execution completed successfully

Completed: 219 files, 11.8 GB

Performance: 646.3 MB/minute (599.6 copy, 700.8 compare)

Duration: 00:55:46 (00:18:30 idle/loading/preparing)

 

This is a full backup and 219 files is awkward.

This is also true for other backups done.

I really don't understand what is happening and have quiet some experience with backups and IT.

 

The other issue I have is that I have a Media set which shows that I have 14 backups and I have 1.1TB used and about 308GB Free.

I've deleted entries in the activities and also deleted the physical files belonging to those backup activities.

Right now I have 9 Activities and the Get Info shows that I have 760GB of space left.

Is there a way to sync the available space so instead of the erroneous 308GB I get 760GB?

Thanks for your reaction

 

Alecio

[Maser]

A couple of comments:

 

1) When you say "a full backup" -- do you mean you did a completely new backup to a new, empty, media set?

 

The Retrospect paradigm is -- using the default settings -- to only backup *incremental changes* from a source in subsequent backups if you've already backed up the source once to a media set.

 

If you want to do a completely *new* full backup, you either have to "recycle" your existing Media Set -- which will delete everything you've previously backed up to that media set --, or create a new Media Set.

 

 

2) the "Full Backup.rbc" file in your message -- this implies you are backing up the media set catalog file -- which would change from the time you started until the "verification" phase. That would be expected. Any "different data size" message would mean that the file was changed from the time the computer was scanned until the file was actually backed up -- the verification phase ("Comparing") tells you that.

 

"didn't compare" -- means just that. For whatever reason, that file wasn't compared.

 

 

3) Your final issue -- when you say you "deleted the physical files" -- you mean from the source? This does not remove them from the "media set" -- you would need to groom out those backups containing those "physical files" if you wanted to remove them from your media set.

 

 

 

You might want to skim the newly released manual for the program as to how it functions.

 

[alecio]

Thank you so much for your reaction.

 

I just yesterday saw that there actually was a new documentation for Retrospect 8.

I have not been able to print and go through it yet but will do so.

 

From your answer on the first topic, can I deduct that it is possible to create a script that will always do a full backup for you?

 

I will try and either groom or recycle to correctly reflect the info of the media set.

 

Regards,

 

Alecio

[Maser]

I believe you can create a backup script and in the "schedule" section of the script, there's a media action of "Recycle Media Set"

 

I've never done this, though, as I'm not sure why you wouldn't just delete the media set (or manually recycle it if you are certain you want to do this) and create a new one for this purpose.

 

 

You might look into the "groom" options instead if you are looking to keep the amount of data in your media set to a manageable level.

 

 

 

[cpguru21]

in retrospect GUI, under Scripts, select your script so it is highlighted. click on the options tab toward the bottom, click on the > arrow beside the word Backup, and click on Matching. To have Retrospect NOT perform incremental backup, uncheck the option Match source files against the Media set.

 

I learned this the hard way and ran out of space on my media set.

 

**

Unchecking this option for me created a new backup of all the files/drives I selected in my script, on my media set, every time the script was run.

 

Hope this helps.

Edited April 6, 2010 by Guest
added words "on my media set"

[rhwalker]

Alecio,

 

The Retrospect paradigm is different from that of other backup programs, and it's hard for new users to understand. Once you come to understand it, I believe that you will find that it's a more natural way to view backups.

 

See my explanation in another thread:

Retrospect paradigm

 

Your initial post seems to indicate that you don't yet quite understand the Retrospect paradigm (see the link above to my response in another thread), and also that you've got some files changing during the backup.

 

Steve Maser explained this message that you saw:

 

4/4/2010 8:34:36 PM: Execution completed successfully

Completed: 219 files, 11.8 GB

Performance: 646.3 MB/minute (599.6 copy, 700.8 compare)

Duration: 00:55:46 (00:18:30 idle/loading/preparing)

 

This is a full backup and 219 files is awkward.

No, only 219 files needed to be backed up. The others were still in the Media set from a previous backup session. Retrospect only backs up what needs to be backed up. Again, see the link above where I explain the Retrospect "snapshot" illusion.

 

I've started backing up in August last year and of all the backups that I have, I doubt that I have a full or reliable backup.

Each backup has been a Full backup and yesterday I decided to do a full backup without cache files.

I suspect that each backup has been a "Normal" backup, and that Retrospect has done the "right thing". Again, read the explanation at the link above regarding the illusion that Retrospect presents with its "snapshot" paradigm.

 

It won't work if you try to use the model presented with other backup programs of "full" and "incremental" backups.

 

*File "/Volumes/Macintosh HD/Library/Application Support/Retrospect/Catalogs/Full Backup.rbc": different data size (set: 780549096, vol: 781180128)

 

*File "/Volumes/Macintosh HD/private/var/db/ntp.drift": didn't compare

The "ntp.drift" file changes as the operating system updates its timeclock from an external standard. This file is used to make your system clock accurate over the long term, and it might change.

 

All that the warning message did was to alert you to the fact that, during the compare phase, this file on your source didn't match what Retrospect had in its media set. If you use MD5 digests, then Retrospect will compare the file in its media set with the MD5 digest it calculated during backup (rather than comparing with the actual file) to determine if the file in the media set is good. It's all in how you choose to set things up.

 

Likewise, same thing with the .rbc file, as Steve Maser explained.

 

From your answer on the first topic, can I deduct that it is possible to create a script that will always do a full backup for you?

Yes, you can, by erasing the Media Set each time (recycle), but that's not using the power of Retrospect. I suggest that you read my explanation in the other thread about what really happens with Retrospect.

 

Hope this helps,

 

Russ

[cpguru21]

Russ,

 

Thanks for the above link. It deffinitely helps me make better sense of the software. Very helpful.

 

Wish you posted this earlier :content:

[Maser]

I just give lazy off-the-cuff responses. Russ gives the more detailed ones with quotes and pointers to other threads. ;-)

 

 

[cpguru21]

 

You both give well rounded and respectable help to the users of this forum, and I am thankful for both of you!

 

Russ does seem a little more OCD when it comes to his replies ;-)

 

just kidding just kidding!

[rhwalker]

Russ does seem a little more OCD when it comes to his replies ;-)

Odd, that's exactly what my wife says, and I've never been able to figure out why.

 

Glad to have been able to help.

 

Russ

[alecio]

Thanks again for your reply.

I've not been able to continue with the backup "project" as I am busy with some issues that are taking all my time.

I'm aware of the backup needing to be done and have planned it for this weekend as I want to move to version 6.3 of MAC OS.

I have bought an additional backup drive to keep the original one as a spare for some time until the dust has settled but from what I understand of the reactions “Retrospect only backs up what needs to be backed upâ€, I think that when I deleted the physical backup files, I may have damaged the media set. (I haven't read up yet on the link by Russ)

I will be looking into your suggestion. I started with printing and browsing through the Retrospect 8 manual.

I will go through all the info supplied by you and the other forum member who were as kind as to take their time to direct me into the right direction.

Regards,

 

Alecio

[alecio]

Thank you so much for your reply.

 

Like you described, that's the way I have been doing my backups by overwriting an older backup set. I had one directory where I kept my full backup and another holding my incremental backups.

Every week I was doing a full backup in the weekend and incrementals from Monday to Friday.

I thought that this would be the same case with Retrospect 8 and created both a Full backup and Incremental backup media sets, to work in the same way.

Not knowing the product and missing an adequate manual, I followed my old habits.

Unfortunately This is not the way to go and I'll go through the indications of Russ and have the manual handy for additional information.

It is good to know about the Matching as explained in your reaction.

I will surely look into how this would be an option in this new backup paradigm.

Regards,

 

Alecio

[alecio]

Hello Russ,

 

Thanks a million for your input to this thread.

I was totally lost and your explanation shed some light on the issue.

 

After reading your explanation referred to in the link you provided, I now can see what you mean by backing up modified files even though I haven't read up on the manual.

 

I had backup's of over 160GB at one time, not being the first full backup.

This I take was that I made some change in the .vdmk file which has a length of 99.8GB and that together with other files did cause the backup to be that big.

There were also other backups before and after that only backup up files for a total length between 12GB and 40GB.

This is what threw me off, not knowing how the backup worked.

I have deleted backup (physical files and backup activities) that didn't complete.

Maybe not immediately after the backup failed but possibly after a new backup session.

Will this not be a problem, as long as I have the original (first) backup of the media set?

I imagine that if the file existed in the original Backup session, the file changed, it will be different and as such it will be backed up.

I assume that the same counts if the file was non existent in the beginning.

 

I however have one thing in your explanation I cannot understand which is:

 

I suspect that each backup has been a "Normal" backup, and that Retrospect has done the "right thing". Again, read the explanation at the link above regarding the illusion that Retrospect presents with its "snapshot" paradigm.

 

[color:red]It won't work if you try to use the model presented with other backup programs of "full" and "incremental" backups.[/color]

Can you explain what you mean with the text in red above?

 

To be more specific, the activities were created by clicking on the backup icon and making the proper modifications by going through the tabs needing to be altered.

 

I will read up on the manual to get a better understanding and thank you for being there for those who need help and turn to to a forum to get a helping hand to solve their problems.

 

I'm really impressed with the response of other fellow forum members to help, as I've been on forums where you never get an answer to your problem even though the answer may seem obvious but you don't see it.

Regards,

 

Alecio

[rhwalker]

What I meant is that, unless you get the right model in your mind about how Retrospect works, you will never get it to do what you expect.

 

If you keep trying to understand Retrospect from the point of view of "initial full backup" followed by a series of "incrementals", it will never do what you expect, and you will always be fighting the program.

 

Retrospect presents the illusion of making a complete backup of your source at every backup session. In reality, it only backs up the files that have changed, and it remembers where the unchanged files are in its database.

 

Retrospect's media set (what the backup set is now called, using the new terminology - grumble) should be treated as a magical black box. It's a database. The "snapshot" interface shows you what was on the source at the time of the backup. Under the hood, as explained in the link above, the snapshot really has pointers into the database (media set) telling Retrospect where the files were backed up in the long backup history of past sessions.

 

If you try to use Retrospect like other backup programs with which you may be familiar, you will never be happy, and it will never do what you expect.

 

Let your mind go. Adopt the new paradigm. You will see the benefit when you need to restore, because you will have the illusion of the "snapshot", and will view the source as it appeared at the time of each backup session, through a succession of "snapshot" illusions. You will come to realize that it's the natural, and only, way to do backups.

 

Understand also that some of your (and everyone else's) difficulty right now is caused by the bugs in the program during the infancy of this new code base. It's taking baby steps right now, and it falls down a lot.

 

Clearer?

 

Russ

[Tree]

Retrospect's paradigm isn't all that much different from the "full" vs. "incremental" dichotomy. In other programs, you order it to perform one or the other; in RS8 you just let the program decide. Actually, I think it can be said that RS8 is always doing "incrementals", it just so happens that the first time a script runs there's no data already present and so it must capture the equivalent of a "full" backup. If you tell RS8 to "recycle" a media set, then it's basically telling it to zero out any data it already has and start from scratch, thus doing another "incremental" that looks like a "full" backup. So, one still does have a way to force RS8 to do a "full" backup, but only at the expense of any prior backup sessions, which is might not be what one wants.

 

As Russ said, the snapshot paradigm does the magic of making all these "incremental" backups look and act like "full" backups to the user. So it's best to just let RS8 do its thing (ie. always incremental) and should you need to do a full restore then any one snapshot from the entire backup history will suffice. You should only need to "recycle" (ie. force the equivalent of a "full" backup) in rare circumstances, understanding that it wipes out the previous sessions.

[rhwalker]

Retrospect's paradigm isn't all that much different from the "full" vs. "incremental" dichotomy.

I think there is a difference, and the difference matters (both in time and results) when you do the restore.

 

With other programs (at least the Unix programs I have used), it can be difficult to do the restore because you either have to know which specific incremental session backed up the file version that you want, or else you have to restore the "full" backup and then all incrementals forward from that point, so that versions of the desired file(s) keep getting overwritten, leaving you with the desired file(s).

 

This process of restore "full" then all incrementals forward from that point can leave you with flotsam and jetsam (transient files that became deleted at some point, but then were restored from some incremental along the way). Because of Retrospect's "snapshot" paradigm, only the files (and directories / folders) that were present at the time of the restored snapshot get restored.

 

Additionally, at least in theory, Retrospect's restore process can be faster than restoring the first full backup and then all incrementals forward because Retrospect knows where each file in the snapshot is located, and only has to retrieve that specific file. However, Retrospect 8 has a bit of code tuning that needs to be done before it begins to match the speed of other backup programs.

 

Russ

[davie_chen]

Respond to Tree:

it is not necessary to recycle the media set to do a full backup. We can just disable matching option in script->option

[alecio]

Finally did a backup after running recycle and getting rid of all the old backups.

It backed-up all files with the exception of the caches as requested.

I take that now I will be OK with my backups.

I however have on issue still that I have on a daily basis.

This is that I receive on a daily basis a warning from Retrospect which reads as follows:

[color:red]"You have not been backed up since Sat, August 15, 2009. Contact your backup administrator for more information."[/color]

 

As mentioned I schedule the backup and execute it immediately.

How do I get rid of this message?

 

Thanks,

 

 

Alecio

 

 

[Maser]

You can turn that off on your client application. I usually do.

 

I think if you want to reset it, you can toggle the Notification off and back on.

 

 

I've only seen this not respect the settings on Windows client and for those, I either turn off the Notification, or I uninstall/reinstall the client to fix it. I've not seen that be broken on a Mac client before...