Are File Backup Sets Risky????

[lmiller]

Hi folks,

 

I used to back up to "Disk backup sets" with multiple files. A coworker convinced me to go with "File Backup Sets", and I haven't had any problems yet, but it seems so risky to have only one huge file (35 GB) as my backup. Anyone have any insights on this?

 

I am using Windows 2000 (Retrospect 6.5), backing up to a neighboring computer, with external hard drives as off-site backups.

 

zostera

[natew]

Hi

 

As long as the disk has no errors either one is fine. But we all know disks can fail.

 

I agree with you. Many smaller files is better. My tests have also shown it is a bit faster too.

 

Thanks

Nate

[lmiller]

Thanks for the reply Nate!

[kaikow]

Quote:

---

Hi

 

As long as the disk has no errors either one is fine. But we all know disks can fail.

 

I agree with you. Many smaller files is better. My tests have also shown it is a bit faster too.

 

Thanks

Nate

 

---

 

Me thinks the real question os recovery of a damaged backup set.

 

With a disk backup set, the catalog is not stored as part of the backup set, so it is likely easier to recover if:

 

1. THe cartalog gets clobbered, in which case one just reconstructs the catalog from the backup set.

 

2. The backup set gets partially munged. In that case, I would ASSuME that the separate catalog could be used to fix the backup set, at worst losing only the damaged snapshots.

 

With a file backup set, as I understand it, the catalog is stored as part of the backup set, so recovery just has to be less robust.

 

Did me thinks right?

[awnews]

I also think that recovery & fault tolerance of a damaged File or Disk backup set is largely up to Dantz (i.e. how robust they made the formats). If a File or Disk backup set has some redundancy or error-recovery features, that could be useful (detection and/or correction). If Retrospect can note that a particular backup file in a multi-file Disk backup set is bad *but* at least recover all the user files in the *remaining* backup files that might say that a Disk backup set is more tolerant than a File backup set. Can Retrospect scan past a damaged section of a File or Disk backup set and recover files that aren't in the damaged area? (I've, unfortunately, used backup programs that couldn't do this--as soon as *one* backup file was bad, the program would complain that it couldn't recover *anything* or complain that it couldn't recover anything from that point on).

 

On the the issue of a damaged Catalog for a File or Disk backup, I thought that Retrospect could repair or regen a new Catalog from the files in the backup set (? Tools).

[natew]

Hi

 

With a Disk backup set Retrospect can see around damaged or missing backup segments and restore/rebuild catalogs from the rest of the segments.

 

Depending on the severity of the corruption in an individual segment it may be partially readable.

 

File backup sets are not as resillient in this regard. If a portion of a file backup set gets damaged you will be able to restore files that are not in the damaged area. As long as the catalog portion of the set still has accurate information about the set contents you should be able to restore "around" the damaged files. Unfortunately its a lot of trial and error figuring out which files can be restored and which cannot.

If the catalog were also to be corrupt you could run a rebuild. However the rebuild is likely to stop when you hit the corrupted part of the file. Anything beyond that will not be restoreable.

 

 

 

Thanks

Nate

[kaikow]

Quote:

---

I also think that recovery & fault tolerance of a damaged File or Disk backup set is largely up to Dantz (i.e. how robust they made the formats). If a File or Disk backup set has some redundancy or error-recovery features, that could be useful (detection and/or correction). If Retrospect can note that a particular backup file in a multi-file Disk backup set is bad *but* at least recover all the user files in the *remaining* backup files that might say that a Disk backup set is more tolerant than a File backup set. Can Retrospect scan past a damaged section of a File or Disk backup set and recover files that aren't in the damaged area? (I've, unfortunately, used backup programs that couldn't do this--as soon as *one* backup file was bad, the program would complain that it couldn't recover *anything* or complain that it couldn't recover anything from that point on).

 

On the the issue of a damaged Catalog for a File or Disk backup, I thought that Retrospect could repair or regen a new Catalog from the files in the backup set (? Tools).

 

---

 

No backup program is going to have redundant data to recover corrupted portions of the backup set.

 

Best one can hope for are checksums. etc., that allow identification of the bad spots and recovery of the rest.

[awnews]

Quote:

---

No backup program is going to have redundant data to recover corrupted portions of the backup set.

 

Best one can hope for are checksums. etc., that allow identification of the bad spots and recovery of the rest.

 

---

 

Even if "no backup program is going to have redundant data" currently, that doesn't mean that one *couldn't."

 

Simple checking (e.g. CRC/checksums) is *one* thing that can be done. The problem with this is that all it can do is tell you that something is wrong and that you're SOL.

 

On the other hand, a backup program *could* build "redundancy" in the form of correction capabilities into the file backup structures. This doesn't mean that it has to provide complete redundant copies of the data (e.g. 2x the space) but rather enough correction-oriented redundancy to work around a certain amount of corruption. Examples of some techniques in current use would be PAR for file corruption and ECC for memory corruption.

 

I know people who bundle up their backup data as RAR + PAR files just in case the backup media (magnetic media or even CDRs and DVRs, with long-term ink issues) has problems down the road. In my case, mainly because it's easier, I just maintain multiple full backup sets on different media.

 

So "doesn't correct errors" is very different that "**couldn't** correct errors."

 

In an unrelated way, in the case of Retro's Disk Backups, just having the ability to mark a *single* file in the file set as bad *but* recover all the data in the remaining files is a good thing--you've lost some stuff but not everything. But this is not a form of redundancy.

[kaikow]

Quote:

---

Quote:

---

No backup program is going to have redundant data to recover corrupted portions of the backup set.

 

Best one can hope for are checksums. etc., that allow identification of the bad spots and recovery of the rest.

 

---

 

Even if "no backup program is going to have redundant data" currently, that doesn't mean that one *couldn't."

 

Simple checking (e.g. CRC/checksums) is *one* thing that can be done. The problem with this is that all it can do is tell you that something is wrong and that you're SOL.

 

On the other hand, a backup program *could* build "redundancy" in the form of correction capabilities into the file backup structures. This doesn't mean that it has to provide complete redundant copies of the data (e.g. 2x the space) but rather enough correction-oriented redundancy to work around a certain amount of corruption. Examples of some techniques in current use would be PAR for file corruption and ECC for memory corruption.

 

I know people who bundle up their backup data as RAR + PAR files just in case the backup media (magnetic media or even CDRs and DVRs, with long-term ink issues) has problems down the road. In my case, mainly because it's easier, I just maintain multiple full backup sets on different media.

 

So "doesn't correct errors" is very different that "**couldn't** correct errors."

 

In an unrelated way, in the case of Retro's Disk Backups, just having the ability to mark a *single* file in the file set as bad *but* recover all the data in the remaining files is a good thing--you've lost some stuff but not everything. But this is not a form of redundancy.

 

---

 

A backup program will not have a copy of data. Best it will do is have checks on its own structures and a means to salvage the unbroken parts of a catalog or backup set.

 

One could always create extra backup sets, and one can use RAID to copy things.

[awnews]

Quote:

---

 

On the other hand, a backup program *could* build "redundancy" in the form of correction capabilities into the file backup structures. This doesn't mean that it has to provide complete redundant copies of the data (e.g. 2x the space) but rather enough correction-oriented redundancy to work around a certain amount of corruption.

 

 

---

A backup program will not have a copy of data. Best it will do is have checks on its own structures and a means to salvage the unbroken parts of a catalog or backup set.

 

 

---

 

I have no idea what you meant by that. I *explicitly* stated that I wouldn't expect a backup file to include a "copy of data." I'm pointing to other file structures & methods (PAR is a great example, http://www.slyck.com/ng.php?page=6, especially given the "multiple small files" created by Disk Backups. Imagine generating N PAR files at the end of Disk Backup set that could recover M missing or corrupted Disk Backup files) which include detection *and* correction methods to recover lost files/sections of files. The "checks" you mention will only tell you something is wrong, and a "salvage" will generally only rebuild a lost catalog, not recover a damaged or missing file (user files or backup set files).