How long should a network backup take?

[kimo]

I know this appears to be "how long is a piece of string?" but I would be interested to see some sort of indication on performance and times to see if I have got the configuration right.

 

My setup has 5 PC's networked with a 100MB switch on a stand-alone LAN. They all are running Win 2K Pro, except for a new laptop testing out XP Pro. I have an HP Datastor DLT70 tape drive on one machine. I do not have an machine designated as a server because the GIS software I use does not support shared files effectively. Hence each machine has large disks (7200 rpm) of 400 GB running in Level 0 RAID arrays for speed. So it is important to have a backup in case a disk fails.

 

I did have a motherboard failure recently and the restore worked extremely well, - the trouble is the backups take a very long time, even the incremental backups because the files are easily changed.

 

I am just looking gloomily at the prediction that the backup of 100 GB of changes will take 6 days at 9.6MB/min! Is this reasonable? Surely not! There must be something that I can do to improve on this time?

 

I do notice that the data is not streaming properly, the tape drive starts and stops, so the data is not being passed across to keep the tape rolling. I therefore conclude that it is my network, or the client software that is the limiting factor, not the hardware, which I admit is obsolete, but I have a lot of tapes that cost a lot of money that would be nice to use.

 

For my smaller machine with only 20GB of disk, and email, the backup runs in a short time, and it all runs very satisfactorily. I do not notice much improvement when backing up the machine with the tape drive attached, but the tape does not stop and start so much.

 

Possibilities that I have considered:

faulty network card causing bottlenecks

missmatched packet sizes, not optimised for a LAN

defragged disks

hardware compression is turned on

[kimo]

I can answer my own question now.

There is a whole chapter in the Version 7 User Guide (363 pages PDF) which

discusses network performance issues and has useful calculation examples.

 

I have started to fix a number of hardware issues, and also software.

 

1. Upgrade Retrospect to V 7 (better catalog indexing) My catalog is 300MB

The Dantz suggestion is to start again with a new backup set, but that will be

very slow still because of the size.

 

2. Defrag the source volume disk. I had found that defragging had been switched off

for an overnight process and not turned back on. The disk was so badly fragmented

that the built-in defragger would not fix it. I have had to install Diskeeper.

 

3. Become an Administrator. This means RTFM and generally become more expert in

managing backups, archive off more files, exclude temporary datasets and tune it.

There is a danger that this will make a restore more complex, and disaster recovery

will be impossible to just run, but few people just mirror back to a new disk, rather they

tend to have an image of the software, and restore the data.

 

4. Consider moving the tape drive to the most active machine.

 

5. tar and compress large groups of small files into a single file before backing up.

This might get up the network transfer speed. One trial of 10,000 files taking up 18 GB compressed down to a 2.7 GB single file. Can't use it in that form of course.

[kimo]

Defragging the data disks on one machine has made the most difference.

After archiving off quite a lot to leave 30% free space, the defragger eventually succeeded after a couple of days.

 

Backup speed is running at a pleasing 361 MB/min with 6 hours and 146 GB to go.

 

So now a fresh backup set is practical, and is only (!) taking 18 hours for starting a new set and I can recycle 12 tapes from the old set since I have archived on to DVD.

 

Even simple copies across the network using Explorer have returned to normal.

 

But have the DVD archives worked?? I tested out one and found - shock horror - I could not read the new DVD. There are some questions about how reliable the media is in the long term with dyes fading away prematurely, but I did not expect it to be faulty immediately. Verification is clearly needed, and even a trial restore to see if it really can read the data back before its lost from the tape recycling.