Performance on gigabit

[jaduke]

I've got Retro 8 running on a Mac Mini (newest model w/4 Gig RAM) backing up to a external RAID 5 box (LaCie Biggest Quadra) via FW800.

 

Connection to the network for all systems is gigabit.

 

Software data compression is enabled in the disk backup set.

 

I'm seeing totally abysmal performance on the two backups I've had running since last Thursday (3/26). I'm only trying to backup about 500 gigs of data between the two servers and with 6.1 I would have been done with both backups by now.

 

Performance I'm seeing:

File-2 MB/min this AM; 9 MB/min now

~300 gigabytes of data

EMail 5 MB/min this AM; 10MB/min now

~175 gigabytes of data

 

I saw, for a moment, a glimmer of hope with 50mb/min but that went away almost as quickly as it appeared.

 

I've got my server limited to 4 thread per the knowledgebase article, but should that be lower?

 

Is Retro 8 the bottleneck here? I've got 6.1 loaded on the same machine, but haven't had a chance to test its performance compared to 8 on like hardware.

 

Thanks.

 

Cheers,

Jon

[Mayoff]

We know that Retrospect 8 is not giving the best performance for client backups. I am already running an internal version designed to boost backup speeds.

 

How are the local backup speeds for you? This should be fast and will help identify bottlenecks.

 

With 2 threads I have seen client speeds in the 300 to 400 MB/Min range.

 

Do you use compression or encryption?

[jaduke]

We know that Retrospect 8 is not giving the best performance for client backups. I am already running an internal version designed to boost backup speeds.

 

How are the local backup speeds for you? This should be fast and will help identify bottlenecks.

 

With 2 threads I have seen client speeds in the 300 to 400 MB/Min range.

 

Do you use compression or encryption?

 

I just tested local backup, 167 MB/min to a disk backup set with software compression. This is while the two server backups are running.

 

I am using software compression for the two server backups.

 

Thanks.

 

Cheers,

Jon

[jaduke]

Robin,

 

One more question-is v8 doing the comparison at the same time as the backup or is that still a separate step like in 6?

 

Thanks.

 

Cheers,

Jon

 

[Mayoff]

You get 2 choices. The old style verify or Media Verification.

 

We write MD5 data to all backup media by default. So you can turn verify off and schedule a verify for later. it will then check the MD5 hash.

 

The Retrospect 7.5 user's guide has good info on the 2 options. The 8.0 Guide is still in progress.

[joelande]

Performance is atrocious.

 

I have been trying to compare v8 to BRU.

 

However the backup that I started on Saturday hasn't completed yet (it is now Tuesday).

 

I can tell you the same volume took seven hours and six minutes to backup under BRU.

 

 

[Mayoff]

Our engineers are working hard to improve backup speeds. Internally I used a version today with client backup speeds that are twice as fast as the release version. Compare speeds seemed to be even faster.

[joelande]

Great to hear! That can't come soon enough!

[jaduke]

Our engineers are working hard to improve backup speeds. Internally I used a version today with client backup speeds that are twice as fast as the release version. Compare speeds seemed to be even faster.

 

I'll say this, the speeds from my servers on gig are terrible, but the 2006 MacBook client shows 638MB/min after the script finished. It's only got a 100 megabit connection.

 

Why aren't the servers hitting anywhere near that mark?

 

Cheers,

Jon

[joelande]

I am wondering - are your servers backing up User Home Directory folders? If they are (like mine) that is lots and lots of small tiny files (many of them useless), which will really slow down a backup compared to your laptop that probably only has one or two user folders on it.

 

That might count for the slowdown/difference to servers.

 

I have the Rule - no cache - chosen, but I could eliminate the entire Users/*/Library folder except for the FIrefox and Safari bookmark documents and be happy, I don't need any of that other crap in my backups. It doesn't appear there is an easy way to add a rule for that type of filter however - no wildcard support.

 

Here is what my filter on BRU looks like:

# This file is used by BRU-Server to provide an inclusion/exclusion

# list. For each pathname of a file selected for backup, each line of

# this file is examined for a pattern, and that pattern is applied to

# the pathname. If the pattern matches, the appropriate action is

# taken (the pathname is accepted or rejected). If the pathname makes

# it through all the patterns it is accepted.

 

# Each command line in the bruxpat file (the file you are now reading)

# consists of a control field and a pattern. The pattern

# is separated from the control field by whitespace. Control field

# characters are:

#

# i Include this pathname if pattern matches. The

# pathname is accepted and no further patterns are

# applied.

# *** NOTE ****

# It stops trying on the first pattern match found

# and passes the filename. Since it scans patterns

# in the order listed, "include" patterns normally

# should be listed before any "exclude" patterns.

#

# x Exclude this pathname if pattern matches. The

# pathname is rejected and no further patterns are

# applied.

#

# z Exclude this pathname from compression if pattern

# matches (if the -Z option is specified).

#

# s The pattern is a shell style wildcard pattern except

# that '/' characters are not treated as special characters.

#

# r The pattern is a regular expression (same as used by the "grep"

# command).

#

# l The pattern is a literal string.

#

 

# Exclude all core files

xs */core

xs core

 

# Exclude all files and subdirectories in the temporary directories.

# Handle files specified with relative and absolute pathnames

#

# -- NOTE -- the actual directory names will still be backed up,

# only the files within the directories will be

# excluded.

xs */usr/tmp/*

xs /usr/tmp/*

xs */tmp/*

xs /tmp/*

xs */dev/fd/*

xs /dev/fd/*

xs /.Trash/*

xs */.Trash/*

xs /.Trashes/*

xs */.Trashes/*

xs */*.cache

xs */Caches/*

xs */Cache/*

xs /Network/*

xs */Network/*

xs /automount/*

xs */automount/*

xs /var/vm /*

xs */var/vm/*

xs /var/run/*

xs */var/run/*

xs /proc/*

xs */proc/*

xs */.ipspot_update

 

# Don't compress files that end in ".z" or ".Z"

zs *.[Zz]

zs *.gz

zs *.tgz

 

# This line excludes the /Network hierarchy from being backed up on

# OSX machines. If you're looking for a way to back up a remote OSX

# machine, use a Bru-Server agent. The OSX automounter does not

# work the same as most unix automounters, and you will get

# permission errors that will cause the backup to fail.

 

xs /Network/*

Edited April 2, 2009 by Guest

[Mayoff]

The server OS may have a lot more small files then a workstation OS (which also has lots of small files). Our performance changes coming soon really shine for small files. The changes will be very helpful