Splitting a backup into multiple jobs.

[misleb]

Hi,

 

I'm currently trying to backup about 400GB of data from an OS X server which contains user home directories. As you may know, OS X creates a LOT of files and folders for each user by default. I have about 700 users and nearly 2.5 million total files. Retrospect simply cannot backup that many files without getting "error -625 (not enough memory)" . My backup server (Windows 20003 32bit) has 4GB of RAM. I believe the error actually occurs while it is building the snapshot.... or whatever it does after it is done copying the the files.

 

Anyway, I've given up on trying to get the full backup to work. I've tweaked memory settings and other things. Retrospect (7.5) just won't do that many files. Right now I want to try to split it into two jobs. Maybe users A-M and users N-Z. I wish there was a different structure, but the reality is tht all these users are in a single directory so I can't just make a couple subvolumes and be done with it.

 

Is there any way to group the user directories in a script? I don't want to have to manually maintain a list of directories to backup every time a user is added or removed. As far as I know the backup "Include" selection selects files for the whole tree. So I can just say bakcup all files and foldera that start with A through M because that rule would apply to ALL files and folders.. not j ust the first level.

 

-matthew

[Lennart_T]

The OS X itself has many files, too.

Perhaps you can backup the entire server EXCEPT the "Users" folder in one job and the "Users" folder in another job? It may not cut the number of files in half, but I'm sure it will help.

[misleb]

I am just backing up the Users folder. That is where all 2.5+ million files are.

 

Anyway, I ended up writing some scripts to move all user directories into directories called A-J and K-Z. This meant making mass LDAP updates as well as doing a string substitution on .plist files to make sure user preferences weren't screwed up (yes, OSX apps often use absolute paths for user preferences, grrrrr!)

 

-matthew