Jump to content

Splitting a backup into multiple jobs.

Recommended Posts



I'm currently trying to backup about 400GB of data from an OS X server which contains user home directories. As you may know, OS X creates a LOT of files and folders for each user by default. I have about 700 users and nearly 2.5 million total files. Retrospect simply cannot backup that many files without getting "error -625 (not enough memory)" . My backup server (Windows 20003 32bit) has 4GB of RAM. I believe the error actually occurs while it is building the snapshot.... or whatever it does after it is done copying the the files.


Anyway, I've given up on trying to get the full backup to work. I've tweaked memory settings and other things. Retrospect (7.5) just won't do that many files. Right now I want to try to split it into two jobs. Maybe users A-M and users N-Z. I wish there was a different structure, but the reality is tht all these users are in a single directory so I can't just make a couple subvolumes and be done with it.


Is there any way to group the user directories in a script? I don't want to have to manually maintain a list of directories to backup every time a user is added or removed. As far as I know the backup "Include" selection selects files for the whole tree. So I can just say bakcup all files and foldera that start with A through M because that rule would apply to ALL files and folders.. not j ust the first level.



Link to comment
Share on other sites

I am just backing up the Users folder. That is where all 2.5+ million files are.


Anyway, I ended up writing some scripts to move all user directories into directories called A-J and K-Z. This meant making mass LDAP updates as well as doing a string substitution on .plist files to make sure user preferences weren't screwed up (yes, OSX apps often use absolute paths for user preferences, grrrrr!)



Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Create New...