Jump to content

What's the best backup approach...

Recommended Posts

OK, I'm trying to figure out the most efficient backup approach, with specific regard to file matching, for this situation...


I'm backing up a machine with a designated backup folder on it, so any files within said folder get backed up on a nightly basis.


A user on this computer is working on a job with the following details (stored within the designated backup folder)...


JobCode12345_JobTitle (this is the main job folder name)

    - ArtworkFile_v1.indd

    - Image01.jpg

    - Image02.jpg

    - Image03.jpg


That night, the backups run and all of the above is backed up.


The next day, they're informed that the project name has changed, so they change the name of the main folder and the job now looks like this...


JobCode6789_NewJobTitle (this is the revised main job folder name)

    - ArtworkFile_v1.indd (NO CHANGE)

    - Image01.jpg (NO CHANGE)

    - Image02.jpg (NO CHANGE)

    - Image03.jpg (NO CHANGE)


The backup script has the following Matching settings...


• Match source files against the Media Set = CHECKED

• Don't add duplicate files to the Media Set = CHECKED

• Match only files in the same location/path = UNCHECKED


On the second night, the backup runs. Because of the above Matching settings on the script, none of the files get backed up again. But, since the parent folder has had a name change, that parent folder itself would/should (?) get backed up.


So, I attempt to pull JobCode6789_NewJobTitle from the backups. I go to Restore > Search for files in selected Media Sets and search for a folder with that job name. Nothing comes up.


Why does nothing come up?

I assume because only the parent folder name changed, and the contents didn't, for whatever reason, the folder doesn't get backed up.


So, I add a file to the folder (image04.jpg) and re-run the backup. Then, I search for JobCode6789_NewJobTitle again. I go to Restore > Search for files in selected Media Sets and search for a folder with that job name. This time, that folder appears, and within it, Retrospect just shows image04.jpg as the contents of the folder.


Maybe I'm missing something, but how do I retrieve the complete contents of the folder JobCode6789_NewJobTitle?




  • Like 1

Share this post

Link to post
Share on other sites

I don't think folders get backed up, per se. The (new) path to the (old) files are stored in the snapshot. So in the latest snapshot, you can restore the contents of JobCode6789_NewJobTitle (but not the (empty) folder itself). Just try a restore (of "selected files and folders") to an empty disk or USB memory and see. In fact, it doesn't have to be empty. You can restore to a new folder (at root level).


You should not use "search for files" to restore files. You would only find their first location. If the same file resides on more than one location on the same source volume, you would only get one of these locations. The one that Retrospect happened to find first. The snapshot then keeps track of all the folders that file resides in.

Share this post

Link to post
Share on other sites

That works if I need something recent, but what if you have no idea what snapshot the job you're looking for is on?


For example, say I get a request to pull a job from the archived backup tapes from 3 years ago, and it's not known what machine had the final version of the job on it. it's possible it was on one of 25+ design/production machines, 3 years ago. How would I find it?

Share this post

Link to post
Share on other sites

Also, I tried the Restore of selected files and folders. The initial "select a point-in-time" backup window only shows a random collection of restore points. I have no idea how Retrospect decides what to include in this list (do you)? It's a mash of random restore points of various different machines, going back as far as 2012 (when we moved to Retrospect 9). Nowhere near a full list of restore points. It doesn't even show all the restore points for all machines over the last 24 hours?


So, I then need to click the More Backups... button. Then I'm displayed with a (very) long list of restore points (for my person backup folder alone, I have over 1000 restore points, going back to when we moved to Retrospect 9 in 2012, so it's a looooooong list when you take into account the 30+ machines that get backed up nightly). Also, once I'm in this new window, there's no option to browse the restore points. The only option is to "Retrieve a backup". Clicking the Retrieve option seems to do two things...


1. It loads the selected Restore Point back into the previous, at which point I can browse it.


2. Back in the main Retrospect console window, under Activities, Retrospect is requesting that I load the media that contained the Restore Point (not sure what it's trying to restore, since I haven't actually requested any files be restored yet?).


And even then, the above is completely inconsistent. For example, once I've completed a go at the above, and realized the job I'm looking for isn't part of that restore point, I then need to click the More Backups... button again, since I need to look at another restore point. At this point, Retrospect completely dies. Sometimes it doesn't die, but it only lists a handful of restore points. Only way to get it to work is to do a complete reboot.


So, I'm at a loss?


If I shouldn't do a  "search for files/folders" to find the job I'm looking for, and I can't browse more than a single restore point (of 1000s) without Retrospect dying, I'm not sure what I'm supposed to do? Is there no way to search my backups?

Share this post

Link to post
Share on other sites

That is a completely different question than the one(s) in your original post. :) In that post you mentioned just "a machine", for instance.


If you know the name of one file with a unique name from that project, it would be easy to find. Then you could find which snapshot it was and restore the entire folder.


At the very least you should keep a list of what projects were finished when on which machine.

At best, you should use a versioning system. :) I can recommend http://www.perforce.com/


I assume you delete the projects from the machines ones they are done, right?

Share this post

Link to post
Share on other sites

I didn't see your post #4 before posting my #5. And now I can't seem to be able to quote your post(s).


Anyway, Retrospect need the media to read the snapshot. Keeping all the snapshots in the catalog file would make it extremely large.


You don't mention what kind of media you have, or why you have thousands of backups on the same media set. How many members are there? Remember that the loss of one member may affect many snapshots.

Share this post

Link to post
Share on other sites

Oh, I forgot.


How many files do you have in the media set?

Retrospect may die due to lack of (free) RAM if you have millions of files in the media set.

Share this post

Link to post
Share on other sites

One more thing:

Have Apple's Activity Monitor running (before) the next time Retrospect dies on you. Please post a screenshot of the memory pane from the moment Retrospect has just died, before you do ANYTHING else.

Share this post

Link to post
Share on other sites

We're using LTO4 tapes. And because Retrospect chokes easily, we never go beyond two members per media set (with two, overlapping/rotating media sets being used at any one time)—we rarely use the second member in a set—it's only used for a single night, once a backup tape fills mid-backup. It'll then grab a blank LTO4 tape from the second tape drive and finish that nights backup. At that point, the media set is retired and a new media set is started (and a bunch of other stuff is done to ensure all the files that are already on that media set don't get included in the next night's new media set...but I don't need to get into those details).


We have various protocols in place for proper file management, archiving, etc., and when a job is completed, it's archived to onsite and offsite media for long term storage. But as I'm sure you're well aware, you can have as many protocols and standards and systems in place that you want, but in the end, people don't always follow those protocols.  And that's when I get the request, because a protocol wasn't followed, or someone messed up in the archiving process, or just because people are lazy and it's "faster" to just ask me to pull from the backups than do the legwork themselves, etc.  ;)


For the most part, it's 99% of the time because someone moved the files to another machine, but didn't properly documents the movement in the file tracking system. And so, I just use Retrospect (via Restore > Search for files/folders) to search through the backups and see where the latest version of the job exists. And 99% of the time, that's all I need as I then know where the file is (and I can close Retrospect down).


Ironically, just as I was posting my last reply, I got a request for this exact situation. "Can you pull job 11600 from the backups? We can't find the files—they're supposed to be on Jack's machine, but they're not." Jack hasn't worked here in over half a year. So, here's what I ended up doing (spoiler alert: the job was moved to another machine without properly being documented)...


- I had a PDF of the final artwork for the job, so I knew the name of the InDesign file. All our jobs/files follow a naming convention of 123456_FileName_vX.indd (with vX changing each time a new version of the file is generated...so v1, v2, v3, v4, etc.—in the end, once the job is approved/completed, the vX becomes FA). So I could extract the name of the InDesign file that generated the PDF).


- I used the Restore > Search for Files option to search for the InDesign file name.


- Retrospect returned multiple backups of the file name (all the vXs and final FA) and from that, I could see what machine the FA existed on. I didn't need to use Retrospect any further since the job was sitting on a machine 5 stations away.


Luckily, it's a rare occasion I need to actually pull the job from the tapes (there's enough redundancy in place to not actually need to go to tape), but when I do, I forget how long in the tooth our Retrospect system is (compared to CrashPlan, which is what I use for everything else except these machines).


Anyway, you said..."If you know the name of one file with a unique name from that project, it would be easy to find. Then you could find which snapshot it was and restore the entire folder." How would you go about determining which snapshot the job existed on, post searching the file? What I do is look at the latest "modified" date (and computer name) in the search results and then go to the snapshot for that computer on that date. Is there a better way?

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now