Supervising Proactive Backup

[pjonesvcc]

User manual in hand, I've digested a large number of forum posts here, and have jumped into Retrospect 8 with both feet again.

 

I've got 6 active scripts (5 proactive for my clients and 1 scheduled backup script to self-backup the R8 server), 6 media sets (one for each script), and 119 clients.

 

Here's the problem. Some of my clients have not yet been fully backed up, even though they were added/configured last week, can be browsed from the client list on the Retrospect console, and are visible on the network for large chunks at a time.

 

When I look in Activities, filter for All, and sort by Source, I've got at least half a dozen of these users where one backup attempt has been made. That attempt was interrupted part way through (these are all laptop users, and they likely put their computers to sleep mid-backup). The only other activity listed for that user is a Proactive Backup scheduled in the past, whose status is "Waiting to retry failed attempt".

 

The User Manual indicates that the least recently backed up client is prioritized to the top of the Proactive Backup queue, but I am seeing never-backed-up clients available and not being backed up. And, the server seems to go periods of time without even firing off backups on anyone - in need of incremental or original backup.

 

So, how do I supervise this process? The logic appears to be completely hidden. The only window into the world of proactive backups appears to be when it actually fires off a backup. But, if it is not, how can I tell what it is planning/trying to do, and where it is stuck?

 

How do I manage the proactive backups? How do I get it to trigger a backup on an available client in need of a backup? How do I get it unstuck?

 

Engine is on: Mac Mini Server 2.53 GHz Intel Core 2 Duo w/ 4 GB RAM on Mac OSX 10.6.2.

 

Console: Retrospect 8.1.626 Console running on my MacBook (2 GHz w/ 4 GB RAM) on MacOSX 10.5.8.

 

Clients: Versions are all over the map on machines from G5 iMacs to MacBooks to MacBook Pros, from 10.3 to 10.6. Mostly wireless laptops.

[bioheath]

Are the machines that aren't backing up all using an older version of the client? Are any of them failing with the newest version installed?

 

I haven't seen any problems with proactive backups here, but I run the newest version of the client on all my Macs/PCs. My only old client is on a linux machine (newest client offered, but still old) and it has all sorts of strange issues backing up. The files get backed up, but Retrospect lists it as "Never Backed Up," and always has.

[Maser]

Try this -- for the clients that have not been fully backed up -- but are on-line...

 

Confirm you can see the clients by going to Sources and hitting "Refresh" for them -- the information should update.

 

You indicate your clients are primarily "wireless laptops" -- does that mean they are getting different IP addresses each time? How did you add the clients to the engine?

 

I think people have had fair-to-poor luck trying to have clients stay in the system when they keep getting different IP addresses with 8.1 (no idea if this will improve in the next bug-fix release...)

 

But, assuming you *can* "Refresh" the clients and get new information...

 

 

Go to Activities and delete the *last activity* for one of the clients that didn't finish backing up all the way.

 

This will mark the client in the proactive list as being needed to be backed up "ASAP" and -- assuming their is enough RAM available to fire off another thread -- it should get backed up sooner than later. For me (with 8 proactive scripts running, but only about 40 clients), that takes about 10 minutes for that client to kick in it's backup.

 

 

But I would suspect that maybe your clients are pulling different IP addresses from when you added them to the system and the "refresh" is going to fail accordingly and that would explain why the clients aren't being seen by the proactive script.

 

 

But, then again, I only ever set my clients up with *static* IP addresses, so I can only comment on what does/doesn't work with Proactive backups that way.

 

 

[Maser]

Oh, and if you *do* want to see the logic, you can set up logging for proactive by:

 

1) Stop the engine.

 

2) With Terminal, do this command: "touch proactive_backup_log.utx" in the directory /library/application support/retrospect

 

3) Turn engine back on.

 

4) That log file should start filling after you do this.

 

 

However, this log is filled with lots of gobble-de-gook. I've had to create these logs for bug reports and they make zero sense to me when trying to decipher them.

[pjonesvcc]

Thanks for the tip, Maser. Most of my affected users have gone home for today, but I deleted the most recent past activity for one of them, and within 5 minutes the server was running a backup on her computer. So, there seems to be some kind of hiccup on interrupted backups that is preventing future activity. I'll have to try that on the others (as well as grab the version number on each of them to see if that's it).

 

Question about removing/deleting past activities from the Activities area. Does this affect the backups/catalogs in any way, or are we just removing what is essentially a log entry?

 

Oh, and btw, all the employee computers are on static IPs.

 

I will have to eventually get around to updating all the clients, as Heath suggests. Due to time constraints, I've been only updating the ones that had issues during configuration (add as Source, define Favorite Folder).

[Maser]

When you remove past activities -- that's just removing something from the *console* log (actually from the "config80.dat" file) -- not from the actual "operations_log.utx" file. This has no effect on the backups/catalogs.

 

But for proactive backups (assuming you run the default daily backups), if you remove the most *recent* activity, then the client doesn't think it was backed up in the last 24 hours, so it marks it internally as "ASAP". I find this frequently works better to fire off a client's backup *right now* than doing a "schedule" and setting the time to the current time..

 

 

As far as updating the clients -- you should be able to "push" a client update through Sources. I've done this to update all of my on-line clients at once (well, at least the *mac* clients -- I don't think I did the PC clients this way the last time...)

 

I recommend, though, that you pause/stop all active scripts (proactive or not) before doing this.

[sussox]

But for proactive backups (assuming you run the default daily backups), if you remove the most *recent* activity, then the client doesn't think it was backed up in the last 24 hours, so it marks it internally as "ASAP". I find this frequently works better to fire off a client's backup *right now* than doing a "schedule" and setting the time to the current time..

 

One of those things that should have been in the manual...

[pjonesvcc]

As far as updating the clients -- you should be able to "push" a client update through Sources. I've done this to update all of my on-line clients at once (well, at least the *mac* clients -- I don't think I did the PC clients this way the last time...)

 

I recommend, though, that you pause/stop all active scripts (proactive or not) before doing this.

 

I'll give that a shot. Back in R6, I tried updating clients from the server once. All of the server-updated clients stopped launching on bootup, and I just went back to updating ones that experienced issues.

Question: Is there a way to see Sources' client versions from the console?

 

Nm, Answered my own question. I see that I can add a column to the Sources section that will list the agent version. Handy.

 

Also let's me answer Heath's question. Most of the clients experiencing Proactive Backup issues are 6.2.234, but two are 6.3.019 and one is 6.3.027. Which means that all are out of date. Current is 6.3.028.

[Maser]

This is unlikely a client issue -- but more likely an issue with Proactive logic -- or more likely proactive *response*.

 

I was told once that having more proactive scripts makes the logic perform harder. I once tried 40 proactive scripts (one for each client) and I can testify to the fact that I brought the engine to it's knees when I did that.

 

There's some sweet-spot for how many activities you can run with how much RAM you have -- and that affect how the Proactive logic works.

 

It's best (?) to not think of this in Retrospect 6 terms where you could set a "Backup Server" client to "ASAP" and it would back up ASAP.

 

With threaded activities, the engine has to do a lot of work to determine what can be run in the available memory, etc...

 

Not to say EMC can't make this better, but I think this is a design issue hitting free RAM limitations.

[pjonesvcc]

Your feedback in earlier threads with regards to the number of Proactive scripts was very valuable to me. I had also brought my engine to its knees and determined that R8 was useless in its current form. Once I reduced the number of scripts and media sets, however, it has been (mostly) functional and responsive (with a daily reboot).

 

I'm in agreement that this is not likely a client issue, but if updating is straightfoward, then it is likely a good use of time to get it done. If nothing else, it will simplify further troubleshooting.

 

I would love to have some insight into the RAM limitations. If I am running with 4 GB on the engine, how many concurrent threads are recommended? How many Proactive scripts? etc. Have you seen EMC weigh in on that?

 

Right now I have preferences set to 8 threads, although with 6 active scripts, that won't happen. I might just bump the pref down, though, in case any resources are being pre-allocated...

[Maser]

I don't have this handy (it's in some other forum), but I know with 4G of RAM (and the console not running), I can usually get 4-5 proactive activities running concurrently.

 

I think with 2G RAM, you only get 2, but it's not directly a 1G-1 activity correlation.

 

The default "8 threads" -- won't actually run 8 threads. It's limited to how many threads can *really* run by how much free RAM is available.

[tgfn]

a late and great thank you for your work.

deleting the last activity for a given proactive script not showing up in the "running activities" tab actually gets it running again.

i wonder, whether a normal (formally "immediate") backup would do the same thing