Proactive backup 'checking source' polling time

[jef]

Hi all,

 

I seem to be having a couple of machines that are momentarily unavailable. They do however need to be backed up pretty urgently according to the proactive schedule.

 

The problem I'm having now is that these machines are on top of the ASAP list. Retrospect is trying to access them one by one, which takes a LONG time, before giving the 'source' status. This results in a long wait before reaching a machine that is 'available' for backup. And after that one backup it starts polling from the top of the list again, which would mean the same wait.

 

Any ideas?

 

Is there a possibility to adjust the timeout for trying to reach a client?

 

Thanks

[Lennart_T]

With 20-30 proactive clients, it should take no longer than a few minutes to check the list through. How long time are you talking about?

[jef]

At the moment, it's about 20 minutes for 20 unreachable destinations.

Which would mean 1 full minute to try and reach a machine that in most cases just isn't turned on.

[jef]

Apparently, you can change some parameters on the interface level. Including the timeout for connection to a client.

 

The minimum timeout is 30 seconds. Less would be better, but the program doesn't allow it.

 

What's the effect?

 

The 'source' status comes up faster now, which is good. More work is being done.

 

BUT

 

After EVERY succesfull job completion the proactive script is RESET, which makes the scanning start from the top again. This is no good, because the clients that need backup asap and are -available- hardly get reached. A complete traversal of the proactive job list is NEVER completed!

 

The proactive script options allow for a inter-source poll time to be set, but it doesn't seem to do what I thought it would. I assumed it would ignore the source for a certain stated time period when not reachable. Unfortunately it doesn't.

 

Is there a solution to this problem?

 

Perhaps a way to turn off the 'start from the top when job script has run'?

 

Thanks,

 

Jef

[mauricev]

You are right, proactive works only if you have clients that are ordinarily available only intermittently, in other words, mobile computers. I use it to backup mobile computers during the daytime and it works very nicely. That is what it's designed for. For regular desktop backups, I use ordinary scripts. It is an order of magnitude more efficient than proactive.

 

 

 

Quote:

---

ith 20-30 proactive clients, it should take no longer than a few minutes to check the list through. How long time are you talking about?

 

---

 

 

 

Say it's 25; 12 are off; 13 are on. That works out to 156 minutes just to reach every computer in the entire list excluding the time it takes to back up the 13 working computers. Of course, if all 25 are on, then it will be much more efficient.

[jcapslock]

I'm observing the same sorts of issues. It makes me wonder why it is recommended to use for all of your clients in the disk-to-disk-to-tape tutorial. That tutorial leads me to believe others are able to turn proactive on for all computers and then do a snapshot transfer to offline media at the end of the week. But I find it so slow that it wouldn't complete enough backups of all clients for the week, especially not servers. I think proactive needs to be reworked to take better advantage of the multithreading. I can never get more than 4 proactive scripts to go at a time before the first one finishes, where I can get 8 going with normal scripts. It seems to waste a lot of time waiting around. I think there should be a polling process for each execution unit so that more work can be done. If the server is sitting around waiting 60 seconds or more for a client to respond, another polling process should be looking for a client from another source group for one of the other execution units. I also agree that the timeout is painstakingly long. Especially since the nature of the proactive backups is that it rescans the machines that haven't been backed up the longest each time a backup is completed. There also should be a setting in the script to affect that. I'd rather just continue on down the line and finish the backups of the systems that are online and return back later. Or perhaps a value that puts a "wait n seconds before polling again timer" on each source found to be offline, which is what I thought "Check source every ..." option was for. It seems, instead, to behave as a value of time in between polling each source? Which adds even more time onto the scanning process.

[Lennart_T]

Quote:

---

Say it's 25; 12 are off; 13 are on. That works out to 156 minutes just to reach every computer in the entire list excluding the time it takes to back up the 13 working computers. Of course, if all 25 are on, then it will be much more efficient.

 

---

 

Let me see if I get this straight. You have 13 connected clients. All of them are now backed up and the proactive backup does nothing except polling the clients. If you connect a previously unconnected client, it takes Retrospect 156 minutes to find the new client and start the backup????

 

Here it takes about 10-15 seconds to poll each client and a minute or two between "polling sessions". So worst case it's about 5 minutes before the backup starts.

[natew]

Hi

 

Are these clients logged in by IP address or multicast? Multicast should pick up all the clients at once.

 

It sounds to me like there is some corruption in the proactive database. To clear it up, open the backup report under the reports menu and delete all report items. This will reset the proactive backup schedule completely and may help with the problems you are having. Just be aware that this will set all of your clients to ASAP status.

 

Thanks

Nate

[mauricev]

Quote:

---

Let me see if I get this straight. You have 13 connected clients. All of them are now backed up and the proactive backup does nothing except polling the clients. If you connect a previously unconnected client, it takes Retrospect 156 minutes to find the new client and start the backup????

 

---

 

 

 

I believe since the 13 have been backed up, it does just one sequential scan of the list , so it could take up to 14 minutes depending on where that new client appeared.

 

 

 

Quote:

---

Here it takes about 10-15 seconds to poll each client

 

---

 

 

 

Each client that is not connected?

 

 

 

I just watched R check a client that was not available. Took 53 seconds to give up.

 

 

 

Quote:

---

Are these clients logged in by IP address or multicast? Multicast should pick up all the clients at once

 

---

 

 

 

How would that impact checking an individual client in the proactive list?

 

 

 

Quote:

---

It sounds to me like there is some corruption in the proactive database. To

 

---

 

 

 

I'm confused. What's he describing sounds like normal Retrospect operation.

[mauricev]

Quote:

---

Here it takes about 10-15 seconds to poll each client

 

---

 

For a clients that have been backed up, regardless of whether they are available or not, the scanning time is about 22 seconds. The 53 above was for a client that was "ASAP".

 

I'm using proactive with default time settings. I don't think the times are impacted by machine or network speed unless they present a bottleneck , but rather these times are hard-coded into the program.

[Lennart_T]

Quote:

---

Quote:

---

Let me see if I get this straight. You have 13 connected clients. All of them are now backed up and the proactive backup does nothing except polling the clients. If you connect a previously unconnected client, it takes Retrospect 156 minutes to find the new client and start the backup????

 

---

 

I believe since the 13 have been backed up, it does just one sequential scan of the list , so it could take up to 14 minutes depending on where that new client appeared.

 

---

 

Now we have gone down from 156 minutes to 14 minutes. A considerable difference.

 

Quote:

---

Quote:

---

Here it takes about 10-15 seconds to poll each client

 

---

 

Each client that is

not

connected?

 

---

 

I just timed it. 13 clients NOT connected, 11 clients connected. It took 5 minutes, 40 seconds to come back to the same client again, after checking all others. 5*60+40=340 seconds for 24 clients is less than 15 seconds per client.

 

 

Quote:

---

I'm confused. What's he describing sounds like normal Retrospect operation.

 

---

 

Ditto.

[RonaldL]

How are the clients being added? Through multicast, subnet broadcasting or direct?

[mauricev]

Quote:

---

Now we have gone down from 156 minutes to 14 minutes. A considerable difference.

 

---

 

14 minutes applies when the connected clients have already been backed up. If they haven't the scanning time increases practically exponentially because it rescans the entire list after each client has been backed up.

 

Quote:

---

I just timed it. 13 clients NOT connected, 11 clients connected. It took 5 minutes, 40 seconds to come back to the same client again, after checking all others. 5*60+40=340 seconds for 24 clients is less than 15 seconds per client.

 

 

---

 

How can your Retrospect be so fast? It gives up very very easily.

 

Based on my previous timings with default settings, the numbers you (and everyone) should get are

 

13 clients NOT connected = 53 * 13 = 689 seconds.

11 clients connected = 11 * 22 = 242 seconds

 

689 + 242 = 931 seconds.

 

That's 15.5 minutes. So what's causing the discrepancy?

 

Does anyone else care to post timings?

 

Quote:

---

How are the clients being added? Through multicast, subnet broadcasting or direct?

 

 

---

 

What's that have to do with it?

[waltr]

Quote:

---

How are the clients being added? Through multicast, subnet broadcasting or direct?

 

 

---

 

What's that have to do with it?

 

---

 

let's say you had several subnets on your lan and added the client through subnet broadcasting. Retrospect caches the ip address of the client and if it can't find it there it will search through the subnets, one by one, until it finds the client. this can take time.

 

direct ip should not have this issue. you could see something similar through multicasting if you had more than one subnet though.

 

be nice maurice.

[mauricev]

Quote:

---

et's say you had several subnets on your lan and added the client through subnet broadcasting. Retrospect caches the ip address of the client and if it can't find it there it will search through the subnets, one by one, until it finds the client. this can take time.

 

direct ip should not have this issue. you could see something similar through multicasting if you had more than one subnet though.

 

 

---

 

This is very interesting. Most of my clients use multicasting, Retrospect's default method. So I tried a non-connected client needing backup that is accessed by direct IP and it took just 22 seconds, not 53 seconds! So it seems that Retrospect is doing some extra work in those 39 seconds.

 

Lennart, were all your clients in your test accessed by direct IP?

 

However, even though it would account for a good part of the difference, there is still my 22 seconds versus his 14 seconds. Seven seconds is a lot of time on a network.

[Lennart_T]

Quote:

---

Lennart, were all your clients in your test accessed by direct IP?

 

---

 

I don't think so. As far as I can remember, they were all added using multicast. There may be one or two added by direct IP because Windows XP SP 2 firewall caused some grief before we figured out how to deal with it.

[gvanden]

My servers spend 43 seconds "checking source" before determining that the client isn't there and moving on to the next one. Has anyone figured out how to speed this up? All the clients were added via multicast. I've played with the "check source every" polling option, but it doesn't seem to make a difference. The "search timeout" in the Advanced Interface Configuration is at the default 10 seconds. Thanks....

[Lennart_T]

I have now discovered that upgrading from version 6.5 to 7.0 made the polling time triple or quadruple from 15-20 secs to about a minute per client. (sigh).

[natew]

Hi

 

Is this occuring on the same machine that had slow closing times?

Does it also happen if you install Retrospect on another machine?

 

Thanks

Nate

[Lennart_T]

Quote:

---

Is this occuring on the same machine that had slow closing times?

Does it also happen if you install Retrospect on another machine?

 

---

Hi.

 

Yes and yes.

In the latter case, it's a HP Proliant ML350 server running on Intel Xeon 3GHz processor with 3GB RAM, Windows 2003 server. The polling time was about 50 seconds per laptop client that is currently offline.

 

Regards

Lennart

[jef]

gentlemen,

 

It's been a long time. There doesn't seem to be a resolution for the problem yet, am I right? Today I figured, fair enough, let's give ordinary scripts a go. As opposed to managing everything with proactive. It's been noted that every time a proactive backup execution finishes, the polling starts from the top instead of continuing. Not good. BUT! This seems to be happening with EVERY SINGLE finished execution, including the normal script ones. So:

 

I run a script to backup all my 80 or so clients and put all the laptops in a proactive pool.

Whenever a backup execution in the script finishes, the proactive polling starts from the top.

Where is the logic?

[Lennart_T]

Quote:

---

It's been noted that every time a proactive backup execution finishes, the polling starts from the top instead of continuing. Not good.

 

---

That is as designed. On top of the list are the clients most in need of a backup. One of them might have connected while another client was being backed up.

[jef]

Even if you ask retrospect to only recheck every xx minutes/hours?

 

And as to "Whenever a backup execution in the script finishes, the proactive polling starts from the top." part? Is that part of the design as well? I agree there might be some logic in the fact that -every- finished execution frees up a backup set that could potentially be used to backup the proactive client with the greatest need, but other than that...

 

 

thanks,

 

jef

[Lennart_T]

Quote:

---

Even if you ask retrospect to only recheck every xx minutes/hours?

 

---

Don't you mean backup interval?

If not, where is the setting you mean? (I have never seen it.)

 

Quote:

---

And as to "Whenever a backup execution in the script finishes, the proactive polling starts from the top." part? Is that part of the design as well?

 

---

I have never seen that happen, so I can't comment.

[jef]

Quote:

---

Quote:

---

Even if you ask retrospect to only recheck every xx minutes/hours?

 

---

Don't you mean

backup

interval?

If not, where is the setting you mean? (I have never seen it.)

 

 

---

 

---

You can change these settings in the proactive backup script options under polling intervals

 

Quote:

---

Quote:

---

And as to "Whenever a backup execution in the script finishes, the proactive polling starts from the top." part? Is that part of the design as well?

 

---

I have never seen that happen, so I can't comment.

 

---

 

I'm not sure, but I have the feeling it is easily reproduced

 

jef