Jump to content
ragge

Quicker, smarter and remote client polling

Recommended Posts

I think the client polling scheme has several problems and should be changed. As much as I enjoy someone actually using IP multicast for something useful, and has done so for a long time by the way, I think the current polling method is inappropriate.

 

We backup some 250 clients, mainly laptops, using one backup server. We split the clients in 8 different groups, and have 8 different proactive backup scripts, each backing up one group of clients to a separate disk backup set each.

 

The current client poller always start polling the oldest client first. We have clients that are away for weeks or months, so the first 50 clients or so haven't been there for weeks and probably won't be today either. In addition, the cyclic poller resets and starts form the beginning every now and then, as when a backup ends I think. This means that our backup server spend most of it's time polling for clients that probably isn't there anyway, and it can take many minutes or tens of minutes for it to find a client to backup, even though there are many available at all times. Requesting an ASAP backup from the client is quite meaningless, because either it already is recently backed up and our server will not get that far in the list until it due again and has wandered up the list, or the client is due for backup anyway.

 

I have two suggestions for changes to the polling scheme, that could be implemented separately or together.

 

1. Still send multicast polls, but let ALL clients answer on each request (and report in their ASAP-backup-request status). To not get a burst of replies, the clients could randomize a delay under, say, a few seconds, for the reply. The server could give the delay time to spread the replies over in the poll request, based on the expected number of replies (for example, number-of-clients/25 seconds, giving an average of 25 replies a second, this could be a user setting). The polls should probably be repeated a few times, as is done now, with a sequence number or such so that a client only replies to each poll once. This would let the backup server get an almost immediate picture of the client situation, and the starving problem explained above would be solved. This should be a minor change to both the server and the client, and the server would have to know which clients are of the older kind that still have to be polled individually.

 

2. The client should remember the IP address of its server and itself report home. This would allow us to backup machines that are away from campus but that still have IP connectivity, such as machines that are at home or at other campuses, which our machines tend to be for shorter or longer times.

 

To work with a NAT, as at many peoples homes, it would have to either just open a normal connection to the server when it is due for backup, or would have to work with the different NAT-tunneling schemes that are available. I would suggest that the client just connects to the server and asks when it is next due for backup, and when that time comes, or when the user requests an ASAP backup, it connects to the server again and waits for backup.

 

/ragge

 

 

Share this post


Link to post
Share on other sites

I would find no. 2 very useful. The amount of time our server polls recently unused or unconnected machines is shocking. Backingup large network is all about optimising backups in the available window.

 

All the client app has to do is send the machine name to the server on a dedicated Retrospect port and then the server can add that to a 'connected' client list and ONLY try and back them up. I'm not quite sure why the product wasn't designed like this in the first place, so I'm assuming there is a reason?

 

On top of this, it would be useful within the client settings to add a 'weight' of importance to the machine. This would enable me to backup important machines on the network rather than waiting their turn or creating yet another proactive script with another group.

 

For example, I would no longer have to split laptop and desktop scripts as I could prioritise laptops machines as they generally have a smaller 'online' window. Another handy feature would be to switch off desktop machines after the backup is complete for night time, out of office hour backups.

 

i.e. priority should be:

 

Filtered by online machines only...

Least recent backup

Laptops

Desktops

 

So, if two machines that were last backup up on the same day... the laptop would get priority over the desktop (or whatever settings were put in the machine settings).

 

Rich

Edited by Guest

Share this post


Link to post
Share on other sites

Item #2:

That is planned for the next major release. It is my understanding that the code to make this work already exists in the client but has not been implemented. Changes to the backup server will need to be made too.

Edited by Guest

Share this post


Link to post
Share on other sites
The current client poller always start polling the oldest client first. We have clients that are away for weeks or months, so the first 50 clients or so haven't been there for weeks and probably won't be today either. In addition, the cyclic poller resets and starts form the beginning every now and then, as when a backup ends I think.

 

Seconded :glasses:

Share this post


Link to post
Share on other sites

This proactive scheduling problem has been around for ages. I've been pointing at it since I first tested retrospect. Don't have the feeling it changed yet. A couple of remarks

 

- First answer you get is 'proactive backup is not meant for standard clients'. Why not? If it worked like it should then it's the best backup strategy out there. Let the scheduler worry about the scheduling.

 

- How come it takes -that- long to try to locate or poll a source? A second should do, no? Allowing the admin to change the timeout for these should solve half the problem already, regardless of the other quirks in proactive backup

 

- Why is there seemingly only ONE poller? If you have different proactive scripts and different backup destinations and multiple cores to do all the processing there doesn't seem to be a good reason to do it all sequentially.

 

Here's hoping for a fresh retrospect for windows in 2009 without proactive problems,

 

cheers,

 

jef

Edited by Guest

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×