How To Run Tasks Sequentially as quickly as possible.

[jotrago]

There are occasions when you need to force tasks to run sequentially, and as quickly as possible, ideally immediately one after the other.

{EG: Each morning you might want to Verify the previous nights' backups, then initiate a Snapshot Transfer Task to Tape for offsite copy.}

 

The usual compromise is to take a guess at how long each Task will need and then schedule them in time sequence with a gap in between to allow for variation in run time.

Whilst this works (provided you at least have an estimate of the run time for each job) it is not very efficient or fast, and very difficult if you have no idea how long a task may take.

 

Try this method to force tasks to run sequentially with each one starting immediately the previous one has finished.

 

The trick is in forcing them all to use the same execution unit.

 

Schedule each job to start a short time, (say 1 minute) after the previous one. This is just to get them in the right order. It doesn't matter how long they may actually take to run.

At the bottom left of the Script window for each task is the Execution Unit selector, which defaults to ANY. Set it to the highest numbered Execution unit, (so it won't clash with any other task which will pick the lowest numbered available unit by default) for each of your Sequential Tasks.

 

At the appointed time the first of your sequential tasks will start with the specific Execution unit.

1 minutes later the next one will try to start, but since the execution unit is in use, it cannot, and will be added to the Waiting Queue. The rest of your sequential tasks will follow at 1 minute intervals.

 

You should now see them in the Waiting Tab of the Activity Monitor, all waiting,  in order,  for the Execution unit to become free.

As soon as the first task completes, releasing the Execution unit, the next one will start.

 

Your tasks will now execute, sequentially, as quickly as possible.

 

See the User Guide, Chapter 7, Controlling Operations > Multiple Concurrent Executions > Assigning Execution Units.

 

 

 

[Scillonian]

For the benefit of those coming across this via forum search the Professional and Desktop versions of Retrospect only have a single Execution Unit available so there is no option shown for setting the Execution Unit and all scripts will therefor always run in schedule order.

Edited July 12, 2014 by Scillonian

[jotrago]

This is true. And that is why this is posted in the Server, SBS, & Multiserver forum

[Lennart_T]

Historically, the waiting executions were executed in alphabetical order, by script name. (That is, not necessarily in the order they were scheduled.) This may have changed in later versions, I still run 7.6.

[jotrago]

The manual mentions that if you simply submit a whole bunch of scripts, or "Run Documents", then they run in alphabetical order, but I have never used that method.

 

I do regularly use the method described above. I assume that by submitting them with a schedule in a specific order, forces that order on them.

 

I like to do my verification during the day, as a separate task after the backups have run overnight (saves on Backup time), this is typically followed by a snapshot transfer to generate offsite copies to tape or removable disk, which is then followed by Catalog, Logs, & Config Duplication jobs for  DR copies. I typically set all these up using the above technique, to ensure they run in the correct sequence but in the shortest time.

Works for me.

[Lennart_T]

It's been a while since a saw any complaints about this, but according to them, as long as a number of scripts are "waiting" when the previous script ends, the next one will be the first in alphabetical order, regardless of when it was scheduled.

 

But I can't verify this with anything but version 7.6. I have been forced to schedule them with only a slight overlap in time in order to keep the third script (in order) to run prematurely.

 

How are your scripts named? Are they (by chance) named in the order they will run?

[jotrago]

Could be, I've used Retro 7, extensively in the past and seem to remember using this technique. Currently I am on 9.

Here's an example of 3 scripts scheduled at 1 minute intervals using the same execution unit #4

I rigged the names so we can see whether they run in scheduled or alpha order.

 

https://app.box.com/s/y6du7m7fnm9v1al1wkb3

 

Here is the first task running in exec4

 

https://app.box.com/s/qig2o5imiudi7lmvo4ix

 

here are the other two waiting in order, If it is Alpha order then the transfer should run before the verify

 

https://app.box.com/s/4c74r2vxbcrsngqo86g1

 

The key here is the Status, we can see the the Transfer Task is waiting for the Verify task. I think if the status indicates a task is waiting for another task, then that is due to them being allocated to the same Exec Unit.

 

So the Backup Ran,

Then the verify

Then the Transfer

Each immediately after the previous one finished.

 

 

[Lennart_T]

Good. It looks like Retrospect fixed so the scripts do run in the order of scheduling

 

Thanks for the info.

[Don Lee]

That's a good thing. I've been depending on the ordering, and have not been disappointed. If the scheduling actually ran in alpha order instead of the initial schedule time, that would be sub-optimal. (in my opinion)

[jotrago]

Cobbler's shoes!

The screen shots in the above links are no longer available.

I accidentally deleted a bunch of files in my "Box" including those. No Backups! Sorry