Stop "catchup" backups after starting Windows 10

[rbratton]

Retrospect 16.1.2.102 and Windows 10 1903.  Not using Proactive.

I have backups for 2 Windows 10 computers scheduled for 2 and 3 AM each day.  I will often shutdown these computers over the weekend, so no backups are run those days.  But when I restart these computers on Monday, Retrospect immediately tries to run the scheduled backups.  This slows down the computer doing the backups and I'm not comfortable working on the other computer being backed up.

How do I tell Retrospect that if a scheduled backup is missed because resources aren't available, just wait until the next scheduled backup time to try again?  I don't want to block off specific times when backups are allowed as I might need to run a manual backup any time of the day.

Also (not critical, but an annoyance):  Computer A backs up itself and then runs the script for Computer B (who has the Retrospect Client).  This works 99.999% of the time for scheduled backups.  For these "Monday catchup" backups, even though Computer B is started first and completely booted before I start Computer A, when Computer A runs the script for Computer B, it will run but then stops after a minute with nothing done and marked as successful.  If I need to start a new topic for this, I will.

If you need more information, please let me know.  Thanks!

[x509]

rbratton,

When Windows first starts up, the LAN may not be active.  In my experience, with years and years of Windows networking, it is the most frustrating part of a Windows environment.

Before Retrospect tries to back up the LAN client, I suggest that you first check if Windows on Computer A already detects Computer B on the LAN.

 

x509

[rbratton]

1 hour ago, x509 said:

rbratton,

When Windows first starts up, the LAN may not be active.  In my experience, with years and years of Windows networking, it is the most frustrating part of a Windows environment.

Before Retrospect tries to back up the LAN client, I suggest that you first check if Windows on Computer A already detects Computer B on the LAN.

 

x509

Thanks for the reply!  I'll try this the next time I skip a backup and restart the computer.

[mbennett]

9 hours ago, rbratton said:

How do I tell Retrospect that if a scheduled backup is missed because resources aren't available, just wait until the next scheduled backup time to try again?  I don't want to block off specific times when backups are allowed as I might need to run a manual backup any time of the day.

I suggest you simply remove the systems from the backup schedule on the weekend, therefore when you boot the systems on Monday morning there won't be a slowdown.  You obviously already know how to launch a backup job manually, so just do that.

[x509]

Is it definite that the OP never uses those systems on the weekends?

[DavidHertzberg]

rbratton,

To solve the first problem in your OP, just duplicate your 2 a.m.-scheduled script.  Have one script scheduled only for Mon.-Fri., and the other script scheduled only for Sat.-Sun..  If your computers are shut down on Sat.-Sun., the second script won't run that week—but the first script will run Mon.-Fri..

To solve the second problem in your OP—if the above paragraph doesn't eliminate it, precede your "real" Monday execution of the first copy of the script with a "sacrificial" script scheduled for 1:55 a.m. Monday.  This will be an exact duplicate of the "real" script, but with the No Files Selector;  Selectors, including Built-In ones, are documented on pages 435-437 of the Retrospect Windows 16 User's Guide.  I've used a "sacrificial" script for years to bypass Retrospect Mac Engine startup delays; if the "sacrificial" script runs before the LAN is active, as x509 says, the "sacrificial" script fails at 1:59 but the "real" script" runs.

 

[rbratton]

14 hours ago, x509 said:

Is it definite that the OP never uses those systems on the weekends?

I wouldn't say "never".  I almost always (95%) turn off the computers before turning in on Saturday night to avoid the temptation to work on Sundays.  They run 24 hrs/day most of the week, so I like to do a clean restart on Monday.  If I have projects to work on (I do remote software development), then yes -- I'm probably working on Saturdays.

And I'll probably shut everything down after I respond to these posts -- expecting some nasty thunderstorms for the rest of today.

[rbratton]

@mbennett, @DavidHertzberg

Thanks for the replies.  I'll give those suggestions a try and report back.

Great idea on the "sacrificial" script to let the remote computer completely boot!

Thanks!

[DavidHertzberg]

On 1/11/2020 at 3:06 PM, rbratton said:

@mbennett, @DavidHertzberg

Thanks for the replies.  I'll give those suggestions a try and report back.

Great idea on the "sacrificial" script to let the remote computer completely boot!

Thanks!

rbratton,

I originally got the idea for the "sacrificial" script from this 2016 post by hgv.  He/she wrote "This suggests there's an issue with Retrospect daemon being initiated too soon at startup. This is particularly annoying to me because my backup computer is restarted daily. As a workaround I created a launchd script to do a 60 seconds delayed stop/restart of both daemons at startup. The problem seems to have disappeared."   I don't know enough about macOS to write a launchd script, so I came up with the "sacrificial" Retrospect script instead.

I noticed, in re-reading that thread, that I originally scheduled my "sacrificial" script 10 minutes before the real script.  That was sufficient to nullify the effect of the -530 error for the "sacrificial" script then.  I still schedule "sacrificial" scripts 5 minutes before my real Backup scripts, but—since I changed to using Add Source Directly (the Retrospect Windows equivalent is Direct Access Method)—I'm no longer getting any -530 errors on the "sacrificial" scripts even when I boot my "backup server" machine (with my MacBook Pro "client" machine already booted) several hours after their scheduled 3 a.m. start time.

Wikipedia says launchd is "an init and operating system service management daemon created by Apple Inc. as part of macOS to replace its BSD-style init and SystemStarter".  If, instead of a "sacrificial" Retrospect script, you want to write the equivalent of a launchd script for Windows, I have no idea how to do that—but it would probably have to do a delayed stop/restart lasting more than 60 seconds.

[rbratton]

Interesting...  I setup 2 sets of scripts for each computer last Saturday.  One script runs Saturday and Sunday at 2 AM; the other runs MTWTF at 2AM.  I shutdown the computers around 16:00 on Saturday and restarted them around 13:41 today (Monday).

Both "Weekend" scripts started at 13:43 today (using Execution units 1 and 2).  The script that backs up Computer "B" using the Retrospect Client finished immediately, and then the "Weekday" script for that computer ran (using the same Execution unit) and finished immediately.  The "Weekend" script for Computer "A" which runs on "A" completed around 15:00 and then the "Weekday" script started.  It completed around 15:56.

My assumption is that Retrospect saw that the backup scheduled for Sunday at 2AM ("Weekend" backup) was not run, so it immediately ran it.  It then saw the backup for Monday at 2AM ("Weekday" backup) was also not run, so it immediately ran it after the first one finished.  It did this for both computers.

I think I'm a little worse off than before.  What I'm really looking for is a setup option that says "If a backup is missed for any reason, just wait for the next scheduled backup."

But I'm open for other suggestions.  Thanks!

[rbratton]

On 1/11/2020 at 3:14 AM, DavidHertzberg said:

rbratton,

... snip ...

To solve the second problem in your OP—if the above paragraph doesn't eliminate it, precede your "real" Monday execution of the first copy of the script with a "sacrificial" script scheduled for 1:55 a.m. Monday.  This will be an exact duplicate of the "real" script, but with the No Files Selector;  Selectors, including Built-In ones, are documented on pages 435-437 of the Retrospect Windows 16 User's Guide.  I've used a "sacrificial" script for years to bypass Retrospect Mac Engine startup delays; if the "sacrificial" script runs before the LAN is active, as x509 says, the "sacrificial" script fails at 1:59 but the "real" script" runs.

 

Thinking on this some more and looking at today's results, I'm not sure if this will work.  The only time the backup using the Retrospect Client "fails" (i.e., it runs, but doesn't actually do any work) is when Retrospect thinks it has to run a "catchup" backup.  On normal daily backups, it works fine.

When doing the "catchup" backups, I'm not sure we have any control on how Retrospect orders the backups for execution nor inserting any delay between the "sacrificial" script and the "normal" script.  It looks like Retrospect tries to run the "catchup" as quickly as possible.

Or I could be missing something.  (It happens -- haha!)

[DavidHertzberg]

rbratton,

You work in remote software development, but you don't normally back up work you do on Friday until the following Monday (assuming you're not working  a Transylvanian schedule)?  😮  Please PM me your employer's company name, so that I can short-sell the stock. 🤣  My 7-days-a-week backups are scheduled for 3:05 a.m., but I boot my MacBook Pro "client" and then my "backup server" whenever my BPH wakes me up after that time—and let the backup run while I do my teeth. 

IMHO you're not professional enough to be using Retrospect; you should instead be using a Windows backup program that does near-continuous backup.  That's why Retrospect doesn't have the feature you want.  However, if you persist, I suggest creating one or more run documents (pages 236-238 of the Retrospect Windows 16 User's Guide).  Then, if you work on a Saturday or Sunday, just use a run document to start the appropriate Backup script after you've finishing working.

[rbratton]

17 minutes ago, DavidHertzberg said:

rbratton,

You work in remote software development, but you don't normally back up work you do on Friday until the following Monday (assuming you're not working  a Transylvanian schedule)?  😮  Please PM me your employer's company name, so that I can short-sell the stock. 🤣  My 7-days-a-week backups are scheduled for 3:05 a.m., but I boot my MacBook Pro "client" and then my "backup server" whenever my BPH wakes me up after that time—and let the backup run while I do my teeth. 

IMHO you're not professional enough to be using Retrospect; you should instead be using a Windows backup program that does near-continuous backup.  That's why Retrospect doesn't have the feature you want.  However, if you persist, I suggest creating one or more run documents (pages 236-238 of the Retrospect Windows 16 User's Guide).  Then, if you work on a Saturday or Sunday, just use a run document to start the appropriate Backup script after you've finishing working.

Go back and read my previous post -- my Friday work is backed up early Saturday morning.

Plus ALL s/w changes are checked into a version control system offsite before I stop work.

[DavidHertzberg]

rbratton,

Sorry, but I read "the other runs MTWTF at 2AM" in your previous post as meaning you normally run what you call your "MTWTF" script at 2 a.m. on Monday through Friday.  I now think you mean "MTWTF" as referring to the days you did the work that is backed up, not the days the script is actually run.  Thus you actually run the "MTWTF" script at 2 a.m. on Tuesday through Saturday, even though it is backing up work you did Monday through Friday.

Assuming that what I wrote in the paragraph above is correct, I hereby absolve you from the charge of not backing up your Friday work until the following Monday.  So you are indeed professional enough to use Retrospect; you're just not always the most professional tech writer.  It's good to hear you're using a version control system promptly.

Making that same assumption,  your "Weekend" script is actually scheduled for Sunday and Monday at 2 a.m..  But maybe your "Weekday" script is also scheduled for 2 a.m. on Monday, not just Tuesday through Saturday.  If so it ran yesterday—Monday—right after the "Weekend" script ran when it was unable to run on Sunday or Monday at 2 a.m..  Or something along those lines.

[Nigel Smith]

On 1/10/2020 at 7:08 PM, rbratton said:

How do I tell Retrospect that if a scheduled backup is missed because resources aren't available, just wait until the next scheduled backup time to try again?  I don't want to block off specific times when backups are allowed as I might need to run a manual backup any time of the day.

Have you tried the "Options" section of your script? There's also scheduling options there, which only apply to that script (though the defaults reflect the Schedule settings in General Prefs, which might make you think otherwise...) and so would have no impact on manual backups. Set your "Start", "Wrap up" and "Stop" times to suit your working practices and required backup window and you should be good.

[rbratton]

6 hours ago, Nigel Smith said:

Have you tried the "Options" section of your script? There's also scheduling options there, which only apply to that script (though the defaults reflect the Schedule settings in General Prefs, which might make you think otherwise...) and so would have no impact on manual backups. Set your "Start", "Wrap up" and "Stop" times to suit your working practices and required backup window and you should be good.

I haven't yet -- but it looks promising.  I set the backup window to 00:00-07:00.  For testing -- On one computer, I'll perform a manual backup tonight outside the backup window to ensure manual backups are allowed.  I'll then shut it down overnight to skip the scheduled nightly backup.  I'll restart tomorrow morning again outside the backup window to see if Retrospect tries to run a "catchup" backup.  From the documentation, it looks like it won't -- so we might be good.  I'll let you know.

Thanks!

[rbratton]

On 1/14/2020 at 9:07 AM, Nigel Smith said:

Have you tried the "Options" section of your script? There's also scheduling options there, which only apply to that script (though the defaults reflect the Schedule settings in General Prefs, which might make you think otherwise...) and so would have no impact on manual backups. Set your "Start", "Wrap up" and "Stop" times to suit your working practices and required backup window and you should be good.

Here's what I did:  The backup window when backups are allowed is from 00:00 to 07:00 every day.  I ran manual backups on both computers which ran fine.  I shut them down overnight so both scheduled nightly backups would be skipped.  I restarted both computers around 9:30 AM today which is not in the allowed backup time window. Retrospect did not automatically start "catchup" backups when the computers booted which is good -- it honored my backup time window.  But it did schedule "catchup" backups of both computers starting tonight at 00:00 -- the earliest allowed backup time.  I think I can live with that; I'll just have to remember to delete these redundant backups (as the scheduled ones will occur a few hours later).

Thanks for all the help.

[DavidHertzberg]

 

On 1/15/2020 at 1:09 PM, rbratton said:

.... Retrospect did not automatically start "catchup" backups when the computers booted which is good -- it honored my backup time window.  But it did schedule "catchup" backups of both computers starting tonight at 00:00 -- the earliest allowed backup time.  I think I can live with that; I'll just have to remember to delete these redundant backups (as the scheduled ones will occur a few hours later).

....

rbratton,Page 398 of the Retrospect Windows 16 User's Guide says "Schedule lets you define a window during which scripts are allowed to execute."  So the result of your experiment is exactly what I would have predicted, if I had had time to post.  If you prefer to remember on most Monday mornings to delete the "catchup" backups that have been scheduled but not yet executed, that's up to you.  If I were you,  I'd prefer to remember on a few Sunday or Monday mornings to execute a pre-prepared run document.  Different strokes for different folks.

Edited January 19, 2020 by DavidHertzberg
I had added a P.S, but I moved it to the first paragraph of my later post below Nigel Smith's

[Nigel Smith]

Assuming incremental backups, no need to delete -- it'll just make the "proper" backup run faster because most has already been done.

And consider using Proactive (unless standard scripts do something you need that Proactive doesn't), which is made for exactly this "sometimes here, sometimes not" situation.

Re-reading your OP, it sounds like both computers get shut down and one is the RS server while the other is the client. Have a play with the "Look ahead time" in the general (rather than script) Schedule Preferences. I'm starting to think it's *because* you shut down the server that you are getting the "catchups" -- look ahead sees you've got something scheduled within the next 12 hours so makes sure it runs at the next opportunity (I'd assumed that you had the server running 24/7 and it was two clients you were restarting). It may be that setting "Look ahead" to 0 solves your problem, but that might require you to leave RS running on the server rather than quitting/autolaunching for the next scheduled run.

[DavidHertzberg]

rbratton,

I had thought about what Nigel Smith says in the first paragraph of his post directly above.  However I assume that you are bothered by the additional scanning time for regular backups  that follow "catchup" backups, even if the backups are incremental.  Moreover I'm not sure his second-paragraph suggestion of Proactive is applicable, because Proactive (which I don't use) assumes a dedicated "backup server" machine that is "always here" whenever the "client" machine is "sometimes here"—which (as Nigel Smith recognizes in his third paragraph) is not the situation described in your OP.

Unfortunately your current solution will IMHO become impossible as of March or August 2020 (maybe with a month's delay if Retrospect engineering gets overwhelmed by other StorCentric-dictated enhancements).  That is because, per this post in another thread (second sentence in P.P.S.),  Retrospect "Inc." AFAICT intends getting rid of Retrospect Launcher in Retrospect Windows 17—as they did with retrorun (its old name) in Retrospect Mac 8.0 in early 2009. 

The Retrospect Mac 16 User's Guide describes no general Schedule Preferences equivalent to what is on pages 397-399 of the Retrospect Windows 16 User's Guide.  Those Schedule Preferences are dependent on the Startup Preferences described on page 401, which affect the behavior of Retrospect Launcher—if the first checkbox in Startup Preferences is checked to enable Retrospect Launcher.  This post is part of another thread that discusses the woes of using Retrospect Launcher in Retrospect Windows; and the post's P.S. says "And now that my [Retrospect Mac] backup script has finished running, the combination of Retrospect Engine and Retrospect Console has dropped to 250MB real memory.    IMHO derek500's [Retrospect Windows] installation ought to be able to afford reserving that continuously."  And in Retrospect Mac one can stop and start the GUI Console while the Engine continues to run.

Edited January 20, 2020 by DavidHertzberg
Add additional sentences to first and third paragraphs

[rbratton]

On 1/19/2020 at 2:30 AM, DavidHertzberg said:

rbratton,

I had thought about what Nigel Smith says in the first paragraph of his post directly above.  However I assume that you are bothered by the additional scanning time for regular backups  that follow "catchup" backups, even if the backups are incremental.  Moreover I'm not sure his second-paragraph suggestion of Proactive is applicable, because Proactive (which I don't use) assumes a dedicated "backup server" machine that is "always here" whenever the "client" machine is "sometimes here"—which (as Nigel Smith recognizes in his third paragraph) is not the situation described in your OP.

Unfortunately your current solution will IMHO become impossible as of March or August 2020 (maybe with a month's delay if Retrospect engineering gets overwhelmed by other StorCentric-dictated enhancements).  That is because, per this post in another thread (second sentence in P.P.S.),  Retrospect "Inc." AFAICT intends getting rid of Retrospect Launcher in Retrospect Windows 17—as they did with retrorun (its old name) in Retrospect Mac 8.0 in early 2009. 

The Retrospect Mac 16 User's Guide describes no general Schedule Preferences equivalent to what is on pages 397-399 of the Retrospect Windows 16 User's Guide.  Those Schedule Preferences are dependent on the Startup Preferences described on page 401, which affect the behavior of Retrospect Launcher—if the first checkbox in Startup Preferences is checked to enable Retrospect Launcher.  This post is part of another thread that discusses the woes of using Retrospect Launcher in Retrospect Windows; and the post's P.S. says "And now that my [Retrospect Mac] backup script has finished running, the combination of Retrospect Engine and Retrospect Console has dropped to 250MB real memory.    IMHO derek500's [Retrospect Windows] installation ought to be able to afford reserving that continuously."  And in Retrospect Mac one can stop and start the GUI Console while the Engine continues to run.

Thanks for the info!  I'll keep an eye on future Retrospect plans.

[DavidHertzberg]

19 hours ago, rbratton said:

Thanks for the info!  I'll keep an eye on future Retrospect plans.

rbratton,

Cheer up; even if my informed guess about Retrospect engineering plans to switch Retrospect Windows 17 to a Launcher-less Retrospect-Mac-like interface proves correct, you will have the following two options:

The first option is the equivalent of your current solution.  First, change your Sat.-Sun. backup scripts to both use the same execution unit (see page 272 of the Retrospect Windows 16 User's Guide), and change the scheduled time of those Sat.-Sun. scripts that backup your "backup server" machine to 5 minutes after those Sat.-Sun. scripts that backup your "client" machine.  Then, on every Monday following a weekend on which you have not worked both Saturday and Sunday, boot your "backup server" machine first.  Immediately, before you boot your "client" machine, start the Retrospect Engine and the Retrospect Console on your "backup server" and cancel every Backup activity awaiting execution.  The Backup activity that has already started should be trying to back up your "client" machine; it will fail by itself because your "client" machine—not having yet been booted—is not yet visible on your LAN.  After you have done the necessary cancellations, you can boot your "client" machine,  By then no Sat.-Sun. Backup activities should be executing or awaiting execution, so you can leave the Retrospect Engine running  on your "backup server" machine—where it will be using mostly virtual memory until the scheduled time for your midnight-Tuesday-morning Backups of your Monday work ( you can quit the Retrospect Console, restarting it to monitor those Tuesday-morning Backups).

The second option is the equivalent of the one I said in a post above I personally would prefer.  First, create scripts for Sat.-Sun. backups of your "backup server" and "client" machines, but do not schedule them.  Then, on every Saturday or Sunday you work, when you have finished work simply start the Retrospect Console and Run those scripts using the appropriate Backup Set destination.  When the Console shows those scripts have finished execution, you can shut down both machines.  If you want, you can instead create a single two-Source script that does a Backup of both machines.

I predict you'll like the new Retrospect Windows GUI after you get used to it.  Based on the current Retrospect Mac 16 User's Guide, the Retrospect Windows 17.x User's Guide will be half—less than 300 pages—the length of the Retrospect Windows 16 User's Guide.

[Nigel Smith]

On 1/19/2020 at 7:30 AM, DavidHertzberg said:

Moreover I'm not sure his second-paragraph suggestion of Proactive is applicable, because Proactive (which I don't use) assumes a dedicated "backup server" machine that is "always here" whenever the "client" machine is "sometimes here"

Not at all -- you can, for example, schedule Proactive to run only for certain hours of the day.

So OP could set Proactive to run from 2am-6am every day, with an 20 hour interval. If the server is running during that time the server will be backed up, and it will also back up the client if that's available. No client is a "graceful fail", no server and nothing happens 😉 

What you can't do with a single Proactive script is set the order in which clients should be backed up, so no good if that's important. You can't shut down the backup server, as part of the script, when it's finished. And using a schedule as above would mean you couldn't use the "Early backup" request to get a daytime backup, so you'd have to make another script for that -- you *might* be able to set a second Proactive script, running from 6am-2am, with a ridiculously large interval setting, that allows earlys, but I haven't tried that myself...

Proactive is very flexible -- which is sometimes a boon, sometimes a pain -- and is always worth considering in any situation where backup routines can vary (presence of clients, volumes, target sets, etc).

[DavidHertzberg]

Nigel Smith,

You're correct in the immediately-above post, but to "schedule Proactive to run only for certain hours of the day" requires that the "backup server" be already running to act on that schedule.  For the next few months,  that can be done via the Retrospect Launcher.  But, as I said in the second paragraph of the same post you quote, "Retrospect 'Inc.' AFAICT intends getting rid of Retrospect Launcher in Retrospect Windows 17".  That would mean that as of March or August of 2020 rbratton's "backup server" machine would  have to either be constantly running the GUI-less Retrospect Engine whenever it is booted, or rbratton would have to remember to start the Engine each midnight—a reversion from what he is going to do.  I think the second alternative might be more burdensome than the only-on-most-Mondays cancellation procedure I described in the second paragraph of this post, which is applicable to regular Backup scripts.  The first alternative would, of course, also work with Backup scripts individually scheduled to run only at certain hours.  We'll see in a few months—but see the next two posts.

 

[Nigel Smith]

21 hours ago, DavidHertzberg said:

That would mean that as of March or August of 2020 rbratton's "backup server" machine would  have to either be constantly running the GUI-less Retrospect Engine whenever it is booted, or rbratton would have to remember to start the Engine each midnight

Or, since he knows the the time the script needs to run, he could use Windows Task Scheduler to launch RS when appropriate.

Or even, if his BIOS supports it and he doesn't mind the security implications: set the PC to boot at a certain time and auto-login, set Windows Task Scheduler to fire up RS, use script hooks to to monitor and shut down both PCs when complete!

These are computers -- we should be getting them to do things, instead of having to remember to do them ourselves! 😉