Jump to content


Photo

Verifying success on two independent backup schemes for clients


  • Please log in to reply
17 replies to this topic

#1 saskia

saskia

    Member

  • Members
  • PipPip
  • 10 posts

Posted 20 July 2017 - 06:02 PM

My two schemes are:

1) Proactive backup to tape of all non-cache files less than 100 megabytes in size. This gets all of the important research files which are text anyway.

 

2) Groomed backup to disk of all files over 100 megabytes. This avoid filling up tapes so fast with big files that tend to be junk but also includes bloated Microsoft Outlook .pst files that might have something useful buried in them.

 

Retrospect will report that a client is backed up if either scheme happens, but I want to be sure that both have been successful recently.

 

What is my least effort way of regularly checking this?



#2 David Hertzberg

David Hertzberg

    Occasional Forum Poster

  • Members
  • 410 posts
  • LocationNew York, NY

Posted 20 July 2017 - 06:49 PM

I don't know what saskia's definition of "least effort" is, but how about sending himself/herself e-mails upon the completion of each backup of a source via a Proactive script and upon the completion of each Scheduled script?  Instructions are on pages 168-169 of the Retrospect Mac 14 User's Guide.



#3 saskia

saskia

    Member

  • Members
  • PipPip
  • 10 posts

Posted 20 July 2017 - 08:43 PM

This is too much email. Am looking into script hooks so I can automate processing with my own bash scripts (running on Mac OSX). I can't find a definitive description of the arguments passed although I am now logging them as a preliminary.



#4 David Hertzberg

David Hertzberg

    Occasional Forum Poster

  • Members
  • 410 posts
  • LocationNew York, NY

Posted 20 July 2017 - 11:50 PM

Has saskia looked at this Knowledge Base article, and downloaded the .ZIP file it links to under the "Example scripts" section?  There are a couple of Bash script examples that hook into external monitoring systems (one of which saskia may have), but the example that sounds most promising for him/her from the description is sample_sh.

 

If that's not sufficient, IMHO saskia should file a Support Case request for documentation for the arguments.  My next post in this thread will be a boilerplate explanation of why and how to file a Support Case.  If saskia is still within the trial period of Retrospect use, he/she can also phone Retrospect Tech Support.

 

P.S.: Although the Knowledge Base article doesn't say so, script hooks are a feature added to Retrospect Mac 14 (and Retrospect Windows 12); I presume saskia already knows that.



#5 David Hertzberg

David Hertzberg

    Occasional Forum Poster

  • Members
  • 410 posts
  • LocationNew York, NY

Posted 20 July 2017 - 11:53 PM

If you think this is a documentation deficiency that should be fixed by Retrospect Inc., you will have to submit it as a Support Case.  For English speakers, that is done by going here http://www.retrospec...support/contact, and filling out the form (sorry, I don't know what the equivalent addresses are for non-English speakers, but they can figure it out from their appropriate Retrospect website address).  IMHO this is quite reasonable; obliging you to fill out the form provides Retrospect Inc. with useful details about your Retrospect installation that they would otherwise have to query you for.

 

As a result, Retrospect Inc. will pay no attention to your post in this forum.  On 12 December 2016, in response to a letter I snail-mailed to Mayoff,  I received an e-mail through a Mayoff account that was signed by JG Heithcock, CEO, Retrospect, Inc. http://www.retrospec...m/en/about#exec.  In it he says "From reading your letter, I think the main issue is that you view the forums as a good place to talk to us, Retrospect, Inc. But we view the audience of the forums as restricted to our customers [my emphasis]. The one caveat we have made on that is for feature requests, largely as we would like to see if other customers also agree on the desirability and feature set for these requests."

 

That means that the only audience for "Retrospect documentation requests" in this forum will be other administrators of Retrospect.  Nevertheless, by posting in this forum you are providing a useful service to us fellow administrator peasants.  Thank you.

 

Please be aware that the "description of your issue" in the Support Case form is IME limited to about 2000 characters by the Support Case software.  If you go over that limit your "description" will be broken up into a "description" plus one or more "additional notes".  The same is true for any additional notes you may later post yourself.  I suggest that, to avoid the appearance of choppiness in your Support Case, you create your case in a post in this forum and then copy it paragraph-by-paragraph to your Support Case. 

 

Note that, despite the new dialogs in the Retrospect Inc. Support Case system urging you to sign up for Annual Support and Maintenance, Mayoff has verbally assured me that you don't need to be signed up for ASM to request more documentation—only to get personal assistance with coping with the problem you need it for.

 

If this post sounds formulaic, that's because I intend it to be.  I intend to post it in every new thread that appears in this forum, unless the OP indicates that he/she has or will open a Support Case for the documentation request that the thread reports.  Of course, Mayoff could take 5 minutes of his time to post a slightly-more-polite version of this post as a  "sticky thread" that will always appear at the top of the forum.  I don't intend to hold my breath until that happens (insert appropriate smiley here).

 

 



#6 saskia

saskia

    Member

  • Members
  • PipPip
  • 10 posts

Posted 21 July 2017 - 03:04 AM

So it looks like there is no proper documentation on the arguments passed. So I will have to wing it. Support gave me this link:

http://www.retrospec...kb#script-hooks

and

http://www.retrospec...kb/script_hooks

gives a bit of information, but in logging with my own minimal

/Library/Application Support/Retrospect/retroEventHandler

it is passed a lot of arguments I shouldn't have to guess about...

 

-StartScript Copy Assistant - 2017-07-20, 3:31 PM 7/20/2017 16:36 /Library/Application Support/Retrospect/intv2775.reh
-StartSource Copy Assistant - 2017-07-20, 3:31 PM My Computer/Home /Volumes/Home/ My Computer /Library/Application Support/Retrospect/intv2776.reh
-EndSource Copy Assistant - 2017-07-20, 3:31 PM My Computer/Home /Volumes/Home/ My Computer 0 0 0 7/20/2017 16:36 7/20/2017 16:36 7/20/2017 16:36 Backup Clients/alpha-dhcp/OS/ada Normal My Computer 0 0 successful true
-EndScript Copy Assistant - 2017-07-20, 3:31 PM 0 0 successful
 
for example, what is 
/Library/Application Support/Retrospect/intv2775.reh
it is not a file
 
and for example, what is 
 0 0 successful true
trying to tell me


#7 David Hertzberg

David Hertzberg

    Occasional Forum Poster

  • Members
  • 410 posts
  • LocationNew York, NY

Posted 21 July 2017 - 09:05 PM

The information Saskia wants is no longer available in the form it was available last night, because of an "anti-miracle" that occurred sometime between 7:50 p.m. yesterday evening and 9:30 a.m. this morning.  The "anti-miracle" is that the contents of the downloadable .ZIP archive I mentioned in the first paragraph of post #4 in this thread no longer contains what this Knowledge Base article says it should contain.  In particular, there is no longer a free-standing sample_sh file in the archive.

 

That free-standing sample_sh file in last evening's version of the archive contained a Bash shell script with functions that would be called for various Retrospect Events. Most of the functions didn't have much executable code—only success and error messages, but the parameters for the various functions were understandably named and IIRC described in comments at the top of the file.  The obvious reason the functions in that file did not have much executable code is that they were not designed to feed into any monitoring system.  By contrast, the only Bash shell scripts in this morning's version of the archive are in folders named "IFTTT" and "Slack", where they are named RetroEventHandler.sh.  For understanding parameters, the file sample_c.cpp in the folder "Sources" in the folder "Sample C" may be helpful.

 

One possibility is that I imagined reading the sample_sh file in last evening's version of the archive, but that doesn't explain why sample_sh is still listed under the "Example scripts" section in the Knowledge Base article—even though it no longer exists in the archive linked to at the top of that section.  Here follows my hypothesis, based on [a] the timing of the change in the archive and [b] the fact that Retrospect Technical Support isn't allowed to author or change Knowledge Base articles.  My hypothesis is that Retrospect T.S. got wind within the last 24 hours of what saskia wanted to do, and decided it would cause a lot of trouble for both him/her and T.S..  So they modified the archive, but were not allowed to modify the Knowledge Base article.  They probably think saskia would be better off using IFTTT, or Slack if he/she is willing to spend the money.  Again that's purely my hypothesis.

 

Sorry, saskia.



#8 saskia

saskia

    Member

  • Members
  • PipPip
  • 10 posts

Posted 21 July 2017 - 10:56 PM

At this point I am monitoring data passed to retroEventHandler.sh by Retrospect to understand the syntax. BUT unless I am sorely mistaken, some of the successful backups don't seem to be flagged and some of the messages seem to come at the wrong time (StartSource at the end of the backup for example, which is a bit late if your want to make a database quiet, not that I need to). I will grab more info as my usual backups come and go and then update this message.

 

BTW, I have an existing web interface for our computer stats and I will not be using any 3rd party tools to complicate my setup.



#9 saskia

saskia

    Member

  • Members
  • PipPip
  • 10 posts

Posted 22 July 2017 - 03:17 AM

Definitely some backups are not reported.



#10 David Hertzberg

David Hertzberg

    Occasional Forum Poster

  • Members
  • 410 posts
  • LocationNew York, NY

Posted 22 July 2017 - 05:11 AM

At this point I am monitoring data passed to retroEventHandler.sh by Retrospect to understand the syntax. BUT unless I am sorely mistaken, some of the successful backups don't seem to be flagged and some of the messages seem to come at the wrong time (StartSource at the end of the backup for example, which is a bit late if your want to make a database quiet, not that I need to). I will grab more info as my usual backups come and go and then update this message.

 

BTW, I have an existing web interface for our computer stats and I will not be using any 3rd party tools to complicate my setup.

 

saskia should make sure he/she is monitoring data passed to one of the two RetroEventHandler.sh shell scripts contained in the latest version of the archive.  There is always the possibility that the archive was updated at least partially because of error(s) in the previous version, notwithstanding my hypothesis.

 

Definitely some backups are not reported.

 

saskia should make sure that he/she is running version 14.1 of Retrospect Mac for both Engine and Client, because the Release Notes for that version say "Fixed Mac client hooks for external scripting with event handlers (#6750)".  If some backups are still not reported, then I would suggest temporarily enabling e-mails as I suggested in post #2 of this thread.  If the resulting e-mails do not report all backups, and/or if the results of saskia's monitoring data passed to the RetroEventHandler.sh shell script do not match what the e-mails report, then saskia should file a Support Case per post #5 in this thread.



#11 saskia

saskia

    Member

  • Members
  • PipPip
  • 10 posts

Posted 22 July 2017 - 02:45 PM

Retrospect up to date 14.1.0 (138). Handler is barebones simple:

 

>>>>>>>> /Library/Application Support/Retrospect/retroEventHandler.sh:

#!/bin/sh
tmpfiles="/datalog/tmp/retrochk"
num="$#"
cmd="$1"
all="$@"
date "+%D-%T ($num) $all " >> $tmpfiles.log
exit

 

>>>>>>> Excerpt from retrochk.log saved by above script:
07/21/17-22:48:19 (4) StartScript Daily 7/21/2017 22:48 /Library/Application Support/Retrospect/intv0013.reh
   (successful incremental backup of a windows server by this script missing here from log)

07/21/17-22:54:01 (6) StartSource Daily My Computer/OS/datalog /Volumes/OS/datalog/ My Computer /Library/Application Support/Retrospect/intv0016.reh
07/21/17-22:54:29 (18) EndSource Daily My Computer/OS/datalog /Volumes/OS/datalog/ My Computer 5754 5 7 7/21/2017 22:54 7/21/2017 22:54 7/21/2017 22:48 LT-C Normal OS 0 0 successful false

 


#12 David Hertzberg

David Hertzberg

    Occasional Forum Poster

  • Members
  • 410 posts
  • LocationNew York, NY

Posted 24 July 2017 - 08:02 AM

Maybe bug #6750 "Fixed Mac [my emphasis] client hooks for external scripting with event handlers", but didn't fix Windows client hooks.  If I were saskia, I would [a] make sure his/her Windows server is running the latest Retrospect Client software and—if that doesn't cause the incremental backup of the Windows server to show up in the log—[b] submit a Support Case as per post #5 in this thread.  If [b] is necessary, and saskia acts fast, maybe the Retrospect engineers can put the fix into Retrospect Mac 14.5/Retrospect Windows 12.5, which based on past history are due to be released in early September.



#13 saskia

saskia

    Member

  • Members
  • PipPip
  • 10 posts

Posted 26 July 2017 - 03:05 PM

Message from Support:

We have reproduced the issue with the external scripting and we have written a bug for the problem.  We hope to fix this in our next update. 



#14 David Hertzberg

David Hertzberg

    Occasional Forum Poster

  • Members
  • 410 posts
  • LocationNew York, NY

Posted 26 July 2017 - 09:03 PM

Message from Support:

We have reproduced the issue with the external scripting and we have written a bug for the problem.  We hope to fix this in our next update. 

 

 

Congrats, saskia.  

 

You might consider adding to your Support Case—which I assume you have already created—an Additional Note requesting that Retrospect Support add back to the .ZIP file archive mentioned in posts #4 and #7 in this thread the stand-alone sample_sh Bash file that used to be there.  If they could add additional comments explaining the arguments for the functions, that would be great.

 

When you get your Bash shell script fully working, you might consider making it available to those of us Retrospect Mac administrators who don't want to use IFTTT or Slack to monitor the results of complicated combinations of Backup scripts.  You might even consider charging money for it, especially if you can develop a version that would work for Retrospect Windows administrators, although I doubt it would make you independently wealthy (insert appropriate smiley here).

 

BTW, folks, when copying from other documents—such as e-mails or Support Cases—into Forum posts, your pasted text will frequently end up in a smaller-size font.  See, for instance, Saskia's post #13 in this thread.  The way to counter this is to select the pasted text, and then use the Font and Size dropdown in the Forums editing window to change the pasted text to font Arial and size 14.  I did that to the quote from post #13 at the top of this post.



#15 David Hertzberg

David Hertzberg

    Occasional Forum Poster

  • Members
  • 410 posts
  • LocationNew York, NY

Posted 29 July 2017 - 09:02 AM

I now have an explanation for the "anti-miracle" I referred to in post #7 in this thread, and probably also for some of the problems saskia is having.  I was tipped off to it when I looked at this Cult of Mac article, which is linked to from Retrospect.com's own Press page.  The article—which is about Retrospect Mac 14—is "Sponsored", which evidently means that Retrospect Inc. itself essentially wrote it.

 

The article says "Speaking of integrations, the Mac version of Retrospect now features the same script hook functionality as the Windows version."  And, by Gadfrey, the bottom of the Knowledge Base article linked to in posts #4 and #7 in this thread says "Last Update: January 27, 2016".  Sure enough, there is an "External Scripting" section starting on page 488 of the Retrospect Windows 11 Users Guide; in case you didn't know, Retrospect Windows 11 is a now-obsoleted version released in early 2016.  The same section now appears on page 484 of the Retrospect Windows 12 UG, which is the current version.

 

So script hooks are not new for Retrospect Windows, only for Retrospect Mac 14 and only the term "script hooks".  IMHO this means the Retrospect engineers probably only tested the Retrospect Mac script hooks for Mac clients—and probably not that thoroughly, and didn't also test whether script hooks for a Mac "backup server" Engine would also work with Windows clients.  It also explains why the .ZIP file linked to in the Knowledge Base was only updated on the evening of 20 July 2017, and likely updated in a hurry—which IMHO would explain why the free-standing sample_sh file was deleted instead of receiving any necessary revisions.



#16 David Hertzberg

David Hertzberg

    Occasional Forum Poster

  • Members
  • 410 posts
  • LocationNew York, NY

Posted 25 August 2017 - 03:38 AM

The "anti-miracle" I referred to in post #7 in this thread has now been reversed by a "positive miracle".  I happened to go through recent entries in the Retrospect Blog tonight, and came across a duplication of the Knowledge Base article.  When I clicked on the link halfway-down the article copy, it downloaded a .ZIP file that does expand to examples that now include a Sample sh folder.  That folder contains a RetroEventHandler.sh file that looks to my misty memory like a Bash file.  It's got lots of comments.  The first of the echo lines at the bottom of that file says "echo "This is a sample Retrospect external script written in ruby.""  However, after taking a fast look at the Wikipedia article on Ruby, I don't think the language of the file is Ruby; I think the language is some variety of Bourne shell scripting.   

 

Have fun, saskia.



#17 saskia

saskia

    Member

  • Members
  • PipPip
  • 10 posts

Posted 06 September 2017 - 04:29 PM

It appears the problem is not fixed in the 14.5 release, bug re-opened.



#18 saskia

saskia

    Member

  • Members
  • PipPip
  • 10 posts

Posted 12 September 2017 - 06:56 PM

Windows client backup events are only recorded when only part of a volume is tagged for backup. Client version appears irrelevant, same behaviour with the 8.2 and 12.5 clients.






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users