6.3.019 Clients still turning off

[jonnybrad2001]

Had a long running problem with the clients turning off. Thought a earlier update was meant to fix this but still getting this now, including on desktop machines with Airport turned off.

 

Any methods to fix this for good?

 

TIA

Jon

 

Edit

Clients turned back on today

iMac C2D 2.0GHZ 10.4.11

iMac C2D 2.4GHZ 10.5.4

Both with Airport off, both running 6.3.019 Agent

Edited September 7, 2009 by Guest

[CallMeDave]

Clients turned back on today

 

You post is a bit confusing; two minutes after reporting an issue with "clients" (no reference to how many) you report that two clients turned "back on."

 

Since the retroclient daemon still uses the now-depreciated /Library/StartupItems/ folder for boot-time launching (instead of the preferred launchd process Apple now recommends for daemons) it would be impossible for retroclient to restart itself after a crash.

 

A (much) more complete description of your observations, experiences and actions would be warranted.

[jonnybrad2001]

Sorry, maybe what that should have said is 'Clients I've had to turn back on today' or 'Clients that have turned themselves off today'.

 

I'll try and keep updating as I note any further clients turn off. Seems easier to track now in v8 than it was under v6.

[Tree]

I have reported this occurence as well - clients installed on machines that otherwise stay powered on 24/7 that, seemingly randomly, toggle to "off" and I must manually go to that computer, launch the Retro Client app, and toggle it back to "on". This has happened to clients installed on both 10.4.11 machines and 10.5.7 machines, both iMac G5's and Intel iMacs. It has happened without any rebooting or crash of the computer, as far as I know. Curiously, it does seem to happen to particular machines only; certain machines, with the same client installed and the same OS, same iMac configuration, seem to never have a problem.

[Tree]

Okay, so in an effort to try to diagnose a common factor related to the Client reverting to "off", I decided that one thing I can try is to isolate the precise time when a given client disappears. To do so, I thought of creating a "pinging" script that would backup the same trial folder with its contents of one little text file, every hour. That way, I can see when the red "X" shows up and narrow things down to see whether anything else happened in that time frame, that might coincide with the failure.

 

I invoked the Backup Assistant to create a script, after having set up the target folder and file on the client machine (this is one of the machines that has had a failing client repeatedly). I was walked through the process, then at the end I chose to "Save" my little script. You wanna know what got saved?

 

It saved the name of the script. Pretty much nothing else. It forgot the Source because I had selected the files manually - there was a handy warning right there when I clicked Save to tell me that this would happen. Okay, so I really do need to declare that target folder as a Source beforehand, can't save a manual file selection as part of a script, I guess I can tolerate that.

 

But it also had no Media Set, after I specifically created a "RetroPing" file media set while walking though the script wizard! So I had to go select that media set.

 

And, it had no Schedule, although I had fiddled with it just prior to hitting "Save" to set it up to run every 1 hour starting in a few minutes! So basically, all that handholding by the Wizard was worth next to nothing, I've had to recreate the script entirely, save only typing in its name.

 

Now that the forum has been down a couple days, I have had a chance to run this script and so far, the client has not lost connectivity. I'm going to add in another of the problematic clients to this same script, and see if maybe this frequent pinging is sufficient to keep the client turned on. Obviously, I'd prefer to know that the underlying problem is dealt with in either a patch or fresh client version, but for now I'll make do with this. I'm still not sure this actually works, as I said I've only been running it since Sept. 9; I'll chime in here again later on to let you know if it works for a full week or more.

[rhwalker]

Consider the possibility that the "off" reversion is related to client going to sleep and then waking up. Your test might prevent that.

 

Russ

[jaduke]

Consider the possibility that the "off" reversion is related to client going to sleep and then waking up. Your test might prevent that.

 

Russ

 

Russ,

 

I've got a similar problem on my mail server where the client just is off. The machine is running 24x7 with processor, screen and drives set to never sleep. And regularly, the client just turns off.

 

In my case, the specs are:

PowerMac G4 Dual 800 Quicksilver

1.5GB RAM

Mac OS X 10.4.11 with all current patches

Retro Client 6.1.130

 

I realize that the client isn't the newest out there, but it was running fine for the longest time (2 years) before starting to shut off without warning.

 

Cheers,

Jon

[rhwalker]

I've got a similar problem on my mail server where the client just is off. The machine is running 24x7 with processor, screen and drives set to never sleep. And regularly, the client just turns off.

 

In my case, the specs are:

PowerMac G4 Dual 800 Quicksilver

1.5GB RAM

Mac OS X 10.4.11 with all current patches

Retro Client 6.1.130

 

I realize that the client isn't the newest out there, but it was running fine for the longest time (2 years) before starting to shut off without warning.

Well, my suspicion is that the problem is happening at the client, not at the Retrospect server, and is caused by the sleep/wakeup cycle.

 

Even though the client has been "running fine" for two years, the client's environment has changed because of the updates to Mac OS 10.4.x during that time. I've never seen this, but, again, our environment is a bit different and the clients don't ever experience this sleep condition. We wake the clients by a script prior to their backup time, and then have them shut down when the backup is finished.

 

Retrospect runs on our mail server, like yours, and the mail server never shuts down.

 

Russ

[CallMeDave]

...toggle to "off" reverting to "off"

...the client just is off

...starting to shut off

 

The Retrospect Client application (Retrospect Client.app) has what is probably the simplest, feature-minimilest application on my entire computer.

 

There is the Window title bar, the EMC logo, an on/off pair of radio buttons, application name (repeating the window title bar), version number, preference button, client name, status, and history. That's it.

 

And yet, in all of these reports, not -one- poster has seen fit to include the message displayed in the "Status" field.

 

Which just seems odd to me.

 

I mean, when asking others to try and visualize a problem, why would someone leave out information about the status of the program that is misbehaving?

 

This has been going on for years, so there must be some reason; I just can't fathom it.

 

 

Dave

[Tree]

Regarding the Status field, all I am seeing is that it reports "Not Running" while the radio button is toggled "Off", and then "Ready" once toggled to "On". I haven't reported this because it seemed insignificant and even redundant; neither status message is other than what I would expect for the given state.

 

As far as sleep cycles go, that was my first thought a long time back... like, back before ever upgrading to R8. We were having problems with client shutoffs when using R5.175, and from scouring forums I figured out that it was a common problem and one that seemed even worse in R6. I adopted R8 thinking that, as it had been a common complaint in R6, it might be somehow addressed. Anyways, back in R5 days I made sure that all machines were set to "Never" go to sleep, only allowing displays to sleep, in an attempt to deal with client access issues. And I have made sure that all problem clients today are on machines set to "Never" go to sleep.

 

Progress update - over the weekend, none of the "pinged" clients has lost connectivity, so this may be a viable workaround. Although a failure might be better, as it would help to pinpoint what was happening to cause the reversion to "off"! I'll keep ya posted.

[CallMeDave]

neither status message is other than what I would expect for the given state.

 

1- Launch Retrospect Client.app

2- If Status reads "Ready," Click on the "Off" radio button

3- Note Status

 

In other words, the "Status" field is the reflection of the true "state" of the client backup software daemon; the radio button is not.

 

 

 

 

 

Dave

[Tree]

What I have been doing, after discovering that a backup has failed, is to go to the client machine and 1. Launch it, 2. See that the Status says "Not Running" as reported above, with the radio button in the "Off" position, then 3. Toggle radio button to "On" at which point it says that the Status is "Ready". I had not, until now, tried going the other direction, as you request.

 

Now that I have done so, I do indeed see a different status reported. After 1. Launch the Client and 2. see that it reads "Ready", I click the radio button "Off" and 3. note that the status says "Turned Off". So, there is a difference between "Turned Off" and "Not Running", as far as the Status nomenclature goes.

 

Still, when it fails to that "Not Running" status, the On/Off radio button does change from On to Off, so it is not inaccurate to say that the client turned off - something caused that change to happen, and that change did happen. It just so happens that the most prominently visible symptom is a bright shiny button labeled "off", which is why people are gonna show up here and report that their client is turned off.

 

But I can fairly confidently now state that my problem clients are reverting to a "Not Running" status, that just happens to resemble an "Off" state.

[gmerritt]

Same problem as original post. This is happening on about a dozen Macs running 6.3.019.

 

I see that the highest version of Client 6.* for Mac is now only "Retrospect Client for Macintosh 6.2.234 2008-10-31" -- see here:

 

http://www.retrospect.com/supportupdates/updates/?path%5B%5D=updates#UPDATETYPE8

 

Was 6.3.019 pulled due to this problem, and replaced by 6.2.234? Should I downgrade to 6.2.234? Is 6.2.234 safe from the recently-revealed security hole in 6.*?

 

Thanks!

 

-Greg

[rhwalker]

Same problem as original post. This is happening on about a dozen Macs running 6.3.019.

 

I see that the highest version of Client 6.* for Mac is now only "Retrospect Client for Macintosh 6.2.234 2008-10-31" -- see here:

 

http://www.retrospect.com/supportupdates/updates/?path%5B%5D=updates#UPDATETYPE8

 

Was 6.3.019 pulled due to this problem, and replaced by 6.2.234? Should I downgrade to 6.2.234? Is 6.2.234 safe from the recently-revealed security hole in 6.*?

Scroll further down the page on the same URL link you provided, and you will see that the current Retrospect Mac client is 6.3.023 (released September 14, 2009). The direct link, without scrolling, is:

Retrospect 8 for Macintosh - updates

 

Of course, there are no release notes. Documentation seems to be a thing of the past for Retrospect 8. Sigh.

 

We've never had the "Client turning off" issue, but our configuration and use may be different from yours.

 

Russ

[CallMeDave]

the recently-revealed security hole in 6.*?

 

I try to keep up on this sort of thing, but could you tell us exactly what/where/by-whom has been "recently-revealed" about "6.*" (something) ?

 

Dave

[rhwalker]

Recent is relative.

 

There's this item from 2006:

Retrospect client updates for buffer overflow vulnerability

 

And also this item from a year ago:

EMC Retrospect 7 Backup Client 7.5.116 and Backup Server vulnerabilities

Universal Macintosh client 6.1.230 now available

 

There was a hasty update of Retrospect 8.1.150, whereby it could erase all of the tapes in your library if you selected an empty slot and clicked "erase", but that's probably not considered a "security" update:

 

Fixes in EMC Retrospect 8.1 build 150

 

22919: Selecting empty library slot and clicking erase, erases all tapes in library

 

There were also the fact that the Retrospect 6.x client's pitond was installed setuid root and world writable back in 2004-2005. While fixed back in 2005 or so, subsequent updaters and installers did not correct this issue, so some long-time users may have an old pitond that is still setuid root and world writable. See:

OS X client 6.0.108 installed setuid root and world-writable

 

Of course, there is the issue that you can do weak (or no) encryption on a backup set that could be cracked given time, and I haven't seen any analysis whether the easy-to-cause crashes of Retrospect 8 console and engine might open a possible vulnerability hole.

 

Russ

Edited September 30, 2009 by Guest

[Tree]

Here is some further information which might help figure out what is going on with the "Not Running" (off) clients:

 

I just upgraded to build 8.1.525 / 8.1.526, and also upgraded clients to 6.3.023. I had hoped that maybe the new client (or even just the fresh install) would fix things, but I still have the same problems. I can reproduce it at whim now - on certain machines, every time they restart, after restarting the client's status is "Not Running". This persists after upgrading the client.

 

Moreover, when I click the radio button to turn it back "on" (status = ready), I am prompted yet again as to whether or not to allow the client to receive network connections (i.e. set a firewall exception). This had not been happening before the client upgrade; I just set the firewall exception once when the 6.3.019 client was installed, and thereafter all I did was click the radio button to "on" when needed. There was no prompt for the firewall, and there was no problem with connection between Engine and Client after clicking the "on" button.

 

Another thing I noticed just recently, prior to upgrading clients, was a new twist (as far as I know) - it was a client that still showed "ready" status and was "on", yet was reported in the console as being "source unavailable". I clicked the "+" button to try to add that source over again, but rather than follow through with that I saw a button to test an address. I clicked that and tested, and it reported the client fine, no real delay that I saw, so I thought maybe the test cleared up whatever logjam was making the console report "unavailable". But nope, when I tried to browse the source again, I got the "unavailable" message again. All of that was prior to upgrading both Engine and Client (and Console, of course), and since upgrading I've not seen it happen again.

 

One other thing I saw was that, while most clients I just installed from a locally-mounted DMG file, requiring a restart, for one client I decided to use the "update" feature built into the console. The user at that machine was prompted about the firewall exceptions, and fortunately made the correct choice on that dialogue, but then was not required to restart. However, on my end, the console reported that client as now being "Busy". This did not clear up after the client user restarted his machine, as I asked him to, but did clear up when I quit and then relaunched the Console.

 

Console and Engine are both installed on my machine, an Intel iMac 2.8ghz Core 2 Duo, and the client that I was testing for "not running" status was on a PPC iMac G5, both running OS 10.5.8. There are other machines in our office, some running 10.4.11, and a mix of Intel and PPC iMacs, and I've seen the "not running" after restart issue on others, however not on all machines.

[rhwalker]

That's the most concise statement of the issue I have ever seen. The reason that I haven't seen it, I suspect, is that all of our client stuff here is running 10.4.11 (or Server 10.4.11).

 

That should enable EMC to find and fix the problem (which is at the client end, from your clear post and analysis).

 

I suspect that you might be able to solve the issue on that one client by manually adding the client to the Leopard firewall exceptions. There are some posts here by CallMeDave on that process, because it was seen early on after Leopard was introduced.

 

Russ

[CallMeDave]

I can reproduce it at whim now - on certain machines, every time they restart, after restarting the client's status is "Not Running".

 

That's not reproducing the client process crashing, that's reproducing the client process not starting.

 

If you cold-start the machine (without even logging a user in) and "retroclient" doesn't make it to the short list of administrator owned processes, you have a different (fairly common, user instigated) issue.

 

Dave

[rhwalker]

If you cold-start the machine (without even logging a user in) and "retroclient" doesn't make it to the short list of administrator owned processes, you have a different (fairly common, user instigated) issue.

Ok, let's investigate that.

 

while most clients I just installed from a locally-mounted DMG file

Where did you put the client?

 

Did you move or rename the client?

 

Russ

[Tree]

I copied the DMG file to the user's desktop, double click to mount it, run the installer and accept the defaults to install into Applications folder, then upon finishing was prompted to restart. After restarting, obviously the DMG was no longer mounted, I go to the folder in Applications and double click the client to bring it up and see whether it is working, and see the "not running" status.

 

Maybe it is significant, too, that all of our machines have multiple users defined, though only 2 to 4 users per computer. At least two of these users have admin privileges, including the login used during the install, and one other "skeleton key" common login that I can use anywhere in the office. But I'm not installing into a specific user's folder, I'm using the general Applications folder, nor am I moving anything related to Retrospect subsequent to installation.

 

The one machine that was upgraded via the Console rather than DMG still exhibits the same behavior; i.e. I have both DMG-installed and Console-upgraded clients that revert to "not running" upon restart.

[rhwalker]

Shouldn't be an issue because the run portion of the client is setuid root. In fact, this has been an issue with earlier versions of the client because the client was not only setuid root, but also world writable. That was fixed, I believe, back in about 2005:

OS X client 6.0.108 installed setuid root and world-writable

 

To check, in Terminal:

sudo find /Applications -perm -4000 -ls

 

Russ

 

[CallMeDave]

After restarting, obviously the DMG was no longer mounted, I go to the folder in Applications and double click the client to bring it up and see whether it is working, and see the "not running" status.

 

What do you see within:

 

/Library/StartupItems/RetroClient/ ?

[Tree]

I find these two files in StartupItems/Retroclient/:

 

RetroClient

StartupParameters.plist

 

Both are dated 3/06/08, and I discovered this: when I have turned a given client "on" by clicking the radio button, everything working as it should, then if I go and double-click on RetroClient here it will report in Terminal that the Client is already running. However, after a restart, when the Client Application itself reports "not running" as status, if I go into StartupItems and double-click RetroClient, I get the Terminal window just reporting "Process completed", and the Client status becomes "ready". I.e., invoking RetroClient in StartupItems has the same effect as launching the application and clicking "on".

 

So now the question becomes, why is this apparently NOT getting invoked upon restart, since it is actually sitting there in StartupItems?

[CallMeDave]

So now the question becomes, why is this apparently NOT getting invoked upon restart, since it is actually sitting there in StartupItems

?

 

Any other applications using /Library/StartupItems/ ? Do they work as expected?