6.3.019 Clients still turning off

[rhwalker]

could you post the contents of those two files in /Library/StartupItems/RetroClient/ and also a listing from Terminal ( ls -al /Library/StartupItems/RetroClient/ ) of their permissions?

[Tree]

I did some hunting for how to enable/disable things sitting in StartupItems, and came across what appears to be a solution!

 

To enable a startup item that has been disabled, you have to remove a hidden file called ".disabled". To do so, the instructions I found were to open Terminal, and get into the folder by typing

 

cd /Library/StartupItems/RetroClient/

 

Then, type the following:

 

sudo rm .disabled

 

It will require your admin-level password, so make sure you know it! Then quit Terminal, and restart the computer.

 

Having done this, the computer will now attempt to process those startup items. What happened for me is that it detected improper security settings (i.e. unix permissions) and prompted me to Decide Later, Disable, or Fix. My guess is that somewhere along the line, a user just got tired of seeing the same message every restart and clicked "disable", thus setting that hidden file. I know it hasn't happened since this past week, when I reinstalled the Client from the DMG, but I guess the hidden file persists in the RetroClient folder, the install process doesn't remove it for us (or just wipe out the whole RetroClient folder in StartupItems to create a new one).

 

Anyways, this time around I chose "Fix" instead, which required a restart, and after restart the Client showed "Ready" status as hoped. I restarted a couple times after that, too, to make sure I wasn't going to have to repair permissions ("Fix") with every restart, and that it came up to "ready" status each time. So far so good!

 

So after all that, it comes down to incorrect security settings for the RetroClient in StartupItems (it also prompted me to Fix StartupItems/.DS_Store). That makes me wonder how they could get set incorrectly to begin with; and perhaps the reason I don't see this issue on all computers is just because some users have selected "Fix" while others have selected "Disable". But this dialogue box shouldn't come up in the first place, right? It should install with the correct permissions, Owner, and Group, right?

 

As to the other question, regarding other apps in StartupItems, on the particular machines I'm paying attention to there is nothing else in StartupItems.

[CallMeDave]

Apple added the feature in Tiger to check for specific permissions for items within /Library/StartupItems. At the time, the Retrospect Client installer wasn't setting things the way it should, so the OS was offering the choices you see.

 

That makes me wonder how they could get set incorrectly to begin with

Cruft.

 

Likely at some point you ran that older version of the client installer under 10.4 or later.

 

Should later versions of the InstallerVISE script know about that .disabled file and remove it? I'd say yes, but whoever write the script probably didn't know about it him/herself; I certainly didn't. But even the current installer script obviously uses an existing RetroClient directory if it's there, although the current script also sets permissions correctly.

 

There is supposed to be a new Apple Installer based Client package coming; but if it still uses this legacy /StartupItems/ folder instead of launchd, I'd say this is a potential bug that should be fixed for it.

 

Dave

[Tree]

Well I might have spoken too soon! While the "Fix" option did get it such that the Client does reach its "Ready" state after restart, now something else mysterious is happening.

 

I cannot connect to that specific client any more.

 

The Console reports "Source Unavailable" if I try to browse it, and scripts that were to have run overnight failed. I can check everything at the Client computer and see that it is correct - same IP address with active LAN/Internet connection, Client still "Ready". Yet from the Console, if I try to Add Source and there use the Test Address button, I'll get the message "No client found at this address (-519)".

 

I tried turning the Client off and then on again, including the new trick of Option+Clicking the OFF button, but no change. I stopped and then restarted the Engine, but no change.

 

So, by "Fixing" the Client, have I broken it? I guess I will try to reinstall, see what happens.

 

EDIT: As expected, following a fresh install of Client from DMG, communication with that Client was restored. I did not Restart this computer yet, though, despite the prompt at the end of the install process; this is because that machine is in use and the restart would be disruptive. A consequence of not restarting, though, is that the DMG has stayed mounted; if I try to unmount it I get "disk is in use" message, even though I option-quit the Client and relaunched it from the Applications folder. I will restart the machine whenever there is a window to do so, and I'll have to see if I get the same choice to "Fix" incorrect permissions then.

Edited October 7, 2009 by Guest

[CallMeDave]

the DMG has stayed mounted; if I try to unmount it I get "disk is in use" message, even though I option-quit the Client and relaunched it from the Applications folder

 

The problem is the "retroclient" unix daemon is running from within the package on the mounted disk image; been a longstand stupidity of the Vise Installer script.

 

Command + Click on the "off" radio button in the Retrospect Client.app application, then unmount the disk image. Then click on the "on" radio button.

 

 

[Tree]

You're right - I got mixed up thinking it was Option+Click OFF; after Control+Click OFF I was able to unmount the DMG fine. Still haven't been able to restart that machine, however, so the question remains whether I'll get that "Fix" "Disable" or "Decide Later" alert after the OS tries to process StartupItems again.

 

Good news is that unmounting the DMG has not negatively affected communication, not that I expected it to, it just offers me confidence that I'm actually communicating with the installed Client and not with something from the installer disk image.

[Tree]

Okay, so that didn't work.

 

This thread seems to be wandering all over the place, but it's because whatever this problem is, it just keeps morphing like a resilient virus into something else just as soon as any potential solution is applied.

 

After restarting, though the Client does resume "ready" state and there is no prompt to "Fix" any security settings (good, that means things are correctly installed, right?!?) I still get the Console reporting to me that there is "no client found at this address (-519)". The only way I can get past that (now) is to Control+Click OFF, then click ON again, and allow the application to receive network communication (i.e. firewall setting) when it asks me about that. Never mind that there is already an application-specific allowance in the firewall settings for Retrospect, there since time immemorial (i.e. the first time I installed the dang thing). But, even after confirming that I do wish to allow such connections, and finally connecting in the Console, if I restart that client machine again, it's back to the same condition: no client found.

 

I manually added an exception to the Firewall settings (to be clear I am talking about Mac OS X's built in firewall, we don't have any other firewall to deal with on this LAN), locating the Retrospect Client application and adding it to the list (retroclient was already included). So... now when I Control+Click OFF then ON again, I get prompted TWICE if I want to allow incoming connections (both indicate that I am allowing "retroclient" to accept the connections). Still, if I do in fact allow both, then Console can talk to the Client - Yay!

 

But I get nothing after restart, until I manually CTRL+OFF & ON; this is Worse than before, because at least I could buzz the user, ask them to check their Client application, and when they said it was "off" (really "not running" but the button is labeled "off") I could just instruct them to turn it "on". Now, when they go to check, it says "Ready" and "On" as if all was well with the world.

 

So what can I do next? Oh golly, who knows. Maybe let's use the Uninstall option on the Client DMG, and then fully erase the folder in StartupItems if it leaves it there, basically purge any trace of the Client. Then try installing fresh once more.

 

Either that or tend to the big red bump, where I've been banging my head up against the wall.

[Tree]

..."Source Unavailable"

 

This just gets worse and worse! What is going on?!?

 

I mount the 6.3.023 Client installer DMG on the client machine, and run the installer, choosing "uninstall" to try to zero things out. It rockets through the process, and gets rid of every scrap of Retrospect that I can see, good. I restart the machine, after checking to see that Library/StartupItems/ is indeed empty. Now I have a blank slate, onto which I can try reinstalling the client to see if it behaves better than before.

 

This time around, I see a difference, as when I last installed the new Client I had not removed the old. Thus, I didn't have to enter the password, but this time, it treated it like a truly fresh install and I had to enter then confirm the password. I used the same password as before. I had to click "On", too, and Allow the client to receive incoming connections, and finally, restart.

 

Now, after restart, its status is "Waiting for first access".

 

Let's go check and see what things look like at the Console! The Source that had been defined previously is still listed there, of course, but I click to Browse it and it reports "Source Unavailable". Okay, maybe I just deleted the client it expected to talk to, and need to let it know that the freshly installed client belongs to this Source, that makes some sense. Only, how do I do that? The durned "Locate" button is frustratingly one of the only ones that stays grey and thus inaccessible; "Refresh" just reports "Source Unavailable" again. I try the Add button, and looky there, the list of potential Sources does NOT include the one I just installed. Shouldn't it?

 

I click the Test Address button, enter the IP, and get "No client found at this address". Say What?

 

I go back to the client computer, check its Firewall settings. There is now a third entry for Retroclient, allowing incoming connections, which must have been added when I affirmed this choice during the most recent install/activation (it gained a 2nd one yesterday when I tried to add one manually, to get the old client communicating). I do my trick, of CTRL+OFF then ON again, get the question as to whether or not to allow incoming and answer yes, and it gets back to "waiting for first access" status. Before I leave, I check Firewall settings for a fourth entry, but such was not created; 3 is fine for now, I guess.

 

Back at the Console, I first try to Browse the hard drive of the old Source, thinking that maybe Retrospect will see the new client at the same address now, and likely prompt me to enter the password to initiate that client, prior to browsing. Nope, instead I continue to just get "Source Unavailable". However, now when I click "Add" I can see that computer showing up as an available Source, and when I "Test Address" I get a prompt and accurate result that reports the correct client name and version. My list of Sources still only includes one iteration of that particular client, which I assume to be the "old" one since it lists the last backup date as prior to my whole reinstall attempt.

 

SO WHAT CAN I POSSIBLY DO NOW? Do I have to (shudder) remove that unavailable Source, screwing up a bunch of scripts, then Add the new version of itself, redefine all the favorites, repair the scripts, and possibly recycle some media sets to boot? Isn't there any way to avoid all of that? And even after I do all of that, I've not made any forward progress because I still had to CTRL+Click OFF then ON again before the Console could see the fresh client!

 

It's getting SO FAR REMOVED from the original problem that I've practically forgotten what the goal was. For my own sanity, I'll state that THE GOAL IS TO HAVE A CLIENT THAT IS ALWAYS FUNCTIONING AND AVAILABLE TO THE BACKUP ENGINE.

[rhwalker]

I mount the 6.3.023 Client installer DMG on the client machine, and run the installer, choosing "uninstall" to try to zero things out. It rockets through the process, and gets rid of every scrap of Retrospect that I can see, good. I restart the machine, after checking to see that Library/StartupItems/ is indeed empty. Now I have a blank slate, onto which I can try reinstalling the client to see if it behaves better than before.

I'm not sure that everything is removed - there may be some preferences floating around.

 

SO WHAT CAN I POSSIBLY DO NOW? Do I have to (shudder) remove that unavailable Source, screwing up a bunch of scripts, then Add the new version of itself, redefine all the favorites, repair the scripts, and possibly recycle some media sets to boot? Isn't there any way to avoid all of that?

Sadly, I think not. Thus far, everything seems to have gone how I would have expected.

 

There is not and never has been, as far as I know, a way to tell Retrospect that a new client is really an old (forgotten) client. Forgetting the client releases the license, and it's a new ball game from there on. The backups from the old client are still in the, um, media set, and could be restored to the new client at a later date.

 

Because I've never seen this issue, but others obviously have, I keep asking myself what is different between our installation and others. I truly believe that it's related to firewall issues on the client (we don't have software firewalls turned on - our LAN is pretty secure, and we've got pretty tight hardware firewalls around our perimeter). I seem to recall that these issues started happening around the time that Leopard was released, which has a different firewall operation than previous. Once all older versions of Retrospect client are truly gone, and the new version gets added "correctly" to the firewall (whatever that means), the problem seems to disappear for people. Might want to investigate in that direction.

 

I suspect, when testing is done at EMC (assuming that it is done) on this issue, the machines are relatively clean, without a history of upgrades / updates to the OS or Retrospect. I don't think that I can recall an instance where the client has been reported to turn off by itself where the firewall on the client machine was disabled.

 

If my theory is correct, this points to a defect in the installation process (whether failure to clean up debris from previous installs, etc., or failure to fully set things up for the new install) and not to a defect in the client itself.

 

Russ

[Tree]

So I went ahead and did it... I added the fresh client as if new, set up all the favorite folders and redirected scripts to use the new source, etc. But let me tell you what else I did, because things seem to be working now:

 

At the Client, the 3 different firewall exceptions troubled me; I know I added one myself, but why would the installer add the second instance? So, I turned off the Client (CTRL+OFF), closed it, then removed all 3 entries in the Firewall exceptions. Then, I opened the client back up (still in "not running" status), clicked the ON button, was prompted to allow incoming... TWICE. Clicked Allow each time, and afterwards I had a single entry in the Firewall exceptions.

 

I checked at the Console, and yes I could communicate with this Client. Time to reboot; I do restart the client machine, and afterwards the Console is still able to talk to the new client! In other words, it is not reverting to "Not Running" status, nor is it claiming that it is "Ready" yet still needing to be CTRL+OFF/ON cycled, and the OS isn't bugging me with "Fix Disable or Decide Later" prompts on restart! So the extra effort to redefine all the scripts, etc., seems worth it.

 

Unfortunately, this means that the fix, generally, for all the other computers in my office is to basically do the same things. That means uninstalling all the clients, removing Firewall exceptions, reinstalling them, rebuilding all the scripts, sources, etc... all without really knowing what was going on, though it does seem like it got narrowed down to a ".disable" file sitting in StartupItems, paired with some Firewall oddities that appear after trying to remove that file (or perhaps completely unrelated, just never showed up until I did).

 

Maybe it's time to move on to the next test-case computer, and see if there are any other ways things can get out of whack, while I try to work the Client kinks out.

[rhwalker]

It would be interesting to hear your results.

 

If I understand your steps correctly, it would seem that serious adjustments to the installer are needed to do a proper installation.

 

Some of the folklore in these forums is that you need to use the same version of installer to do the uninstall that was used to install the version being uninstalled. Now, while the reason for this folklore is understandable, considering all the process name changes and changes in launch mechanism that have happened over the life of the Retrospect Client, if this is true, it seems inexcusable to me and an example of incomplete testing and sloppy programming. Current / later versions of the installer need to be able to uninstall all prior versions of the Retrospect Client (and any debris left from prior installations), and then to do a proper clean install of their own version of the Retrospect Client. Just my thoughts.

 

I feel your pain.

 

Russ

[CallMeDave]

prompted to allow incoming... TWICE

 

I have no idea how this auto-add feature works in the Leopard firewall, but perhaps it gives one prompt for TCP and another for UDP, since the client listens on both.

 

 

Dave

[Tree]

Current / later versions of the installer need to be able to uninstall all prior versions of the Retrospect Client (and any debris left from prior installations), and then to do a proper clean install of their own version of the Retrospect Client.

 

Actually, I think the uninstaller needs a bit of a pause to it... I was surprised how it really didn't ask me to confirm anything or give me warning that I was essentially killing an active client by uninstalling it. If there was a prompt to ask me whether I wanted to retain some Preferences file, such that my fresh installation would be recognized as the "same" client as before, I could be spared the need to re-craft all the scripts, etc. As it was, Uninstall just proceeded and left things barren. I think it actually does remove enough for a proper install later.

 

The problem was that I never Uninstalled before, because I wanted to just upgrade the client that was there, without losing scripts and sources. That's the process that isn't getting rid of cruft like a hidden ".disabled" flag. I guess the only thing Uninstall didn't do that perhaps it should is to remove all Firewall exceptions pertaining to itself... if that is even possible.

 

You can't really tell users to Uninstall the previous Client prior to installing the new one, because there isn't any way to get the Console to accept the new one as the old one if you do that. So, the Upgrade/Update process needs to be smart enough to work out any kinks; either that, or give us a way to "bless" a new client as if it were the old (like Locating a Media Set).

 

[rhwalker]

Well, apparently a new Client installer was released today, too:

 

Retrospect 8.x Release Notes

 

FIXES IN EMC RETROSPECT 8.1 BUILD 622

...

 

--New Mac client installer

No indication of what changes were made, though.

 

Retrospect 8 documentation is missing in action...

 

Russ

[Tree]

prompted to allow incoming... TWICE

 

I have no idea how this auto-add feature works in the Leopard firewall, but perhaps it gives one prompt for TCP and another for UDP, since the client listens on both.

 

 

Dave

 

And from another thread I just saw, Mayoff said:

If the test is successful, then something is blocking the UDP packets used to locate clients over multicast/subnet broadcast.

 

Putting 2 and 2 together, I realize now that, at least for the OS 10.5.7 Firewall, the indicator of a successful installation will be that double prompt, right? Because the Firewall has to allow both TCP and UDP packets, and it might only allow one or the other type of connection if it only prompts once. Perhaps that is what I was seeing, when the Firewall exceptions shows one entry already, but then after CTRL+OFF/ON cycling the Client, I get another prompt to allow the incoming connection.

 

In fact, it makes me wonder whether I can bypass the entire Uninstall/Reinstall routine just by removing all the Firewall exceptions, and see if it triggers a double-invocation when I relaunch the Client...

 

Let me go try that on another computer. Will post results in an edit.

 

EDIT:

Well, I didn't get the double prompt.

 

But this procedure did work: First, I removed any ".disabled" flag using Terminal. Then, I reinstalled the Client from .DMG - this got me to the state where the client reports "ready" status yet Console cannot find it (click on Add Source then on the Test Address button, it says "no client found..."); I think it double-prompted me to allow connections at the end of this install, right before requesting the restart, but I didn't jot that down in my notes, might have just been a single. In any case, it got me back to the problem condition I had yesterday on another machine, which is where I needed to be to test the above.

 

After restart, I was at the "ready" but "not found" position, like I just mentioned. As before, by CTRL-clicking OFF and then ON again, I was able to get it to work, as I was prompted once to allow incoming connections, after which time the Console could see the Client. Next, I clicked CTRL+OFF and went to the Firewall settings; I removed the one instance labeled RetroClient (this is on a Mac OS 10.5.8 I should mention). Then, I launched the Client application again ("not running") and clicked ON, and was prompted just ONCE to allow incoming connections. After allowing this, Console could see the Client, and there was much rejoicing!

 

Finally, after restarting the machine, the Client came back to the same "ready" status, visible to the Console, and no prompt to "Fix" any security settings.

 

So, deleting the Firewall exception was effective! It just didn't cause a double-invocation to allow connections, as I was expecting.

 

 

Edited October 9, 2009 by Guest

[BHIAdmin]

Hi,

 

Am having the same problems with upgrading clients, so I thought I would share my experiences.

 

So, deleting the Firewall exception was effective! It just didn't cause a double-invocation to allow connections, as I was expecting.

 

This is what I have found to be the case. Leopard's application-based firewall causes the client install to not work right. Retrospect's installer needs updated to work with this common OS configuration. All 15 non-server Macs I have set to back up to my Retrospect engine on a Mac Mini have this configuration.

 

 

Before any upgrade of the client, I make sure to clear all Leopard firewall exceptions for 'retroclient' then install the new client over the old. I almost always get prompted TWICE for 'retroclient' to be allowed incoming connections. Then I restart. (I agree that the twice notification probably stems from TCP and UDP ports being needed.)

 

Sometimes, after restart, the client is still not seen by the engine. I open Activity Monitor and force quit the 'retroclient' process, clear the firewall again, then relaunch the Client app (will try the Ctrl-Click next time). Then I get prompted for the firewall settings again. This usually works for the problem Macs.

 

 

A better solution is to stop using the app-baased firewall in Leopard and move back to the port-based one using NoobProof to help configure it. If you use Apple Remote Desktop, you can distribute the NoobProof settings via remote shell whenever you want a change. This is what I use for my server setups.

 

The worst is if any of our network switches goes down during the day (we commonly have power outages). The engine completely forgets almost all the clients and the engine's machine needs to be rebooted.

 

Larry

[rhwalker]

Larry, have you tried the process with the new Retrospect Client / installer released today?

 

Retrospect 8.1.622 released

 

EMC Retrospect 8 Release Notes

 

FIXES IN EMC RETROSPECT 8.1 BUILD 622

...

--New Mac client installer

[CallMeDave]

I was surprised how it really didn't ask me to confirm anything or give me warning that I was essentially killing an active client by uninstalling it. If there was a prompt to ask me whether I wanted to retain some Preferences file, such that my fresh installation would be recognized as the "same" client as before

 

That's what uninstalling _is!_

 

The retroclient process lives in the Retrospect Client application package; the only thing that the installer needs to delete is the shell script in the StartupItems folder (which would have no effect if the application were not there) and the retroclient.state file, which holds the password and preferences.

 

If all you want is a fresh copy of the application just delete the one in /Applications and run the installer again. If you actually want to uninstall, which, by definition would include the .state file, then run the installers uninstall script.

 

 

Dave

[rhwalker]

The retroclient process lives in the Retrospect Client application package; the only thing that the installer needs to delete is the shell script in the StartupItems folder (which would have no effect if the application were not there) and the retroclient.state file, which holds the password and preferences.

 

If all you want is a fresh copy of the application just delete the one in /Applications and run the installer again. If you actually want to uninstall, which, by definition would include the .state file, then run the installers uninstall script.

All of which is covered, we hope, in the User's Guide...

 

Russ

[BHIAdmin]

Larry, have you tried the process with the new Retrospect Client / installer released today?

 

Yes. It works great (!), updating the Client and resetting the firewall. So far, so good on the Macs I've manually installed on. Big improvement.

 

However, seeing that it was now a metapackage, I tried to roll it out through Apple Remote Desktop. It installed; but, unless you share screens and manually click 'Allow' when the firewall dialog(s) comes up, the dialogs default to deny when they auto close after awhile, or if you force a restart. This leaves the client updated, but the firewall settings like before. The Firewall pane notes the 'retroclient' as Allow, but this is not the case. (I think it is referring to the previous retroclient.)

 

I have tried, in vain, to script a command to auto-add the newly updated 'retroclient' to the firewall by using a command like:

 

sudo /usr/libexec/ApplicationFirewall/socketfilterfw -t '/Applications/Retrospect Client.app/Contents/Resources/retroclient'

 

but the command just hangs the socket firewall, and fails. It would be really nice to be able to roll out updates via Apple Remote Desktop, while keeping the Client in Read-Only mode and not bothering the user.

 

The installer just needs a way to auto-click the firewall dialogs, which could be done with an osascript command to click the GUI buttons, and run via the posinstall package script. Better yet, if the firewall could be auto-updated via a shell command, then the client's Mac can be updated without a user being logged in (like during a nightly rolled-out upgrade of the Mac).

 

Larry

[BHIAdmin]

Sorry, spoke too soon. The only reliable way for the application firewall settings to stick on an upgraded client (through restart) is to make sure to erase the old 'retroclient' firewall setting prior to running the new installer, just like with the old one.

 

If I run the new installer (manually or through ARD), the client is updated, I am prompted to Allow the firewall setting (if logged in), and until a restart, is fully accessible by the engine. Once I restart client machine the engine sees the client, but can not access it. (It looks like the firewall setting is NOT being updated, even after the prompts, and probably still references the previous retroclient binary.)

 

If I use the Cmd-click-the-Off-button method of stopping the upgraded Client app (one that was installed without deleting the firewall settings first), delete the old firewall setting, then launch the process again to prompt the firewall dialog, the new setting is back to 'allow incoming'. (Though, some Macs default to block??)

 

Looks like Retrospect should sign their Client/retroclient apps and see about having them added by Apple to the signed app exceptions in the OS's default com.apple.alf.plist like Skype, World of Warcraft, etc. Then we would never have to deal with this.

 

Larry

 

 

[rhwalker]

Looks like Retrospect should sign their Client/retroclient apps and see about having them added by Apple to the signed app exceptions in the OS's default com.apple.alf.plist like Skype, World of Warcraft, etc. Then we would never have to deal with this.

Agreed, that is the right solution for many reasons.

 

Russ