Retrospect Client not being seen on Intel MacBook

[scotty321]

Quote:

---

The symptom is 100% reproducible. Restart any of these 3 after confirming that Retrospect Client is on ... and Retrospect Client will be off. All of these machines used to work fine. Since I didn't see it when it happened, I suspect that the problem crept in with one of the 10.4.x software updates. These machines are all on 10.4.8.

 

---

 

This is not the same problem that I am reporting. My Retrospect Client software always says that it is "on", but the Retrospect Server machine can't see it.

 

In your case, are you running the very latest version of Retrospect Client? Just an idea of something to check..

[scotty321]

Just another update here -- same problem at a completely different office:

15 Macs on the network, a mixture of Intel machines (Core Solo and Core Duo) and PowerPC machines. No laptops, all desktop machines. All have been backed up just fine by Retrospect for months. Today, we just installed a brand new iMac Core 2 Duo machine -- no migration assistant. The first Core 2 Duo Mac on the entire network. Installed the latest version of Retrospect Client from scratch. After the machine wakes from sleep, Retrospect Client says that it is "on" by Retrospect (Server) no longer sees it anymore. So as I guessed before, I believe this issue is isolated to Core 2 Duo Macs only.

Thanks,

Scott

[scotty321]

I just want everybody to know that I have done EXTENSIVE testing on this problem this week at a gigantic cost to myself, and this is the problem that EMC Insignia has yet to fix with Retrospect for Macintosh:

 

IF ANY INTEL CORE 2 DUO MACINTOSH GOES TO SLEEP WHILE RUNNING THE RETROSPECT CLIENT, WHEN IT AWAKES FROM SLEEP, THE RETROSPECT CLIENT SAYS THAT IT IS "ON" BUT IN FACT IT IS NO LONGER BROADCASTING ITSELF TO THE RETROSPECT SERVER. The Retrospect application on the server machine can NO LONGER see the Retrospect Client. The thing that causes this to happen is the Core 2 Duo client machine going to sleep. IT IS SLEEP WHICH CAUSES THIS PROBLEM. This happens on ALL Core 2 Duo Macintoshes, this is reproducible 100% of the time, and it happens in Mac OS X 10.4.8 and 10.4.9.

 

Now how in the heck do I report this to EMC Insignia without paying to report a bug? I'm not about to give them a credit card number and have them charge me, just so I can report a bug to them that their engineers should have caught themselves.

 

And Robin Mayoff, who religiously reads these forums, why are you not reporting this to your engineers?

 

Scott

[rhwalker]

Thanks, Scott. I appreciate how you have narrowed the problem down to such a concise and reproducible issue that should be easy for the Retrospect programmer(s) to address.

 

You are not the only user who has invested time in troubleshooting Retrospect issues.

 

Russ

[scotty321]

Thanks, Russ, I appreciate it.

[CallMeDave]

Quote:

---

I just want everybody to know that I have done EXTENSIVE testing on this problem this week at a gigantic cost to myself, and this is the problem that EMC Insignia has yet to fix with Retrospect for Macintosh:

 

If any Intel core 2 duo Macintosh goes to sleep while running the Retrospect client, when it awakes from sleep, the Retrospect client says that it is "on" but in fact it is no longer broadcasting itself to the Retrospect server

 

---

 

Wow, and it seems like only 17 days ago that you reported: "...as far as I can tell, it seems to happen after the machine goes to sleep."

 

I can only imagine the magnitude of the expense necessary to prove a null hypothesis on this one...

 

Quote:

---

And Robin Mayoff, who religiously reads these forums, why are you not reporting this to your engineers?

 

---

 

Yeah, Robin. It's 6:24 pm Pacific Daylight Time on a Friday; why haven't you logged or updated your bug database with this new specific information yet?

[scotty321]

Wow, Dave, you're so funny. Truly cracking me up over here. Can someone give this guy a comedy award of some sort?

 

Way to go, Dave, defending a multi-million dollar corporation that has put out a faulty product, completely ignored its Macintosh customers for years, and has all but abandoned Macintosh support altogether. There are also very credible reports on the Internet that all Macintosh development on Retrospect has stopped at the company and their Macintosh engineers have been dismissed. Yet here you are, defending EMC Insignia? A faceless corporation that cares so much about you, Dave. I'm sure they send you Christmas Cards every year and pay your monthly mortgage.

 

Meanwhile, this is a bug that has been present ever since the Core 2 Duo Intel Macs shipped in October 2006, so therefore the bug has been around for at least 5 months without even a single update from EMC Insignia. In fact, they haven't issued a single update to the Retrospect Client for Macintosh in almost a year. And it's consultants like myself who have to bear the brunt of EMC Insignia's disdain for Macintosh customers, because we're the ones who recommended the product to all of our clients and we're the ones who have to deal with the nightmare when the product fails to work as advertised.

[CallMeDave]

> Way to go, Dave, defending a multi-million dollar corporation...

 

I did no such thing. I was merely ribbing a post suggesting that it was costly to conduct what seems to me, to be an easily proven theory ("Retrospect will loose connectivity to any Core2Duo client that has awoken from sleep (to an ethernet connection)"). Especially considering that I suggested that you focus on answering this specific question weeks ago.

 

Then I defended a particular employee of that corporation, someone who has proven himself to be an advocate for that corporation's product(s).

 

> that has put out a faulty product,

 

If the newest hardware from Apple includes a specific incompatibility with the Retrospect OS X Client software, that doesn't mean that the software was "faulty" when it was released. It simply means that things have changed and the developer has not addressed the change.

 

> ...completely ignored its Macintosh customers for years,

 

They may not have provided as much attention as you and I wish that they had, but "completely ignored" is an insult to the men and women who have been providing what ever attention the company gave over the last few years.

 

> ...and has all but abandoned Macintosh support altogether.

 

As best as can be discerned from news reports and public announcements, the Windows product and the Macintosh product appear to now share about the same level of support. It's not very great.

 

 

> There are also very credible reports on the Internet that

> all Macintosh development on Retrospect has stopped at the

> company and their Macintosh engineers have been dismissed.

 

What does it take to make a report on the Internets "credible?"

 

Even though Larry Zulch left EMC shortly after writing his public letter to the Retrospect community, I have no reason to doubt his assertion that Laurie Gill is still involved in the Retrospect program. I don't hold out much hope that there'll ever be a true "new" version of the program, but I also don't doubt Robin's public announcement that there will be an update for Leopard. That takes development effort.

 

> Meanwhile, this is a bug that has been present ever since the Core 2

> Duo Intel Macs shipped in October 2006, so therefore the bug has been

> around for at least 5 months without even a single update from EMC.

 

If this is, in fact, a bug with this specific hardware, as you claim your tests confirm, then your timeline seems reasonable. Now I'm really angry!

 

> it's consultants like myself who have ... to deal with the

> nightmare when the product fails to work as advertised.

 

Oh. My. God.

 

You recommend products to your clients based on how they're _advertised?_

 

And they pay you money for that?

 

Dave

[scotty321]

Okay, truce. Dave really is funny (I mean that sincerely, not sarcastically).

 

Sorry for being an ass on some of these posts. I've just been really riled up about the issue because I've been getting kicked around by 2 very high-maintenance clients about this issue.

 

Dave is right that what little staff is left working on Retrospect has gotten the product to successfully work up until there was a new hardware release by Apple in October 2006.

 

And yes, if it takes them 5 months to still not address a bug caused by sleeping, then we're dealing with an EXTREMELY scaled-down staff here.

 

You're right, it would've been easy for me to test the sleeping thing, but I was actually testing a whole bunch of other things first to make sure that I could isolate it to just sleeping.

 

Ah well, I've said enough on the topic. I hope they fix this bug soon.

 

If anyone can get their voice across to the (remaining) engineers, can you please tell them about this Core 2 Duo bug?

 

Thanks especially to Russ & Dave for your comments.

 

Best,

Scott

[CallMeDave]

Quote:

---

if it takes them 5 months to still not address a bug caused by sleeping, then we're dealing with an EXTREMELY scaled-down staff here.

 

---

 

Or, the staff simply didn't have the steps to reproduce a problem that was reported in the field.

 

Often, a software developer depends on complete user generated reports to discover bugs. These can be received from tech support interactions (real-time and two-way). Or they can be found in online discussions, as long as those discussions are complete and accurate.

 

It was a couple of years or so ago that a problem was reported here on the Forum that Retrospect was painfully slow for some users. It took some back and forth before the common configuration was found to be RAID arrays, and only after that online discovery was made was Dantz (I _think_ it might have been before EMC) able to address it.

 

Claimed confirmation that this is a sleep/wake issue has only been reported this weekend. Perhaps EMCInsignia testing can try it now. Or perhaps they don't have ready access to Apple's newest machines, and will have to test it at the Apple third party testing lab in Cupertino.

 

Dave

[retrohaterz]

Hey guys... I have the same issue with Intel xServe's, and these certainly do not go to sleep.

 

The xServe's run 10.4.8 server (don't tell me to go to 10.4.9, that breaks some other AFP stuff) and the latest version of Client. The Retro server's run 10.4.9 and 6.1.126. ACL's don't have any effect on this issue cause I've tried it both ways, but for the record they're on. I could care less if the ACL is properly backed up or not, and I know about the issue with Retrospect not working with ACL's on 10.4.8 unless you use that hack.

 

So it starts on boot, I can verify that. I can launch the desktop client tool and it says Ready. I go to Retrospect Server on another Mac and I can connect to it from the Client window (and the client says "Connected"), I can configure it, and I can see what Volumes there are and what the clock offset is, so it's clearly communicating.

 

Than I go to the Volumes window, select the volume presented from Client, and try to subvolume it. I expect to see the contents of that volume appear in a window, just like this Mac does with each and every other copy of Retro Client on other servers.

 

I get the spinning gears and it monopolizes my CPU till it times out and complains about "Network Communication Failed, scanning incomplete, error -519".

 

I've tried the apple-click off then back on to recycle it. I have to do this frequently on my other servers where Client randomly turns itself off, but thats another story with no fix. It turns off, says "Not Running", then I can turn it back on and it says "Ready". Retrospect Server still complains about 519 errors, and none of my backups will run.

 

The funny thing is this used to work for a couple days, now it just doesn't. No, I changed nothing on the Xserve's, and nobody else did either. The Retrospect Servers would lose communication with the Client (only on my 2 new Intel servers) and I would go to the Xserver's to see what's up. If I try to launch the desktop app, it just bounces and bounces for a couple minutes then just stops. From the terminal, i can see that pitond is running (well, status sleep to be exact), and retropds.23 is running, with a capital R, status running, eating CPU. If I kill (-9) that process, the desktop app immediately pops up on screen (assuming it was still bouncing) and says "Ready", even tho nothing can talk to it.

 

I'm working on some more testing, cause this is a new issue for me, well the Intel part anyway.

 

I sympathize with the parent poster. EMC Dantz will only talk to you if you pay them (*so* arrogant), after the first 30 days. I've had this version for many months now, cause, well let's face it, no new versions have been released, development has essentially stopped (the company/devs are in denial about that; an OS compatibility shim or device driver update does not a new version make). This is the only avenue left and the compaay, officially, doesn't even read this. So what recourse do we have to get bugs fixed on a dead product?

[CallMeDave]

Given your previous angry rant that resulted in the Forum moderator locking the thread (a first for this board; congratulations), I'm hesitant to reply. But there seems to be some previously un-acknowledged behavior being seen that warrants further exposure in the chance that the root problem can be revealed.

 

> From the terminal, i can see that pitond is running (well, status

> sleep to be exact), and retropds.23 is running, with a capital R, status

> running, eating CPU. If I kill (-9) that process, the desktop app

> immediately pops up on screen (assuming it was still bouncing) and says

> "Ready", even tho nothing can talk to it.

 

The retropds.23 process will only launch when the Retrospect application is asking a client to provide file listings. If Retrospect isn't currently connected to or backing up a client, or if you've stopped attempting to Define a Subvolume, then retropds.23 should not be running.

 

So if you have in fact canceled out of Retrospect's Subvolume define attempt, and retropds.23 is still eating cycles, then it's apparent that the problem lies with this process.

 

- Is there any change in behavior if you kill pitond _and_ make sure retropds.23 is not running, then launch pitond again, then try and get a file list (via Configure-Volumes to make it easy)?

You can do it all remotely from the teminal via ssh, so the Retrospect Client.app behavior is not part of your test.

 

Dave

[Pants]

I've been suffering from this Core 2 Duo Retrospect Client turning itself off problem too. I'm afraid I don't have any firm information to reproduce it (I'm not near the machines at present), but it seems like scotty321's prognosis matches my experience too.

 

I'm not at all experienced in running shell scripts or whatever, but is it possible to have a scripted solution that - say, every hour - checks to see if Retrospect Client is turned on and, if not, turns it on?

 

If it is possible, then clear instructions on how to do it would be very much appreciated!

 

Unless, of course, Dantz decide that they might as well fix it...

 

Many thanks,

[scotty321]

Yep, this is a serious problem with absolutely no fix (except for restarting the computer -- try telling your clients to restart their laptop after every time they put it to sleep). If any Core 2 Duo Mac is put to sleep for a reasonable amount of time, Retrospect Client stops working even though it says it's "on" when the machine re-awakes from sleep. I've noticed that you can sleep a machine for a SHORT amount of time without this problem happening, but any lengthy amount of time causes this problem. It affects ALL Core 2 Duo Macs (laptops & desktops), and I have seen this happen on 15 different machines at different client locations.

 

 

[rhwalker]

There has been extensive discussion of this issue this past week on the retro-talk list. The workaround that seems to work is to make a launchd item that will keep pitond running, relaunching it every time it dies. The easiest way to do this is to use the freeware utility "lingon":

lingon

 

You can search the retro-talk archives for the discussion.

 

I suggest that EMC support may want to write up a KnowledgeBase article on how to do this workaround if the bug is not going to be addressed.

 

Russ

[Mayoff]

This issue will get addressed. We have been taking info from the Retro-Talk thread and adding it to our bug report.

 

I think the workaround is the best option for right now.

[rhwalker]

For those of you who don't get the retro-talk list emails, here is the message with the workaround (although it's much simpler just to use lingon to create a launchd item for pitond rather than crafting a plist, etc.):

retro-talk message with workaround

 

Here is the start of the full thread for anyone who wants to read it by cycling through the "next message" stuff. There are details of tests and, shall we say, a bit of testiness between some posters:

retro-talk thread

 

Hope this helps. Just use lingon if you've got an Intel Retrospect client and are seeing this problem:

lingon

 

Russ

[CallMeDave]

Quote:

---

There has been extensive discussion of this issue this past week on the retro-talk list.

 

---

 

No, I'm not sure that the current discussion on RetroTALK is the same as this, although getting detailed, accurate and standardized information from the people effected is like hearding cats.

 

On the mailing lists, users are reporting pitond dying when physical network interfaces are changed in certain ways. That has been a known issue in the KnowledgeBase for a while, yet there has been no indication, so far, in this thread, that network connections are being changed in advance of seeing the problem.

 

And what scotty321 wrote in an early post:

> If I launch the Retrospect Client, it actually says that it is ON and READY to go!

would suggest that pitond is still running.

 

> If any Core 2 Duo Mac is put to sleep for a reasonable amount of time,

> Retrospect Client stops working even though it says it's "on" when the machine

> re-awakes from sleep.

snip

> I have seen this happen on 15 different machines at different client locations.

 

What, exactly, have you seen? What, exactly, do you do when you see it?

 

- Are the physical network interfaces changed before, during or after you see what you see?

- Is pitond running, as shown via the shell's "top" command?

- Is any other "retro" process, such as retropds.23, running?

- Was Retrospect connected to one or all of these 15 machines prior to them going to sleep?

 

What. Were. The. Exact. Steps. Taken. When. You. Saw. Something. Unexpected. On. Any. Or. All. Of. The. 15. Machines. You. Have. Seen. It. On?

 

It's understood that the Retrospect application can't connect to your machines after those machines wake from sleep. But knowing more about what's happening on those clients, above and beyond the marginal information provided by the client application's "on" radio button, would go a long way towards identifying what's happening.

 

 

Dave

(who gets testy around cats)

[scotty321]

Here's what happens:

 

1. Retrospect on the server machine sees the Core 2 Duo client machine just fine.

2. Core 2 Duo machine running Retrospect Client goes to sleep. It MUST be asleep for a lengthy amount of time. I have not pinpointed the exact time yet, but I think it's at least 30 minutes.

3. Core 2 Duo machine wakes from sleep after its lengthy sleep. Launching the Retrospect Client application shows that the "on" button is still highlighted, but...

4. Retrospect on the server machine NO LONGER SEES the Core 2 Duo client machine anymore. It is listed in the Backup Client Database temporarily, but when you click on the "Network" button, it does not show up.

 

There are NO physical network changes because this is happening on desktop machines that are assigned static IP addresses!! (I have informed my clients to not sleep their desktop machines so this is no longer a problem with the desktop machines b/c they just stay awake all the time. However, this does happen on laptops when their lids are closed. With the laptops, their physical network location MAY CHANGE because they may take the laptop to several different wireless & wired locations before returning back to the office where it is getting backed up.)

 

I would be happy to give you the information on whether pitond is running or any other retro process is running, but I do not know how to run the shell top command or how to interpret the results of such a command. If you can give me detailed instructions, I can try to troubleshoot this further.

 

As far as I know, Retrospect is NOT connected to any of these machines at the same time that they are going to sleep.

 

(These 15 machines are not at the same location. These are 15 Core 2 Duo machine spread out at different client locations, who are all experiencing this same problem. Please note that this problem does not happen on any Intel Macs besides the Core 2 Duo Macs. It is only a Core 2 Duo problem. It does not happen on the Core Duos.)

[rhwalker]

Quote:

---

I would be happy to give you the information on whether pitond is running or any other retro process is running, but I do not know how to run the shell top command or how to interpret the results of such a command. If you can give me detailed instructions, I can try to troubleshoot this further.

 

---

One easy way is to open a Terminal window, make it "real wide" (because the ps axl output is wide, and fgrep will get a truncated line otherwise) and then use the command:

ps axl | fgrep pitond

 

Here's an example (shows both pitond and the fgrep command processing it):

Code:

---

rhwimac:~ rhwalker$ ps axl | fgrep pitond

0 163 1 0 31 0 30848 1084 - S ?? 5:29.98 /Applications/Retrospect Client.app/Contents/Resources/pitond

501 1925 1921 0 31 0 18052 288 - R+ p1 0:00.01 fgrep pitond

rhwimac:~ rhwalker$

---

If the terminal window is not wide, then you will not get any output because only the first so many columns are sent to fgrep from ps, depending on how wide the window is (I call that a bug, in that ps should not truncate its output if it is sending to a pipe, but that's a personal preference).

 

Russ

[CallMeDave]

> Launching the Retrospect Client application shows that the "on" button is still highlighted

 

- What is the text display in the "Status" field directly below that magic "on" radio button?

 

russ writes:

> One easy way is to open a Terminal window

 

*nix geeks love Terminal. As a GUI Baby, I'm more comfy with pretty applications.

 

Launch Activity Monitor (in your Utilities Folder), make sure the drop down box is set to either 'all processes' or 'administrator processes' and then filter for pitond (works just like iTunes!).

 

On a machine that is exhibiting the problem (_while_ that machine is exhibiting the problem) check for the status not only of pitond, but also for any other process that begins with "retro" (such as retropds22 or retropds23).

 

Retrohaterz noted that retropds.23 was alive and using CPU cycles, even though Retrospect was not currently connected (at leat that was implied; no further posts have been provided). If this process is getting "stuck," that would be uneffected by the launchd workaround suggested above, and would be a valuable datapoint for the developers (who, thankfully, _are_ paying attention to this issue).

 

Dave

[rhwalker]

Quote:

---

russ writes:

> One easy way is to open a Terminal window

 

*nix geeks love Terminal. As a GUI Baby, I'm more comfy with pretty applications.

 

---

Sigh. The reason I suggested using the shell interface is that it can be done from ssh into the problematic client without having to walk to that machine, and it can be done without disturbing the user who may be working at that machine.

 

If you want to see the "retro" processes, then you could:

Code:

---

ps axl | fgrep retro

---

 

russ