Friday, July 22, 2016

OSX’s “powerd” daemon has some bugs...

There are manthreads on the internet about problems with the “powerd” daemon in OSX using up huge amounts of CPU time until everything else grinds to a halt. For me this problem started in Yosemite and has continued even through El Capitan 10.11.5. There is a .6 update for that now that I will install on my server shortly and see if that makes any difference.

On one site someone offered a bash shell script to regularly monitor the CPU usage of powerd and to kill it if it goes above 30%. I’ve wrapped that into an executable file that actually starts up in bash and doesn’t throw errors trying to run in tcsh which is the default shell on my system. I also added some code to send the CPU usage to XTension as well as keep track of the restart count. This way I can graph the CPU usage which looks like this over the last 2 days:

It takes just short of 11 hours for the spikes in activity to exceed the 30% threshold that causes the script to kill it and the system restarts it. I have no idea what the problem is, but something is definitely going on there. This graph is 24 hours of samples taken every 10 seconds so what you can’t see is that still most of the time in between those peaks it’s at 0% usage. Then it goes to do something periodically and each time it does it it takes a little bit more time. Some table not getting cleared? I don’t know what the cause is but it’s definitely real looking at that. Eventually if not killed the CPU usage becomes constant as it doesn’t finish servicing it’s queue before it has to service it again and the CPU usage becomes constant and debilitating to the machine.

If you’re also suffering from problems while having to continually kill powerd you can use this script. Cut and paste the code into any text editor, I like TextWranlger rather than using text edit built into the system, but any of them will work. Save it with the filename “powerd_monitor.command” the name is important! It must end in “.command” (and not .command.txt which text edit might try to add to the end of it) and it must be named powerd_monitor. This is because the script searches through the list of all processes for those containing the term “powerd” and it has to know how to exclude itself from the list that is returned. If you name it something else that also includes the term “powerd” then it will find multiple matches when searching and everything will error out and fail to do anything for you. So name the file “powerd_monitor.command”

You have to add the execute bit to the file. without that it won’t run when you double click it. If you’re terminal aware then thats easy. For anyone who isn’t you can do so by opening a terminal window and typing “chmod a+x “ (note the space after the x!) and then dragging the text file you’ve just created into the terminal window and pressing return.

If you aren’t an XTension user then you’ll want to remove the 2 lines that echo an applescript into osascript. Otherwise you’ll get an error printed for that with each check. Remove the 2 lines that begin “echo “tell app \”XTension\””

Lastly you must fill in your admin password into the first line of code where is says “sudopass=“ since the powerd is owned by the system and not the user you can’t kill it without executing the kill command with sudo to get the proper permissions to do so. Otherwise it will fail to actually stop the process.

The output in the terminal is multiplied by 100. So 0.1% comes out as a 10 and so forth. I don’t know why the original script author had to do that, but I suspect it’s because bash is bad at math and wasn’t able to compare against floating point numbers properly. 

The interval is currently set to check once a minute, but you can extend that or reduce it as necessary by changing the sleep value at the end. Of course, running the script too often will cause it to use far more CPU time than is necessary and make your problem worse not better. I generated the graph above with a check happening every 10 seconds.


# fill in your sudo password below or it won't be able to kill the process
# in XTension create a dimmable pseudo unit named "powerd cpu usage"
# in XTension create another dimmable pseudo unit named "powerd restart count"
# checks the cpu usage of powerd and if it's more than 30% it will kill it for you
# the output of the echo value is *100 so will say 40 for 0.4%!

sudopass='your sudo password here'
while true ; do
  cpu_usage=`ps aux | grep powerd | grep -v grep | grep -v powerd_monitor| awk 'BEGIN {ORS=""} {print $3*100}'`
  echo "powerd is using $cpu_usage"
  echo "tell app \"XTension\" to set value of \"powerd cpu usage\" to ($cpu_usage / 100)" | osascript -
  if [ $cpu_usage -ge 3000 ] ; then
    echo 'killing powerd!!!'
    echo $sudopass | sudo -S /usr/bin/killall powerd
    echo "tell app \"XTension\" to set value of \"powerd restart count\" to value of \"powerd restart count\" + 1" | osascript -
  sleep 60

There is some talk in some of those forum posts that just not running Activity Monitor or not running some other app that is listening to events from the powerd daemon solves the problem. I am testing that now on my server but it hasn’t been off long enough to tell yet. In the first hour since I quit it it seems like it might be making a difference but it’s too early to tell. I will update this afternoon when I have some more info. I have personally spoken to people who never ran activity monitor on their machines during the issue though, so that is not the only cause even if it does help mine. Perhaps something subscribed to the events from it is enough to make it start to leak CPU. Perhaps it’s calculating the energy impact which is faulty that some of those apps cause it to have to do constantly. I will play with some of those other issues after I’m done with a cycle without the activity monitor running.

UPDATE: it sure looks like just not running activity monitor is keeping it from climbing away in my case.

Since about 10:30am it’s CPU usage hasn’t climbed above about 0.5% of the CPU that it’s running on. It shows no signs of continuing to leak CPU time.

This is definitely not the only problem that causes this as my day job work neighbor Michael has the problem and has never left activity monitor running on his machine. I expect that just quitting the program won’t solve the problem, but it might keep it from continuing to escalate until you restart or kill the powerd. I will continue to watch and post any further observations as it’s a very frustrating problem!

Wednesday, June 22, 2016

A return to projects and a Vera upgrade...

It appears it’s been more than a year since I last blogged about a project. It’s been a fascinating year and all good things, but so busy. I’ve concentrated almost entirely on bringing the core and UI of XTension up to snuff with the more recent OSX updates. The UI especially has been languishing a little bit and lots of cleanup and just making things work better was necessary. I’ve been doing a weekly build and getting so much stuff done it’s felt really great. The software is starting to shine again now and I believe is on par or better than anything else thats out there. We’ve always had better internals than anyone else ;) but the UI was sometimes hard to use or understand. That is much better now.

Then there was the construction project around here. About 7  or 8 months ago we finally got a contractor who was able to start working on some small additions we wanted done. When we moved into this house it was a little smaller than the last house, which was good, but it didn’t have a couple of things that we really needed. As soon as we sold the old house we were going to add on to the garage for me to have a little workshop and put on a screened in porch for my wife to enjoy. Neither of which you’d think was that huge a project. It took 3 years to get the old house sold which was WAY longer than we expected and then it took a further 2 years of mucking about to get a contractor to actually do anything other than a few vague drawings and then disappear for months at a time. Most of them just never returned calls after that initial meeting. One of them who finally did some design work disappeared for 4 months and then came back, returned our check to him and said he was going to retire that he couldn’t do it anymore! The next one would only return my emails after I would follow up when I would suggest that he might recommend someone with more time on their plate. Finally we landed a company who was excited about the project and who’s estimate only came in about 20% over what I wanted to pay for it so we ran with them! The first set of framing subcontractors had to be fired because they never showed up. The second set of framing subcontractors who came in to fix everything and rebuild from scratch some of the stuff the second group did were terrific. They showed up every day they said they were going to, they were skilled and friendly and they had a foreman on site with the carpenters all the time to make the important decisions and make sure everything was going to plan.

The addition to the garage also gave me an opportunity to extend my office and lab which is above the garage over the new space. Four months and change ago I took apart my professional life, packed it in boxes and stacked it up in the center of the room while they came through the wall and very slowly finished the new section of my space! It’s finally done up here! I am very slowly starting to make sense of where I put everything and am going to rebuild all my workbenches. In another few days I’ll actually have a place to solder things that isn’t the kitchen table! It’s a huge job just given the volume of stuff that I need to organize. I’ll be putting kits together and getting ready to release more XTension kits for more things in the very near future and I’m very excited. Right now it’s a horrific mess, but that will change.

Lastly I finally got around to something I’ve been meaning to do for a long time. XTension and I are huge fans of the Vera device as a ZWave interface. We’re not huge fans of the user interface to it on it’s web pages nor it’s flexibility as far as scripting and logic are concerned, which is why we recommend it as the Z-Wave interface for XTension and not as your only home automation device. I had pre-ordered the new Vera Plus which has Zigbee functions, Z-Wave plus support, BlueTooth LE and all sorts of new low level things that will be very nice. I knew that the backend connections were compatible with XTension as we already have users using them without any difficulty, but some things are subtly different and I really needed it to test with. XTension connects to it via it’s excellent and well documented REST API and JSON interfaces. Having a documented protocol also makes it possible for us to support things without having to continually reverse engineer the changes that they make. Going forward I’m really only going to support devices with a documented protocol. Their support is pretty good too. I was previously running a Vera Lite interface with a much earlier version of the UI firmware. For any other XTension users that are upgrading I’ve not written a wiki article yet on the process, but here’s the gist.

Vera has on their site several support articles for upgrading from different boxes to the latest one. Since I had the Lite I followed this one. The only trouble I had was that the new Vera would not accept the transfer until I rebooted it one more time. After that it did and all was well. Both devices will show in their status lines that something is happening if it’s working. It will take the new box some time before the devices start to show up in the devices tab, so don’t panic. It will just say that it’s configuring the ZWave devices or some such. When all the devices are visible, even if they aren’t all reachable yet, you can upload the backup file you saved off in an earlier step of the process on the web page above.  Initially things aren’t going to work very well. There were several devices that simply were not responding that evening, but this morning it’s finished it’s re-routing magic and as far as I can tell everything is working again.

The only thing you need to know about upgrading XTension is NOT to create a new interface and point it to the new box. When the units are automatically created you’ll end up with 2 units for each real device, one assigned to the old interface and one on the new interface.

The proper method is to keep the connection to the old box open during the upgrade and transfer process. When the process is complete simply disable the old interface, change the IP to the new box and re-enable the interface. Thats all.

Friday, May 8, 2015

XTension Smart Meter Reader Kit

I am pleased and proud to announce the XTension Smart Meter Reader Kit

The arduino IDE compatible board reads the calibration pulses available on many smart meter models and outputs to XTension the instantaneous usage, the peak usage, the wh used since the last packet and the wh elapsed since the last nightly reset of the counters.

Communications back to XTension is done via an xBee radio, any other “bee” device with the same pinout and power requirements or any other means you wish via the serial data pads on the board.

Kits are available for shipping now starting at $45USD without xBee radios. The included electric meter attachments are 3D printed to order.

Though this kit is designed to interface with the XTension for Macintosh home automation software and does not come with any kind of display system, you could certainly make that a second arduino receiver project if you wished to do it yourself. The firmware describing the data output is open and available on the MacHomeAutomation wiki linked to above.

Sunday, April 26, 2015

XTdb update and Lightning Sensor for XTension

Lots of work done recently on the day job over at XTension.

An updated version of the graphing and database add on to the program that provides motion reports and lovely graphs. I replaced the simple graphing engine with a commercial one that has already added several features like transparency and on/off graph zones. Very useful and a great update. Many more possibilities are coming in future builds. You can get the new version from the XTdb home page

Today I finished a tutotial for connecting a very nice Lightning sensor to XTension by way of an Arduino and an xBee radio. I’ve been enjoying the storm alerts from this on my phone for a long time now and am finally getting around to posting the tutorial for everyone. XTension Lightning Tutorial

UPDATE: 4/28/15 and digi liked the lightning tutorial so much they added it to their gallery site!

Monday, February 9, 2015

A Comcast/Xfinity modem upgrade story that doesn’t end in tragedy.

Comcast/Xfinity is working hard to get everyone off Docsis 2 modems and to install Docsis 3 ones. They are so hell bent on it that they have an almost phishing level attack on you, if you click the link in their emails you’re forwarded to a page that automatically sends you a new modem whether you want one or not.

So thats where I started. I have have zero problems with my docsis 2.0 telephone modem through them. It has NEVER needed to be restarted. It’s never hiccuped or dropped a call or a data packet in 4 years. I did not want to replace it. They started sending me letters claiming that if I wanted my full speed that I needed to upgrade. Finally I clicked on one of those links and found myself with a new modem in the mail. OK, whatever, I’ll look into it when it gets here.

What arrived was their standard Technicolor TC8305C modem. Wow, phone lines and built in wifi. I wonder if I could use it to extend my current wifi or something useful or just turn it off? Well no, you can’t do either of those. The whole point of these modems is that they are complete control of XFinity and will always share that “xfinity wifi” signal that you see when you’re wandering around town. This isn’t necessarily a bad idea where people might be walking by your house and can get on the internet through wifi rather than over the phone if they log in. But it doesn’t make sense out here at all and I want to turn it off. You can’t turn it off. You also can’t set the modem in bridge mode and handle the NAT and other sharing things yourself. You have to use the tools built into the modem. I always run an airport for NAT sharing and wifi and I want to continue to do that. You CAN put the modem into bridge mode, but you have to call comcast and ask them to do it remotely! Thats just outrageous to my mind, and after reading the number of horror stories about this modem continually forgetting that setting and having to call in all the time to get it reset not to mention the lousy wifi range and other problems people have had I wanted nothing to do with it.

So I got an xfinity’s chat help person who was very nice and suggested that there was a simpler bridge mode modem they could send me instead. Ah, I would be happy to have that please go ahead and send it to me and I’ll take both other modems to the comcast place in town later.

2 days later A big box arrived with yet another Technicolor TC8305C in it. I now have 2 identical modems that I don’t want. So I decided that I had to do the research after all and got myself an Arris model that does nothing but bridge mode thats on their supported modem list. Telephone and Docsys 3 and NO wifi or NAT stuff, just a bridge to my airport.

Now I had to get it to actually work though, and there are as many horror stories about that out there as anything else. In my case the replacement of the old modem was almost completely painless and worked great. If you can help it do NOT call them at all.

Grab a cable bill or write down your account number before you start this process or you’ll not be able to go online to get it.

Disconnect the old modem and hook up the new modem. You’ll need to have a computer connected directly to the modem for this initial setup. Wait for the lights to come up, it may reboot a couple of times but eventually you’ll have a real routable IP address on your computer and any website you try to bring up will redirect you to comcast’s setup pages. Enter your account number and telephone numbers. The account number needs to be formatted differently in this page than how it was printed in my bill. Leave out all the spaces and dashes and also I had to leave off the trailing 0 in order to get it to go through.

Then it thinks for a while, reboots a couple more times. And in just 15 minutes or so I had the internet back up and running and my phone service was working too. I know they say it can take hours, but it didn’t in my case. I did have to reboot my airport after I connected it to the modem. This isn’t surprising, but my browser was redirecting me to the setup pages again which was alarming. So I just rebooted the airport and everything has been just great since then.

So if you can avoid having to call and talk to their support you might be able to get your upgraded modem working without that much pain or wasted time at all. I was pleasantly surprised that their automatic upgrade stuff actually worked and worked for me. I was not expecting it to go easily.

Monday, January 5, 2015

My pick for the best LED bulb to use with home automation dimmers

Phillips and Cree seem to be in tight competition for your LED bulb money. They both have cheap and cheaper versions of their bulbs now in more and more interesting packages but so far very few bulbs available will work in automated dimmer fixtures. The one reliable exception that I’ve found that is available locally at a reasonable cost are these Phillips bulbs.

The model info on the packaging says “AmbientLED A19  model # 9290002268A(Upgraded 06/2013)” with an ordering code of BC11A19/AMB/2700 DIM 120V

The light color is 2700k, but they dont feel that warm to me. The light is a little brighter than the sick yellow you can expect from a label saying 2700k. 

They dim perfectly on every remote controlled home automation dimmer that I’ve tried them on. They work on dimmers that do not have a neutral which forces them to run a little current through the load to power themselves. They also will run as a single bulb on an old style X10 push button switch! This is amazing as there aren’t many other LED bulbs that can run in that configuration. Sometimes you can build a large array of bulbs that wont be on dim all the time on a no-neutral switch but these work as a single bulb.

They are silent, never making even a hint of a buzz and in the last couple of years that I’ve been buying them I haven’t lost a single one.

Dont be fooled by the Phillips label though, they have several other kind and formats of bulbs, like the very interesting flat ones, but those will not work with no-neutral dimmers. They will be on low all the time and cannot be turned off. The bulb pictured above does not have that problem.

Cree makes a bulb that is a little bit less expensive than this bulb and they now come in 60, 75 and 100 watt versions in both warm and cool color temperatures. I dont have a picture of these but they are the ones with the glass envelope coated in what feels like a dip in silicone rubber to protect you if the bulb breaks. These also work great on remote controlled dimmers and will also work as a single bulb on even an old style X10 push button switch. The downside of them is that they are noisy, buzzing very audibly when on a remote dimmer and even worse they flicker. These bulbs flicker even when there isn’t a dimmer anywhere around. I’ve got some of their flood lights that I am going to soon replace as even when on an appliance module with no dimmer around randomly they seem to think they need to flicker horribly. I cannot recommend these bulbs even though they do work on the dimmers because of the noise and flickering. 

The phillips bulbs are available here at Home Depot and online in many places.

Tuesday, September 16, 2014

Serial2tcp sharing a serial port on your computer to a remote device

There are dozens of ways to connect to remote devices from XTension but it occurred to me that there isn’t a simple way to just share one connected to another Mac. This little app will connect a local serial port on any Mac to a TCP port. Listen for a connection to the port and then forward data between the two. It can connect multiple serial ports and TCP ports with only one process. I’ve made no attempt to code for the opposite side of the equation, turning a TCP stream back into a virtual com port because thats not necessary for XTension. Anything XTension can connect to via a local port it can also connect to via a TCP stream.


(since the code was fairly straightforward I was able to load the source into an older version of the compiler and create a fat binary with both Intel and PPC code for anyone wanting to recycle a really old machine. I’m not entirely sure how far back in OS versions this will run, but it would be worth trying on 10.4 intel or ppc machines.)

download ppc:

download and unzip. I would recommend placing the serial2tcp folder inside the Applications folder. There is a folder of libraries inside it as well that must remain next to the application regardless where you put it. Then if you’re the kind of person who likes to run things from the command line you can make an alias to your /bin directory.

command line options:

--help   prints out the help
--list    prints out a list of available serial ports on your computer
--share [serial port name] [serial port baud] [tcp port number]

--share is the main entry point for getting it to do anything. Get the serial port name from the --list function, a standard baud rate and a TCP port above 1024 to run it on.

For example to share a CM11a on a remote machine you’d use something like:

/Applications/serial2tcp/serial2tcp --share usberial_FT5434 4800 2000

would connect to an FTDI based usb/serial adaptor with that name, open the port at 4800 baud and share it on port 2000.  If you wish to share more than one port you can use multiple --share parameters. But they must have unique serial port names and TCP port numbers.

The program will allow only 1 connection at a time for any given serial port but will re-listen if a connection goes away. So you should be able to reconnect to it without restarting it or any such thing. It also makes a valiant effort to reconnect to a lost serial port. If someone pulls the USB plug and plugs it back in on the remote machine it will work very hard to reconnect to that port and then reconnect it to the already open TCP stream. If you’re lucky you wont even miss any data. On 10.9 with the latest FTDI drivers I had no trouble with this apart from the garbage data being sent down the serial port making my CM11 freak out until being power cycled. On earlier OS versions though it was not uncommon to get a kernel panic if an open usb/serial adaptor was pulled while in use. So YMMV.

To use it then in XTension just select “remote tcp port” instead of a serial port from the available port popup. Enter the IP address of the computer doing the sharing and the port number you’ve chosen, in the above example 2000. The computer hosting the port should be configured with a static IP address or a reserved DHCP address so that it doesn’t change. If you want to share across the internet you’ll have to be able to create the proper NAT pass throughs to reach it. It’s a server listening for an incoming connection on the port not a client that can make an outgoing connection.

Thats all well and good, but the real value would be in having it launch at computer startup and run no matter who is logged in or if anybody is logged in to the computer. This is possible by setting up a system launchd configuration file for it. The following instructions are for OSX 10.9 and later, but it should work for the last few versions. Earlier than 10.7 and you should verify the proper launchd plist configuration because I believe it has changed since 10.6.

Inside the serial2tcp folder you’ll find a file called “com.sentman.serial2tcp.plist” this is a boilerplate launchd configuration file that you will need to edit for your own use. If you have apple’s dev tools installed then you can use the excellent plist editor, otherwise any text editor can do it but pay attention to the XML structure. Screenshots below are from the plist editor.

The “label” must be unique and should be the same name as the file but without the .plist at the end. You can leave it like it is unless you’re starting multiple instances.

the first program argument must be the link to the program you wish to run, in this case I put it in the Applications folder, but if you put it somewhere else then change that link.

The following parameters are the same as if you were adding them at the command line. In this example I’m sharing usbserial-FTG3LGPX at 57600 baud on port 2000. You’ll want to change those for your own settings. You can add more program arguments if you wish to share more than one port. Just repeat items 1 through 4 with the new info in them.

Once you’ve saved this back you’ll need to place a copy into the system launchd folder which is at /Library/LaunchAgents you can use the command line to get there or the “go to folder” option of the finders “Go” menu and enter that path. It will ask you to enter your password in the finder because this is a system protected folder and you need to run as an administrator to move files in and out of it. Once the file is there you’ll need to edit the owner of the file. It must be owned as root in order to be loaded while nobody is logged in. I could not get this to work from the finder’s file permissions panel and so it must be done from the command line.

Open a terminal window and type:

cd /Library/LaunchAgents


sudo chown root ./com.sentman.serial2tcp.plist

again it will ask you for your password to make sure you’re actually allowed to make a file owned as root.

You can test without rebooting by using the launchctl command from the same command line, type:

sudo launchctl load ./com.sentman.serial2tcp.plist

and you should just get the cursor again if no errors occurred. The program will log it’s output to the system log while running in the background so look there for connect/disconnect errors or other messages. If all went well you should now be able to see it running with the activity manager and connect to the port from XTension and share the device.

This article is cross posted at