Saturday, November 30, 2013

Comcasts bandwidth notice breaks everything

It was 2 days before the end of the month of November 2013 when I first approached the 300 gig bandwidth limit imposed by Comcast. My daughter has been watching a lot of Dr Who on netflix to try to get caught up for the Day of the Doctor and thats the only thing I can think of that is different than any other month. I’ve never come even remotely close before and certainly do not run any file sharing or other such systems here.

They alert you by sending you an email, but they also start actively filtering your web traffic and inserting code that creates a popup window in every browser session that you try to create. This window will not go away until you log into your comcast account and acknowledge the fact that they are about to start charging you extra.

This altering of the HTML code coming from every server is a huge problem. It will wreck havoc on the “internet of things” and other such devices that happen to communicate on port 80. All of a sudden my internet thermostats were not parsing the fact that they were connected to their service, my phillips hue bridge complained that it had lost internet connectivity, my iDevices would not load any web content at all. The HTML was so broken that I got only a white page with nothing displayed at all.

I finally resorted to rebooting my router and power cycling the hubs that the hue bridge was connected to all without an improvement in what was going on, yet some other things were working so I knew I wasn’t just offline.

I finally logged into my actual desktop computer and got the popup and was able to log into my account and turn it off. But you can’t opt out of the popups entirely, they are “required” for 100% of the usage, though you can turn them on for other amounts.

Net neutrality aside (bandwidth spent watching netflix or hulu counts against my usage, but bandwidth used watching their xfinity app does not, thats cheating Comcast) the implications of their ability to watch not just caching server logs of my html traffic but actually insert code into it, code that is malicious in how bad it is even if it’s purpose is not actually to do harm, is huge. They could alter anything, change what adds are being displayed, silently filter things they dont like. Forcing all port 80 traffic to have this popup inserted even when many devices making such connections dont have the ability to display any such popup and are not sophisticated enough to realize that their JSON or SOAP or XML or other documents have been totally corrupted by Comcast shows a real lack of thought went into this.

Obviously they wish to make sure you know that you’re about to be charged extra so that you can’t get mad at them later when I claim I never got their email. I wish to opt out of your corrupting the HTML data flowing into my home and yet I cannot.

Overall I’ve had very little difficulty with Comcast and the service. It is not overpriced, at least not by the standards of other internet service in America, and it has been reliable. But I will be watching as technology continues to render their privileges of easements to run their wires less important. When other ways of getting connected are available to me I will remember how you broke my house for no good reason and how you filter and insert things into the packets coming into my house.

Wednesday, October 16, 2013

Thoughts upon returning from a short vacation at Disney World


Reviewing the various features and foibles of Disney World is a full time job for an army of bloggers and review sites. Though this is just a single breath in a veritable hurricane of writing on the internet, it is my breath.

We went for just 3 days. My children are 9 and 13 as of this writing and had a wonderful time. It's not necessary or even enjoyable to spend a full week there.  You run out of steam and blister bandaids too quickly. Especially with smaller kids, a couple of days is plenty and take them again in a couple of years instead of blowing all your dough in one huge vacation that they will be too tired to enjoy.

We stayed in the park for the first time this time. Previous visits we've rented a small house somewhere nearby and driven in (and saved THOUSANDS of dollars). Disney runs the best managed and accessible parking lots in world. Do not be afraid to drive in, you will not be delayed. We stayed at the lower tier "resort" The Art Of Animation. I was not initially filled with love and happiness when I first got a look at the industrial big square box buildings with a few lion sketches on the side and a couple of decorations on the endcaps. but they are really OK. The room we had was very nice, with a sofabed and a Murphy bed for the kids in the main room and a separate master bedroom and bath for us to actually close the door and get some sleep. If you're staying in the hotel to make use of "extra magic hours" make sure they actually fit into your schedule. The parks we were visiting never overlapped with an extra magic hour that we could actually visit, so that was a wash.

There is no good way to unload your luggage at these resorts. There are bellhops with trollies at the main checkin but not enough of them and you'll wait for a long time. Or you'll carry your own which is what we did. The parking is very close to some of the buildings, and very far away from others. There is no door to the building facing it either. You pretty much have to walk all the way around to get to the elevators, or use the stairs. Your wristband wont open the doors by the stairs though even if you wanted to use them until you've used it to open your room door which unlocks it for other uses. So you have to walk around to the main entrance which is in the middle of the complex of buildings at least the first time.

I must take a moment and talk about the toilets installed in this hotel. They are obviously air pressure driven and are the loudest flushing toilets I have ever encountered in my life. The first few times you flush them they will make you jump, it takes getting used to. You can also hear every other person in your side of the building when they flush, just as I'm sure everyone could hear you. The volume level is truly impressive. I would also hang onto small children if they are standing nearby when you flush else they be sucked down. These are powerful turbo-charged toilets.

Food service at the hotel was adequate. Disney does everything as good as can possibly be done, and even they can only do just better than passable food service on this scale. The food was not bad, but it was impossible to get your tray and your kids tray definitely from another separate line through checkout and to a table while your food was still warm.

If you buy into the food service plan do the research ahead of time and know what you get and get it before reaching the register. If you're lucky enough to have a cashier that knows what you're entitled to you'll end up going back to get a different drink or another pack of grapes or carrots or a dessert. If you're cashier doesn't know then you just wont get those things that meal. In either case you'll be unhappy.

We also bought the infinitely refillable cups which were a waste of money. They can only be refilled at the resort that you're staying at, there is no point in carrying them to the parks with you as they wont work there. So we bought them and used them the first night we were there. Then the next morning we didn't carry them with us to breakfast because we were going to the parks without returning to the room which is what we then did the rest of our visit. I believe I used them 3 times while we were there. Not worth the money unless you are going to be hanging around the pool at the resort a lot or are willing to walk from your hotel building over to the cafeteria a lot, which can be quite a little walk. After coming back from the parks and taking off my aching shoes the last thing I wanted to do was walk back to the cafeteria to get soda, we just got ice from the ice machine down the hall and had nice cold water.

The cups are interesting from a technological standpoint. They have an RFID chip in the base that has to be programmed or read from the register. There is a reader under every single spigot in the pop dispensers and you have to set it down and wait a heartbeat before you try to fill it. That way there is no black market in used cups as they are programmed only to work while you're there. But nobody really explains how to use them and that you have to set them down, I watched many frustrated people holding them under the spigot and nothing would work till you set them down. Disney is going all RFID for everything and it's not going completely smoothly yet. I'll take the chip out of my cup soon and post some takeapart picts to try to figure out just what the system is based on.

We got "Magic Bands!" which are a cool idea, no more being afraid of losing your card, they easily stay around your wrist (but you'll keep looking at them trying to find the watch face to tell you what time it is, now THAT would be an upgrade to them :) When they say "near" field what they really mean is touching. the range on them is such that if you even press them a little out of perfectly on the front of the sensor they dont work. At least on your hotel door, other readers at the park may be higher power. The absolute best use of this technology is when you combine it with being able to schedule your fast passes on the iPhone app before you even get to the park. This is so fantastic I can't even rave about it enough. Schedule your visits to the 3 biggest rides you want to see before you even leave for the park, or better yet if it's something like soarin do it the day before. At 10am there was already a 2 hour wait in the standby line for soaring the day we were there. We walked right on with a scheduled visit. I really think that the technology is there to schedule more and more of the visits and reduce the walk in line considerably. This would be great, standing in line sucks, just tell me when to come back thats fine. I don't understand why every person in the park isn't doing this.

We visited Blizzard Beach the first day there, then spent that evening at Hollywood Studios because there were only a couple of things we wanted to see there (hello, fast pass!) The second day we spent at epcot and lastly at the Magic Kingdom and that evening did Mickey's not so scary halloween party which is fantastic but you have to book 6 months or more in advance if you want to get in. They sell out regularly now.

At Blizzard Beach the first day we managed to get our first and only minor injury of the trip. My daughter managed to fall off her float on the lazy river and scrape her hand up pretty good trying to hold onto the side. There is an application for nano tech out there somewhere for someone to invest a coating for the bottom and sides of swimming pools so that you have some traction but not so that it rips up your hands and the bottom of your feet while you're playing in it. In any case the very sympathetic life guard handed us a couple of bandaids and we were back on our way. (I think he was glad she hadn't hit her head as he told of witnessing several such events, probably the most dangerous ride at the park because people try to stop themselves and the tubes want to keep going and they flip over into the side of the embankment!)

Epcot has some great stuff too, but it's a LOT more walking than any of the other parks. Epcot though serves beer for just walking around. Which is great in theory, but it means that older "children" end up there behaving like "children" sometimes too which is never pretty.



Our only real negative was trying to find a space around the halloween parade route. Disney has adult fans so rabid that they wont step back 2 feet and let your kids sit on the ground in front of them. So my kids can't see as grownups without any children in tow snap flash pictures in the dark that will never come out of chip and dale dancing to "this is halloween" really people. You could see over my kids, my kids can't see over you. We've had to work around people like this at every parade at Disney that we've ever been too. Thats not to say that we've not met hundreds of perfectly nice and happy to help out or move over people too, but it's the jerks that you remember.

The new Fantasy land in the Magic Kingdom wasn't finished yet, they are still building the new dark ride, I think based on snow white? but the new Little Mermaid rid was up and running and was definitely worth the walk around the back of the new area. The animatronics have made huge leaps and it's amazing stuff. Ariel even had animated hair so that it looked like it was floating around under water. A little bit freaky but just so cool. I felt the ride was a little rushed though, it could have run slower and let us enjoy the details a bit more. It also felt a little sterile to me, there wasn't as much detail to see and it was all very brightly lit. I suppose it will gather details and upgrades over the years, but it's still definitely worth the trip. I think it could have used one more story setting, it went right from Ursela using her voice to woo Eric to the real Ariel's wedding scene. I would have loved to see Eric empaling Ursela on the sailing ship, but I suppose that would be too dark even for a dark ride? In any case, it was a great new ride and you should go on it (unlike the nemo mini-ride they built into the sea, that is totally lame)

The haunted mansion has undergone some updates. Some interactive things to play on in the queue are there now and I think a welcome improvement. The insides as well seem to have been constantly updated and replaced over the years, everything is still there but there is more and more to see. I love it. Same with Pirates which has lost some of it's rather overt movie themeing. Capt Jack is still all over, but the mist screen with the nasty from the movie right before the initial fall is gone (or wasn't working when we were there) making it more like it was prior which I liked better.



Disney theme parks are awesome. Take your children for a couple of days, not when they are too little or they wont like the dark rides and loud noises. (bring earplugs for anyone who might be startled! It makes a huge difference in their enjoyment) Take them back to the hotel in the heat of the afternoon and come back in the evening after you've rested up. Even if you hate corporate Disney and the stupid that is the Disney Channel and the stupid that is Disney sponsored tween targeted rock and roll. The parks are wonderful.

Thursday, October 10, 2013

Larson Scanner Hat for Halloween


one Larson Scanner from Evil Mad Science but instead of mounting the LED's to the board I glued them to my hat. I spend so much time helping the kids with their costumes that I dont normally get to wear anything fun myself, but at least this year I'll have a Cylon hat!

Tuesday, October 8, 2013

XTension saves my crawlspace

What is home automation good for other than remote control of your lights is an oft asked question. I collect a lot of data about things, especially things I can't see easily without it and good information is hidden in there.

I have a very fancy and expensive encapsulated crawlspace under this house. It's one of my favorite things. No longer is it moist, and moldy and wet and full of bugs under there. It's sealed up and dehumidified and it's so clean we could have tea under there when you come to visit.

One of the things I monitor is the duty cycle of the dehumidifier. That device has a dedicated circuit that runs under the house to power it so just outside the electric box I added some monitoring equipment. Since the dehumidifier has a passive humidistat it's almost totally off when it's off so that makes it easy to tell if it's on or not. I used a 120v HVAC sensor. They are for turning on a booster fan when the main blower turns on. You just pass one wire of the feed to the main blower through it's sensor loop and it will switch on it's load when that load reaches a certain point. You can't really adjust them so they aren't useful for things with a more complicated power usage structure (like the sump pump down there which I also tried to do this with but which never worked because the charger circuit on the battery backup pump kept it on all the time) So I used that HVAC switch to control a 120v relay which provides a dry contact which you can read into XTension in many ways. Mine is connected to a weeder digital card.

The duty cycle of that has been around 4% since a few weeks after it had been installed and dried out the space under there. I still display it on my Web Remote because that number going up could mean equipment problems or some other issue under there.

Yesterday I noticed that the duty cycle was at 15% and knew there was a problem. Sure enough the little hooks that held the access panel in place from the outside had finished rusting through and the door had fallen open. It had probably been open only a few hours before I noticed it and was able to fix it keeping me from having bugs and skunks and god knows what move in under the house.

I think I'll place a door sensor on the access panel now too.  You can't get from there into the house without a sawzall but still it would be good to know if the door comes open.

Friday, October 4, 2013

creating an applescript record with user defined labels in Xojo

Perhaps this really is documented somewhere, but I couldn't find it. Or perhaps it's so obvious that everyone else has been doing it for a decade but I just figured it out.

In Xojo (IDE previously known as REALbasic) you can create enumerated records with 4 byte codes that are translated into labels by the applescript dictionary.

dim MyRecord as new AppleEventRecord

MyRecord.StringParam( "xxxx") = "my data here"
and if you have an enumeration for the xxxx in your dictionary that says it means "cool stuff" then you can access that record in applescript by saying

set myThing to cool stuff of ThatRecord

and you'll get that data out again. But in order for that to be useful you have to know what those labels are ahead of time. In a script itself you can create a record with labels defined at runtime like:

set myRecord to {label1:"value1", label2:"value2", etc...}

and then you can access them via those label names

set myString to label1 of myRecord

but until yesterday I've never been able to create a record that had user defined labels in Xojo code. It's a simple matter of creating a AppleEventDescList with alternating label/value pairs in it and then placing it inside an AppleEventRecord with the 4 byte code of "usrf"

dim theRecord as new AppleEventRecord
dim theList as new AppleEventDescList

theList.appendString( "label1")
theList.appendString( "value1")
theList.appendString( "label2")
theList.appendString( "value2")
etc...

theRecord.DescListParam( "usrf") = theList

now if you return that record from an event the calling script will be able to treat it like a user defined list and use it the same way like:

set myString to label1 of TheRecord

Specifically i'm using this in a JSON parser where you just wont know the labels before hand so there is no chance to pre-build them as enumerations into the scripting dictionary.

Monday, September 9, 2013

adding animated weather radar to your XTension Web Interface pages

Another monday another tutorial on the Mac Home Automation site. This time how to embed data from other sources into the displays, for example animated gif's of weather maps.



http://machomeautomation.com/doku.php/webremote/radar

These instructions apply to what I do at my day job as a programmer for the Macintosh based home automation software XTension and the new WebRemote system.

Monday, August 19, 2013

xBee battery powered motion sensor, part the second.

This is a followup to an earlier post as came out in the comments you can't wake up an xBee directly with the output from the motion sensor, it's the wrong polarity. It connects to Vcc when motion is detected and floats when there is no motion. Just adding a pulldown and a transistor didn't work at all to connect it to ground and wake up the xBee. Too many timing issues.

So I decided to go with an ATTiny85 chip that can sleep most of the time and be woken up itself when the sensor goes high, and it can then handle the timing of waking up the xBee and having it sample it's pins and send the data. This actually worked lovely until I tried to add back in the motion sensor.

It turns out these motion sensors just dont work below 3v or so. They have an onboard voltage regulator for the 3.3v that they need to work so you really must power them with 5v to get them going. There is some talk on teh intertubes that you can just short across the regulator on the board. I think those instructions were written before they switched to all surface mount parts on these boards which makes it much more difficult. I did try doing that though and it did work then at 3.3v quite well.


you can see my tiny little wire loop across the voltage regulator. Unfortunately at about 3v and lower it stops behaving at all which means that 2AA batteries are not going to power it for very long. And it also stops behaving above 4.5v which means that 4 batteries are a no go. The ATTiny has a good range of voltages in this area which makes it excellent for a long term battery operated product but the xBee is very sensitive to voltages above 3.3 volts. I wouldn't want to run it on 3 batteries without a regulator. Which means that would require a regulator which would waste a lot power and significantly reduce battery life.

I've looked around quite a bit, and there are some excellent canned solutions to motion sensing out there, including some fantastic chip/sensor combinations from Zilog (who also offer an SMT version of the Z80 CPU chip which somehow I find funny :) but they use too much current and still dont operate at a low enough voltage for a good battery operated project.

So we're left with separate power supplies for the sensor and the xBee or regulators or rolling it from scratch. I might experiment with a 3AA battery system where the sensor runs with it's regulator off 3 batteries and see what the current usage is and tap just 2 of the batteries to power the xBee. They will run down out of sync but that might still work and 3 batteries in the device isn't out of the question. Or it really might be time to start studying how these sensors work and roll one from scratch. The otherwise quite good except in radio range and reliability X10 motion sensors work for years on 2AAA batteries. I've glanced at their insides and I think they do a op amp comparitor thing between the 2 outputs of the motion sensor and then have a tiny PIC to make decisions on the output.

At least I understand why the newer ZWave motion sensors are so expensive and why the battery life in them sucks so badly overall.

But all this does mean that without the motion sensor the combination of an ATTiny and an xBee radio work great. So other sensors should be a piece of cake, something like a door/window contact closure device that might also sample temperature and light levels on a regular basis should be quite possible. I might concentrate on that for the moment and stew over the motion sensor problems a little longer.

Tuesday, August 6, 2013

Tankless Water Heater Musings

One way of saving money and energy that is touted greatly is a tankless water heater. As I understand it there is still some controversy over just how much money this will save you, and they tend to be rather expensive so the payback can be long. A year ago or so I strapped a temp sensor to the inlet and outlet pipes of my traditional tank water heater so I could measure it's usage and see just how much it's running when nobody is using water to give myself some idea of the use of a tankless water heater here.  Specifically they are dallas 1-wire sensors connected to a Barix, Barionet 50 and finally reading their data in to XTension.




Here is a few recent days worth of data. Yes, I know I have the temperature turned up too high, but this is the kitchen, laundry and master bedroom heater and not the one in the kids rooms. My kids are big enough now that it's not that much of a danger to them. The second water heater that serves their rooms  is still turned down to 120 though so don't worry about us.

The blue line is the cold inlet temp and the red line is the hot output line. The first thing I see is that the cold inlet temp rapidly reaches the same temp as the outlet as soon as you stop using water. That makes me suspect there isn't a heat trap or whatever that fixture is that is supposed to reduce heat loss through the pipe on that line. I'll have to investigate that. But the other thing you see is that in these 3 days there isn't a single point where the cold water inlet temp spikes without a corresponding spike in the outlet temp. Meaning that the water heater hasn't run once not in response to demand in the last 3 days. As it turns out I have to go back to July 24th to find a graph that shows a "maintenance" cycle just to keep the tank warm.


There, at 2:30 in the morning the cold temp spikes with no hot water use measured. I have to go back several weeks again from this to find another one. If I switched to a tankless heater those would go away, and any energy wasted by them. But how much would that be? 2 or 3 waste cycles a month isn't very much to pay back the high cost of a tankless water heater.

The other factor to consider is efficiency or how well does it put the energy that it uses into the water. It was my understanding years ago that electric tank water heaters are nearly 100% efficient, meaning that almost none of the energy is lost, it all goes into the water. Gas water heaters like mine do lose quite a bit of energy up the smokestack, but so do gas tankless. This is an area for further research, but I do not believe that it's a huge difference at this point. At least not enough to make me replace a currently functional hot water heater. When it's time to replace it though I will look seriously again at these figures and it may make sense in the future.

There is an additional consideration as well in cost. Gas tankless water heaters require a lot of gas, they have a minimum pipe size in order to get enough gas. At the placement in my attic where it would replace the existing water heater I would have to have new gas piping run to support a tankless of enough capacity for my home. At considerable expense. The same holds true for electric, you have to have a high amp service run to where it is, they can't just be plugged in to a handy outlet. They use a huge amount of power. If you're on a peak metering system that changes you more if you use large amounts of power your electric tankless is very likely to put you close to or over the limit so the power you use for it may be much more expensive. Do the math on this or you may be very unpleasantly surprised.

Just thinking out loud here, everyones situation is different and it may make perfect sense for you to get yourself one of these. If you use a LOT of hot water you'll never run out, and if you use very little hot water you dont waste energy keeping the tank warm. Though if the tank is like mine the number of those phantom cycles might be very few.

Tuesday, July 30, 2013

Getting data out of WeatherMan and into XTension

Another day another WebRemote tutorial showing how to get a live weather report onto your XTension Web based remote control page. WeatherMan is an excellent program available for $.99 from the Mac App Store and has a great scripting dictionary.


The scripts also show how to parse out the forecasted high and low temperatures for each day and the chance of precipitation. Those are all placed in XTension pseudo units and you can either just display the data or use it for other calculations like running your sprinklers or not.

WeatherMan and XTension

Monday, July 29, 2013

a Live Motion Report display for your XTension Web Remote



Just finished the scripting and a short tutorial over on Mac Home Automation that shows how to create a live updated display of motion sensor activity. This is probably one of the most useful things I've got on my web remote page.  MacHomeAutomation/livemotion

Friday, July 26, 2013

SSL/HTTPS passthrough server for XTension

The various ways of making a web interface in XTension have suffered from a lack of an easy way to get an SSL/HTTPS connection going. The older interface using apache and X2Web could be encrypted by installing a certificate into Apache but that was "non-trivial" as they say in the business.

UPDATE: The new interface now has HTTPS and certificate generation built in! Setting up this software is no longer necessary for a secure connection to the XTension Web Interface, but it may still be useful for encrypting the connections to other services.

So in the meantime I have created a simple SSL passthrough server. It will allow a browser (or any SSL socket really) to connect to it, handle the decryption and forward the connection on to either apach on your local or other server, or to the new Web Remote. With the password set on the new web remote this should offer a good measure of security for your system.

To make this work you would run the program on the same server as XTension and set the port of the SSL Passthrough server to a high port. Say 8080. Then setup your NAT Router or airport to forward traffic to the SSL Passthrough servers port and NOT the native port that your web server or web remote session is running on. The passthrough server will forward the hits on to the unencrypted server on your local machine or across your local network and sniffing your password on an unencrypted wifi or other network will be as difficult as any other HTTPS encrypted connection.

The server can be used with any web server or remote product that you're using on the local network.

The first thing you must do is to create your own ssl certificate. You can do this with the built in tools on MacOS. Follow the first 4 steps on this web page. Once you have the resultant files you need to use any text editor to past 2 of them together for the sslpass program to use. First open the "server.key" file. it should begin with a line like: "-----BEGIN RSA PRIVATE KEY-----" then open the "server.crt" file which should being like: " -----BEGIN CERTIFICATE-----" cut and past both blocks of data one after another (a couple of lines of clear space between them in the text file is fine) into a new file. Save that file into the same folder as the sslpass program and call it "certificate.txt"

the folder with the application and the certificate should look like this:



the program is a command line application that you pass the parameters to from the command line. A typical setting might be something like:

/path/to/program/folder/sslpass -listenport 8080

that would be the simplest setting, it will listen on port 8080 for HTTPS connections and forward them to the localhost on port 80. The last 2 parms are the default. If you are using this with X2Web then thats really all that is necessary. If you wish to forward it to a different port say for a WebRemote install you would do something like:

/path/to/program/folder/sslpass -listenport 8080 -remoteport 10023

assuming that your web remote instance was running on port 10023 on the localhost. You can run as many instances of this passthrough server as you wish for encrypting multiple web servers on multiple ports. Or even pass through to unencrypted sources on entirely different machines by adding the optional -remoteaddress 192.168.x.x parameter.

if you wish these passthroughs to be started automatically you can include them in a terminal file and make them part of your startup items. Paste the command line as you would type it into another text file and save it with a file extension of ".command" so it might be something like "startsslforweb.command" when double clicked that will run it as if you had typed it into the command line and you can add those to the startup items in the user control panel for launching upon startup.

Remember though that you can't set it to listen on a port less than 1024 without some dangerous permissions magic. Dont do it.

The program requires an Intel Mac, and it should run as far back as OSX 10.5 though I have not tested that. The program is freeware for owners of XTension, please consider purchasing a license for one of the XTension add on programs that I sell like the web remote or XTdb.

download: sslpass.zip version 1.0 July 26th 2013

PS: this program uses only the open SSL libraries as available all over the world and used in every web browser currently shipping in every country. If there are other import restrictions to your country from mine please do not download this program.


Monday, July 15, 2013

Building and interfacing an Arduino barometer sensor with XTension

With the initial release just the other day of the Arduino interface for XTension I find that I want to do just heaps of examples and demos. This is the perfect startup project as it's very simple build and fairly simple software. Though I've added features to the software that adds functionality and tries to get around the limitations of the sensor the basics should be easy to see in there.

The Arduino interface is just a simple ASCII protocol that sends and receives name=value pairs to XTension which end up in the standard units in that home automation program.

This project is going to build a barometer and put the data into XTension as well as take a crack at measuring a rising/falling trend and report that as well.


You're looking at all the necessary hardware. The code is specific to this kind of barometer sensor, but any arduino uno or compatible should work. The only other needed accessory are 4 female to male jumper wires. They aren't strictly necessary, but without them you have to do more soldering.
The barometer breakout board is the MPL115A2 from Adafruit for just shy of $12 as of this writing. There are other models but they were more expensive (and also more accurate) but if you use a different one you'll need to alter to code to include their libraries and different access methods. 

UPDATE: I've redone this with the $20 BMP085 also from adafruit. It gives much more resolution and usable data. Everything else is the same, I've uploaded the arduino project at the end to support the different board but wiring it up is virtually identical since it's also an IC2 device. So the same 4 pins need to be connected but they are in different locations on the board so read the labels and not just go by the positions in my pictures here. This is the new board:



Adafruit has an excellent library making the reading of the pressure (and it has a built in temp sensor too) just a single line in code, you don't have to worry about any of the low level stuff at all. You just include the library and say read pressure. 

Since we're going to plug this directly into the XTension server via USB you'll want an Arduino with a built in USB/Serial adaptor. Apart from that any of the Uno types will do.

The only soldering you have to do for this project is to connect those headers to the barometer board. If you're doing without the molded cables you can skip that and solder directly to the pads. 


 Standard soldering precautions apply. Don't overheat the board or the surface mount components might come off, don't overdo the solder or you may bridge pins, wash the lead off your hands if you're using leaded solder before dinner... But this is easy.




Lets make the power connections first, connect the "vdd" labeled pin to the +5v pin on the arduino and the gnd pin to a ground pin on the arduino. 




(REMEMBER that the BMP085 board is different than the one pictured. The pins have the same labels but are physically in a different order than in the pictures I posted above)

Next up the data connections. Follow the info on the Adafruit site for connecting them but on mine the barometer pin labeled "SCL" goes to analog pin 5 and the barometer pin "SDA" goes to analog pin 4. This is actually an "I2C" data bus which is an interconnect protocol for small computers like this. Theoretically you can hang many devices off those same 2 pins and talk to them by address. In reality most devices, like this one, have a fixed address and you have to desolder or add surface mount resisters to change it so that more devices can be connected. But that doesn't matter here.

After you get the wires plugged in it's a good idea to drop on a small amount of hot glue to hold these connections in place. They are really just meant for temporary breadbording and prototyping but with a little glue you can keep them from getting unstuck while still making it easy to take them apart on purpose should you ever wish to recycle them.




 don't over do the hot glue, if you get it in other holes you'll be stuck if you want to use them later. Just a little will hold them together.


now we need to load in the firmware.  If you're already old hat with the arduino IDE you can skip this bit but I'm going to document a few of the things that were confusing to me. Download and install the latest version of the Arduino IDE from their download page. Next you'll need to download the Adafruit library that simplifies the communication with the barometer. They use a github repository as of this writing and it's confusing if you've never used it. The github page is here and you're looking for the "download ZIP" button to get the whole thing.

Decompress the resultant file and you'll have to remove the "-master" from the end of the folder name. Arduino doesn't like the "-" character at the end of it. Now you need to find or create the hierarchy of folders on your Mac (assuming you're using a Mac which most of you will be since XTension is a Mac app...) When you run the IDE it will create an "Arduino" folder in your documents directory. Inside of that you'll need to find or create if it's not there a folder called "Libraries" and place the "Adafruit_MPL115A2" folder into that. If the IDE is already running you'll need to quit and restart to make it pick up on the new folder. If it worked you should see a folder for it under examples in the file menu something like:


If you see that then all is well and you can download and open the arduino source code. Place that folder inside the same Arduino folder in your documents folder. Projects must live there. If you click the checkmark and it says compiled and gives you a size for the compiled project then everything worked and you're ready to program the Arduino. 

Connect the arduino to the Mac in question and you should now be able to set the board type and the port name under the "tools" menu. then click the upload button and program your reader. after the upload is complete if you click the magnifying glass icon in the upper right hand corner of the program window you will get the serial console. Make sure it's set to 9600 baud and you should get the arduino barometer startup message and in 20 seconds or so start to see updates being sent. If thats working then you can move on to the next step and connect it to your XTension machine and create the necessary units there.


My install just looks like this at the moment, I want to be able to remove it and easily reprogram it as I refine the firmware. Pegboard is also dead useful for hanging all your home automation doodads off of. 

First step is to create the interface. Open the preferences window and create a new interface, set the device type to "Arduino" and select the silly and non-descriptive serial port name that is created for it in the port popup. Add a port command to it so you can more easily find it again later once you know what it is. Depending on the kind of arduino it may have the same name as on the machine you used to program it, or like the Uno i used it has a different name on that machine, I have no idea why. 

Don't enable the interface yet though, just save it. First we need to create a couple of units to hold the data. One for the pressure and one for the temperature. You may not need to use the temperature for anything and I dont know how accurate it is but you get it for free so you may as well record it.


The pressure unit should look something like this, you can call it anything you like but the interface needs to be the same as the one you just created and the address must be "pressure" Add the "hPa" unit of measure for better display. The unit needs to display numbers and not just on and off so check the dimmable checkbox and both the receive only and ignore clicks in list to keep you from accidentally changing it's value while using the UI.


The temperature unit is very similar but the address must be "temp" in the arduino code I am converting the temp to F since this is America, but if you wish to keep it as C see the arduino code comments for info on changing that. Use "option-k" to create that degree sign. Go ahead and enable the interface in the preferences and you'll start to collect data.





The description will initially say "calculating" for about half an hour as it collects data and starts watching for a trend. I'm not sure how good thats going to actually work. the logic I"m using is that I look back 15 and 30 minutes from now and compare them with the current reading. If they are all going up I say rising, if they are all going down I say falling otherwise I say stable. If anyone knows the convention for calculating the trend let me know and I'll adjust the code.

I'm also not entirely sure about the proper resolution and hysteresis for the pressure value. It reports directly from the device with 4 decimal points, but they bounce all over the place. I'm taking a reading every second and averaging the last 20 of them together and then rounding to just one decimal point to get any thing that doesn't just bounce uselessly around. It may be my unit, or it may be that the math just works out to far more decimals than it's accurate to. When I look up the pressure on my local weather site this seems to keep it in good sync and the resolution seems similar to what they display but if anybody knows more about this and what it should be let me know. 

If you have XTdb installed you can generate interesting graphs. The output of the new board is in hPa, not kPa but I might have missed some of that when updating the article above. In this graph I have the temperature output and the pressure for the last 24 hours here at the house.



Saturday, June 29, 2013

is it even possible to huff alcohol with dry ice?




My kitchen sports an odd tableau today. Dry ice, bubbles, kitchen scales and vodka. I saw a fear and uncertainty and oh my god wont someone think of the children article today that claimed teenagers were getting dangerously drunk by inhaling alcohol vaporized by putting dry ice into a bottle and breathing the vapors. I was immediately skeptical that the alcohol would vaporize that way in any meaningful amount and since I had to go to the grocery store today anyway I decided to pick up some dry ice and sacrifice a few ounces of vodka to testing this.

I ran 2 tests with dry ice and one control bowl just to see if it vaporized just sitting in an open bowl at room temperature.

I weighed the bowls and added 2 oz of vodka and then a chunk of dry ice. In the first test I used tongs to move around the dry ice piece but when I realized I was removing drops of the sample each time I did this I stopped and only gently sloshed around the bowl in the second to keep the dry ice in contact with the vodka. I didn't agitate or move the control bowl at all between weighing it and weighing it again at the end of the experiment.


I recorded the weight of the empty bowl first as this kitchen scale wont stay on long enough to let the ice completely disappear and so the tear would get lost. This is me measuring in the 2oz of vodka. Then I added in a couple of small pieces of dry ice.

 


And I let them sit there and bubble. If you're going to get enough alcohol into you through any route even inhaling, then a significant physical amount of this liquid must get carried away along with the CO2. It's not homeopathic alcohol after all.

While I did not try to huff any of it during the experiment the fog did smell strongly of vodka and I thought for a while that perhaps that meant that a significant amount was being liberated! I was wrong.

While we waited my son and I performed all the other necessary experiments and delights that having extra dry ice around make possible. And I finally got that forming a bubble over the entire perimeter of a bowl to work making HUGE dry ice bubbles that pop in a cascade of fog that is just so much fun to watch over and over again :)

It took around 10 minutes for the dry ice to bubble away. Experiment 1, the one I messed with and lost a few drops, lost .05oz. Thats 5 hundredths of an ounce. (incidentally, also the smallest granularity of my scale, so it may have been slightly more or less) The 3 drops on the table under my tongs are more than enough to explain that. The second test lost nothing, it weighed exactly the same amount after the ice as it did before the ice. The control bowl lost .05oz as well, just sitting there at room temperature the alcohol was evaporating, but nobody got drunk off that. So it seems that putting the booze in wth the dry ice actually keeps it from evaporating, dropping the temperature so that the vapor pressure of alcohol didn't let hardly any at all escape into the air.

So I'm calling this one busted. There is no way that you can get drunk off inhaling the fog from a bottle of dry ice and booze. What you might get is hypoxic however, which could be very dangerous if you breathed only from the bottle for an extended period of time. But most of us have taken a few hits off helium balloons in our day and there is no oxygen in there either and as long as you kept it to 1 or 2 lungfulls you didn't pass out. Having been a teenager once myself I witnessed and even once in a great while took part in some things that would have qualified as sublimely stupid. Teenagers do this sort of thing. But huffing alcohol in this particular manner just can't do anything to you. There almost certainly are ways of actually vaporizing the alcohol though, if you wanted to heat it and bubble air through it, or do something really painful and stupid like put it into a cold mist humidifier you could probably make a cloud of it that had plenty of alcohol in it. But at least in that last example I challenge anyone to take a second breath after taking the first. You'd be choking and telling your stupid buddy that you were drunk enough and it was his turn and you'd also get it in your eyes. Ouch...

So tell your kids that if they see anybody doing this to leave the party, not because they are doing something really dangerous, but that they are too stupid to trust with other necessities of keeping themselves alive through the rest of the day.

Now.. if you'll excuse me I have 4oz of very well chilled vodka and a martini glass waiting for me to finish this entry.

Sunday, June 16, 2013

Arduino, xBee and PWM lighting with XTension

Back on Halloween Ben and I built him a gigantic lightup inflatable pumpkin costume that was a great deal of fun. In addition to the el wire face on it we also built him an electric t-shirt with LED strips attached to an arduino, some buttons for lighting effects and a little 12v battery pack to power them.

You can see them there under his costume, at night they put on a great show, flickering and fading in and out and flashing brightly when he pushed the button that let them accentuate his awesomeness while trick-or-treating.  We used these cool white LED strips from adafruit.com. After halloween I removed them from the t-shirt and saved them for future projects. A future project revealed itself shortly thereafter in the form of aquarium lighting for his new fish tank. We didn't do any fancy remote control or PWM on them for version 1. We just taped 2 of the 4 little strips to some shelf support material I had and powered them from a generic 12v wall wort.

They worked great for a while, but the constant splashing of water on them and the metal bars soon took it's toll. I think that the power supply may have been a bit high too for they rather quickly started to degrade and individual LED's started to go dim. Even so they lasted several months, but finally died and it was time to rebuild. We had 2 more strips left over from the costume and this time we decided to do it right. PWM to control the levels so it could be dimmed, an xBee radio so that it could be wired into the Home Automation System and some buttons so he could set the level there. I had just finished an arduino interface for XTension and so decided that this should be a demo project on how to bring some of these things together. 

Here you can see how far the old LED strips degraded. The metal shelf support is rusty, the LED strips have gone opaque and you can see where some coper was exposed and turned bright green.


Yuck... This time I would build something with proper heat sinking and proper protection from the elements. I used a  piece of aluminum extrusion U channel as the base for them. It's available at HD and others as a shelf support. It's meant to go over the long edge of a plywood or particle board shelf to keep it from bowing in the middle. It wasn't quite the perfect size for 2 LED strips, they were a tight fit, but I added some extra adhesive behind them as well. 


Also you can see one of the bezel pieces. I printed those on the MakerBot out of translucent PLA. They are actually less translucent that I had though and I might replace them with just cut pieces of plastic soon, but they work for now to protect the lights from the water.

For the Arduino I went with a Fio. I'm not using the lipoly battery charging ability of it, just the fact that it's an arduino with an xBee socket built in. Both running at 3.3v so no other circuitry is required. Or so it seemed... The LED strips need to run at 12v, the Fio is rated at 12v input and initially booted and started up OK, when I had it wired in circuit and applied 12v the regulator on the board burst into bright flame or arcing or something. I removed the power with some alacrity thinking I had cooked the arduino and probably the xBee as well. Testing from USB power though showed them both to be just fine. Now thats impressive, a piece of gear that can actually catch fire and not break down! So instead of relying on the on board regulator I added a beefier external 3.3v regulator. That also released it's magic smoke upon startup. So at this point I took a step back and reviewed the rest of my wiring. Had I wired the mosfet backwards and was putting 12v into one of the input pins or something? Nope, everything looked good. So I broke out an anyvolt micro adjustable regulator and hooked that up instead. Adjusted it's output down to 3.3v and all was well. I guess 3.3v linear regulators just dont like an input as high as 12v. There are many dc/dc converters on the market now, and anyvolt also make a 3.3v version, but the adjustable was what I had on hand. Connecting that to the 3.3v input and bypassing the internal regulator completely brought it all happily back to life.



The mosfet in the picture makes PWM'ing high loads so very easy, no external parts required, not even a resister. This particular model is an N-channel power mosfet and I wired it according to adafruits excellent examples that are linked to from that store entry. Thats actually the same one that powered ben's halloween costume so it's still got some lumps of hot glue attached.  A $1.25 saved is a $1.25 earned ;) You can also see the LED strips powered up and running behind their makerbotted water protection. The plastic strips are just hot glued gently on top of the aluminum channel. I want to be able to replace them when they get a scuzzy from being in the water.


I mounted all three devices into a tiny plastic parts case with a hole drilled through the top/bottom for the xBee antenna. If you were using a lower output one with a chip antenna then even that wouldn't be necessary. 


The buttons in their final install position on the fish tank. Not very pretty, this end of the project will likely get a reworking.

The arduino interface for XTEnsion just provides a simple text based interface for sending values back and forth. basically just address=value with a return at the end. It can be extended of course ;) but for this that wasn't necessary. I opted to make the buttons and the light totally separate in the device itself, requiring the intervention of the computer in order to send back the light level in response to a button push. This means that you can't control the light if the computer if offline, but if the computer is offline I've got bigger problems than the fish tank lights. So in this code the buttons just send an ON when they are pushed (and no off, though I could do that easily enough if it's useful) and then the computer responds by sending a level for the lamp. There is a heartbeat delay between pushing a button and the response coming back, but this way I can set the levels in the computer without having to recompile the arduino code, or without having to write a lot of code to save the levels into the eeprom. So there are other solutions if you like that. You could also build this without any remote computer in the mix at all by just setting the lamp PWM value in the processButtons() handler. But this is an example of how to write arduino code that talks to XTension.


This is how the units are setup in XTension. The buttons receive the commands send from the arduino like:
  Serial.println( "button1=on");

and if you change the aquarium light units level it ends up sending a command like "LAMP=160"

XTension will also convert the address to be all caps, so keep that in mind in your arduino when comparing strings. In the code you'll see some commented out map commands as I was going to map the standard dim level in XTension, 0%=100% onto the range of PWM outputs available on the arduino, 0-255, but decided not to, just let us control it from here. In experimenting with it we discovered that 160 was plenty bright for the fish and the plants and struck a nice comprimise between light and heat output. Any more than that and the light bar started to border on hot rather than just warm. I want to keep these LED's for a long time so we are not going to let them overheat. And a value of just 4 is bright enough on the dim setting for Ben to use as a nightlight. It really does look like moonlight at that level and is quite pretty.

The computer control is important separate from the button control so that the lights can be turned on remotely when we are on vacation, or if he fails to turn them on in the morning. Since feeding the fish and turning on the light is part of his morning ritual he rarely forgets. But now I know when he last turned the lights on, and if it doesn't happen I can turn the lights on automatically and send me an alert so that I can feed the fish if he forgets. He's 8 and is really very good at taking care of them, but it does happen. 

Since the arduino has plenty of inputs and outputs available we have plenty of room available for future expansion. I think a water temperature sensor is in the near future, that will be easy. Perhaps some UV or deep blue LED's to make his fluorescent fish light up at night too. We're also starting to think about an automatic feeder that we would design and print on the makerbot and control from this same arduino.

Setting up xBee radios to work with XTension needs an entry of it's own as well. That will come shortly. Here is the source code free and open use it as you wish. download the arduino project

/*

  Ben's aquarium light 
  
  using an xBee to talk to XTension

  June 10 2013  james@sentman.com
  
  In this example, the 3 buttons send their ON to XTension when they are pushed, and XTension replies
  with the proper change to the light level. They do not directly control the level of the lamp.
  the lamp is only controlled from XTension by way of scripting from the button units.

*/


//input and output pin definitions
const int lampPin = 3;
const int upButton = 8;
const int midButton = 7;
const int downButton = 12;
const int ledPin = 13;

//these globals are used by the serial interface for talking to and getting commands
//from XTension via the Arduino interface. All commands are sent as NAME=value pairs with a 
//carriage return at the end. Set your buffer sizes to be at least as big as the longest command
//you expect to receive. These are oversized as only a single value is sent.

const int commandBufferSize = 19;
const int valueBufferSize = 22;
char commandBuffer[ commandBufferSize + 1];
int commandBufferIndex = 0;
char valueBuffer[ valueBufferSize + 1];
int valueBufferIndex = 0;
boolean gotCommand = false;


//debounce necessities
unsigned long lastButtonMillis = 0;
unsigned long currentMillis = 0;
int currentButtonPushed = -1;




void setup(){
  
 //make sure xBee is set to same serial speed. Could use faster of course 
 Serial.begin( 9600);

 //set the pinmodes for in or out as appropriate
 pinMode( lampPin, OUTPUT);
 pinMode( upButton, INPUT);
 pinMode( midButton, INPUT);
 pinMode( downButton, INPUT);
 pinMode( ledPin, OUTPUT); 
 //turnon the pullup resisters
 //to simplify the wiring I'm using internal pullup resisters for the buttons
 //that way I only have to connect them to ground.
 digitalWrite( upButton, HIGH);
 digitalWrite( midButton, HIGH);
 digitalWrite( downButton, HIGH);
 //make sure we startup off 
 analogWrite( lampPin, 0);
 //could always save the current value of the light to the eeprom 
 //and retrieve it at startup so that a power outage for a few seconds 
 //wouldn't result in the light being off.
 //give the led a flash
 digitalWrite( ledPin, HIGH);
 delay( 500);
 digitalWrite( ledPin, LOW);
 //write to XTension that the system is ready.
 Serial.println( "log=Aquarium Ready");

 //and update it's unit so that it knows that it's not on
 Serial.println( "LAMP=0");
  
  
  
}


void loop(){
  
   //currentMillis is used in the button processing in several places
   //so instead of constantly calling to millis() just do it once and put it
   //in this variable.
   currentMillis = millis();
   //check the serial buffer for any waiting command data and process it
   //if there is any
   processSerialCommands();
  
   //check for any button state changes
   processButtons(); 

  
  
}

/*
  this is part of the Arduino interface to XTension
  it can easily be reused in future projects.
  this just manages the command and value buffers and reading of data
  from the port. Any actual received commands will be passed to the next
  processCommand() method. Add any other incoming value handlers or special
  commands there. This handler shoulnd't need to be changed for most things
*/

void processSerialCommands(){
   int workByte;
  
    //if no data available then just return, nothing to do this cycle through loop
    if (Serial.available() == 0){
     return;
    }
   //read a byte
   workByte = Serial.read();

   //since all data is sent and received as name=value pairs
   //initial data is added to the command buffer. When a 61 or "=" is recieved
   //we finalize the end of the commandBuffer with a nul and start adding future characters
   //to the value buffer.
   if (workByte == 61){
      commandBuffer[ commandBufferIndex] = 0;
      commandBufferIndex = 0;
      gotCommand = true;
      return;
   }
   
   
   //this is looking for the carriage return at the end of the command line
   //if it finds the return then it finalizes the value buffer with a nul and 
   //resets the buffer indexes to get them all ready for the next command
   // and finally passes the most recently received command=value pair to the process command
   if (workByte == 13){
     valueBuffer[ valueBufferIndex]=0;
     valueBufferIndex = 0;
     commandBufferIndex = 0;
     gotCommand = false;
     processCommand();
     return;
   }
   
   
   //if we haven't finalized the command yet (or address as it's now called in XTension)
   //then we add the byte to the commandBuffer and increment the index to wait for the next one
  if (!gotCommand){
    commandBuffer[ commandBufferIndex] = workByte;
    commandBufferIndex++;
    
    //make sure that we dont overrun the buffer though and reset it if we're receiving garbage
    if (commandBufferIndex > commandBufferSize + 1){
      commandBufferIndex = 0;
      return;
    }
    
  } else {
    
    //we've already got the command because gotCommand has been set to true by the
    //check above for a 61 value or "="
    //so now add instead to the value buffer, increment it's counter and continue
    valueBuffer[ valueBufferIndex] = workByte;
    valueBufferIndex++;
    
    //make sure that the value buffer doesn't overflow and if so then
    //reset both the command and value buffer due to garbage on the line.
    if (valueBufferIndex > valueBufferSize + 1){
      commandBufferIndex = 0;
      valueBufferIndex = 0;
      return;
    }
  }
  
}

//XTENSION here is where you will receive incoming commands use the string objects to manage your response
//  add as many addresses as you need.

void processCommand(){
  String theCommand = String( commandBuffer);
  String theValue = String( valueBuffer);
  
  if (theCommand == "LAMP"){
    
    if (theValue == "on"){
      setLampLevel( 100);
    } else if (theValue == "off"){
      setLampLevel( 0);
    } else {
      setLampLevel( theValue.toInt());
    }
    
    return;
  }
  
  //add as many more comparisons for theCommmand as you need to for your application
  //XTension will always send address=value pairs if it's a change to a unit, though
  //any data may be sent with the "send data" verb including commands that have no value
  //though in that case you should still send the = and then immediately the return. like
  //reboot=/13 do not omit the equal sign.
}


void setLampLevel( int newLevel){
 //int adjustedLevel;
 //level from XTension is 0 to 100, map it so that we never go to all on 
 //adjustedLevel = map(newLevel, 0, 100, 0, 210); //keep it from going all on as that burns out the LED's
 //Serial.print( "log=level is ");
 //Serial.println( adjustedLevel);
 analogWrite( lampPin, newLevel);
  
  
}


void processButtons(){
  boolean debounceTimeout;
  
  if ((currentMillis - lastButtonMillis) > 100){
    debounceTimeout = true;
  } else {
    debounceTimeout = false;
  }
  
  
  if (digitalRead( upButton) == LOW){
    //upbutton is pushed
    if ((debounceTimeout) && (currentButtonPushed != upButton)){
      currentButtonPushed = upButton;
      lastButtonMillis = currentMillis;
      Serial.println( "button1=on");
      return;
    }
  } else if (currentButtonPushed == upButton){
      //the button is up but the last button pushed was ours so we can reset it
      if (debounceTimeout){
        lastButtonMillis = currentMillis;
        currentButtonPushed = -1;
      }
  }

    
 if (digitalRead( midButton) == LOW){
    //midButton is pushed
    if ((debounceTimeout) && (currentButtonPushed != midButton)){
      currentButtonPushed = midButton;
      lastButtonMillis = currentMillis;
      Serial.println( "button2=on");
      return;
    }
  } else if (currentButtonPushed == midButton){
      //the button is up but the last button pushed was ours so we can reset it
      if (debounceTimeout){
        lastButtonMillis = currentMillis;
        currentButtonPushed = -1;
      }
    }
    
  if (digitalRead( downButton) == LOW){
    //upbutton is pushed
    if ((debounceTimeout) && (currentButtonPushed != downButton)){
      currentButtonPushed = downButton;
      lastButtonMillis = currentMillis;
      Serial.println( "button3=on");
      return;
    }
  } else if (currentButtonPushed == downButton){
      //the button is up but the last button pushed was ours so we can reset it
      if (debounceTimeout){
        lastButtonMillis = currentMillis;
        currentButtonPushed = -1;
      }
  }
}