Tuesday, July 30, 2013

Getting data out of WeatherMan and into XTension

Another day another WebRemote tutorial showing how to get a live weather report onto your XTension Web based remote control page. WeatherMan is an excellent program available for $.99 from the Mac App Store and has a great scripting dictionary.

The scripts also show how to parse out the forecasted high and low temperatures for each day and the chance of precipitation. Those are all placed in XTension pseudo units and you can either just display the data or use it for other calculations like running your sprinklers or not.

WeatherMan and XTension

Monday, July 29, 2013

a Live Motion Report display for your XTension Web Remote

Just finished the scripting and a short tutorial over on Mac Home Automation that shows how to create a live updated display of motion sensor activity. This is probably one of the most useful things I've got on my web remote page.  MacHomeAutomation/livemotion

Friday, July 26, 2013

SSL/HTTPS passthrough server for XTension

The various ways of making a web interface in XTension have suffered from a lack of an easy way to get an SSL/HTTPS connection going. The older interface using apache and X2Web could be encrypted by installing a certificate into Apache but that was "non-trivial" as they say in the business.

UPDATE: The new interface now has HTTPS and certificate generation built in! Setting up this software is no longer necessary for a secure connection to the XTension Web Interface, but it may still be useful for encrypting the connections to other services.

So in the meantime I have created a simple SSL passthrough server. It will allow a browser (or any SSL socket really) to connect to it, handle the decryption and forward the connection on to either apach on your local or other server, or to the new Web Remote. With the password set on the new web remote this should offer a good measure of security for your system.

To make this work you would run the program on the same server as XTension and set the port of the SSL Passthrough server to a high port. Say 8080. Then setup your NAT Router or airport to forward traffic to the SSL Passthrough servers port and NOT the native port that your web server or web remote session is running on. The passthrough server will forward the hits on to the unencrypted server on your local machine or across your local network and sniffing your password on an unencrypted wifi or other network will be as difficult as any other HTTPS encrypted connection.

The server can be used with any web server or remote product that you're using on the local network.

The first thing you must do is to create your own ssl certificate. You can do this with the built in tools on MacOS. Follow the first 4 steps on this web page. Once you have the resultant files you need to use any text editor to past 2 of them together for the sslpass program to use. First open the "server.key" file. it should begin with a line like: "-----BEGIN RSA PRIVATE KEY-----" then open the "server.crt" file which should being like: " -----BEGIN CERTIFICATE-----" cut and past both blocks of data one after another (a couple of lines of clear space between them in the text file is fine) into a new file. Save that file into the same folder as the sslpass program and call it "certificate.txt"

the folder with the application and the certificate should look like this:

the program is a command line application that you pass the parameters to from the command line. A typical setting might be something like:

/path/to/program/folder/sslpass -listenport 8080

that would be the simplest setting, it will listen on port 8080 for HTTPS connections and forward them to the localhost on port 80. The last 2 parms are the default. If you are using this with X2Web then thats really all that is necessary. If you wish to forward it to a different port say for a WebRemote install you would do something like:

/path/to/program/folder/sslpass -listenport 8080 -remoteport 10023

assuming that your web remote instance was running on port 10023 on the localhost. You can run as many instances of this passthrough server as you wish for encrypting multiple web servers on multiple ports. Or even pass through to unencrypted sources on entirely different machines by adding the optional -remoteaddress 192.168.x.x parameter.

if you wish these passthroughs to be started automatically you can include them in a terminal file and make them part of your startup items. Paste the command line as you would type it into another text file and save it with a file extension of ".command" so it might be something like "startsslforweb.command" when double clicked that will run it as if you had typed it into the command line and you can add those to the startup items in the user control panel for launching upon startup.

Remember though that you can't set it to listen on a port less than 1024 without some dangerous permissions magic. Dont do it.

The program requires an Intel Mac, and it should run as far back as OSX 10.5 though I have not tested that. The program is freeware for owners of XTension, please consider purchasing a license for one of the XTension add on programs that I sell like the web remote or XTdb.

download: sslpass.zip version 1.0 July 26th 2013

PS: this program uses only the open SSL libraries as available all over the world and used in every web browser currently shipping in every country. If there are other import restrictions to your country from mine please do not download this program.

Monday, July 15, 2013

Building and interfacing an Arduino barometer sensor with XTension

With the initial release just the other day of the Arduino interface for XTension I find that I want to do just heaps of examples and demos. This is the perfect startup project as it's very simple build and fairly simple software. Though I've added features to the software that adds functionality and tries to get around the limitations of the sensor the basics should be easy to see in there.

The Arduino interface is just a simple ASCII protocol that sends and receives name=value pairs to XTension which end up in the standard units in that home automation program.

This project is going to build a barometer and put the data into XTension as well as take a crack at measuring a rising/falling trend and report that as well.

You're looking at all the necessary hardware. The code is specific to this kind of barometer sensor, but any arduino uno or compatible should work. The only other needed accessory are 4 female to male jumper wires. They aren't strictly necessary, but without them you have to do more soldering.
The barometer breakout board is the MPL115A2 from Adafruit for just shy of $12 as of this writing. There are other models but they were more expensive (and also more accurate) but if you use a different one you'll need to alter to code to include their libraries and different access methods. 

UPDATE: I've redone this with the $20 BMP085 also from adafruit. It gives much more resolution and usable data. Everything else is the same, I've uploaded the arduino project at the end to support the different board but wiring it up is virtually identical since it's also an IC2 device. So the same 4 pins need to be connected but they are in different locations on the board so read the labels and not just go by the positions in my pictures here. This is the new board:

Adafruit has an excellent library making the reading of the pressure (and it has a built in temp sensor too) just a single line in code, you don't have to worry about any of the low level stuff at all. You just include the library and say read pressure. 

Since we're going to plug this directly into the XTension server via USB you'll want an Arduino with a built in USB/Serial adaptor. Apart from that any of the Uno types will do.

The only soldering you have to do for this project is to connect those headers to the barometer board. If you're doing without the molded cables you can skip that and solder directly to the pads. 

 Standard soldering precautions apply. Don't overheat the board or the surface mount components might come off, don't overdo the solder or you may bridge pins, wash the lead off your hands if you're using leaded solder before dinner... But this is easy.

Lets make the power connections first, connect the "vdd" labeled pin to the +5v pin on the arduino and the gnd pin to a ground pin on the arduino. 

(REMEMBER that the BMP085 board is different than the one pictured. The pins have the same labels but are physically in a different order than in the pictures I posted above)

Next up the data connections. Follow the info on the Adafruit site for connecting them but on mine the barometer pin labeled "SCL" goes to analog pin 5 and the barometer pin "SDA" goes to analog pin 4. This is actually an "I2C" data bus which is an interconnect protocol for small computers like this. Theoretically you can hang many devices off those same 2 pins and talk to them by address. In reality most devices, like this one, have a fixed address and you have to desolder or add surface mount resisters to change it so that more devices can be connected. But that doesn't matter here.

After you get the wires plugged in it's a good idea to drop on a small amount of hot glue to hold these connections in place. They are really just meant for temporary breadbording and prototyping but with a little glue you can keep them from getting unstuck while still making it easy to take them apart on purpose should you ever wish to recycle them.

 don't over do the hot glue, if you get it in other holes you'll be stuck if you want to use them later. Just a little will hold them together.

now we need to load in the firmware.  If you're already old hat with the arduino IDE you can skip this bit but I'm going to document a few of the things that were confusing to me. Download and install the latest version of the Arduino IDE from their download page. Next you'll need to download the Adafruit library that simplifies the communication with the barometer. They use a github repository as of this writing and it's confusing if you've never used it. The github page is here and you're looking for the "download ZIP" button to get the whole thing.

Decompress the resultant file and you'll have to remove the "-master" from the end of the folder name. Arduino doesn't like the "-" character at the end of it. Now you need to find or create the hierarchy of folders on your Mac (assuming you're using a Mac which most of you will be since XTension is a Mac app...) When you run the IDE it will create an "Arduino" folder in your documents directory. Inside of that you'll need to find or create if it's not there a folder called "Libraries" and place the "Adafruit_MPL115A2" folder into that. If the IDE is already running you'll need to quit and restart to make it pick up on the new folder. If it worked you should see a folder for it under examples in the file menu something like:

If you see that then all is well and you can download and open the arduino source code. Place that folder inside the same Arduino folder in your documents folder. Projects must live there. If you click the checkmark and it says compiled and gives you a size for the compiled project then everything worked and you're ready to program the Arduino. 

Connect the arduino to the Mac in question and you should now be able to set the board type and the port name under the "tools" menu. then click the upload button and program your reader. after the upload is complete if you click the magnifying glass icon in the upper right hand corner of the program window you will get the serial console. Make sure it's set to 9600 baud and you should get the arduino barometer startup message and in 20 seconds or so start to see updates being sent. If thats working then you can move on to the next step and connect it to your XTension machine and create the necessary units there.

My install just looks like this at the moment, I want to be able to remove it and easily reprogram it as I refine the firmware. Pegboard is also dead useful for hanging all your home automation doodads off of. 

First step is to create the interface. Open the preferences window and create a new interface, set the device type to "Arduino" and select the silly and non-descriptive serial port name that is created for it in the port popup. Add a port command to it so you can more easily find it again later once you know what it is. Depending on the kind of arduino it may have the same name as on the machine you used to program it, or like the Uno i used it has a different name on that machine, I have no idea why. 

Don't enable the interface yet though, just save it. First we need to create a couple of units to hold the data. One for the pressure and one for the temperature. You may not need to use the temperature for anything and I dont know how accurate it is but you get it for free so you may as well record it.

The pressure unit should look something like this, you can call it anything you like but the interface needs to be the same as the one you just created and the address must be "pressure" Add the "hPa" unit of measure for better display. The unit needs to display numbers and not just on and off so check the dimmable checkbox and both the receive only and ignore clicks in list to keep you from accidentally changing it's value while using the UI.

The temperature unit is very similar but the address must be "temp" in the arduino code I am converting the temp to F since this is America, but if you wish to keep it as C see the arduino code comments for info on changing that. Use "option-k" to create that degree sign. Go ahead and enable the interface in the preferences and you'll start to collect data.

The description will initially say "calculating" for about half an hour as it collects data and starts watching for a trend. I'm not sure how good thats going to actually work. the logic I"m using is that I look back 15 and 30 minutes from now and compare them with the current reading. If they are all going up I say rising, if they are all going down I say falling otherwise I say stable. If anyone knows the convention for calculating the trend let me know and I'll adjust the code.

I'm also not entirely sure about the proper resolution and hysteresis for the pressure value. It reports directly from the device with 4 decimal points, but they bounce all over the place. I'm taking a reading every second and averaging the last 20 of them together and then rounding to just one decimal point to get any thing that doesn't just bounce uselessly around. It may be my unit, or it may be that the math just works out to far more decimals than it's accurate to. When I look up the pressure on my local weather site this seems to keep it in good sync and the resolution seems similar to what they display but if anybody knows more about this and what it should be let me know. 

If you have XTdb installed you can generate interesting graphs. The output of the new board is in hPa, not kPa but I might have missed some of that when updating the article above. In this graph I have the temperature output and the pressure for the last 24 hours here at the house.

.code { background:#f5f8fa; background-repeat:no-repeat; border: solid #5C7B90; border-width: 1px 1px 1px 20px; color: #000000; font: 13px 'Courier New', Courier, monospace; line-height: 16px; margin: 10px 0 10px 10px; max-height: 200px; min-height: 16px; overflow: auto; padding: 28px 10px 10px; width: 90%; } .code:hover { background-repeat:no-repeat; }