DIY projects of woodworking, home automation, electronics and software development for the Macintosh.
Monday, August 3, 2020
Power Supplies for ESP32 Arduino Based Projects
Friday, July 3, 2020
1-wire Temperature Sensors and Lightning (and other sources of interference)
if (future value) is equal to 185 then
change future value to (value of (thisUnit))
write log "ignored bogus 185 value for porch temp sensors"
end if
futureValue tells you the value that the unit will be set to when the script is finished running. The (value of (thisUnit)) portion returns the current value of the unit. This is sometimes initially confusing to people new to working with XTension but it allows you to do things like this or anything where you need to compare the current value with the new value and potentially override or change it. It also lets you do very specific smoothing of readings or throwing out ones that are out of range such as this. The call to ThisUnit just returns the name of the unit that you would normally have to pass to the value of verb so that if you renamed the unit you wouldn’t have to remember to edit the script and if you had many of the devices you want to do this to you could either cut and paste the code, or create a single script that you could call for all of them.
This 185/85 reading is actually a known issue with some 1-wire temp sensors. I don’t see a difference between powered or parasitically powered networks. The sensors in the above graph are parasitically powered but the one that is on the power supply is powered. The parasitically powered ones are on an absolutely horrible network of wires flying through the attic and to say that it is not a single bus or a standard star configuration would be a dramatic understatement. There is no shielding though they are on a single twisted pair of bell wire. Anyplace I wanted to add more sensors I just spliced into it and extended it. That attic network is my test of the absolute worst case I can imagine for testing 1-wire reading devices. The barix does exceptionally well under almost all circumstances except for these lighting storms.
That network now has 10 sensors on it all over the attic and it does sometimes drop a reading for one of them entirely. It also reads the intake and output temperatures from 3 different AC units so that I can verify the temp differential and know if the units are leaking gas and will need a recharge (or in one of the older ones a replacement is coming up I’m afraid) before they actually stop working and we have to make an emergency call to the HVAC folks. That reading also lets me run the fan after an AC or Heat cycle the best length of time to get all the cold or hot out of the unit. I turn the fan to On when a cycle completes and then start watching the temp sensors. As soon as the 2 temperatures are within a few degrees of each other I turn the fan back to Auto.
I am also going to update the hardware in the next revision of the wifi temp sensor to use the better interface that is outlined in the 1-wire best practices documentation to see if that will help with bad values or interference. In any case, overall 1-wire sensors work great, are inexpensive and there are a lot of options for getting them into your home automation system.
Friday, January 3, 2020
Python Motion Detection on a Raspberry Pi Zero
So in the interest of sanity and preserving the small quantity of it with which I have been granted I took a short vacation from everything MacOS. I have been fighting with the horrible firmware of even the best available IP security cams for some time as I struggle to write software that can connect to them and insert their video streams into the web and other interfaces of XTension. I believed that there should be software for the Raspberry Pi that could leverage that camera for a much more reasonable price that would be terrific. Sadly even the best software out there for it just wasn’t doing the things that I wanted or at least with the quality of image and recording that I wanted.
The idea of controlling everything about a security camera is compelling! Not having to setup a firewall to keep it from contacting it’s own “cloud” and making sure it isn’t setting up a UPnP passthrough even when you told it not to have revealed to me several times that even commercial products don’t really do what you think they are. I wanted one based on software that I have written myself and so knew exactly what it’s doing.
It should also run well on a Pi Zero W so that you can make something small, though if it runs better on the latest multi-core Pi4’s thats OK too. Those little machines are worth every penny and very very nice!
The built in “motion” system is really interesting, but it’s remarkably difficult to get it to run at a high framerate and a larger frame size. Who wants a non-HD security cam now days anyway? The separate OS versions available may be spectacular but I want a software package for the stock Pi. After fighting with trying to get openCV to compile on a Pi Zero for the last week I wanted something that didn’t need that either. (Which I suspect is why they went with a totally separate image for their security cam OS as getting that stuff to install properly is almost hilarious)
The python camera support for the Pi is absolutely awesome. I can stream 1080p video and I can record it to disk and then post process it to standard MP4 files in almost real time. Those were not insignificant accomplishments ;) But the real sticking point was going to be the video motion detection. Using the pycamera code as a starting point and the hardware H264 encoders ability to output it’s changes from one frame to another makes it absolutely possible to do this at least as well as any commercial product. I’ll eventually open source this project but for the moment I’m working on it just myself. Here’s the 720p stream, part of the web interface and also a stream of just the motion sensing. This works as high as 1080p but at lower framerates though I’m sure I can optimize the calculations a bit more for that.
Thursday, October 17, 2019
RGB Fairy Lights with only 2 wires (part the first)
I am a huge fan of all things blinky and light up. I’m less interested in things I cannot control remotely by writing a plugin for XTension or by building a new controller for it. While looking for RGB “fairy” lights on Amazon (“fairy” lights here being defined as some strands of what looks like magnet wire with surface mount LED’s soldered between then and then sealed into a little blob of epoxy) I came across these RGB ones and was intrigued since they appeared to have only 2 wires. There was no control wire or separate PWM lines for the RGB channels. (Amazon Link) These are by “Yihong” but there may be others that are the same.
They came with an RF remote which is nicer than the IR ones that some similar things come with. Since you don’t have to point it at the controller box. Interestingly enough the remote also has an IR led and so could be used by either type of system. I have not tested to see if it is running on the standard 433mhz band or does something special. Given that it appears the same as the thousand other such remotes I’ve gotten over the years for other LED displays or just for remote arduino projects I would guess that it does.In spite of the picture on the box showing different colors on the strand at the same time, you cannot control the LED’s individually, only as a whole strand.
This is the very tiny control box. The strand is powered from a USB plug and this box does the work of receiving the remote signals and adding whatever data is necessary to the power lines going down the strand. I have not broken the case open yet to see whats in there.
While there is tons of information out there about the “neopixel” type LED’s and the newer “dot star” that Adafruit sells and are amazingly wonderful there is very little information about anything similar to this. If anybody knows the type of chip or LED that they are made from I’d be interested to hear about it.
This you tube episode from Julian Ilett shows him reverse engineering a similar protocol. The system used by my lights looks to be very different however. So the first step to seeing what it’s doing is to get it on the scope. Since the wires are just enameled I scraped some of the insulation off them at the end of the strip. It occurs to me that I might have gotten a cleaner signal by tapping it at the front of the string instead. I’ll try that in step 2 and see if it makes it easier for the scope to trigger on it.These LED’s are different than Julians as they don’t pulse all the way to off. The data pulses drop the voltage to the lights down to 1.5 volts, but not all the way to off. This potentially makes sense as it would let the little processor in the LED chip continue to operate as it’s being updated.
Zoomed in on those a bit you can start to see the data.
Tuesday, May 7, 2019
WiFi Signal Strength as presence indicator, part 2
When the variance goes above 1, or whatever preset threshold, I turn on the presence register which shows in green. You can see that it definitely does work to know when I’m coming and going, but at this sample rate, buffer size and threshold it also comes on randomly with non-people moving through the room wifi interference.
In order to register anything reasonable at all you have to have the device very close to the person you want to know if they are there. This may be of use for this particular application as I plan to have the device sitting on my desk to provide some display of info for me while sitting here. It might even be able to blank and wake up it’s own display based on that. That would be useful and fun. Unfortunately if I sit very still it eventually thinks I’ve gone away. It’s decided that while I’ve been typing this paragraph actually. Lost of tweaking of settings would be necessary to make it at all useful.
I’m not sure that it will ever be useful as a generic motion sensor. You just have to be too close to the device to get it to see enough variance in the signal. Still an interesting experiment and I will keep fiddling with it to see if something useful can be done with it. It’s also possible that with the newer ESP32’s or other devices you could get better resolution or even access to things like the path being used to send the info or other similar data that would let you do a much better job. But for something that wakes up when you walk up to it and then goes to sleep shortly after nobody is standing next to it this could potentially be used.
Calculating a variance is a standard statistical technique so that you can gauge if you’ve got enough data points to realistically represent what you’re looking for among other thing. The formula is:
Which translates to the Variance squared being equal to the sum of each value in the dataset minus the average of the dataset squared and then divided by the number of points in the dataset minus one. So the first thing you need is an average of the dataset. It’s slightly complicated by the fact that my buffer array of read RSSI values may contain 0’s or values that are not a valid RSSI. As I place the values into the buffer I am making them all positive as I am not worrying about the actual db value, only the variance of the data. So the first thing you need to do is to call the scanRSSI function every second. To do this you need to compare millis() with the calculated next time you want to do something inside the loop. You can’t just compare simple unsigned longs as are returned from millis() as they will overflow back to 0 at some point which will break everything. So you can do the clever trick of casting it to a long and then subtracting it later which will yield the proper result of timing even during an overflow condition.
#define MAX_RSSI_ARRAY 128 #define RSSI_PRESENCE_THRESHOLD 1 bool currentPresence = false; byte rssiArray[MAX_RSSI_ARRAY]; byte rssiIndex = 0; // the index into the array where we are now unsigned long nextPresenceScan = ((long)(millis() + 5000)); // 5 second start
the code above sets up the buffer array as bytes, since the numbers will never be more than 255 there is no reason to waste memory by using 2 byte ints or anything else. We also need an index into the array so we know where to save the next value and the nextPresenceScan value which is where the trick comes in. Casting the value of millis() + 5000 as a long integer and then putting it into an unsigned long variable we get around the overflow problem as far as the comparisons later are concerned. Very clever and described in great detail on the Arduino website if you’re interested.
Next we need some code to run inside the loop() to check the time and call us when it’s time to run. Something like:
void loop( void){ if (((long)(millis() - nextPresenceScan)) >= 0){ nextPresenceScan = ((long)(millis() + 1000)); scanRSSI(); } }
Same trick as before, sets up the nextPresenceScan and then calls the function to do the work.
void scanRSSI(){ float currentRSSI = WiFi.RSSI(); if (currentRSSI > 0){ // sometimes the WiFi.RSSI() call can return a number greater than 0 to indicate // an error or that it simply couldn't get the info at this time. If that happens // just ignore it. return; } // add the new value into our buffer at the current rssi buffer index // multiply by -1 to turn it into a positive number rssiArray[ rssiIndex++] = currentRSSI * -1; // if the index is now beyond the size of our buffer reset it back to // the beginning of the buffer for the next read if (rssiIndex >= MAX_RSSI_ARRAY){ rssiIndex = 0; } // first value we need for the variance calculation is the average float accumulator = 0; // since the buffer may not be filled with valid data yet // we need to count how many non-zero values there are for the // average calculation rather than just divide it by the // MAX_RSSI_ARRAY int averageCounter = 0; float average = 0; for (int i=0; i<MAX_RSSI_ARRAY; i++){ // ignore values of 0 if (rssiArray[ i] > 0){ if (rssiArray[ i] < mn){ mn = rssiArray[i];} averageCounter++; accumulator += rssiArray[ i]; } } // if we don't have at least 10 valid values in the array then just drop out // there is no point in calculating the rest unless we have enough values to // look at if (averageCounter < 10){ return; } average = accumulator / averageCounter; // now loop again to calculate the SUM portion of the variance // reuse the accumulator accumulator = 0; // loop through the buffer array again, this time subtracting the average and then // squaring the result adding it to the accumulator for (int i=0; i<MAX_RSSI_ARRAY; i++){ if (rssiArray[ i] > 0){ // dont want to waste the memory to include the floating point or C math // libraries, so just put the calculated value into a local variable and // multiply that by itself float work = (rssiArray[i] - average); accumulator += work * work; } } // here we divide by n-1 float variance = (accumulator / (averageCounter - 1)); // the variance may be less than 0? if so take absolute value if (variance < 0.0){ variance *= -1; } // now do something clever with the variance data. }
That is actually pretty horrible code. I also do realize that I’m never taking the square root of the variance as calculated. That doesn’t seem necessary at this point and the poor arduino is already working hard with all those floats. If I make the value even smaller it will be harder to watch whats happening. It’s also not memory compact as it needs a full buffer. I’m sure some other kind of rolling average could be used. To get anything useful you’ll have to adjust the threshold and the scanning frequency and probably a lot of other stuff too. An interesting starting point to something potentially useful.
The device in question here is sending it’s values back to XTension for graphing and potential use. The graph above was generated by that program. XTension: Home Automation for the Mac, is my day job.
Sunday, May 5, 2019
Using wifi signal strength as a motion or presence indication
The device is sitting on my desk just a couple of feet away, so the range might not be very good, but still there might be instances where this is useful. You can quite plainly see in the above graph that I was in the office on and off until just after 7, and then it is fairly smooth while I was attending the local Philharmonic concert to hear my wife sing Mahler’s 2nd, and then picks up in variability as soon as I started going in and out of here this morning.
A lot of hysteresis would be necessary since there are many things that can affect the signal strength. Since it would take some time to know that you were there for sure it might not be useful to turn on lights or make adjustments when you arrive, but it could definitely be used to figure out when you’ve gone. Adding a Wifi calculated presence detection to such IOT devices would be very helpful. It would not work very well for battery operated devices however since you would need to wake up the wifi radios far more often than for the regular functions of the device.
Part two of this thought has code examples and is here.
Wednesday, February 6, 2019
Fancy Crown Molding for New Bookshelves...
I haven’t written about this yet because even so many years later it’s still not quite finished. We’re getting closer though. I just wanted to share this piece of test molding I made, oh, a little more than a year ago now, that serves as the template for the rest of it as I slowly completed the work. More info will follow as it’s even closer to being done now.
I’ve always been a bit intimidated by fancy crown molding. I didn’t know how to design it or make it work or make it go around the things that were there. This project has forced me to come to terms with some of that.
Here is my test piece for the multi-part crown molding that will go around the top of the new bookshelves. I’m so happy with the way it turned out I can’t even tell you. It even looks great in the finished product that I’ll upload another article about shortly :)
It’s all commercially available molding from the local Home Depot or Lowes. I tend to buy molding from Lowes because they sell it in 6 and 8 foot sections where at Home Depot I need to know how much I need ahead of time and cut it from a 10 foot segment that will by no means in the known universe fit into my car. I could probably save a few bucks by figuring that all out. But it makes too much work ahead of time. I can just calculate how many 8 foot sections I need and add one more to fill in the gaps :)








