Sunday, July 29, 2018

Arris model DG3270A SNMP and backup decoding

I have a remote install of XTension setup at my mother in laws condo. The internet and tv company there started with DSL modems in the building, but it’s an older building and the wires in the walls never resulted in a very good connection. They finally got around to putting in a cable head end in the basement and switched everyone to cable modems. They did this without calling me first or otherwise letting me know that they were going to show up and drop in a new modem and make everything I have installed there home automation wise to drop off the network and kill all my incoming passthroughs and such. I got there last night to have a look at the new modem and reset everything up and I discovered some interesting things.

The modem is an Arris DG3270A and actually works pretty nicely. The wifi is significantly better than the old modem actually supporting a 5ghz wifi network as well as 2.4ghz AND it will allow you to set both the 2.4ghz and 5ghz to the same network name. This is the preferrable way to do this as your devices will migrate between the 2 as needed and you won’t have to be using the faster 5ghz one and then walk upstairs or into bedroom and have it drop out and have to manually switch to the other. Your device will just always connect to the strongest one.

They have the interface to it setup so that you can do all the configuration of almost everything you’d want to yourself. The default password on the box was not changed which in this case was just admin/password. There are many different models and many providers lock you out of these settings. With the old modem I had to actually call the company to get the password to it and they told me that they would give it to me but would charge me for the service call if I screwed it up ;) I didn’t screw it up ;)

I was able to change the DHCP range to carve out the same block of static IP’s that I give to her XTension machine and the Vera there as well as setup the couple of pass throughs so that I can ssh into the machine and the webcam if necessary. It even has dynamic dns support so I was able to add the modem to my account there and have it update that info too. I also have the client running on the Mac in her closet but it never hurts to have a backup to such things, without that updated I can’t talk back to her machine very easily.

With the previous modem I had written a little script that polled the status page of the modem periodically and dumped the values of the noise and signal level on the DSL line into XTdb for graphing. It was interesting to watch how bad the wires were in the building and how they got slowly worse over time. It really was time to either pull more wires or switch to the cable system as they had been promising to do for so long. So I went looking for the same status into for the cable mode to see if there was anything interesting there to parse out and track just for fun.

Even though the modem is on the standard router address of 192.168.0.1 there is another page that gives all the status information on address 192.168.100.1 which seems strange to me, why run it on a separate IP like that and not just on a different link on the main router address? I suppose it’s possible that is bound to that network and ip address regardless of what NAT network you have the rest of your network set to? So perhaps that will still work even if your network is set to use the 10.0.0.xxx range? I don’t know but it seems needlessly complicated.

There is a lot of fun info there that I’ll probably parse out in the future though I hope the cable modem will be operating well enough that I won’t really care. At the very end of the tab bar across the top though is a tab called “Advanced” and so I had to have a look at that only to be provided with a password field. Obviously setup with a daily password system that takes a known salt and does some kind of hash or encryption against the date to come up with a daily changing password. This is really a good idea as even if the user is looking over your shoulder and sees the password, or you have to send a password to one in an email or something after the day you did it it’s no longer valid and they can’t publish it and let the entire internet into the machine to screw everything up. Indeed, searching on the internet led to several sites with information about this and even daily password calculators like this one arris password of the day generator that one didn’t work as it seems each company changes the salt value for it. I never did get into that screen but I did learn some interesting things in trying to find out what it was. On that site above he talks about decoding the router.data file that you can download from the regular interface as a backup which I had already done. On his page he talks about having to untar it and then decrypt it. None of that was necessary for my file from this machine. It turned out that each line was just base64 encoded. Each line has to be decoded separately you can’t do the entire file. There was no other encryption.

The file that resulted had some initial setup stuff that starts with MAC address like data and long hex strings. None of which were the right length to have been the salt for the daily password unfortunately. And then a user section with a long list of SNMP looking addresses and all the configuration data that I had entered into the machine. Again nothing that looked like the salt but if the device supports SNMP then it may be possible to scan more values in there and find that as well as lots of other information. Of course there are no comments or anything else in the file that would help you figure it out I present mine here with any real data or password information changed. If you would benefit from seeing the actual data in these fields I’ll send you the real file, I doubt anyone really cares about getting on the wifi at my mother in laws house but I’m still not posting the passwords to the internet.


<POSTDATAVER>
version 4
</POSTDATAVER>
<CLIENTDB>
version 1
8c:ec:4b:e8:be:0f 0 4c2d31303639 7e6e6f636f6d6d656e747e 00000000
00:0e:8f:92:e4:f1 0 756e6b6e6f776e 7e6e6f636f6d6d656e747e 00000000
48:bf:6b:71:89:b3 0 4d696e64794e6168736950686f6e65 7e6e6f636f6d6d656e747e 00000000
40:4d:7f:1c:2a:a6 0 4d696e64797341706c655761746368 7e6e6f636f6d6d656e747e 00000000
dc:a9:04:8c:f2:c8 0 4a616d65732d4d6163626f6f6b 7e6e6f636f6d6d656e747e 00000000
48:d7:05:d9:14:bf 0 4d696e64794e616841697232303134 7e6e6f636f6d6d656e747e 00000000
</CLIENTDB>
<USR>
1.3.6.1.4.1.4115.1.3.7.1.1.10.0=admin;4;&_=@<20180419 233953>TR69 (likely the admin password user name, though I do not see the actual
admin password here. Likely it will only be present if you set it to something
other than the default which was just "password")
1.3.6.1.4.1.4115.1.3.7.1.1.6.0=10800;66;&_=@<20180606 120200>TR69
1.3.6.1.4.1.4115.1.20.1.1.3.22.1.3.10001=1;2;&_=@<20180727 085921>USER
1.3.6.1.4.1.4115.1.20.1.1.3.22.1.3.10101=1;2;&_=@<20180727 085923>USER
1.3.6.1.4.1.4115.1.20.1.1.2.2.1.11.200=%24C5A81164;4;&_=@<20180728 182904>USER (5 hex values, altered from what they really were)
1.3.6.1.4.1.4115.1.20.1.1.2.2.1.19.200=1;2;&_=@<20180728 182904>USER
1.3.6.1.4.1.4115.1.20.1.1.3.26.1.2.10001=wifipassword;4;&_=@<20180728 183201>USER (one of these is the password for the 2.4 ghz and one for the 5ghz
1.3.6.1.4.1.4115.1.20.1.1.3.26.1.2.10101=wifipassword;4;&_=@<20180728 183446>USER since I have them set to the same I don't know which is which)
1.3.6.1.4.1.4115.1.20.1.1.4.21.0=1;2;&_=@<20180728 183615>USER
1.3.6.1.4.1.4115.1.20.1.1.3.22.1.2.10001=wifi SSID;4;&_=@<20180728 184019>USER (the SSID of the 2.4 and 5ghz wifi networks, again I have them set
1.3.6.1.4.1.4115.1.20.1.1.3.22.1.2.10101=wifi SSID;4;&_=@<20180728 184104>USER to the same value so I don't know which is which)
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.2.1=name of a NAT map;4;&_=@<20180728 184933>USER (this is a NAT port mapping
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.3.1=start port;66;&_=@<20180728 184934>USER there are 4 port setting associated with each mapping, the external
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.4.1=end port;66;&_=@<20180728 184934>USER start and end port and the internal start and end port. I don't know
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.5.1=1;2;&_=@<20180728 184935>USER which of these 4 values here is which as I am passing through only one port
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.7.1=%24C1A81042;4;&_=@<20180728 184936>USER so all 4 values were the same.)
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.9.1=start port;66;&_=@<20180728 184937>USER
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.10.1=end port;66;&_=@<20180728 184938>USER
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.11.1=1;2;&_=@<20180728 184939>USER
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.2.2=another port map;4;&_=@<20180728 192237>USER (another port mapping here)
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.3.2=start port;66;&_=@<20180728 192237>USER
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.4.2=end port;66;&_=@<20180728 192238>USER
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.5.2=1;2;&_=@<20180728 192238>USER
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.7.2=%24C1A84012;4;&_=@<20180728 192239>USER
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.9.2=start port;66;&_=@<20180728 192240>USER
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.10.2=end port;66;&_=@<20180728 192240>USER
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.11.2=1;2;&_=@<20180728 192240>USER
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.2.3=another port map;4;&_=@<20180728 192319>USER (yet another NAT mapping)
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.3.3=ext port start;66;&_=@<20180728 192319>USER (in this case I can see this is the external port)
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.4.3=ext port end;66;&_=@<20180728 192320>USER
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.5.3=1;2;&_=@<20180728 192320>USER
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.7.3=%24C5A80BC4;4;&_=@<20180728 192321>USER
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.9.3=int port start;66;&_=@<20180728 192322>USER
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.10.3=int port end;66;&_=@<20180728 192322>USER
1.3.6.1.4.1.4115.1.20.1.1.4.12.1.11.3=1;2;&_=@<20180728 192323>USER
1.3.6.1.4.1.4115.1.20.1.1.4.18.1.0=1;2;&_=@<20180728 202234>USER
1.3.6.1.4.1.4115.1.20.1.1.4.18.2.0=1;2;&_=@<20180728 202235>USER
1.3.6.1.4.1.4115.1.20.1.1.4.18.3.0=dyndns username;4;&_=@<20180728 202235>USER
1.3.6.1.4.1.4115.1.20.1.1.4.18.4.0=dyndns password;4;&_=@<20180728 202236>USER
1.3.6.1.4.1.4115.1.20.1.1.4.18.5.0=dyndns hostname;4;&_=@<20180728 202236>USER
</USR>


Note that Ive munged the data above including the hex values just so I dont expose something important. Not that I think there is anything anyone could do with this info who wasnt already standing in her living room but still. Since I dont know what it is I altered it but kept the layout and length and format the same. My comments in parentheses are also obviously not part of the actual file either.

So it may be possible to setup these devices via SNMP including all the port mappings you wish as well as any other user data. Thats potentially useful if you wish to dynamically turn on and off port mappings or something like that. Im sure that much more data is available to a real SNMP dump of the thing assuming it doesnt require other passwords or encryption to set the data. In future visits I will play with the SNMP properties of the modem but I didnt have a chance to do that this trip. I might be able to get the connection quality and error rate that way which would be interesting to graph.

If someone can see the salt for the advanced daily password in that Id sure love to know what it is. It may be discoverable via a different SNMP address and Ill experiment with that in the future.

Wednesday, May 30, 2018

Restarting an AirPort via AppleScript

Apples airports are great devices, but like so many “appliances” they need a restart once in a while. Apple has the newer models pretty well locked down. There is no web interface, nothing you can hit with a curl command and no SNMP or anything else to talk to to tell it to restart itself.

After I lost my entire wifi network the other day and I had to restart it manually I decided it was time to tackle the AirPort Utility via User Interface Scripting. It turns out you can do this, sort of... There are either bugs in the AirPort utility and it’s implementation of user interface scripting, even though you’re not supposed to have to implement it, it’s just there because of the OS, or there are some limitations to it that aren’t clearly documented.

The applescript to restart an airport is pretty simple actually, this handler will do it, sometimes:

on restartAirportByName(theName)
tell application "System Events"
tell process "AirPort Utility"
click (first image of group 1 of scroll area 1 of window 1 where its name starts with theName)
click menu item "Restart…" of menu "Base Station" of menu bar 1
end tell
end tell

end restartAirportByName


Note that handler doesn’t actually launch the AirPort Utility first, You should do something like:

launch application “AirPort Utility”
delay( 5)

Before calling that.

There are some gotchas right away, the ellipsis in the Restart menu item name is a UTF8 ellipsis character and not just three periods. If you use three periods it won’t be able to find the menu to click.

It won’t work if the application is hidden or the main window minimized. It does not have to be frontmost, but it cannot be hidden in any way.

The name of the “process” is case sensitive. It must have the same case as the actual application in order to match, even though much else of AppleScript is not case sensitive this is. It must be “AirPort Utility” or it won’t work.

Next the script or the host of the script must have assistive access turned on in order to use the UI scripting at all. You have to do that in the System Preferences, Privacy & Security pane under Assistive Access. This will also break if you are saving the script as an application and make any edits to it. You must go in and uncheck and recheck the checkbox next to the application you created or it will know that you changed it and won’t let it access the UI. Again, you don’t get the helpful error message that you’d get if you hadn’t added it to the list at all telling you that assistive access is not enabled for this program, it just fails to be able to do anything.

Beyond that there is something weird about how AirPort Utility processes the event, or there is a limitation for processes not specifically started by a user which may be the case. I would expect those to generate an error specifically telling you that you don’t have privs. Sometimes, or all the time depending on the OS version you get an error about a bad index when trying to get a reference to window 1. I’ve tried accessing the window by name and several other hacky work arounds but it doesn’t seem to matter. If you run the script from a program that has assistive access turned on, but wasn’t launched by the user you’ll definitely get this error at least once, and then it might start working for the second time. Or it might not.

In my testing the best solution is to embed that bit of script into an applescript saved as an application. Then you can give it privs to use assistive access and just launch it, or use some other system to launch the app. That seems to work. At least on OSX 10.13. There still seem to be problems doing these things on 10.12.

I’ve wrapped that and a whole lot more scanning and checking code including the ability to monitor your internet connection and get your external IP address into a larger script that talks to my home automation software of choice XTension (well it should be my choice as I’m one of the developers) which will definitely be useful for other XTension users as long as they are running an OS version that includes a less buggy AirPort Utility. That code is free and open and if you are interested in how to get the rest of that info out of it it’s also fully documented with comments in the code here.

Tuesday, May 22, 2018

A fix for Stupidly Slow VNC performance on the Mac

On and off I’ve had a problem with my house server where the remote desktop or VNC connection was just so slow that it was almost unusable. I normally load test versions of XTension by dragging them and dropping them onto the VNC window. This was also excruciatingly slow. Like 200k a second slow rather than the 60 or 70 megs a second I would expect.

I’ve spent months fighting with it, restarting seems to help for a few minutes but it comes right back. I’ve run new ethernet cables, I’ve tried plugging directly into the switch. Of course it never occurred to me to test other throughput to the server. Connecting over regular SMB file sharing to it results in more than 70 meg a second throughput. So it had to be something with VNC.

Just gazing at my system logs the other day I watched 3 or 4 zombied windows machines trying to log into my VNC server as it’s exposed to the internet. I have the ports for VNC passed through my router so I can log in remotely if I ever needed to. They are just running through the list of most common passwords and dictionary words hoping that my password is just “secret” and I’ve just ignored them up until now.

Just as a test I turned the VNC passthrough off on my router and guess what? I can now VNC like I could before and I can get 70meg a second throughput when dragging large files onto the VNC window! It was truly awful to try to type in an edit script window on the machine remotely, but now it’s wonderful again.

It seems that a failed password attempt causes the VNC server to hang up for a fraction of a second. Multiply that by 2 or 3 attempts a second from different infected windows machines and it comes to a screeching halt.

For myself I’ll just leave those passthroughs off unless I specifically think I’m going to need them. Moving your VNC install to a higher random port would solve the problem from just the hacked windows machines as they won’t bother to do a full port scan on your machine, but it won’t protect you from someone targeting you specifically as they will do a port scan and find it no matter where you put it. It would definitely help though.

OSX is supposed to have an adaptive firewall that you can turn on that watches the log files for those failed password attempts and if someone has more than a certain number it will temporarily block them. I can’t get this to actually work on my machine though. Much info out there suggests that this is broken on OSX but I will experiment further with it.

Saturday, January 6, 2018

Copper Veneer Is Beautiful

I am in the middle of a project building floor to ceiling bookshelves across one wall of our living room. It’s a rather larger project than I normally take on and has provided many unique and stimulating challenges both in design and fabrication... We decided on a gentle arch at the top of each book case and in order for them not to look ordinary and drab I’m experimenting with some antiqued copper veneers that I got from Veneer Supplies.com The same place I got the wonderful dark burl I used to make our Veneered Mantle. I chickened out and didn’t buy any of the really weird and wonderful copper patinas available, but opted for a sheet of just the antiqued stuff to experiment with.  this stuff is beautiful.

This is just a test run to hang one up so that She Who Must Be Obeyed can get a feel for it and make sure that it’s what she wants before I commit to making a dozen of these things. There are only 6 bookshelves, but there will be an arch at the top section, and an arch below the library ladder rail 8 feet up. With each part of this project I keep thinking that won’t be a lot of extra work, and then I remember that oh, there are six bookshelves so I have to do it six times. And then with these I have to make twice that number. But it will be worth it, these look fantastic.

Now if only I could make the stain on the routed edge look a bit more even. I will have to sand that a lot more aggressively to smooth out the wood before applying the stain on the real ones.

Dont worry about the slightly funky edges. I made it an inch or so wider than it needs to be and will trim those off to perfection on the table saw before putting it into place.









Friday, January 5, 2018

High Sierra hangs your main thread when the screen is asleep!

Thanks to Ben Bird of bensoftware and SecuritySpy fame I was able to work around the problem in High Sierra. If any Mac users are interested in really good DVR software I can whole heartedly recommend his Security Spy app. It works great with XTension and we even have an interface that will let you get motion data out of the program and assigned to units in XTension for taking other actions based on that.

The problem is evident in Mac OS version 10.13.2 and may actually have been introduced in the .2 update as I don’t recall having this problem immediately upon my updating to High Sierra but only more recently. I could be completely wrong about that though.

It turns out it has nothing to do with App Nap at all, I was barking up the wrong tree about that completely. It’s a rather nasty bug in some of Apple's screen drawing code. If you have an app that has an interface and it causes interface elements to update once the screen goes to sleep High Sierra will eventually just hang. Something causes a thread contention with the screen drawing. This might not be a big problem for most user apps as if the screen is asleep then you’re not using them. They will wake up along with the screen as soon as you touch the computer again. For an app that needs to keep doing things while you’re not there like XTension does this is tragic.

My first attempt at a fix was not successful. You can subscribe to events to tell you when the screen is put to sleep or woken up. In the event where the screen is going to sleep I hid all my open windows and in the event waking up the screen I showed them again. This was not enough to stop the thread contention issue. Possibly even drawing the menu is enough to cause the problem. I am not able to disable all the drawing in the app as far as I know. Though there are calls for windows to defer updates. There may be a Cocoa way to disable all screen drawing and I’ll have a look at that later. The pressure is off right now because I was able to get it working with a background helper app to check for the the main thread hanging in the app.

Ben had suggested a thread that pinged or otherwise checked on the regular spinning of the apps main thread. I tried this, but since I’m working in Xojo I’m limited to cooperative threads. If you block the main thread in a Xojo app you also block any other threads. Which is important to know if you’re working with threads and using semaphores to manage access to resources between the threads and the main thread. So I wasn’t very confident that the pinging thread would be effective since I expected that thread to get hung as well. That was indeed the case, the second thread stopped as well and so was never able to wake the screen.

The solution, hopefully temporary until Apple fixes this, was to create a separate terminal app to do the waking of the screen. Since command line apps have no GUI they aren’t affected by the problem. The background app just listens for a single character to arrive via STDIN. If it doesn’t get one from the main app at least every 5 seconds it makes a call to the Power Manager to assert a User Activity. Using the MBS plugins for Xojo this is easy, but I suspect it could fairly easily be done with a declare or external method call to the right library.  To wake the screen I just did this:


Dim userActivityID as integer

Dim err As Integer = IOPMAssertionMBS.DeclareUserActivity( "waking screen for thread contention", IOPMAssertionMBS.kIOPMUserActiveLocal, userActivityID)
   
You’re supposed to keep that userActivityID around and re-use it for any previous calls, so don’t just reset it to 0 each time you make this call. Though I’m not sure if thats really necessary.

The terminal app is copied into XTension’s resources folder during compilation and I’ve created an interactive shell class in XTension that is very simple. It just launches the app and starts a timer that sends a “P” to the shells stdin every second. Since timers fire on the main thread they are all silenced by the thread contention issue and so it stops sending that ping. 5 seconds later the screen wakes up and all starts working again. If I set the timing to much less than 5 seconds then I seem to get a lot of wakeups for slightly longer program operations like regular database saves and such. The 5 seconds seems a good compromise between having it hang all the time and just keeping the screen awake all the time.

Which is the other solution. You could just call that user activity constantly or tell your users to not let the screen go to sleep at all. Both things also “solve” the problem. Interestingly enough making the request to stop app nap and adding the flag for stopping idle sleeping of the screen did not solve the problem. So App Nap is not keeping the screen awake in those cases, or perhaps the screen saver is starting or something I don’t know, but just doing that did not solve the problem.