We Create Games

Chumby Advanced Tricks

Posted January 29th by Jay Crossler in Code, Home Automation, iPhone, Life, Pugs, Web

(check out the above Chumby tiger cover here)

(built on previous Chumby iRemote post here)

Overview
This past month, I extended the web server on my older-style Chumby to add a lot of functionality to my house. For a while, I’ve wanted to automate a number of my home systems – X10 to control our home lights (specifically the Living Room lights), the Chumby to play music with, and web cameras to watch our dogs during the day. As the new Chumby just came out, I thought it would be useful to build all of the control into the Chumby device, so that others could also try the same thing.

I also build a feedback system that would try to monitor the amount of times our dogs jumped and barked at cars, then change music and turn on/off lights to find the combination that calmed them the most. Unfortunately, I found that there is no magic configuration – rather the dogs seem to relax whenever the lights and music change (possibly because they feel that someone is home). Either way, it was a cool experiment that I learned a lot from. It is a bit “fragile”, meaning that there’s a good amount of custom configuration that you would need to use to reuse everything. Nonetheless, I’m posting all code and configuration samples here in case people want to set parts up themselves. The X10 webpage, for example, is pretty useful even if it is the only piece loaded.

ch_brain1 ch_brain2

Details
I call this marvelous contraption the “Puppy Ubiquitous Pacifying Preference Engine” (PUPPE). If you want to be all research, the Technology Goal was to build an automation engine that optimizes environmental support for dogs left home alone. The Research question I set to achieve was: Can an automatic feedback engine be used to optimize relaxation for dogs by controlling home automation electronics? I was partially able to achieve this goal, though in a way I completely didn’t plan for. Rather than find an optimal configuration, I found that the more I randomly changed the lights and music throughout the day, the more relaxed our dogs seemed to behave.

The tools I used for this were:

  • X10 Wall Light Switch
  • X10 Application Switch
  • X10 Home Automation software on Windows XP (old laptop LAP1)
    • X10 Controller software listening on TCP port 6003
  • Mobiscope Desktop Webcam controller on Windows XP (old laptop LAP2)
    • copSSH – ssh server for secure remote connections
    • ipsecCmd and IP Security Policies to filter IP traffic
  • Two cheap USB webcams
  • Mobiscope web site for hosting video
  • Weather Web Service for determining sunset time
  • Chumby running ChumbyOS Linux (CHUMBY3)

Client devices:

  • User Laptops (LAP4 and LAP5)
  • iPhones (PHONE6 and PHONE7)

There were a number of moving pieces to this configuration, and each piece took a while to determine the best approach to implement. I’ll describe the solutions and tricks I came up with below. Many people probably won’t have two extra laptops laying around, so the configuration will likely change depending on circumstance. I spent a good portion of overall effort intentionally trying out different approaches and security option, so there are many other ways of having the same effect. The X10 Commander interface is shown below:

ch_x10cmdr

Technical Architecture

ch_arch

The picture above shows the logical architecture I came up with. All the software actually ran on the Chumby (except the Mobiscope viewing page that is hosted on that company’s web server). I firmly believe that as systems become more complex (some people call this a Service Oriented Architecture – which I’ve studied in depth, but don’t want to go into here), it’s important to build very simple interfaces. I implemented three main types of API service interfaces, each very simple to work with. I call these “ramps” or “loose couplers.”

Loose Couplers:
HTTP GET-based web service. In the simplest configuration, a web service is a web page that a machine returns when it receives a certain input. The Chumby has a CGI-based web server that can be used to run simple sh shell scripts. We pass in variables through an HTTP querystring (the part following a ? in a URL). This way, when an event.cgi?setVolume0 is called, the cgi file shell command executes a command to mute the radio. We use x10.cgi to listen for light-switch control messages, event.cgi for Chumby-specific events, and cameras.cgi for commands to control the web cameras.

The Chumby also can act as a hosting web server, using an index.cgi page that points to all of the other commands. The interface looks like this:

ch_brain1

And the hosting code looks like (the launch javascript is declared in the full file, see attached code):

ch_brain_code

SSH-based camera control service. Secure Shell or SSH is a network protocol that allows data to be exchanged using a secure channel between two networked devices (as described in RFC 4252). Most linux or unix-based machines (including the Chumby) have an ssh client or an ssh server built in. As we want to control services on an Windows XP machine, we needed to add an ssh server, so I chose copSSH after testing multiple other options. By using IP Security policies, we can add a firewall filtering policy to the machine from the command line with a batch file (Cam-Block.bat) that runs `ipseccmd –w REG –p “BlockMobiscope” –x`, where BlockMobiscope is a windows IP filtering policy as shown below:

ch_ipsec

To make remote triggering as simple as possible, we want to enable PKI key exchange on both machines. This allows a single shell command `ssh UserName@192.168.2.3 ‘./Cam-Block.bat’` run on the Chumby to securely turn on or off traffic from USB cameras being sent to the Mobiscope web server. The home router is configured to always map 192.168.2.3 to the camera server (LAP2), which saves a lot of effort when machines don’t properly identify their hostnames.

One of the most painful steps is setting up the SSH server to run without entering a password at every instance. Building this PKI key-exchange process uses ssh-keygen command to generate a client and server key. Because the Chumby does not have ssh-keygen installed, we need to build the ssh keys on another machine and remotely configure both the chumby and LAP2 server with these keys. Once we do this, we can be reasonable assured that only the Chumby can interact with the XP laptop (LAP2). Because this machine controls transferring video from our USB cameras out to the web, we want the most amount of security around this interface.

TCP-based network injection service. By using the X10 Active Home software with the X10 Commander program, iPhones can send TCP messages to a laptop to turn X10 devices on and off. Rather than using the iPhone client, we can spoof TCP messages to emulate the same commands that the client would send. By using WireShark, I found that the TCP commands are simple messages sent on port 6003:

ch_wireshark

Using NetCat ‘nc’ – (my new favorite linux command!), these messages can easily be replicated using the shell command: echo ‘DEVICE~sendplc~”C4 ON”‘ | nc 192.168.2.4 6003 (this is to the LAP1 Home Server). This is not very secure, but is based on inherent security limitations of the X10 Commander software. Given time, I might rewrite a more secure X10 controller. As only the lights are controlled from this server, I found that was a worthwhile privacy tradeoff.
Experimentation

Dog owners frequently ask “What are my dogs doing while I’m at work?” My wife specifically wanted to watch our dogs while she was at work (and her company blocked the firewall connection). To give her this capability of watching our dogs that are blocked off in the kitchen each day, I set up a pair of USB cameras on an old laptop and purchased the $20 Mobiscope software that is iPhone compatible. She is able to watch her dogs anytime she wishes on her iPhone.

ch_mobiscope1

After observing our dogs for a few days, we noticed that they would become more and more agitated throughout the day, and bark at cars and pedestrians walking outside our window. Some days they were more relaxed, and some days they were more agitated. Every day we played different radio stations, and surmised that different music at different times would help calm them. Also, we noticed that with shorter winter days, the sunlight dimming early in the evening seemed to agitate them more. We surmised that if we could modulate lighting conditions along with music, we could help minimize the amount of times they got up to bark out the window.

photo2

The experimentation had two phases. During the first phase, I used the two USB cameras to record the dogs in time-stopped video (3 frames/sec). By trying a number of algorithms, I determined that a good way to rate the number of “agitations” was to identify an active area of the camera view and to count the amount of pixel change over a period of a second. As shown in the picture above, if more than 84% of all pixels in the active area changed in a second (out of a max of 300% as there are 3 frames x 100%), then a recording is triggered and an “agitation” is recorded. Due to varying lighting conditions, this number was not optimal in every case – and recommendations are not to use the Mobiscope to do this process in the future.

ch_agitations_per_hr
Average # of agitations/hour at time-of-day

The # of agitations/hour was then used to build an algorithm to remotely turn on and off lights and to change radio stations in the kitchen. Unfortunately, there was no reliable way to detect agitations as they occurred. The Mobiscope web page hosted only cycling images and passed recording #s through javascript. A few attempts to “Screen scrape” the number of recordings only worked intermittently. I used a greasemonkey script that would read the data, but had many issues with reliability. (As a note, I just realized yesterday that each recording of an agitation produced a new data file in a directory – and that directory could be watched for updates to trigger the algorithm. This would be useful in follow on work).

I worked through 5 algorithms that were all sh shell scripts fed through the web page and greasemonkey script. This proved problematic, as the script never ran concurrently throughout the day. Greasemonkey just wasn’t robust enough to handle web errors or dropped connections from the Mobiscope server. As these were working locally inside my home intranet, I didn’t have a reliably secure method to tweak and monitor these while I was out of the house (and whenever I was in the house, the dogs’ behavior would change dramatically).

When the algorithms did work, it became quickly apparent that every change was a “good” one. Any time there was an agitation, if the radio was changed or a light level was adjusted, the dogs immediately calmed down for a period. It didn’t matter which radio station or which light level was set. Any change seemed to reduce agitations. The algorithm I now use just rotates radio stations and light levels randomly, without any inputs. On average, it seems to have reduced # of agitations by 45% (though I haven’t yet had enough testing to ensure it is statistically accurate or will remain lower).
Privacy Engine

In order to use the system well, we wanted to add two extra capabilities:
1) When either my wife or I come home, turn off the web cams and radio, and turn on the kitchen lights
2) As a backup, turn off any recording at sunset (in case the first doesn’t trigger)

To implement the first capability, we need a way to detect when either of us comes home. I initially tried using an X10 motion sensor in the door way, but it was quite cheap and would sometimes trigger when there was direct sunlight on it, and frequently not trigger when people walked by too quickly.

I’ve noticed through watching Wireshark that when an iPhone connects to a wifi network, it sends out a few IGMP/Multicast DNS packets to 224.0.0.251 (local multicast to all machines, as specified in RFC 3171). By listening for this message, I have the Chumby look for the correct identifiers and send out the “turn cameras off”, “mute radios”, and “turn lights on” commands.

ch_cam_block

I would have liked to detect a method for turning the cameras on when no one was home (as we frequently forgot to do so in the morning), but couldn’t find a reliable method.

To implement the second capability of always turning cameras off at sunset, I had a script that pinged a web service daily and asked for the sunset time of the current locale. The below script illustrates this method (but condenses some of the logic for readability).

ch_sunset
(Note: That should read “sunset” not “sunrise” in the web service request above… otherwise things get a little wonky.)

Conclusion

This was a fun project that had a lot of moving pieces. I built some useful functions that we still use daily. The light and radio controls are very reusable, and I will be releasing all of the source code for others to do so. I was able to partially prove out that modifying the environment modifies our dog behavior. More importantly, I gave my wife tools to feel more attached to her dogs throughout the day (and justify my spending money on technical toys).

The code is here, please download – released fully creative commons/open source (BY 3.0 license).




(required)



(required) (Won't be displayed)


Your Comment: