(Note, this old post is background on some of the things I was doing here before.)

A few weeks ago I lost my rfxcom receiver during a lightning storm. Which meant my old weather station reporting on http://home.dague.net went defunct. My plan had already been to move away from that code (some incredibly hacky and fragile ruby) and do this over again less hacky in python on a raspberry pi. I’d even ordered and received a new rfxtrx receiver for this purpose. However, nothing gets you quite so motivated to make progress as downed infrastructure….

All through the week I’ve been getting data collection working and finally getting it uploaded to Weather Underground, where I’m personal weather station KNYPOUGH7.

I’ve even started to get some local graphing working as well, like the following:

(outdoor temp, outdoor temp in cold frame, bedroom temp (sensor moved in the middle of the day for the spike), refrigerator temp, freezer temp)

The Raspberry Pi has made things interesting…. First, it’s mobile, as I have it connected via wifi to the home network. This has been incredibly useful, as it turns out there is a sweet spot for seeing all my sensors consistently, which is only possible to get to if I don’t have to worry about wired ethernet. The old solution had this sensor plugged directly into my home server, which meant drop outs on some of the sensors.

My old approach to data collection was rrdtool. As soon as I got a sensor reading I dumped it into an rrdtool database, which also made graphs. This had the advantage of being easy, and the disadvantage that rrdtool is a one way black hole for data. Once in, you can’t ever get it out sanely. It’s really designed to make operations centers look impressive and give admins the ability to eye trends, but makes working with that data not very useful. It does have the advantage of having a known fixed size, so you won’t overrun your local storage.

In the new system I’m using mysql, on the pi. It’s slow. Really slow. Often surprisingly slow. That slowness force one change in my thinking, which is to not use d3 to graph things client side, but instead generate static graphs with matplotlib on a cronjob, so user interaction won’t be slow. It means the data might be a couple minutes behind, but for weather, that’s fine.

The code… is currently terrible. However I’ve gone with the point of view that brute forcing working code right now is what I need to do to understand the patterns I want to build in this code. I spent too much time paralyzed on this project thinking through different ways to do it. So this will require some major refactoring and despecializing, already starting to happen, before it’s of any value to anyone else. Once I get there, I’ll have everything up on github as open source.