Start somewhere

When faced with a problem you don’t understand, do any part of it you do understand, then look at it again.

-Robert A. Heinlein, The Moon is a Harsh Mistress

I see a lot of folks that have lofty ideas, and never get anywhere, because they don't want to start until they are sure they can do it right the first time. Here's the thing, you can't do anything right the first time. So just dive in somewhere. Consider the mistakes lessons learned, and the cost of building expertise. Be prepared to feel stupid a lot, because that's the price of learning something new.

You'll never finish a project if you don't start it. You'll never complete a project, if you don't keep working on it. These may seem obvious, but it's impressive how many people don't seem to realize this.

(Thanks to Ed for posting this first.)

Irene's aftermath in the Hudson Valley

The flood waters here in the Hudson Valley have largely subsided. That 12.8' prediction for the Wappingers turned out to be way off. It crested north of 15' (second highest recorded value, above the 2007 spring flood). As that's the only river with a sensor in Dutchess county, it is an indicator of how swollen every little creek was. One of those, the Fallkill, which runs through the City of Poughkeepsie, breached it's banks putting a bunch of city blocks under water.

Our town, LaGrange, fared very well. I give a lot of credit to our municipal government and public servants for that. They were putting rapid updates with critical information on their website as the storm progressed. They preemptively declared a state of emergency to keep people off the roads. They had dry ice and water stations running before the storm even arrived, and they were constantly monitoring and closing off road sections once they became dangerous.

For our particular neighborhood the most important thing the town did was 5 years ago. They replaced the culvert that lets a little stream cross under the main road into the neighborhood with one that was just gigantic. When they did it I was really puzzled, because it seemed like a ridiculous over kill. On Sunday we walked down there, and found the culvert was 3/4 full. So extra kudos to whoever decided that needed to be upgraded, because they were right, and if that hadn't been done, we'd be stranded in our neighborhood. I sent a email of thanks to our officials over this response, because they deserved it.

Another party that did a lot to minimize the effect in our area was Central Hudson. We had a double blizzard here a couple of years ago that completely taxed their emergency response. NY state government called them to task on it, and they started focusing hard on disaster recovery. For the last two years they've been doing an incredible amount of preventative maintenance, new taller poles, backup pole structures, cutting back trees that could be dangerous to main lines. They brought in additional crews from Kansas ahead of the storm, knowing they were going to need more manpower. They also started communicating a lot more during disasters. I've gotten a daily email from them every day since the storm hit with an update of what's going on. Their storm watch website tracks every single outage report, and in 30 seconds you can report an outage via their mobile site.

There is a lot of damage here, mostly caused by the water. But in the grand scheme of things, it's not that bad. And a reason for that is that we've actually got a lot of organizations, our local governments, and our local utilities, investing in advance in resilient infrastructure.

So to all that were a part of that, I just want to say: THANK YOU.

Irene in Poughkeepsie

We're back in NY right now to ride out the storm. There is a pretty constant roar outside as the 40 mile hr winds rumble through the tree tops. My rain guage says it's raining at about 1/2" an hour for the last 4 hours. All told we're at about 4" of rain so far. My guess is we'll get 8" by the time it's all over.

The rains and winds still haven't peaked. That's projected for early this afternoon.

NOAA hourly weather graph for poughkeepsie

The wappingers creek is projected to spike up to 12.8' from this, which is going to mean quite a bit of flooding.

Projected height of wappingers creek

Our town actually issued a state of emergency, so it's currently illegal to be on the roads. For safety's sake, stay indoors.

Revitalizing a software project

I became one of the co-maintainers of the epublish drupal module this year, as we are an active user of it at the Poughkeepsie Farm Project, where we use it for our newsletter. I had about 6 big fixes to it that we needed, and so I was eventually just added to the project so I could commit them myself. Since then I've tried to clear out some of the backlog of issues, and added a few features.

In doing so I realized how fragile some pieces of the code were, as I broke things pretty regularly. Fragile code is the biggest inhibitor to innovation, because no one wants to try anything exciting and new when just keeping things working after simple patches is so exciting (and not in a good way).

So I decided to take on the project of revitalizing the code, in a couple of steps.

Add Automated Tests

Drupal has a framework called SimpleTest that lets you include automated tests in your code. These tests simulate a non javascript browser, much like using mechanize in ruby. There are also a lot of convenience functions for testing the results. In Drupal 7 this is a built in, in Drupal 6 it's an add on module that requires a small patch to core.

Before actually making any substantial changes, I wanted to actually nail down current behavior, so that I know if I'm going to break anything. Over the past three weeks I've been figuring out how SimpleTest works, and writing tests for the code. I'm now up to 798 assertions, and have covered a lot of basic workflow, creating / deleting / updating publications and editions. I'd say I've probably got 1/3 of the coverage I should have, but I've ensured I've covered the goofy bits around article ordering that I broke multiple times in the last six months. I consider this a pretty good start.

As with all testing, the hardest part is the first test. That takes about eight hours to understand the system, and build and run a test that makes sense. I did that on a rainy Saturday before vacation. After that adding tests is easily done in 30 minute increments, which I puttered on in the mornings before Susan woke up during our vacation. In the process of getting these tests to work, I found and fixed a few critical bugs as well (when things didn't work like they really should). Epublish 1.13 reflects these changes.

Extract the Database Layer

Step 2, which I'm starting on now, is to remove the database layer from the core epublish.module. My goal is to have no SQL statements in the core module file, and instead have those all in an epublish_api.inc file. They can be replaced with functions like "epublish_list_publications" and "epublish_add_publication($pub)".

Again, this can be done in small increments, mornings while sipping coffee, or after work. I'm mostly looking for the kinds of SQL statements currently used, and trying to come up with the sane and well understood function name that they would map to.

Additional possible refactoring

One thing I found in doing the lending module is that it's actually pretty nice to isolate the 3 form functions (display, validate, submit) for a specific form into it's own include file. This makes it easy to know you are looking at everything about a transaction in one place. I'm going to see if this makes sense after the database pull out.

Theming functions also make sense to have in a separate place, they are logically quite different than other code in a drupal module, as they are the one place you'll get markup included in the file.

New Features

All of this is really about adding a new and better interface for managing the articles in an edition, as the current interface leaves a lot to be desired, especially because of the automatically imported calendar entries we've got on the PFP site. It will also mean that a port to Drupal 7 can really happen. There are lots of other things holding the PFP site back on Drupal 6, but this is one of the bigger ones.

Brownfield Development

When you are a young and eager developer you always want to start from scratch on projects. The assumption is you can write a replacement in less time than it takes to figure out what the existing code does.

I'm now of the opinion that a whole sale rewrite is almost always the wrong thing to do. It means a very long period of time where you have nothing working, which means there is a good chance you give up on the project all together, with a lot of wasted effort.

This kind of incremental refactoring, adding tests along the way, lets you make pieces of the code better, and continue regular releases on existing function (even finding some bug fixes as you go). If this becomes uninteresting to me at any point, I'll still have made the code easier to read, created a solid battery of tests, and handled edge conditions much better.

And this is how the majority of real software projects work. You aren't there at the beginning, you aren't there at the end. Your impact is really whether you can leave things better for the guy or gal that comes after you.

Cool Tool: Kreg Jig

Kreg Jig Pocket Hole System

One of the reasons that I started getting so excited working with wood this summer was because of my purchase of a Kreg Jig. I was introduced to this tool at the IBM Day of Service by a few folks that built furniture as a hobby. With it we were able to build a quite nice cabinet out of 3/4" furniture grade plywood without any additional supports.

I've used this on the coffee table and the white board frame that I've built so far, and I've got many other projects in mind that this will come in handy for. It's definitely the kind of tool that opens up new avenues of construction, even for the novice like myself.

Highly recommended. Available from Amazon, or locally at Lowes in this neck of the woods.

Radio Silence in Greenbank WV

One of the more impromptu parts of our vacation was a realization that without going too far out of the way (adding only about an extra hour of driving to DC) we could hit up the National Radio Astronomy Observatory in Greenbank, West Virginia.

IMG_1026.JPG
NRAO Greenank Radio Telescope. Dish surface is over 2 acres in size.

The NRAO is home to the world's largest steerable radio telescope. The surface of which is over 2 acres in size. Beyond just the big dish there are a number of other large scale radio observatories, some run by the observatory directly, others leased out to Universities. Because of an act of Congress, the NRAO has sole licensing authority for commercial wireless in a 100 mile radius ("We're the reason your cell phone doesn't work right now"), and complete authority over all types of wireless in a 10 mile range. If your microwave goes hay wire and starts emitting radio signals, they've got a truck outfitted with antennas which will help them find your house. They will then attempt to fix it, or will replace it if it's unfixable.

IMG_1029.JPG
NRAO radio emission locator truck

We managed to arrive just as the 4pm tour was starting. We got a great presentation inside the main visitors center, then a tour of the telescope grounds in our diesel powered van. Nothing with spark plugs is allowed within 1 mile of the Greenbank Telescope, as it would generate enough radio noise to mess up experiments. I also have no digital camera pictures within that radius, as the stray RF from a digital camera (both being on and taking pictures) inside the 1 mile zone will show up in the telescope data. They have an observation platform at 1 mile just for that purpose. Our tour guide was pretty awesome, telling lots of great stories about the challenges of running super sensitive radio experiments.

IMG_1036.JPG
The EM spectrum. Wavelengths are correct scale, which is a nice detail.
IMG_1044.JPG
IR photo of Susan and Me. We also used it to prove her feet are abnormally cold.

Back at the visitors center they had an amazingly good display on the Science of Electromagnetic Radiation. Including displays in IR, Spectrographs, investigations into Pulsars, and many other great exhibits. We spent nearly an hour wandering around there.

IMG_1042.JPG
The Overhead display at NRAO, which covers the ceiling of the entire room. This is by far the best visualization of the milkway that I've seen.

This was definitely a great stop. By it's nature, it's pretty far away from everything, but the hills of West Virginia are beautiful, so it's a nice scenic drive to get there. We might go back in the future, as there is some great camping / hiking in that area of WV.

Cleveland's Beer District - Market Garden Brewery

We just returned from a road trip of a vacation, where we visited many friends and family. It was sort of like the vacations we've taken in far away lands, except instead of jumping on a plane or a train every 2 - 4 days, this time it was our own car. Home to Ithaca, NY to Cleveland, Ohio to Charleston, WV to Greenbank, WV to Greater DC area then home. About 2000 miles in 12 days.

The reason the trip happened at all was because of the Ohio portion. The Market Garden Brewery just opened last month, with Susan's brother Andy as brewmaster, so we had to go check it out. It's an American style beer garden, with a few different inside seating spaces, and roof seating in the works for next year.

IMG_0949.JPG
Market Garden Brewery - Beer Garden
IMG_0948.JPG
Market Garden Brewery - Beer Garden
IMG_0947.JPG
Market Garden Brewery - Bar and Dining Area
IMG_0945.JPG
Market Garden Brewery - Other Bar which opens into a meeting room

They are currently running 10 taps with 10 different brews going at once. Most are what Andy calls "Session Beers", meaning 5% or alcohol or less. This goes contrary to the current trend among internet beer aficionados, who are mostly drinking much stronger stuff at home, but for a brew pub, is the only way you can sanely have more than one glass. Over the course of a couple of nights, plus some growlers that came back to the house and to WV, we managed to sample most of the menu. The Cluster Fuggle IPA and Tart Blanche were probably my favorites, with Pearl Street PM and the Strong Ale as close seconds. They also did a great job with the food at the place.

With the Market Garden Brewery, Bier Mart, and Great Lakes Brewery across the street, the whole area is basically now the Cleveland Beer District. This appears to be working well for all parties involved, because you've got so many good options right across from each other. If one doesn't fit your taste, just go down one door to the next.

I suspect that we'll be making a regular pilgrimage out to Cleveland now. Like I've said many times, it's really a great thing to have a beer maker in the family.

100 MHVLUG Meetings

Last night was the 100th MHVLUG meeting, which makes it a good time to reflect on a lot of things.

Good organizing is invisible, and takes a lot of energy.

Running a group like this for eight and a half years has truly made me appreciate a well run event, as I know how much work it takes. Good organizing, like good infrastructure, is invisible, which also makes it really easy to take for granted by people that have never had to do it themselves. Just like you only notice your plumbing when there is a leak, you only notice the effort required on an event when the equipment fails, or the speaker gets lost, or they turn off the power on your venue and you only find out about it 3 days in advance.

You must take 100% responsibility for the outcome.

Groups break down when no one takes responsibility for the whole. This doesn't mean that you've got to do everything yourself (more on that in a second), but at the end of the day someone has to be responsible for the whole thing coming together. I've seen a lot of groups start to get dysfunctional when every one just sticks to their defined roles, and no one is interested in the whole.

Groups work really well when more than one person feels 100% responsible for making things work. Which is why I'm really happy with the officers we've now got with MHVLUG.

It's too much work to do alone.

For a lot of years I was pretty much lone wolf in organizing MHVLUG, and it really started to burn me out. I wasn't really enjoying it any more, and kind of lost it about a year ago on our mailing list. In the end, our incredible community stepped up, and a number of folks stepped forward into roles to help share the burden of MHVLUG organizing. The results have been fantastic!

Ben, Paul, and Joe have all been putting in great work this year to make things happen. Our attendance has gone from low 20s, to the low 30s at basically every meeting this year (we passed 40 at the Arduino meeting). That's due to a combination of dedicated publicity, a really fresh and exciting list of programming, and activity on our website showing what we've done in the past. All really wonderful stuff. And all stuff that I just didn't have the ability to handle when I was doing this on my own.

The power of this extra help can clearly be seen in the reaction to the venue change yesterday. On Monday we found out that Central Hudson was shutting off power to the street where our meeting takes place. I immediately sent out an email to our officers asking about options. Within an hour I was getting IMs from Ben that they were finalizing a new space at Marist. By the end of Monday afternoon we had the new space booked, Maps and directions on the website, and announcements out. Most of this was done by others, so I only needed to key an eye on things and make sure everything was coming together. It was all incredibly smooth. And even after the venue change, we had 30 folks at the meeting, something which would have been unheard of in the past.

Success breeds excitement, which breeds more success

I'm now more excited than ever about our group. This year has been simply amazing. It used to be that summer spots were largely dead air, and we were lucky if we had 18 people at a meeting in July or August. But now they are vibrant cutting edge presentation, that have massive engagement from the audience. Last night's really excellent Scala talk by Ron Coleman had 30 people, and at least 10 members of the audience had questions or comments during the presentation. That level of engagement really brings an energy which is amazing.

This has been a great journey so far, and I can't wait to see how the group evolves over time. Getting this community off the ground, and managing to not only have it still exist 8 years later, but be as vibrant as ever, is one of the things I'm most proud about. Here's to another 100 meetings, which I'm sure will be every bit as good as the last 100 meetings.

AOS Star Fest 2011

A new moon, not a cloud in the sky, and it's above freezing. That's very rare combination of things in the North East. But this year at AOS we got just that on Saturday night. After 1am we started struggling with dewing (to an extent I'd never seen before). The Milky Way was strong, and high, and even in the moments when we weren't looking at Galaxies, Globular Clusters, and Nebula, we were just staring up, taking in the sky in all it's glory.

Every time I go to one of these events I learn more about the universe, more about equipment, and more about getting the most out of a night of observing. It's an amazing and energizing thing.

While there are no pictures of the observing while it was happening, I got a number of pictures just at dusk, of the setup. Hope you enjoy.