Notes from North Bay Python

North Bay Python marqee at the Mystic Theatre in Petaluma, CA

I had the pleasure of attending the first North Bay Python conference in Petaluma, CA this past weekend. IBM was a sponsor, and I gave a few quick remarks about doing python serverless actions on OpenWhisk. My two days there were full of wonderful talks and interactions with a slice of the python community.

One of the reasons I love low cost (to attendee) regional conferences like North Bay Python is that it makes technology conferences more accessible. For 40% of the 250 attendees, this was the first technology conference they'd ever gone to. Not everyone lives in New York City or San Francisco (or wants to), and having local events is critical to expanding the range of voices in the technology community.

There were tons of great talks, you can watch them all here. But I'll highlight a few moments that I'll keep with me for a while.

Fortran on stage

For a single track Python conference, we actually got to see FORTRAN in 2 different talks. It's probably more FORTRAN than I've ever read before.

Catherine Moroney is part of the team that does analysis of satellite images from the LandSat program. They've got a lot of optimized FORTRAN and C code for processing these images. But FORTAN and C aren't great languages for writing new features to orchestrate these lower level transforms. She uses Python to do this work, and can seamlessly pass data back and forth from Python to FORTRAN for data crunching. It was great to see how and when a hybrid approach like this makes the developers much more effective.

Christopher Swenson tackled FORTRAN from the other side. He hacked together a FORTAN IV interpretter in Python, so that he could run Colossal Cave Adventure (originally written for the PDP-11) as a text message game using the Twilio API. His talk wandered through some of the interesting quirks of now extinct programming languages, and the systems they were written for. This is a world before ASCII as we know it became a standard, and the idea of 32bit integers really hadn't emerged. 36bit integers were used to store 5, 7bit characters, which were later assembled into text streams.

Through the whole thing he showed snippets of FORTRAN that he had to make guesses about what it really meant, as well as be really frank on shortcuts he made to get things to work. There is no more FORTRAN IV code in the world, this didn't have to be a perfect emulator, it just had to run this one single FORTRAN IV program well enough to connect it to the internet.

You can play this by texting to +1 (669) 238-3683 right now if you want to see it in action.

Twitter Bots

Tweet: "What is Machine Learning? Easy! Machine Learning is how you automate your biases so you can think about them even less."

My vote for most hilarious talk was Benno Rice's dive into writing twitter bots. He started with pretty easy template base bots, like one producing plausible plot lines for Mid Summer Murders. This is just a stack of well crafted arrays and a random number generator.

Things got more interesting when he started diving into Markov Chain bots. Especially where you take content from a bunch of different sources. It's really easy for that to just become word salad at worst, or just confusing and "meh". He found you had to keep playing with the content mix as well as the overlap parameters to get the bots to generate something that's amusing at least some of the time. The bots he's got he doesn't let post directly, content is generated offline, and he pushes the good ones after manual review.

Benno also took a dive down the path trying to do machine learning to make these better, but mostly got worse results in his experiments. But, the story of that not working out was funny all by itself. The real lesson here is that playfulness is useful in learning some new things, and that Twitter bots are a lot of fun to build.

Search First Documentation

My vote for most takeaways that I'll personally use goes to Heidi Waterhouse for "Search-First Writing for Developers". Recently there as a mass migration of OpenStack Documentation from a dedicated docs team to all the development teams.

The heart of her message is that to any first approximation, no one reads your documentation. Users end up at your documentation when they have a problem with your software, so they are showing up a) grumpy, and b) through whatever Google terms they could guess for their problem. They are not coming through your carefully curated table of contents, they are coming from Google, and then they are skimming to find their answer. The won't follow links to other pages, this is where they are.

What that means is you need to treat every page as the only page that the user will ever see, you need to optimize your content for skimming, and you need to answer problems people actually have, not the ones you think they might have. Getting real analytics on how folks are reading your docs, and the search terms they are coming in with, is an important part of this.

Hearing all these harsh and practical words from someone that spent 15 years as a technical content author was really enlightening. I'll definitely have to watch this talk again and digest more of Heidi's lessons.

Safe Spaces

Reporting guidelines for Safety Incidents

One of the welcome trends that I've seen at tech conferences over the last 5 years is a real focus on a strict Code of Conduct, clear reporting guidelines, and making sure that folks feel safe at these events. North Bay Python did a great job on that front, and that commitment definitely was reflected in a pretty diverse speaker lineup and attendee base.

The effort they went to was highlighted further by Seán Hanson's talk on Quiet Developers. We've long known that while diversity in Tech is much lower than national averages, it's ever worse in Open Source Software. A big reason for this is members of traditionally marginalized communities really don't feel safe in these environments, or may not have the spare time to devote outside of their normal day jobs. It doesn't mean they aren't great developers, it's just that current systems are optimized for loudness as much as talent. Seán's whole talk was ways to engage and get the most out of your quiet developers, and give them what they need to really succeed. While I did need to leave about the time this talk started, I stuck around and watched from the balcony. His message was really powerful and really important to how we all evolve the tech community going forward.

Double A Plus, Would Come Again

North Bay Python was definitely worth the trip. It had a few normal quirks of a first time conference on scheduling. Being Petaluma, the Theatre didn't actually have heat, so the first few hours the first day were a bit cold in there. But it warmed up pretty quickly with 250 bodies. The biggest issue in my mind was there wasn't much common space outside of the theatre, so a hallway track wasn't really a thing. It would have been nice to have a bit more milling about time to get to know folks there, and ask follow up questions of speakers.

But all in all a great time. Looking forward to seeing how they do next year.


Notes from API Strat

Back in November I had the pleasure to attend API Strat for the first time. It was 2 days of short (20 minute) sessions running in 3 tracks with people discussing web service API design, practice, and related topics. My interest was to get wider exposure to the API Microversions work that we did in OpenStack, and get out of that bubble to see what else was going on in the space.

Events on the Web

Event technologies being used by different web services
Event technologies being used by different web services

There were lots of talks that brought up the problem of getting real time events back to clients. Clients talking to servers is a pretty solved problem with RESTful interfaces. But the other way is far from a solved item. The 5 leading contenders are Webhooks (over http), HTTP long polling, Web Sockets, AMQP, and MQTT. Each has their boosters, and their place, but this is going to be a messy space for the next few years.

OpenAPI's version 3 specification includes webhooks, though with version 3 there is no simultaneously launched tooling. It will take some time before people build implementations around that. That's a boost in that direction. Nginx is adding MQTT proxy support. That's a boost in that direction.

Webhooks vs. Serverless

Speaking of webhooks, the keynote from Glenn Block of Auth0 brought up an interesting point: serverless effectively lives in the eventing space as well.

Webhooks are all fine and good to make your platform efficient and scalable. If clients now have to run their own redundant highly available services to catch events, that's a lot of work, and many will just skip it. The found that once they build out a serverless platform where they could host their clients code, they got much more uptake on their event API. And, more importantly, they found that their power user customers were actually building out important features of their platform. He made a good case that every online service should really be considering an embedded serverless environment.

API Microversions

I was ostensibly there to talk about API Microversions, an approach we did in OpenStack to handle the fact that deployments of OpenStack upgrade at very different cadences. The talk went pretty well.

20 minutes was a challenge to explain something that took us all 6 months to get our heads around. I do think I managed to communicate the key challenge: when you build an open source system with a user facing API, how do users control what they get?  A lot of previous "good enough" rules fall down.

Darrel Miller had given a talk "How to never make another breaking API change". His first 5 minutes were really similar to mine, and then, because this was about Azure, with a single controlled API instance, the solution veered in a different direction. It was solid reinforcement for that fact that we were on the right path here, and that the open source solution has a different additional constraint.

One of the key questions I got in Q&A is one I'd been thinking about. Does GraphQL make this all obsolete? GraphQL was invented by Facebook to get away from the HTTP GET/POST model of passing data around, and let you specify a pretty structured query about the data you need from the server. On paper, it solves a similar problem as microversions, because it if you are really careful with your GraphQL you can ask for the minimum data you need, and are unlikely to get caught by things coming and going in the API. However, in practice, I'm not convinced it would work. In OpenStack we saw that most API usage was not raw API calls, it was through an SDK provided by someone in the community. If you are an SDK writer, it's a lot harder to make assumptions about what parts of objects people want, so you'd tend to return everything. And that puts you right back with the same problem we have in REST in OpenStack.

API Documentation

There were many talks on better approaches for documentation, which resonated with my after the great OpenStack docs migration.

Taylor Barnett's talk "Things I Wish People Told Me About Writing Docs" was one of my favorites. It included real user studies on what people actually read in documentation. It turns out that people don't read your API documentation, they skim hard. They will read your code snippets as long as they aren't too long. But they won't read the paragraph before it, so if there is something really critical about the code, make it a comment in the code snippet itself. There was also a great cautionary tale to stop using phases like "can be easily done". People furiously hunting around your site trying to get code working are ramping up quick. Words like "easy" make them feel dumb and frustrated when they don't get it working on the first try. Having a little more empathy for the state of mind of the user when they show up goes a long way towards building a relationship with them, and making them more bought into your platform.

Making New Friends

I also managed to have an incredible dinner the first night I was in town setup by my friend Chris Aedo. Both the food and conversation were amazing, in which I learned about Wordnic, distributed data systems, and that you can loose a year of research because ferrits bread for specific traits might be too dumb to be trained.

Definitely a lovely conference, and one I hope to make it back to next year.

Getting Chevy Bolt Charge Data with Python

Filed under: kind of insane code, be careful about doing this at home.

Recently we went electric, and got a Chevy Bolt to replace our 12 year old Toyota Prius (who has and continues to be a workhorse). I had a spot in line for a Tesla Model 3, but due to many factors, we decided to go test drive and ultimately purchase the Bolt. It's a week in and so far so good.

One of the things GM does far worse than Tesla, is make its data available to owners. There is quite a lot of telemetry captured by the Bolt, through OnStar, which you can see by logging into their website or app. But, no API (or at least no clear path to get access to the API).

However, it's the 21st century. That means we can do ridiculous things with software, like use python to start a full web browser, log into their web application, and scrape out data..... so I did that.

The Code

This uses selenium, which is a tool used to test websites automatically. To get started you have to install selenium python drivers, as well as the chrome web driver. I'll leave those as an exercise to the reader.

After that, the process looks a little like one might expect. Start with the login screen, find the fields for user/password, send_keys (which literally acts like typing), and submit.

The My Chevrolet site is an Angular JS site, which seems to have no stateful caching of the telemetry data for the car. Instead, once you log in you are presented with an overview of your car, and it makes an async call through the OnStar network back to your car to get its data. That includes charge level, charge state, estimated range. The OnStar network is a CDMA network, proprietary protocol, and ends up taking at least 60 seconds to return that call.

This means that you can't just pull data out of the page once you've logged in, because the data isn't there, there is a spinner instead. Selenium provides you a WebDriverWait class for that, which will wait until an element shows up in the DOM. We can just wait for the status-box to arrive. Then dump its text.

The output from this script looks like this:

Which was enough for what I was hoping to return.

The Future

Honestly, I really didn't want to write any of this code. I really would rather get access to the GM API and do this the right way. Ideally I'd really like to make the Chevy Bolt in Home Assistant as easy as using a Tesla. With chrome inspector, I can see that the inner call is actually returning a very nice json structure back to the angular app. I've sent an email to the GM developer program to try to get real access, thus far, black hole.

Lots of Caveats on this code. That OnStar link and the My Chevrolet site are sometimes flakey, don't know why, so running something like this on a busy loop probably is not a thing you want to do. For about 2 hours last night I just got "there is no OnStar account associated with this vehicle", which then magically went away. I'd honestly probably not run it more than hourly. I made no claims about the integrity of things like this.

Once you see the thing working, it can be run headless by uncommenting line 18. Then it could be run on any Linux system, even one without graphics.

Again, this is one of the more rediculous pieces of code I've ever written. It is definitely a "currently seems to work for me" state, and don't expect it be robust. I make no claims about whether or not it might damage anything in the process, though if logging into a website damages your car, GM has bigger issues.


It came from beyond the stars

Meet 1I/2017 U1 (‘Oumuamua), but make it fast, because it’s already leaving.

Source: First-known interstellar visitor is a bizarre, cigar-shaped asteroid

This is one of the most exciting news stories of the year. We've now actually seen an object that had to have come from another solar system, pass through ours. Data collection by the Hubble telescope continues into December, after which point it's too faint for anything to see again.

Migration by Sea

For decades, students were taught that the first people in the Americas were a group called the Clovis who walked over the Bering land bridge about 13,500 years ago. They arrived (so the narrative goes) via an ice-free corridor between glaciers in North America. But evidence has been piling up since the 1980s of human campsites in North and South America that date back much earlier than 13,500 years. At sites ranging from Oregon in the US to Monte Verde in Chile, evidence of human habitation goes back as far as 18,000 years.

Source: Most scientists now reject the idea that the first Americans came by land | Ars Technica

Pretty huge change in what we were taught growing up, and a great story about how extraordinary claims require extraordinary evidence. And when that evidence is compelling, scientific consensus moves. This also provides more overlap for the mega fauna die off and human habitation, which makes sense.

Catskills Conf 2017

Sunrise club at Ashokan courtesy of Ruthie Nachmany -

Yawp! YAWP!
Yawp! YAWP!
Don't fall in the creek.
Hudson Valley Tech Ashokan Community.
Yawp! YAWP!
Yawp! YAWP!
Don't fall in the creek.
Be as open and present as you can be.

That was the chorus of the theme song for Castkills Conf this weekend. Yes there was a theme song. Every day started with a musical riff on the talks of the day before by Jonathan Mann, who has been posting a song-a-day, every day, to youtube for a decade. You can go watch them for Friday, Saturday, and Sunday (soon). This is one of the many wonderful ways that this event was unlike any tech event I've been to, and why it just became one of my favorite tech events I've ever been to.

At a typical tech event the focus is on getting a bunch of speakers, so much content, splitting folks into tracks on the topics they would be interested in, then packing that in from 9 - 5 (or later). People are exhausted by the end of the day. They also largely attended different conferences. At a 10 track conference, the only shared experience, if it exists, is keynotes. Which for larger conferences are purchased slots.

Catskills Conf was a single speaking track. There were only 10 speakers, plus a lightning talk session with 7 lightning talks. The talks were a shared experience for everyone. They were all about technology, the tech industry, and/or the intersection of tech with other aspects of our lives. And they were all incredible. I considered it quite an honor to be a part of the speaking lineup.

And when it came to speakers, the Catskills Conf team was extremely serious about having a diverse speaker list. Of the 10 speakers, the gender split was 3 men, 6 women, and 1 non-binary. 4 of 10 speakers were people of color. The lightning talks were equally diverse. It was such a stark contrast to what you typically see at a Tech event that it was in your face refreshing.

10 talks doesn't seem like a lot for a 3 day conference, but in between them there were structured Activity times. Saturday afternoon there was a 2 hour activity block after lunch with options including black smithing, letter press, self defense, foraging, hiking (with historic interpretation of the Ashokan site), and bread making. Being a 75 degree sunny fall afternoon, I opted for the 2 hour hike, wandering through woods and along streams. I came straight back from that into my talk more energized than I've ever been for one.

These kind of breaks from sitting and listening to talks and doing something with your hands or feet gave was wonderful for processing what you were hearing. It also meant that by the end of the day instead of feeling like your brain was jelly, you had enough processing time that you were excited to talk about what you heard, or get to know the person sitting next to you at dinner and find out the fascinating things they were doing. Every night ended with a campfire and beer under the stars. Which was another place to talk and get to know more folks. You weren't so overloaded during the day that you wanted to go off and hide and decompress afterwards, even those of us that are more on the introvert side.

A couple of folks even collected for a 6am sunrise hike on Sunday morning. I joined 3 others as we hiked under flashlights, losing the trail a couple of times, to a sugar orchard and sugar shack, while discussing the work one of the hikers was doing around infectious disease modeling and experiments with mosquitos, the bio mimicry work that another was doing trying to take queues from nature and work them into built materials, discussing bird migrations, tech meetups, and just generally exploring a beautiful area.

This was the 3rd year of Catskills Conf, but the first time I could make it. I'm going to be processing the event for weeks to come. There were so many moments that I really loved that aren't here, it just doesn't all fit. But one thing is for sure. I'm extremely excited about attending and participating in the years to come.

I'll leave you with this really cool Catskills Conf 2017 wrap up video. It's not like being there, but it gives you a flavor.

Book Review: The Ends of the World

I'm going to open with, this book is flat out amazing. In school, or even through popular science journalism, we learn a bit about some key points of geologic time. But these are snap shots, Dinosaurs, Ice Ages, even Snow Ball Earth. Really interesting things on their own, but they all seem a little disjoint.

This book brings an incredible visual narrative through life on Earth, by looking at the 5 mass extinction events the planet has experienced. An extinction is only emotionally meaningful if you understand what is lost, so the author paints an incredible picture of the aliens worlds that were Earth in these previous eras. Worlds without life on land, worlds of giant insects, worlds of bus size armored carnivorous fish as apex predators. He does this by road tripping to the scientists and fossil sites where this story is being assembled, talking with experts along the way. A story as old and hidden in the fossil record needs lots of lines of evidence to point to answers, and the author does a great job of doing that, and pointing out what we seem to know, and what we've only got guesses on.

The story of life on earth is the story of carbon and climate. As volcanoes stirred up carbon from the deep, and life reclaimed it, died, and sunk it back into the Earth. When this cycle gets really out of whack, the climate goes nuts, and life is paused on planet Earth, and taken tens of thousands of years to get back on track. There are many points of reflection about how our current mining and burning of ancient sequestered carbon is impacting our world today.

There are also just incredible moments that make you sit and think. The death of the land based mega fauna, 12,000 years ago, in North America, that still leaves ecological holes.

But the menagerie lives on in evolutionary ghosts. In North America, the fleet-footed pronghorns of the American West run laughably faster than any of their existing predators. But then, their speed isn’t meant for existing predators. It might be a vestige of their need to escape constant, harrowing pursuits by American cheetahs—until a geological moment ago. The absence was palpable to me as I rode a train past New Mexico’s Kiowa National Grassland, an American Serengeti, windswept and empty except for a lone wandering pronghorn still running from ghosts.

Other evolutionary shadows of the Pleistocene live on in the produce aisle. Seeds in fruit are designed to be eaten and dispersed by animals, but for the avocado this makes little sense. Their billiard ball–sized cores, if swallowed whole, would at the very least make for an agonizing few days of digestive transit. But the fruit makes a little more sense in a land populated by tree-foraging giants, like the sometimes dinosaur-proportioned ground sloths, who swallowed the seeds and hardly noticed them. The ground sloths disappeared a geological moment ago, but their curious fruit, the avocado, remains.

It will make me never quite look at an avocado the same way again.

There are so many things I learned which made me reconsider my whole view of dinosaurs. Like Dimetrodon, the creature with a large sale on it's back for temperature regulation, is more closely related to mammals than dinosaurs. And without the 3rd mass extinction, we'd never have seen Dinosaurs, and mammals might have ruled the Earth much earlier. And that T-rex showed up really late on the scene, filling the niche that that much more successful Allosaurs held as apex predators for most of the Jurasic era (the Allosaurs all disappeare in a more minor great extinction).

It's not often that you find a non fiction book that both reads fast, and dumps such an incredible amount of information on you. The jumping back and forth from road trip, chatting with scientists, facts, and painting pictures of the world that was, works really well. There is never a dull moment in it, and you come out the far end for a much greater appreciation for life on Earth in all its forms.

My Solar Eclipse Experience

They are right when they say there is nothing quite like a total solar eclipse. I had seen an annular eclipse in my senior year of high school as it cross over Vermont. Wandering out there with our physics teacher, looking through glasses, it was pretty cool, but it was nothing like the Sun going out.

Eclipse day for us was in Springfield, TN, as my friend's Nick and Heather were living there, and it was deep in the path of totality. We made plans to visit in May, booked our hotel then, and framed a trip around this so that we'd do four 8 hour driving days on our great eclipse road trip. Down through Cleveland to visit family, and up through Virginia on the way back to break up the drive.

On the way to Tennessee on Friday, my friend Jack was still sorting out his plans. He and his son were going to be sleeping in their car wherever they went. The weather where he was originally headed was looking dicey, and I said "well this is where we'll be, I make no promises, but I can offer you use of our hotel shower." Friday evening they decided they would join us. They arrived Sunday about 9am. In addition to bringing their wonderful selves, they had brought a solar filtered scope, a hydrogen alpha scope, and a camera rig to shoot the eclipse, and a shade tent.

We scouted a location at Nick and Heather's apartment, a flat grassy section near the apartment complex dog park. First contact was just shy of noon for us. We started setting up all the equipment at 10, were in a good state by 11.

Our setup area for the eclipse
All set up, shade tent, telescopes

First contact. The moment when you stare through those eclipse glasses and say: "wait, is that side just a little flat now? Maybe I'm imagining it. No, I really think it is. Yeah, that must be real." As the eclipse grows, your brain does this funny thing and enhances the boundary. The silhouette of the Sun seems to glow brighter than the Sun itself.

Eclipse glasses on!

After about 20 minutes, and our pacman shape ever growing, I realized we're never going to break for food (the original plan), we should bring it back down here. So I headed back upstairs with Nick, prepared a bit of our picnic food that we'd gotten the night before, and headed back provisions in hand.

A shade tent is a glorious thing when watching an eclipse on a 95 degree sunny day with few clouds. We would spend time inside the tent, drinking water, having a snack, then popping out to see how things were progressing.

Shade tent, and making new friends

With our little camp setup, folks from the complex started stopping by. Including a number of kids. Jack's a pro at the sidewalk astronomy outreach, we introduced people to the scopes, what they were looking at, and kept them pointed and in focus. One of the kids came back out with popsicles for everyone as a thank you for letting everyone see through those scopes.

Popsicle break

It was never going to get cold in Tennessee, but once we got past 75% coverage, the beating hot 95 turned into "a reasonable warm day to be outside". Maybe we dialed back to 80. The sky lost it's deep blue, and was just a muted version of itself. Everything was muted in an erie way that you can't quite describe.

As we closed in on second contact, a cloud creeped in over the Sun. It kept going away, coming back, going away, coming back, with lots of, "is that it?", "no". Then just before totality the cloud cleared, it went black, one of the neighbors yelled, "we're in!". Everyone took off their glasses. And we stared at a hole in the sky with the giant wispy corona spewing out from it.

Our eclipse, courtesy Jack Chastain

Pictures don't ever really capture what the eye sees. Most of them show a small ring around the Sun. But this was a sun flower. The corona extended at least the radius of the Sun again. It wasn't uniform, it was sweepy with a few petals poking out. It was amazing. Everyone was exclaiming in different ways, processing this true wonder of nature in a way personal to them.

Venus, Jupiter, and even dim Mercury came out to meet us. I was so focused on all of those I didn't really take the time to look for other stars. But given Mercury was dialing in at magnitude 3, there should have been plenty.

During totality

We got about two and a half minutes. It's not enough time. Not enough time to soak in this totally bizarre experience. It was about a minute longer than our daughter (not quite 3) was happy with. Both the dark, and the black hole in the sky definitely got her scared. She was not the only one, we heard another boy crying in one of the apartments behind us.

One of the neighbors gave us a countdown for the event ending. As hit the end of the countdown, I looked down at the 2 trees in front of us. No shadow, no shadow, no shadow, then with what appeared to be a swoosh... 2 shadows. Like they were dropped back into place in a cartoon.

A child crying... my child. "I want to see Venus! I want to see Venus!". Venus is a night sky friend, we've been finding it in the sky for nearly a year. Everyone was talking about how you could see it during the eclipse and now Arwen was upset that they didn't also get that chance. But at magnitude 4, Venus was easy to see even back to 90% coverage. We spent some time pointing and looking, and calming down. She's at the age where I don't know if she'll remember all of this later when she's grown up, but maybe there will be flashes of it.

The sun returns, with crescents through the bushes

And as the Sun returned, everyone dispersed. I spent the next hour helping Jack pack up and put things in his car. We all ended up back in the nice AC before the even was properly over, taking a last look at a 15% eclipse before going inside for more food and to crack a few beers.

Jack and son stayed a few hours to let the worst of the traffic clear, then headed off back to Virginia. The google map from that day is amazing. We fortunately had all the food we needed in the apartment for dinner, so dined in, made our goodbyes at about 7, headed to our hotel, and even started watching the PBS eclipse documentary that aired that night. Though after a few pictures of totality were on the screen Arwen proclaimed "that is too much for me, please turn it off."

My plan for avoiding the traffic was leaving the day after. However, there were really more people out for this event that did the same. Our 9 hour driving day, turned into 12 hours of driving thanks to constant stop and go traffic on the I-81 corridor in Virginia. We did get to our Tuesday night lodging that day, at 11:59pm. So we also get to have a traffic story for our eclipse adventure.

There really is nothing like a total eclipse. I'm now excited for 2024 (less excited about New England April weather). And, we'll see if we consider destination adventures to see another one along the way. I was exciting to share this with my daughter and wife, and friends both old and new. What a great end to the summer.


OpenWest 2017 Roundup

When I first discovered the Open West conference, I was told it was the biggest US open source event that I'd never heard of, which is a pretty apt description. Open West brings together technologists interested in Open Technology in Sandy Utah, just south of Salt Lake City. This is a community regional Open Source event, run by volunteers, which means the program is much more varied than what you'd see at an event focused on a particular open source technology stack.

With up to 13 tracks happening simultaneously, there were lots of great moments for me over the course of the week. I'm just going to capture a few of them.

OpenCV Trials and Tribulations

There was a great talk by John Harrison at Lucid Charts about trying to do something interesting with OpenCV, and failing. He was giving the talk in the spirit of the Journal of Negative Results: reporting a hard problem they tried and failed at, and the dead ends they ran into.

It started as a hackfest project, could they take a screen shot with a camera of a flow chart, and use OpenCV to turn that into a symbolic flow chart in their tool. Turns out if you write all connecting lines in red, and all shapes in black, it's not a very hard problem. Also turns out, even in controlled user experiments, you can't get anyone to do that. It fails UX. And while they did build a system that worked with black lines everywhere in controlled lab environments, it worked with 0% of customer taken images, and the path to improvement wasn't clear, so after a 2 month experiment they stopped.

While they are primarily a Java shop, they did this entire project in python, because while "there are OpenCV bindings for every language you can imagine, all the interesting examples are only in python." Which goes to show how import an open and vibrant ecosystem of consuming tools is to the success of a project.

Writing Ethical Software

This was an interesting talk by James Prestwich on writing ethical software, that started with a brief history of schools of thought on ethics over the last 3000 years. The primer was just straight up informative, and the presenter actually did a quite good job being neutral through all of that.

Then we were posed with an interesting question. Software is now mostly about mediating complex interactions between people. If you look at other fields like Medicine and Law there are oaths and codes of conduct that their practitioners take because of how much their work affects people's lives.

We have collectively decided that certain things, like land mines, should not exist in the world. We have treaties on that. But as software eats the world, we're not having the conversation about what software should not exist, for any reason.

There weren't answers during this talk, it was mostly questions and attempting to start a conversation. But for anyone who works in software it's a good thought exercise to have. What are your personal ethical boundaries about software you would create or contribute to? It's also a much better conversation to have well in advance of any actual ethical conflict, because things are rarely bright lines, but long slippery slopes.

Hardware Track

There was a dedicated hardware track on the main stage for the whole conference, at least a third of the talks were related to home automation in some way, and 80% of them centered around a project that used a Raspberry Pi.

Raspberry Pi has managed to go across the entire hype curve and is now climbing away on the plateau of productivity. We went from neat idea, to unobtainium, to toy projects, to boxes full of pis in basements, to real productivity over the last 5 years. Yes, there are lots of other cheaper, neater, more powerful platforms, but the ecosystem around the pi just makes it the no brainer work horse.

I was actually a little surprised how many home grown Home Automation systems people talked about there. I did have pieces of something like that before discovering Home Assistant, but now it's hard to imagine doing all the work that the community is doing for me.

One of the projects I thought was most interesting was air quality monitoring with the esp8662. For about $30 they can build each monitoring unit, then find places throughout the community they can plug them in (need power and wifi). They are collecting it all in a
central MQTT broker and doing reports on it to try to get a better baseline on the air quality in the Salt Lake City area.

Patching People

The stand out keynote of the event was Deb Nicholson on patching people.

Any group of humans, and they ways they interact, have bugs, just like software has bugs. A people bug is like a software bug, it's unintended negative side effects of things that are happening. The point is, patching people is actually not all that different than patching software.

Filing "bugs" against people is a little harder than software, because no one likes to accept criticism. So as such, she put forward the idea of "calling in" vs. "calling out". Take the person aside, privately, and say "I think you were trying to do X, but the way it was said excluded a bunch of these people. Maybe saying it this other way would be more effective?".

The other thing to realize is none of us is above this. We all make mistakes, and need some patching from time to time.

After this talk I'm going to try to be better about calling in when I think it will help. In open source projects, they live or die by the longevity of the community, so patching the community to be more inclusive and welcoming is key.

So many more good moments...

Honestly, there were so many other good moments as well: chatting with folks about Home Assistant after my talk; seeing the state of the world on different AI cloud platforms; thinking about localization and culture in software; getting my head around the oauth model; json web tokens.

This is definitely a conference I'd love to get to again, and a great community event they've built there. Thanks to the OpenWest organizing team for such a great show.

Triple Bottom Line in Open Source

One of the more thought provoking things that came out of the OpenStack leadership training at Zingerman's last year, was the idea of the Triple Bottom Line. It's something I continue to ponder regularly.

The Zingerman's family of businesses definitely exist to make money, there are no apologies for that. However, it's not their only bottom line that they measure against they've defined for themselves. Their full bottom line is "Great Food, Great Service, Great Finance." In practice this means you have to ensure that all are being met, and not sacrifice the food and service just to make a buck.

If you look at Open Source through this kind of lens, a lot of trade offs that successful projects make make a lot more sense. The TBL for OpenStack would probably be something like: Code, Community, Contributors. Yes, this is about building great code, to make a great cloud, but it's also really critical to grow the community, and mentor and grow individual contributors as well. Those contributors might stay in OpenStack, or they might go on to use their skills to help other Open Source projects be better in the future. All of these are measures of success.

This was one of the reasons we recently switch the development tooling in OpenStack (DevStack) to using systemd more natively. Not only did it solve a bunch of long standing technical issues, that had really ugly work arounds, but it also meant enhancing our contributors. Systemd and the journal are default in every new Linux environment now, so skills that our contributors gained working with DevStack would now directly transfer to any Linux environment. It would make them better Linux users in any context, not just OpenStack. It also makes the environment easier for people coming from the outside to understand, because it looks more like what they are used to.

While I don't have enough data to back it up, it feels like this central question is really important to success in Open Source: "In order to be successful in this project you must learn X, which will be useful in these other contexts outside of the project." X has to be small enough to be learnable, but also has to be useful in other contexts, so time invested has larger payoffs. That's what growing a contributor looks like, they don't just become better at your project, they become a better developer for everything they touch in the future.

Exploring and discovering how things are more complicated, with a focus on climate and software