Tag Archives: climate

MQTT, Kubernetes, and CO2 in NY State

Back in November we decided to stop waiting for our Tesla Model 3 (ever changing estimates) and bought a Chevy Bolt EV (which we could do right off the lot). A week later we had a level 2 charger installed at home, and a work order in for a time of use meter. Central Hudson's current time of use peak times are just 2 - 7pm on weekdays, and everything else is considered off peak. That's very easy to not charge during, but is it actually the optimal time to charge? Especially if you are trying to limit your CO2 footprint on the electricity? How would we find out?

The NY Independent System Operator (ISO) generates between 75% and 85% of the electricity used in the state at any given time. For the electricity they generate, they provide some very detailed views about what is going on.

There is no public API for this data, but they do publish CSV files at 5 minute resolution on a public site that you can ingest. For current day they are updated every 5 to 20 minutes. So you can get a near real time view of the world. That shows a much more complicated mix of energy demand over the course of the day which isn't just about avoiding the 2 - 7pm window.

Building a public event stream

With my upcoming talk at IndexConf next week on MQTT, this actually jumped up as an interesting demonstration of that. Turn these public polling data sets into an MQTT live stream. And, add some data calculation on top to calculate what the estimated CO2 emitted per kWh is currently. The entire system is written as a set of micro services on IBM Cloud running in Kubernetes.

The services are as follows:

  • ny-power-pump - a polling system that is looking for new published content and publishing it to an MQTT bus
  • ny-power-mqtt - A mosquitto MQTT server (exposed at mqtt.ny-power.org). It can be anonymously read by anyone
  • ny-power-archive - An mqtt client that's watching the MQTT event stream and sending data to influx for time series calculations. It also exposes recent time series as additional MQTT messages.
  • ny-power-influx - influx time series database.
  • ny-power-api - serves up a sample webpage that runs an MQTT over websocket bit of javascript (available at http://ny-power.org)


MQTT is a light weight message protocol using a publish / subscribe server. It's extremely popular in the Internet of Things space because of how simple the protocol is. That lets it be embedded in micro controllers like arduino.

MQTT has the advantage of being something you can just subscribe to, then take actions only when interesting information is provided. For a slow changing data stream like this, giving applications access to an open event stream means being able to start doing something more quickly. It also drastically reduces network traffic. Instead of constantly downloading and comparing CSV files, the application gets a few bytes when it's relevant.

The Demo App

That's the current instantaneous fuel mix, as well as the estimated CO2 per kWh being emitted. That's done through a set of simplifying assumptions by looking at 2016 historic data (explained here, any better assumptions would be welcomed).

The demo app also includes an MQTT console, where you can see the messages coming in that are feeding it as well.

The code for the python applications running in the services is open source here. The code for the deploying the microservices will be open sourced in the near future after some terrible hardcoding is removed (so others can more easily replicate it).

The Verdict

While NY State does have variability in fuel mix, especially depending on how the wind load happens. There is a pretty good fixed point which is "finish charging by 5am". That's when there is a ramp up in Natural Gas infrastructure to support people waking up in the morning. Completing charging before that means the grid is largely Nuclear, Hydro, and whatever Wind is available that day, with Natural Gas filling in some gaps.

Once I got that answer, I set my departure charging schedule in my Chevy Bolt. If the car had a more dynamic charge API, you could do better, and specify charging once it flat lined at 1am, or dropped below a certain threshold.

Learn more at IndexConf

On Feb 22nd I'll be diving into MQTT the protocol, and applications like this one at IndexConf in San Francisco. If you'd love to discuss more about turning public data sets into public event streams with the cloud, come check it out.

Getting to a Zero Carbon Grid

This talk by Jesse Jenkins at UPENN is one of the best looks at what doing deep decarbonization of the grid really looks like. Jenkins is a PhD candidate at MIT researching realistic paths to get our electricity sector down to zero carbon emissions.

Price vs. Value

He starts with the common and simple refrain we all have, which is that research investments in solar have driven down the cost below that of fossil fuels, that cross over point has happened, and renewables will just take off and take over.

But that's the wrong model. Because of the intermitency of Wind and Solar, after a certain saturation point the wholesale value of a new MWh of their energy keeps decreasing. This has already been seen in practice in energy markets with high penetration.

 Sources of Energy

The biggest challenge is not all sources of energy are the same.

Jenkins bundles these into 3 categories. Renewables are great at Fuel savings, providing us a way not to burn some fuel. We also need a certain amount of fast burst on the grid, today this is done with Natural Gas Peaker plants, but demand hydro and energy storage fit that bill as well. In both of these categories we are making good progress on new technologies.

However, in the Flexible base camp, we are not. Today that's being provided by Natural Gas and Coal plants, and some aging Nuclear that's struggling to compete with so much cheap Natural Gas on the market.

How the mix changes under different limits

He did a series of simulations about what a price optimal grid looks like under different emissions limits given current price curves.

Under a relatively high emissions threshold the most cost efficient approach is about 40% renewables on the grid, some place for storage. The rest of the power comes from natural gas. 16% of solar power ends up being curtailed during the course of the year, which means you had to overbuild solar capacity to get there.

Crank down the emissions limit and you get more solar / wind, but you get a lot of curtailment. This is a 70% renewable grid. It's also got a ton of over build to deal with the curtailment.

But if you want to take the CO2 down further, things get interesting. 

Because of the different between price and value, relatively high priced Nuclear makes a return (Nuclear is a stand in for any flexible base source, it's just the only one we current have in production that works in all 50 states). There still is a lot of overbuild on solar and wind, and huge amounts of curtailment. And if you go for basically zero carbon grid, you get something a little surprising.

Which is the share of renewables goes down. They are used more efficiently, there is less curtailment. These are "cost optimal" projections with emissions targets fixed. They represent the cheapest way to get to a goal.

The important take away is that we're in this very interesting point in our grid evolution where cheap Natural Gas is driving other zero carbon sources out of business because we aren't pricing Carbon (either through caps or direct fees). A 40 - 60% renewables grid can definitely emerge naturally in this market, but you are left with a lot of entrenched Natural Gas. Taking that last bit off the board with renewables is really expensive, which means taking that path is unlikely.

But 100% Renewables?

This is in contrast to the Mark Jacobson 100% renewables paper. Jenkins points out that there have really been two camps of study. One trying to demonstrate the technical ability to have 100% renewables, the other looking at realistic pathways to zero carbon grid. Proving that 100% renewables is technically possible is a good exercise, but it doesn't mean that it's feasible from a land management, transmission upgrade, and price of electricity option. However none of the studies looking at realistic paths landed on a 100% renewables option.

Jenkins did his simulation with the 100% renewables constraint, and this is what it looked like.

When you pull out the flexible base you end up with a requirement for a massive overbuild on solar to charge sources during the day. Much of the time you are dumping that energy because there is no place for it to go. You also require storage at a scale that we don't really know how to do.

Storage Reality Check

The Jacobson study (and others) make some assumptions about season storage of electricity of 12 - 14 weeks of storage. What does that look like? Pumped hydro is currently the largest capacity, and most efficient way to store energy. Basically you pump water behind a dam when you have extra / cheap energy, then you release it back through the hydro facility when you need it. It's really straight forward tech, and we have some on our grid already. But scale matters.

The top 10 pumped hydro facilities combined provide us 43 minutes of grid power.

One of the larger facilities is in Washington state it is a reservoir 27 miles long, you can see it from space. It provides 3 1/2 minutes grid average power demand.

Pumped hydro storage is great, where the geography supports it. But the number of those places is small, and it's hard to see their build out increasing dramatically over time.

Does it have to be Nuclear?

No. All through Jenkins presentation Nuclear was a stand in for any zero carbon flexible base power source. It's just the only one we have working at scale right now. There other other potential technologies including burning fossil fuels but with carbon capture and storage, as well as engineered geothermal.

Engineered Geothermal was something new to me. Geothermal electricity generation today is very geographically limited you need to find a place where you have a geologic hot spot, and an underground water reserve, that's turning that into steam you can run through generators. It's pretty rare in the US. Iceland gets about 25% of it's power this way, but it has pretty unique geology.

However, the fracking technology that created the natural gas boom openned a door here. You can pump water down 2 miles into the earth and artificially create conditions to produce steam and harvest it. It does come with the same increase in seismic activity that we've seen in fracking, but there are thoughts on mitigation.

It's all trade offs

I think the most important take away is there is no silver bullet in this path forward. Everything has downsides. The land use requirements for solar and wind are big. In Jenkins home state of Massachusetts in order to get to 100% renewables it would take 7% of the land area. That number seems small, until you try to find it. On the ground you can see lots of people opposing build outs in their area (I saw a Solar project for our school district get scuttled in this way).

In the North East we actually have a ton of existing zero carbon energy available in Hydro Quebec, that's trapped behind not having enough transmission capacity. Massachusetts just attempted to move forward with the Norther Pass Transmission project to replace shutting the Pilgrim Nuclear facility, but New Hampshire approval board unanimously voted against it.

Vermont's shutdown of their Yankee Nuclear plant in 2014 caused a 2.9% increase in CO2 in the New England ISO region, as the power was replaced by natural gas. That's the wrong direction for us to be headed.

The important thing about non perfect solutions is to keep as many options on the table, as long as you can. Future conditions might change in a way where some of these options become more appealing as we strive to get closer to a zero carbon grid. R&D is critical.

That makes the recent 2018 budget with increased investment credits for Carbon Capture and Storage and small scale Nuclear pretty exciting from a policy perspective. These are keeping some future doors open.

Final Thoughts


Jenkins presentation was really excellent, I really look forward to seeing more of his work in the future, and for a wider exposure on the fact that the path to a zero carbon grid is not a straight line. Techniques that get us to a 50% clean grid don't work to get us past 80%. Managing that complex transition is important, and keeping all the options on the table is critical to getting there.

No Coal this Christmas Season - Personal Climate Action you can take now

"The best time to plant a tree was 20 years ago. The second best time is now."

Over the last year we've done a lot to our house to make it much less energy intensive, bought an electric car, and got  involved in Citizens' Climate Lobby. The research for all of that created a big pile of links for me, which I've tried to summarize here, to really show how many different ways you can make an impact.

This list is customized for New York, because that's where I live, and where I've done all my research. It would be great to see other folks build local guides for their areas as well, and I'd love to link to them.

Where Energy is Used

How do we use energy in the US? Because we measure electricity in kWh, gasoline / fuel oil in gallons, natural gas in cubic feet, sizing them all up and comparing them is hard. And we don't think about them as a single energy system. At a national level our energy is used by [1]:

  • Buildings - 40%
    • Residential - 20%
    • Commercial - 20%
  • Transportation - 28%
    • Cars, Light Trucks, Motorcycles - 16.2%
    • Other Trucks - 6.4%
    • Planes - 2.2%
  • Industry - 32%
    • Petroleum Refining - 10%
    • Chemical Production - 8.6%
    • Paper Production - 3.5%
    • Metals Production - 3%

The bits of this that always surprise me is that buildings are our key use of energy. Buildings are long term infrastructure. Our house was built in 1960, there are plenty of houses in our area build in 1900. Improving existing buildings is critical to making our infrastructure more efficient. Every improvement we've made over the last couple of years will live on beyond us in this home.

The other thing that sticks out is that we use 10% of our energy budget in the US refining petroleum. Much of that to be burned in other parts of the system. Every time we prevent a gallon of gas from burning, we don't only save it's emissions, but the emissions that happened when it was refined.



Average Home Energy use in NY State

Get a home energy inspection

In NY, the NY State Energy Research and Development Agency has many programs to increase energy efficiency. One of the programs is subsidized home energy audits to give you a targeted plan about what the biggest impacts for saving energy in your home will be.

Air sealing and Attic Insulation

Our home was built in 1960, and insulated to the standard at the time (which was not much). A year ago we went forward on our energy audit recommendations and got our attic air sealed, and 8" of cellulose insulation put on top. The results were dramatic. Heating dropped about 15%, my home office (which is the far end of the HVAC), no longer needed a space heater, and summer cooling was also dramatically reduced.

Get your energy inspection first, but realize that proper insulation in your home will dramatically, and immediately change the comfort level, and your energy use.

Replace your Oil Furnace with Geothermal

About 50% of homes in NY State heat with Fuel Oil. It is one of the dirtiest way to heat your home.

If you live in the Hudson Valley or Albany regions, Dandelion is a new geothermal company offering package deals to replace your existing oil system with a ground source heat pump. They put a well or two in your front yard, put a sealed tube down it, then use the 50 degree earth and a compressor to heat your home. Heat pumps get about 4 units of heat for every unit of electricity they consume. Ours has been in for about a month, and so far we're in love. So much quieter, no whiffs of oil smoke, and much more even distribution of heat in the house (it runs the fan slower and longer).

When I did the math, this was the single biggest climate impact we could make. This takes 700 gallons of fuel oil off the table. In comparison, we used about 500 gallons of gasoline in an average year between our cars.

Replace your Oil Furnace with... anything else

Seriously, Fuel Oil is terrible for the environment. While Natural Gas and Propane are still fossil fuels, they emit a lot less both in creating them, and when they burn. If you can't go the full hog to something like a heat pump, changing from Oil to NG or Propane will reduce your emissions on heating to about 1/2 of what they were before.


If you've not yet replaced all your lighting in your house with LEDs, do that now. They only use about 20% of the electricity of incandescent bulbs, are more efficient than even fluorescent, and last an incredibly long time (25 year lifespans are common).

If you are a Central Hudson customer you can get 60W replacement bulbs for $1 each. Just do it. While lighting use is overall a pretty low part of your energy budget, it is also very actionable if you haven't done the conversion to LEDs yet. And, LED lights fit in christmas stockings.


The path to decarbonizing the economy is to electrify everything, while simultaneously making the electric grid less carbon intensive.


NY State's energy production is relatively low carbon, but if we are going to fully decarbonize we do need to reduce natural gas consumed for electricity as much as possible.

Choose a Green ESCO

NY State allows you to choose your energy producer (energy services company, or ESCO). There are a number of companies that provide you with energy from wind farms that they are building regionally. This typically mean a small rise in your energy costs, but that comes with supporting the build out of new renewables.

Two good options in our area are:

Community Solar

NY State has new rules in place that allow for Community Solar in our area. These are small scale (2 Mega Watt or less) facilities that you can sign up and get your power from solar even if you can't put solar panels on your roof (you have bad site, or are a renter).

Solarize Hudson Valley has sign up information for folks in the area. If you are in the Central Hudson power generation region, Nexamp is building a facility in Wappingers Falls. We've signed up, and starting in May of 2018, will be getting our power from solar.

Carbon Offsets

If there is nothing on the list that works for you, but you still want to have an impact on reducing your carbon footprint, consider some kind of carbon offset. Carbon offset projects work to capture carbon, or reduce emissions from something like a landfill. We all share one atmosphere, so any way you reduce emissions helps.

The carbon offset market is a wildly confusing place as an individual. As a NY (or North East) resident, the Carbon Reduction Certificates from the Adirondack Counsil is great. Each certificate is used to buy 1 ton of CO2 off the Regional Greenhouse Gas Initiative annual carbon auction (a carbon trading system for power companies that 9 states have agreed to, and NJ and VA might be joining soon). The price for a ton of carbon on the RGGI is still pretty low, so left over proceeds go to their micro grants program which support local energy efficiency and emissions reduction.


Get Engaged

Right now, you need to do something extra, or out of the ordinary to have an impact on climate change. Citizens' Climate Lobby is a political action group looking to change that, by pricing carbon in the economy. A real price on carbon would make doing the efficient thing, also be doing the cheaper thing, which would make it the default choice in most situations.

We've got a local chapter that meets in Beacon, NY once a month, and so if you want to flex your political muscles, as well as your economic ones, sign up and join us.

It all maters

Every action you make matters. And the exciting fact is that there are so many things you can do now to have an impact (including many things not on this list). So take a minute this holiday season and think about how you can take a little bit of coal out of your own Christmas season.


Book Review: The Ends of the World

I'm going to open with, this book is flat out amazing. In school, or even through popular science journalism, we learn a bit about some key points of geologic time. But these are snap shots, Dinosaurs, Ice Ages, even Snow Ball Earth. Really interesting things on their own, but they all seem a little disjoint.

This book brings an incredible visual narrative through life on Earth, by looking at the 5 mass extinction events the planet has experienced. An extinction is only emotionally meaningful if you understand what is lost, so the author paints an incredible picture of the aliens worlds that were Earth in these previous eras. Worlds without life on land, worlds of giant insects, worlds of bus size armored carnivorous fish as apex predators. He does this by road tripping to the scientists and fossil sites where this story is being assembled, talking with experts along the way. A story as old and hidden in the fossil record needs lots of lines of evidence to point to answers, and the author does a great job of doing that, and pointing out what we seem to know, and what we've only got guesses on.

The story of life on earth is the story of carbon and climate. As volcanoes stirred up carbon from the deep, and life reclaimed it, died, and sunk it back into the Earth. When this cycle gets really out of whack, the climate goes nuts, and life is paused on planet Earth, and taken tens of thousands of years to get back on track. There are many points of reflection about how our current mining and burning of ancient sequestered carbon is impacting our world today.

There are also just incredible moments that make you sit and think. The death of the land based mega fauna, 12,000 years ago, in North America, that still leaves ecological holes.

But the menagerie lives on in evolutionary ghosts. In North America, the fleet-footed pronghorns of the American West run laughably faster than any of their existing predators. But then, their speed isn’t meant for existing predators. It might be a vestige of their need to escape constant, harrowing pursuits by American cheetahs—until a geological moment ago. The absence was palpable to me as I rode a train past New Mexico’s Kiowa National Grassland, an American Serengeti, windswept and empty except for a lone wandering pronghorn still running from ghosts.

Other evolutionary shadows of the Pleistocene live on in the produce aisle. Seeds in fruit are designed to be eaten and dispersed by animals, but for the avocado this makes little sense. Their billiard ball–sized cores, if swallowed whole, would at the very least make for an agonizing few days of digestive transit. But the fruit makes a little more sense in a land populated by tree-foraging giants, like the sometimes dinosaur-proportioned ground sloths, who swallowed the seeds and hardly noticed them. The ground sloths disappeared a geological moment ago, but their curious fruit, the avocado, remains.

It will make me never quite look at an avocado the same way again.

There are so many things I learned which made me reconsider my whole view of dinosaurs. Like Dimetrodon, the creature with a large sale on it's back for temperature regulation, is more closely related to mammals than dinosaurs. And without the 3rd mass extinction, we'd never have seen Dinosaurs, and mammals might have ruled the Earth much earlier. And that T-rex showed up really late on the scene, filling the niche that that much more successful Allosaurs held as apex predators for most of the Jurasic era (the Allosaurs all disappeare in a more minor great extinction).

It's not often that you find a non fiction book that both reads fast, and dumps such an incredible amount of information on you. The jumping back and forth from road trip, chatting with scientists, facts, and painting pictures of the world that was, works really well. There is never a dull moment in it, and you come out the far end for a much greater appreciation for life on Earth in all its forms.

Much warmer summers

The map above, based on a new analysis from the Climate Impact Lab, shows how 95-degree days (35 degrees Celsius) are expected to multiply this century if countries take moderate climate action. In this scenario, countries would take some measures, but not drastic ones, to curb emissions — roughly the trajectory of the current pledges under the Paris climate agreement.

The resulting global warming would still cause significant shifts for many cities. In Washington, from 1986 to 2005, an average of seven days each year had temperatures of at least 95 degrees. By the end of the century, the city can expect 29 of these extremely hot days per year, on average. (The likely range is 14 to 46 hot days per year.)

Source: 95-Degree Days: How Extreme Heat Could Spread Across the World - The New York Times

Good analysis on what the impacts of the hotter days coming are going to be. 95 F is as reasonable an arbitrary measuring point as anything else, we're approaching body temperature there. The article looks at the world under the Paris agreement, as well as without it. The differences are striking.

Interestingly, commercial crop yields (specifically corn and soybeans) start to drop after 84 F. I hadn't realized that, but it makes sense. That's how hotter days, even without drought, have negative impacts on our food supply.

It’s about 50 degrees warmer than normal near the North Pole, yet again - The Washington Post

Extreme temperature spikes such as this one have occurred multiple times in the past two winters, whereas they only previously occurred once or twice per decade in historical records according to research published in the journal Nature.

As Mashable science writer Andrew Freedman put it: “Something is very, very wrong with the Arctic climate.”

Source: It’s about 50 degrees warmer than normal near the North Pole, yet again - The Washington Post

As someone that follows the science, I definitely understand the difference between weather and climate. However, it takes climate change to create aberrations this extreme, this often.

This is all very real. It is unfortunate that many of our elected representatives don't agree with the science.

One of the largest icebergs recorded

The crack in Larsen C now reaches over 100 miles in length, and some parts of it are as wide as two miles. The tip of the rift is currently only about 20 miles from reaching the other end of the ice shelf.

Once the crack reaches all the way across the ice shelf, the break will create one of the largest icebergs ever recorded, according to Project Midas, a research team that has been monitoring the rift since 2014. Because of the amount of stress the crack is placing on the remaining 20 miles of the shelf, the team expects the break soon.

Source: A Crack in an Antarctic Ice Shelf Grew 17 Miles in the Last Two Months - The New York Times

Climate Change is real, and keeps on chugging. The visuals in the NY Times article are quite impressive and give you a more visceral sense of what is going on.

IPCC Debacle from the horse's mouth

There is a good posting on the Real Climate blog about the IPCC AR4 blow back that is happening.  I think this gets to the heart of it:

To those familiar with the science and the IPCC’s work, the current
media discussion is in large part simply absurd and surreal.
Journalists who have never even peeked into the IPCC report are now
outraged that one wrong number appears on page 493 of Volume 2. We’ve
met TV teams coming to film a report on the IPCC reports’ errors, who
were astonished when they held one of the heavy volumes in hand, having
never even seen it. They told us frankly that they had no way to make
their own judgment; they could only report what they were being told
about it. And there are well-organized lobby forces
with proper PR skills that make sure these journalists are being told
the “right” story. That explains why some media stories about what is
supposedly said in the IPCC reports can easily be falsified simply by
opening the report and reading. Unfortunately, as a broad-based
volunteer effort with only minimal organizational structure the IPCC is
not in a good position to rapidly counter misinformation.