Helpful Gerrit Queries (Gerrit 2.8 edition)

Gerrit got a very nice upgrade recently which brings in a whole new host of features that are really interesting. Here are some of the things you should know to make use of these new features. You might want to read up on the basics of gerrit searches here: Gerrit queries to avoid review overload, before getting started.

Labels

Gone are the days of -CodeReview-1, we now have a more generic mechanism called labels. Labels are a lot more powerful because they can specify both ranges as well as specific users!

For instance, to select everything without negative code reviews:

Because we now have operators, we can select for a range of values, so any negative (-1, -2, or any high negative value should it get implemented in the future) matches. Also negation is done with the 'NOT' keyword, and notable that CodeReview becomes label:Code-Review in the new system.

Labels exist for all three columns. Verified is what CI bots vote in, and Workflow is a combination of the Work in Progress (Workflow=-1) and Approved (Workflow=1) states that we used to have.

Labels with Users

Labels get really power when you start adding users to them. Now that we have a ton of CI bots voting, with regular issues in their systems, you might want to filter out by changes that Jenkins currently has a positive vote on.

This means that changes which do not yet have a Jenkins +1 or +2 won't be shown in your list. Hiding patches which are currently blocked by Jenkins or it hasn't reported on yet. If you want to see not yet voted changes, you could change that to >=0.

Labels with Self

This is where we get really fun. There is a special user, self, which means your logged in id.

This is a list of all changes that 'you have not yet commented on', that don't have negative code reviews, and that Jenkins has passing results. That means this query becomes a todo list, because as you comment on changes, positive, negative, or otherwise, they drop out of this query.

If you also drop all the work in progress patches:

then I consider this a basic "Inbox zero" review query. You can apply this to specific projects with "project:openstack/nova", for instance. Out of this basic chunk I've built a bunch of local links to work through reviews.

File Matching

With this version of gerrit we get a thing called secondary indexes, which means we get some more interesting searching capabilities. which basically means we also have a search engine for certain other types of queries. This includes matching changes against files.

is a query that looks at all the outstanding changes in OpenStack that change a database migration. It's currently showing glance, heat, nova, neutron, trove, and storyboard changes.

Very helpful if as a reviewer you want to keep an eye on a cross section of changes regardless of project.

Learning more

There are also plenty of other new parts of this query language. You can learn all the details in the gerrit documentation.

We're also going to work at making some of these "inbox zero" queries available in the gerrit review system as a custom dashboard, making it easy to use it on any project in the system without building local bookmarks to queries.

Happy reviewing!

 

Employment Agreements vs. Open Source

Reading through an interesting, and mostly accurate piece about OpenSSL the following jumped out at me:

The fact that OpenSSL pays next to nothing constrains things further. Those who do help Henson out often juggle coding with full-time paying jobs elsewhere. Others can’t code for OpenSSL: Their employment contracts prohibit it, so they simply act as advisers. That leaves Henson responsible for 60% of the code commits (or sets of changes to the source code), twice as many as the next-most-prolific developer, Andy Polyakov, who’s compensated some. Most of the code added in the past few years has been approved by Henson, or tapped out on his own keyboard.

via The Internet Is Being Protected By Two Guys Named Steve.

Last week a bunch of tech companies signed up to donate $100K to help projects like OpenSSL. However, money really isn't the most constrained resource here, it's people with expertise.

A lot of large companies that aren't incorporated in California have really stringent IP agreements with their employers. Something along the lines of "relate to the actual or anticipated business or research or development" of said company. In a large company, this means, everything. California has a state law that trumps this, and doesn't allow companies to claim anything that you create on your own time, without the use of company equipment.

I wonder what the impact would be if these same companies also pledged to adjust their employment agreements to let their employees contribute to Open Source in their own time. I expect that it would have a much larger impact than the financial pledge.

Bash trick of the week - call stacks

For someone that used to be very vocal about hating shell scripting, I seem to be building more and more tools related to it every day. The latest is caller (from "man bash"):

caller [expr]
Returns the context of any active subroutine call (a shell function or a script executed with the . or source builtins). Without expr, caller displays the line number and source filename of the current subroutine call. If a non-negative inte‐ ger is supplied as expr, caller displays the line number, subroutine name, and source file corresponding to that position in the current execution call stack. This extra information may be used, for example, to print a stack trace. The current frame is frame 0. The return value is 0 unless the shell is not executing a subroutine call or expr does not correspond to a valid position in the call stack.

This means that if your bash code makes heavy use of functions, you can get the call stack back out. This turns out to be really handy for things like writing testing scripts. I recently added some more unit testing to devstack-gate, and used this to make it easy to see what was going on:

The output ends up looking like this:

I never thought I'd know this much bash, and I still think data structure manipulation is bash is craziness, but for imperative programming that's largely a lot of command calls, this works pretty well.

Greed and the Wright Brothers - NYTimes.com

The Wright brothers’ critical insight was the importance of “lateral stability” — that is, wingtip-to-wingtip stability — to flight. And their great innovation was something they called “wing warping,” in which they used a series of pulleys that caused the wingtips on one side of the airplane to go up when the wingtips on the other side were pulled down. That allowed the Wrights’ airplane to make banked turns and to correct itself when it flew into a gust of wind.

But when the Wrights applied for a patent, they didn’t seek one that just covered wing warping; their patent covered any means to achieve lateral stability. There is no question what the Wrights sought: nothing less than a monopoly on the airplane business — every airplane ever manufactured, they believed, owed them a royalty. As Wilbur Wright, who was both the more domineering and the more inventive of the two brothers, put it in a letter: “It is our view that morally the world owes its almost universal system of lateral control entirely to us. It is also our opinion that legally it owes it to us.”

What was Curtiss doing in the meantime? In addition to coming up with the idea of adding wheels for easier takeoffs and landings, he invented an entirely different system for dealing with lateral stability, a system of flaps that went up and down and controlled the wings. (Airplane manufacturers today still use that basic insight.) The Wrights responded by filing a lawsuit, claiming that Curtiss was violating their patents. The litigation would consume them literally until the day Wilbur Wright died.

via Greed and the Wright Brothers - NYTimes.com.

The problem with the Intellectual Property is that it incentivizes people to sit on their laurels once they've captured an idea, so stiffles the next round of innovation. Most of these ideas aren't nearly as revolutionary as you think, as even ground break ideas are typically simultaneously invented by independent groups. There is a great survey of this across multiple major inventions in human history in Steven Johnson's Where Good Ideas Come From.

Robert Muth: Better Bash Scripting in 15 Minutes

Better Bash Scripting in 15 Minutes. The tips and tricks below originally appeared as one of Google's "Testing on the Toilet" TOTT episodes. This is a revised and augmented version.

via Robert Muth: Better Bash Scripting in 15 Minutes.

Some good bits in here. We've implemented some of them in devstack, and I think a few more (like uninitialized and enforcing double brackets on all conditionals) would be helpful. It also makes me think about things to enforce in bash8.

We live on fragile hopes and dreams

OpenSSL isn't formally verified!?

No, neither is any part of your browser, your kernel, your hardware, the image rendering libraries that your browser uses, the web servers you talk to, or basically any other part of the system you use.

The closest to formally verified in your day-to-day life that you're going to get may well be the brakes on your car, or the control systems on a jet engine. I shit you not.

We live on fragile hopes and dreams.

via My Heart Bleeds for OpenSSL | Coder in a World of Code.

At lot of the internet is learning a lot more about how software in the wild functions after heartbleed. I found that statement to be one of the best summaries.

Devstack Vagrant

Devstack is tooling for OpenStack to make it easy to bring up an OpenStack environment based on the latest git trees. It's used extensively in upstream testing, and by many OpenStack developers to set up dev/test environments.

One of my personal challenges in working on Devstack was testing devstack itself. Relying on the upstream gate means we have a limited number of configurations, and when something goes wrong, iterating on a fix is hard. Even more importantly, the upstream gate is currently only a single node test environment.

A month ago I hacked out a new tool - devstack-vagrant (available on github).

DevstackVagrant

Devstack vagrant provides a customized Vagrant environment that will build a 2 node devstack cluster under VirtualBox. The basic model is 2 devstack nodes (a controller and a compute) that bridge through a physical interface on your host. The bridged interface is set as the default route in the nodes so that 2nd level guests created on top of this 2 node devstack can route to the outside world.

The nodes start and build from official Ubuntu 12.04 cloud images, and are customized using the puppet provisioning support in vagrant. There are a few config variables you need to set in a config.yaml, including hostnames, bridge interface, and the password hash you want your stack user to have. Basically enough to bootstrap the environment and then run devstack from upstream git.

I added a bit of additional logic to the end of the normal devstack process that includes installing an Ubuntu 12.04 and Fedora 20 cloud image in your glance, injecting the ssh public key for the stack user into the keyserver, and opening up ping and ssh in all the security groups.

I still consider this an expert tool at this point, as in, if it breaks you get to keep all the pieces. However, this has been useful to me so far, and given the pull requests I got the other day, seemingly is useful for others as well. Patches definitely welcomed. And if it looks like more folks want to contribute I'll happily move to stackforge.

One of the things I'd love to do is sort out a functioning libvirt backend for vagrant (there are 2, they are both a little wonky) because then the 2nd level guests could use nested KVM and not be ridiculously slow.

This tool has already proved very useful to me, so hopefully it will be useful to others as well.

The miracle of a billion cameras

Meteor Fall

It sounds like a remarkable story, almost unbelievable: Anders Helstrup went skydiving nearly two years ago in Hedmark, Norway and while he didn’t realize it at the time, when he reviewed the footage taken by two cameras fixed to his helmet during the dive, he saw a rock plummet past him. He took it to experts and they realized he had captured a meteorite falling during its “dark flight” — when it has been slowed by atmospheric braking, and has cooled and is no longer luminous.

via Norwegian Skydiver Almost Gets Hit by Falling Meteor — and Captures it on Film.

Part of what's amazing about so many people recording things all the time on camera is we get to see things that we know must be, but no one has directly observed before. Like rocks falling from the sky.

I think XKCD sums it up best:

XKCD Settled