Over the past few weeks I’ve been tweaking my emacs configs, and writing some new modes to help with OpenStack development. Here is a non comprehensive list of some of the emacs integration I’m finding useful in developing for OpenStack (especially things that have come up in conversation). URLs provided, though I won’t walk through all configuration in depth.
Tramp is a built in facility in emacs that lets you open files on remote machines via ssh (and other protocols). This means your emacs runs locally, with all the latency gains that has, as configured as you would like, but editing can be done across multiple systems. A remote file url looks like /servername:/file/path. All the normal file completion that you expect works after that point.
I tend to do code that doesn’t need a full stack run locally on my laptop or desktop, but full stack code happens on my NUC devstack box, and tramp lets me do that from an single emacs instance.
More info about tramp at emacswiki.
Emacs has an in-buffer syntax checker called flymake. There are various tutorials out there for integrating pyflakes or flake8 into that. However in OpenStack we have hacking, which extends flake8 with new rules. Also, every project turns on custom ignores. Also, many projects extend flake8 further with custom rules for that repo.
fly-hack uses the flake8 in the .tox/pep8 venv for each project, and uses the tox.ini config for each project, so when in Nova, nova rules will be enforced, when in Cinder, cinder rules will be enforced. Mousing over the error will pop up what it is (you can see the H306 in that screenshot). It has a fallback mode when using it over tramp that’s a reasonably sane flake8 least common denominator for OpenStack projects.
More info at fly-hack github page – https://github.com/sdague/fly-hack
Our testr / subunit / testtools testing toolchain gets a bit heavy handed when trying to iterate on test development. testr discovery takes 20 – 30s on the Nova source tree, even if you are trying to only run 1 test. I became inspired at the Ops Summit in Philly to see if I could do better. And stacktest.el was born.
It’s mostly a port from nosemacs which allows you to run tests from an emacs buffer, however it does so using tox, subunit, or testtools, depending on whether you want to run a top level target, test a file, test an individual test function, and/or use pdb. It works over tramp, it works with pdb, and it uses the subunit-trace tooling if available.
I’ve now bound F12 to stacktest-one, which is a super fast way to both iterate on test development.
More info at the stacktest github page – https://github.com/sdague/stacktest
OpenStack is a lot of code, and uses a ton of libraries. git grep works ok in a single repo, but the moment some piece of code ends up calling into an oslo library, that breaks down.
Peter Portante, OpenStack Swift contributor, maintains a pythonized version of cscope. It parses the AST of all the files to build a quite rich symbol cscope database. This lets you search for definitions (searching down), calling points (searching up), and references (searching sideways). Which very quickly lets you dive through a code path and figure out where you end up.
The only draw back is the AST parse is consuming on something as large as the Nova tree, especially if you index all the .tox directories, which I do to let myself get all the way back through the library stacks that we include.
You can learn more about pycscope at it’s github page – https://github.com/portante/pycscope
Emacs includes a spell checker called flyspell. Very useful for text files. What I only learned last year is that there is also a flyspell-prog-mode, which is like flyspell, but only acts on comments and strings that are semantically parsed by Emacs. This helps avoid a spelling mistake when writing inline documentation.
More information at Emacs wiki.
This is totally gratuitous, but fun. There is a small mode that does a display adjustment of the word ‘lambda’ to an actual ‘λ’. It’s a display adjustment only, this is still 6 characters in the buffer. But it makes the lambda expressions pop out a bit more.
More information at Emacs wiki.
The important thing about having an extensible editor is actually extending it to fit your workflow. If you are an emacs user, hopefully this will give you some fun things to dive into. If you use something else, hopefully this will inspire you to go looking into your toolchain for similar optimizations.
9 thoughts on “OpenStack Emacs Tools”
This is really helpful! I’m excited to take a look at PyCscope.
I do a similar thing regarding testing with the big difference is that I use Elpy and pytest for iterative testing. I keep a local virtualenv for the package and configure that via dir-local. For example, my Designate directory has the following .dir-locals.el
;;; Directory Local Variables
;;; For more information see (info “(emacs) Directory Variables”)
(pyvenv-activate . “/Users/elarson/venv”)
(elpy-test-runner . elpy-test-pytest-runner))
(magit-gerrit-remote . “ssh://email@example.com:29418/openstack/designate”)))
Then, in a test file, I can hit C-c C-t to do a similar thing as nosemacs / pytest.el where it runs the single test, file or full suite.
Magit Gerrit has also been really handy for doing things like basing a review on some other review by downloading the review branch and branching off of that.
Thanks, that’s a nice post. You might be interested by adding interactivity to pycscope: that’s possible with helm-cscope http://wikemacs.org/wiki/Python#Interactivity_with_helm-cscope (which isn’t on melpa for now).
What is your virtualenv workflow?
I’m not sure what your question means in this context. Can you be more specific?
Might not be specific to this context. But to be specific, I am using elpy, jedi and pyvenv in emacs for python development. In openstack we are using tox for creating virtualenv. But when I actually activate one of them say, for eg:- .tox/py27 using pyvenv and then subsequently run tox -e py27, somehow the virtualenv gets wiped out. Are you using any other better approach?
I’m using jedi, but not the others. For normal development I’m not activating a venv very often. If you are really only running a single command or two, there is very little reason to activate the venv, just .tox/py27/bin/TOOLNAME and be done with it. That’s how stacktest.el works.