Tag Archives: email

OpenStack

Processing OpenStack GPG keys in Thunderbird

1 Comment

If you were part of the OpenStack keysigning party from the summit, you are currently probably getting a bunch of emails sent by caff. This is an easy way to let a key signer send you your signed key.

These are really easy to process if you are using Thunderbird + Enigmail as your signed/encrypted mail platform. Just open up the mail attachments, right click, and import key:

screenshot_161

Once you’ve done this you’ll have included the signature in your local database. Then from the command line you can:

gpg --send-key YOURKEYID

And then you are done.

Happy GPGing!

Uncategorized

Moving off GMail

2 Comments

In early December I finally decided it was time to move my primary email out of google. There were a few reasons to do it, though the practical (reaching the limits on their filtering) largely outweighed the ideological.

Movable Email

If email is important to you, you should really register your own domain name, so you have a permanent address. I got dague.net back in 1999 to create a permanent home for my identity. This has meant over they years the backend for dague.net has changed at least 5 times, including me hosting it myself for a large number of years.

My Requirements

  • Can host email on my own domain – as I’d be moving dague.net
  • Web UI – because sometimes I want to access my email via Chromebook
  • Good Search – because there are times I fall back to full text search to find things
  • IMAP – because most of the time I’ll be accessing via Thunderbird or Kaiten Mail
  • Good spam filtering
  • Good generic filtering on the server side – My daily mail volume is north of 1000 messages (40% spam), I need good filtering otherwise I drown

Fastmail.fm

I eventually landed on Fastmail.fm, who I’ve been watching for a lot of years. They are fairly priced ($40/yr), their company contributes back to the open source software they run their business on, and because they are actually an Australian company, you’ll get disclosure if some agency is accessing your accounts. They also give you a 60 day free trial, so you can do a slow migration over, and see if it will meet your needs.

Migration

One I was sure I was going to do this, I created my fastmail.fm account, and then pulled and configured imapsync to sync my existing gmail content over. I have a couple of GB of email, which means an imap sync takes a good 24 hours at this point. Imapsync is rerunable, so run it once, wait until it finishes, then run it a second time, and pick up the changes. Once it seems like you’ve basically closed the gaps between the two accounts, you can change MX records, and start getting email at the new service provider.

For safety the first thing I do once this has happened is build a forward rule from the new provider to the old one. Then if something goes horribly wrong, all my email remains in both locations for a while. A month later I’m still running that forward, though will be disconnecting it soon.

So far so good

The webmail for fastmail is really solid, honestly I like it better than gmail’s web ui, which has become incredibly cluttered over the years. This is just email, which is good. It also has a search facility which is on par with google’s. It’s also available as part of the IMAP protocol, which means real searching from Kaiten mail on Android. Switching from GMail App to Kaiten Mail on my phone was about 10 minutes. And it means I can actually customize things I get alerted to, which gmail broken at some point. Thunderbird transition was simple.

I had gotten used to Raportive on gmail that would give me people’s pictures on their email. I found the Ldap Info Show extension on Thunderbird, which looks people up on various social networks, and gives you pictures they have public.

Lacking APIs

The one complaint I have with fastmail, is that it’s lacking APIs to handle your data. For instance, my filtering rules are complex. 342 lines of sieve and counting at this point. This is managed via a web form, but copy / paste on every change is something I’m not really into. I solved this by writing a python mechanize sync script so I can manage the rules locally, version controlled, then sync them up afterwards.

Address book has some issues, and I’ve not built a work around. The sieve rules they give you whitelist your address book as spam sources, so it’s something I’d like to keep in sync. However, without an API it’s not really worth it.

Overall: Very Good

Overall I’m very happy with the move. My biggest complaints are around the API issue, which I hope they correct in the future.

Uncategorized

Migration to Google Email

Leave a comment

In late 1999 I claimed my last name as a domain, and have had various email and web solutions hanging off of it ever since. This past weekend I migrated off of self hosted email to Google Apps for Domains. My email address remains the same, but the infrastructure is Google’s. There were 3 main converging trends that drove me there: spam, client innovation, and protocol integration.

Because most people host email on one of the big three (Google, Yahoo, Microsoft), spam fighting techniques for the little guy have largely stagnated. If you are big enough there are other algorithms you can apply to patterns affecting your millions of subscribers at once. For individual filtering, especially with an email address that’s been constant for over a decade, the best you can do is spamassassin, which last released in 2011, and realistically hasn’t done anything innovative in the last 3 years. So recently my false possitive and negative pool have been overlapping in a way that means a lot of manual work. Thunderbird’s bayesan filtering is quite good, and makes up for some of this. So when my laptop is running while I’m getting mobile email, my spam rate in mobile is low. When it’s not, about 80% of what gets to my mobile inbox is spam.

Spam by itself wouldn’t have pushed me over the edge, but Mozilla deprecating new development on Thunderbird was another blow here. Email is powerful because it is universal, can be accessed by any device at any size, multiple clients interacting with the same data, at the same time. Desktop email has gotten the short end of the stick in recent years, again because most people are hosting with the big 3, but Mozilla was still making a valiant attempt to keep email open. They’ve now decided it’s more interesting to chase Chrome than provide value here. That’s their call, but it’s sad to see desktop email take that hit.

Lastly, there are lots of quite interesting tools growing up to integrate with email in a new social world. Give you profiles of your contacts via social networks, make it easy to convert email into tasks. All great stuff. None of it works with IMAP or Desktop email. All the innovation around email right now is using the GMail API and Chrome extensions to modify GMail web interface.

So the migration is on, I’ve nearly got my email history dating back to 2000 into Google now. In the process I found I’d actually lost 2008 and 2009 archives, which I’ve mostly restored via backup. That would explain why some things weren’t showing up in search that I expected. Already, the spam filtering is a huge win, and eventually I’ll get used to the web UI for some things (still going to keep using Thunderbird in combo for a while).

The biggest challenge in this whole process is that because of how Google has wedged Plus into everything, having both a gmail and an apps email causes some real confusion on the Plus side, because there is no way to tell Google they are the same. That’s just going to be confusing for a while, and if anyone has best practices around that, let me know.

Uncategorized

Hello Thunderbird

1 Comment

Push finally came to shove, and I’ve now entered the 21st century by making Thunderbird my email client (I actually tried Evolution for a day, but after 20 crashes gave up. But that’s a different story.) Previously I was using mutt. There were a bunch of reasons to do this, though the biggest one for me was getting to turn off a box at home that was my IRC proxy, gateway to my home network, and ssh point for reading my email in mutt. That should save us at least a few hundred watts.

The New Configuration – Server Side

I’ve moved to using dovecot as my imap server. This has the advantage of being able to handle a home directory full of mboxes nicely (which courier could not). This means I can keep my perl based dynamic mail filtering working on the server until I manage to rewrite it as a thunderbird extension. I was using IMAPS before just as a secure POP, but now I’m actually taking full advantage of having imap remote folders.

My IRC proxy moved to my linode, which was probably a better place for it to be anyway. I even bothered to package it as a ppa for ubuntu, which means you can easily install as well.

Lastly, my gateway box is now a kvm guest running on my big home media / backup server. I was quite impressed by how nice virt-manager made the system install and setup from an ubuntu 8.04 iso. I had to do a little manual effort to configure bridge networking correctly, and deal with conflicting dnsmasq instances, but after that all was good.

The New Configuration – Client Side

I’ve now got thunderbird setup for 3 IMAP accounts (dague.net, gmail, and work), plus news groups (all work ones). This gives me a really nice consolodated view of my email. I was pretty impressed by how well thunderbird handles the 4 identies, and routes outbound correctly quite nicely. For dague.net and gmail email is filtered server side, I’ve client side filters for work because it’s sieve, and I really don’t want to learn another filtering specification language.

On top of that I’ve got a ton of extensions. I found that thunderbird out of the box was ok, but I lost a lot of mutt functionality. After a hunt through the extensions I got most, if not all of that back. For the record here are the extensions I currently have installed:

  • Attachment Reminder – this fires off a warning and prompt if you hit it’s heuristic rules of an email that might need an attachment but you don’t have any. I’ve seen the warning 4 times now, though they were all false positives. I do like the idea though, so I’ll keep it around.
  • Colored Diffs – brilliant if you are on mailing lists where patches are sent around
  • Display Mail User Agent – because I’m curious on who uses what. I always had this header visible in my mutt configs.
  • Display mailing list header – way more useful than I thought. It basically puts a set of links across the top of the email for Subscribe, Unsubscribe, Archive, etc. It makes it a lot easier to get off lists that you realize you don’t really care about any more.
  • Enigmail (from ubuntu package) – there was no way I was giving up pgp. It also has the advantage of making pgp policy setting much simpler.
  • Extension Developer – more on this later
  • Import Export Tools – because I had a lot of saved off mbox files that I needed to get back into thunderbird.
  • keyconfig – actually works on all mozilla base tools, but I needed it to redo a few key bindings
  • Lightning – this is the Sunbird callendar program as an embedded addon. It’s actually quite nice for callendaring and task lists.
  • Mnenhy – this gave me more control over mail headers. IIRC display mailing list header needs it to function.
  • Mutt Keys – my own extension, more on that in a bit
  • Nostalgy – gives you a set of nice key bindings and input field for save & copy of email. Very handy.
  • Provider for Google Calendar – a lightning plugin that lets you have good 2-way google calendar support. This is something evolution promissed, but it didn’t work. It works great on thunderbird with this extension.
  • Quote Colors – if people both to follow standard quoting models for email this does a really nice job of coloring the different posters to make it much easier to read.
  • Track Package – gives you highlight + right click to track packages based on emails. While it’s not everything I want, it is pretty useful.

But it could be better…

Thunderbird is now very useful to me, but I have found ways in which I could make the whole thing better. Mutt keys was a quick dive into making my own thunderbird extension that was nothing much more than key bindings (based on the now unmaintained mouseless extension). It’s rough, but it let me figure out some of the basic structure of writing thunderbird extensions.

Since then I installed extension developer, which has a great tab completable javascript shell, and have been exploring making an extension that lets me quickly make a calendar task out of an email. I have a bunch of ideas queued up behind this, but that is a short term useful one to dig into. I actually quite like the component interface model that thunderbird has, though I wish there were a few more API docs or examples to figure out what possibilities exist.

As I figure out more, I’m sure I’ll post it here. I have definitely found that developing thunderbird extensions is pretty tall grass, as very few folks have really written down much on it. I’m going to try to be a good citizen and stick stuff in the mozilla wiki as I figure it out.