Tag Archives: hacking

User modifiable Canon firmware

Through dave hansen’s blog I found the CHDK project, which creates custom Canon camera firmware that adds a lot of features to existing cameras. While I’ve only had this loaded for a day, I’m really psyched with the results so far.

It immediately meant that I got RAW support, an OSD battery sensor showing % left, exposure settings, and this zebra mode thing that let’s you know when you’ve maxed out the CCD, and where specifically is the issue. The procedure to add it is pretty simple, and it’s breathed new life into my SD500, which has seen a lot of good days with me.

I get really excited when devices evolve and improve after you have them. My Logitech Harmony Remote, XBMC on my old xbox, and now my Canon camera all fall under that heading. This also guaruntees that I’ll only buy Canon cameras that support this in the future. This kind of freedom is hard to give up. 🙂

The switch from xemacs -> emacs

I’ve always felt the root of the emacs vs. vi holy war (which is one of the longest standing holy wars in free software) basically came down to the following key point:

When you first were learning Linux / Unix, did your mentor use vi or emacs?

The answer to that is at least 90% correlation to your preferences. Much like most people share the politics of their parents, most people share the editor preferences of their mentors. Switches don’t tend to happen unless mentors switch as well.

And in that camp, I’m an emacs guy. I learned it in college when I took my first programming class (which was in lisp). Our professor gave us a starting .emacs file, pointed us at the tutorial, and built macros that helped us out in our efforts.

A near decade with XEmacs

Then I graduated from college, and my first mentor at IBM was also an emacs guy, except he was an xemacs guy. It was emacs, but prettier. So I piled on, and was there ever since. Over the years I tried a couple of times to go back to emacs, but their font handling was never as good. I love programming in arial, as it’s just really pleasant on the eyes (this shocks and horrifies people that line up = signs in declarations, but I don’t much care. 🙂 ).

A few years ago, just as emacs was getting reasonable variable width font support, xemacs integrated anti-aliased fonts into their CVS tree, and now I had another reason to stay on xemacs, because now everything looks pretty. Using xemacs was sometimes a pain, as a number of modes didn’t really work right on it. I never had a reasonable html mode working that did indentation like I wanted.

Steve Yegge’s Rant

Last month Steve Yegge had a post entitled Xemacs is Dead, Long Live Xemacs which was basically a call to unify around emacs because it had finally caught up, and it is being very actively maintained. I was skeptical, but decided to try again. Using the Ubuntu packages I lost my anti-aliasing, which meant this was a failed experiment.

But, after some research, I realized that emacs cvs not only has xft support in the tree, but that since March it’s been the default. This is what will be emacs 23. I was already running xemacs out of cvs, so taking the same leap with emacs cvs wasn’t such a big deal.

It’s taken a couple of weeks to tweak my configuration to get me the same, or better, results with emacs as I had with xemacs. Last night I finally understood what I needed to get nxhtml to do my html.erb files correctly (ruby and html bits independently highlighted, and mode switching automatically when moving between code blocks). Minus 1 font issue with planner, I can definitely say I’m fully converted.

I’m also enjoying diving into elisp again. For whatever reason, life seems a bit more stable on emacs than it did on xemacs. And once emacs 23 actually makes it to distros, I won’t even need to have my own binary builds. 🙂

Perfect Rails Development / Deployment environment with mercurial and passenger

A couple weeks ago I found phusion’s passenger (aka mod_rails), and it’s great. Passenger compiles an apache module which manages a rails app server for each application. It is on par with mongrel on speed, and so much easier to integrate into an average Linux environment for rails app hosting. Here’s my new standard rails development environment (3 applications right now, with a couple more in the works).

Development with Mercurial

Distributed Source Code Management is the wave of the future, and I can’t say enough good about it. My prefered system is Mercurial (aka hg), which I got to know why working on the Xen project. While git is getting a lot more hype of late, I still think hg is easier to use, and an easier switch for people that know subversion.

Rails makes it very easy to run a server locally for development, so having versioning locally makes perfect sense. I can hack away for days making changes on my laptop until I get to a point where I want the code to see the light of day. Then it is an easy hg push to put my code either into production, or into a repository for sharing.

Deployment with Passenger

is a god send. Getting mongrel to do the right things on init on an ubuntu box was just a pain in the rear end. I like apache, I know apache, having to configure a web app outside of apache sucks. Passenger builds an apache plugin that is an rails app server. You don’t need to know any more than that, because it just works.

Actually, you need to know 2 things, both of which add to passenger’s awesomeness.

  • The rails app will run as the uid of the owner of your environment.rb file. This means you need to take care on how your permissions work on you deployed rails app. This is a good thing, as we’ll see in a minute.
  • You don’t need to restart apache to restart the rails app. You just need to touch tmp/restart.txt in the rails directory, and the app server restarts. This is very handy.

Bringing it all together with Mercurial Hooks

On my box where I’m deploying applications I created a new rails user, with a scrambled password, and just my ssh key to get in. This is the account in which I push production versions of apps to. It means that the rails apps run as user rails, which is ever further issolated than user www-data. You could even go a step further and have each rails app under a different user, but for me that’s overkill.

The .hg/hgrc for one of my local projects looks as follows (with myhostname, myrailsapp, and myrailssite replaced to protect the innocent):

default = ssh://myhostname.org/code/myrailsapp/
production = ssh://rails@myhostname.org//data/site/myrailssite/

Now my push targets are in place.<br /><br />The last bit is adding a set of hooks on the target repository so that on each push the repo is updated, migrations are run, and the app is restarted. I’ve created /data/site/myrailssite/.hg/hgrc as follows:

incoming = hg update -C && RAILS_ENV=production rake db:migrate && touch tmp/restart.txt

And that’s it. Now an hg push production force syncs the remote repo, runs any new migrations, and restarts passenger. A 1 step command, and everything is live.

While I’ve been a huge Ruby on Rails fan for the last year, deployment always sort of sucked with it. Now I’m a very happy camper with this setup which makes for very seemless development of rails applications.

Weekend Rails Hacking

For the past 4 years I’ve been using evite to manage the RSVPs for our memorial day weekend party. Given that it’s a pretty large scale pot luck event, it’s helpful to have a system where people can respond with a message that others can see. The reduces my need to field “what should we bring” questions, as you can easily see what everyone else is bringing and react accordingly.

Evite sucks. While it doesn’t force attendees to make accounts, it makes it look like it does. The evite.com emails tend to catch as spam. And the interface is now dubious under firefox. The idea is still good, but it hasn’t really ramped with the trends in the rest of the service web application space.

One of the key things I wanted in an evite replacement is getting rid of user logins. Given an event, and an email address, you can come up with a unique key that qualifies that person for that event. That means the user just follows a link, and they are in. Links are unique for people. If you make the key a hash of the person’s email and some secret seed key for the event, you’ve got something cryptographically strong as well. No one can modify another person’s entry because the key needs to match before you get any info.

Saturday was a rainy day, so I built this system. By Saturday night I had most of it working, and had rolled this out live by Sunday afternoon. This was my first rails 2.0 app, so I needed to catch up on a few things along the way. Things I learned:

  • Rails 2 creates scaffolds in a slightly new way. That threw me for a bit, as I had already built models for most of my objects before creating scaffolds. The new way (putting attributes on the command line) looks like it is designed to make rails tooling easier.
  • ActionMailer is crazy easy. It even does multipart mixed emails really easily. My mhvlug mailer script for month announcements is going to need to be converted to this at some point.
  • Rails has a word_wrap function in the view context. Of course it does, why did I even doubt that.
  • The google maps API is impressive. I had maps based on event location within 60 minutes of signing up for my Google Maps API key.
  • The f= param on maps.google.com is which fuction to drive. q: location query, d: directions. That took a little bit of reading urls to realize.
  • It’s pretty easy to integrate mercurial push to auto restart a rails app if it’s running under passenger.
  • If you are running multiple versions of rails applicatoins under passenger, delete all the rails links in vendor/ so that it picks up the right rails environment.
  • arround_filter in rails is really handy to catch generic exceptions and dump people off to an error page that isn’t the default rails one.

All in all, I was really happy how this turned out. As soon as I get some free time I’ll genericize the bits of the app that I coded just for our event, and get this out on rubyforge. I only wish there was a rails equiv of gems, as I’ve still found that it isn’t entirely clear how to best package a rails application as an easy to download open source component.

Dague.net move

Once upon a time I said I would never host my own email (or email in general), as it was a pain I didn’t want. Then, I ended up hosting email for mhvlug.org because it turned out to be the simplest solution. A week later I installed postgrey, and watched the spam rates drop by 80%. And it was good.

A couple things changed in the last year. Linode went from UML to Xen, which definitely makes each linode more powerful. My shared hosting company stopped being helpful. I had a couple of small outages. They had moved from a knowledgable support staff, to a support pool that was clueless, and never seemed to understand the ways in which their system was broken. And, after hosting mhvlug email for a while with no issues, it seemed reasonable that dague.net email would be safe there as well.

Backups (thanks to backuppc) have been ramped up from every 24 hrs to every 6 hrs on the box, to narrow my window in which I can screw things up. Only one set of email delays so far, mostly because I set a wrong postfix param over the weekend, which may have been blocking mhvlug.org email as well. But that is resolved now. Dan will at least thing I’m a real man now. 😉

Mono 1.9 install script

Unfortunately no one has made ubuntu packages yet, however here is a script that I built based on Dirk’s post to automate mono 1.9 installation onto Ubuntu environments.


# This is needed to pick up our built mono for commands
export PATH=/usr/local/bin:$PATH 

apt-get install build-essential bison gawk
apt-get install libglib2.0-dev
apt-get install libpng12-dev libx11-dev libfontconfig1-dev
apt-get install libfreetype6-dev libjpeg62-dev libtiff4-dev
apt-get install libungif4-dev libexif-dev libcairo2-dev
apt-get install libpango1.0-dev libgtk2.0-dev libglade2-dev
apt-get install libgnome2-dev libgnomecanvas2-dev libgnomeui-dev
apt-get install libgnomeprint2.2-dev libgnomeprintui2.2-dev
apt-get install libpanel-applet2-dev libgtksourceview-dev
apt-get install libgtkhtml3.14-dev

mkdir -p $BUILDDIR

wget http://go-mono.com/sources/libgdiplus/libgdiplus-1.9.tar.bz2
tar xvf libgdiplus-1.9.tar.bz2
cd libgdiplus-1.9
./configure --prefix=/usr/local
make install
cd ..

wget http://go-mono.com/sources/mono/mono-1.9.1.tar.bz2
tar xvf mono-1.9.1.tar.bz2
cd mono-1.9.1
./configure --prefix=/usr/local
make install
cd ..

wget http://switch.dl.sourceforge.net/sourceforge/nant/nant-0.86-beta1-src.tar.gz
tar xvf nant-0.86-beta1-src.tar.gz
cd nant-0.86-beta1
make install --prefix=/usr/local
cd ..

wget http://go-mono.com/sources/gtk-sharp210/gtk-sharp-2.10.4.tar.bz2
tar xvf gtk-sharp-2.10.4.tar.bz2
cd gtk-sharp-2.10.4
./configure --prefix=/usr/local
make install
cd ..

wget http://go-mono.com/sources/gnome-sharp2/gnome-sharp-2.16.1.tar.gz
tar xvf gnome-sharp-2.16.1.tar.gz
cd gnome-sharp-2.16.1
./configure --prefix=/usr/local
make install
cd ..

wget http://go-mono.com/sources/gtksourceview-sharp2/gtksourceview-sharp-2.0-0.12.tar.bz2
tar xvf gtksourceview-sharp-2.0-0.12.tar.bz2
cd gtksourceview-sharp-2.0-0.12
./configure --prefix=/usr/local
make install
cd ..

cd mono-1.9
wget  http://go-mono.com/sources/monodoc/monodoc-1.9.zip
unzip monodoc-1.9.zip
cd monodoc-1.9
./configure --prefix=/usr/local
make install
cd ../..

wget http://go-mono.com/sources/mono-tools/mono-tools-1.9.tar.bz2
tar xvf mono-tools-1.9.tar.bz2
cd mono-tools-1.9
./configure --prefix=/usr/local
make install
cd ..

wget http://ftp.novell.com/pub/mono/sources/mono-debugger/mono-debugger-0.60.tar.bz2
tar xvf mono-debugger-0.60.tar.bz2
cd mono-debugger-0.60
./configure --prefix=/usr/local
make install
cd ..

wget http://ftp.novell.com/pub/mono/sources/heap-buddy/heap-buddy-0.2.tar.gz
tar xvf heap-buddy-0.2.tar.gz
cd heap-buddy-0.2
./configure --prefix=/usr/local
make install
cd ..

wget http://ftp.novell.com/pub/mono/sources/mono-addins/mono-addins-0.3.1.tar.bz2
tar xvf mono-addins-0.3.1.tar.bz2
cd mono-addins-0.3.1
./configure --prefix=/usr/local
make install
cd ..

wget http://ftp.novell.com/pub/mono/sources/monodevelop/monodevelop-1.0.tar.bz2
tar xvf monodevelop-1.0.tar.bz2
cd monodevelop-1.0
./configure --prefix=/usr/local
make install
cd ..

Hacking on OpenSim Infrastructure: Mantis Improvements For The Win!

Among other roles in the OpenSim project, I’m reluctant admin for most of the opensimulator.org infrastructure. Infrastructure being defined as: scm repo (subversion and mercurial), bug tracker (mantis), and wiki (media wiki).

Recently I decided to do some hacking on our mantis tracker to make it work better for the project. There was a very real reason to do this, our mantis has over 350 open and unassigned issues, and was starting to get massively ignored by most developers as it was far too overwhelming to route out real issues in mantis vs. stale issues vs. user errors in such a large sea.

The first change in this area was creating an osmantis bot. This is an IRC bot that spits out a message in our #opensim-dev channel on every mantis change. This brings the same level of visibility to our bugs as to our svn commits (which get the same treatment). This involved a rather brutal amount of hackering in mantis, and a perl IRC bot that runs on opensimulator.org. If you’ve been on IRC at all in the last couple of weeks you’ve seen it.

This morning I introduced 3 new states related to patches (patch included, patch feedback, patch ready). The agreed policy of OpenSim is that patches come in via mantis. However, as nothing really makes patches stand out more than regular issues, we were loosing a lot of patches in mantis. I believe our oldest patch in mantis which clearly hadn’t been looked at by anyone was over 3 months old. Not a great state to be in. With the new set of states mantis issues with patches can be set to patch included as part of triage. This will make them pop up to the top of everyone’s attention.

Another issue we’ve had is that users can’t close their own issues. I just fixed that (I think), so if you have reported issues via OpenSim mantis I’d ask to go check to make sure they still look valid. If they aren’t, please close them, to help us clean up the current state of things.

I’m hoping these changes make it easier for us to manage issues coming into the project, and make using mantis more effective for everyone.