Sean’s Mental Walkabout

I definitely don’t have this anywhere near I like it, but I did manage to just dump out a bunch of info from my thermostat and turn off the fan with this script:

I need some nicer symbolic constants for state setting, and pull together a rails site just to keep track of thermostat data over the course of the day.  All this code is going on my newly registered rubyforge project.

Popularity: 17% [?]

Xplanet is this great program on Linux that does projections of the earth (including all kinds of possible overlays to include things like real time clouds, earthquake activity, and major storms.)  For years I’ve used it as the background image on my Linux desktop.  On the way home last night, while chatting with my friend Trey, it occured to me that through creative use of OpenSim, we could easily do this this in world with dramatic effect.

Below is a couple of screen shots of my 10m globe.  I had to modify the code paths for osSetDynamicTextureURL to get 1024×1024 textures into the pipe, as 512×512 doesn’t actually look good wrapped on a 10m globe.  There is still a bit of work to make this really good (like completing osSetDynamicTextureURL so we don’t need another timing loop in LSL to do refresh), but the initial results are quite nice.

The circles with numbers are earthquakes in the last 24 hours plus magnitudes (this information comes from the TotalMarker project).

Here you can see the storm track of the named storm Ivan (also from TotalMarker).

Popularity: 13% [?]

Since my dbus post last week, I’ve been playing around more with dbus whenever I get a few minutes. The modern Linux desktop is pretty good, but with minor tweaks, you can make things even better. (all this code is now up in a mercurial repository called dbus-hackery).

Automating Inactivity

Pidgin makes sounds on every message to me, xchat makes sounds on certain key words. Without these cues, I’d never remember to go check these applications. Because I have a tendency to leave my laptop on overnight, I found that I’d often have xchat ringing away at midnight when someone was looking for me. If I forgot to mute my machine before that, it would often wake me up.

One of the programs sending signals on dbus is gnome-screensaver.

The connect will look exactly as expected from the previous look at dbus. The ActiveChanged signal outputs a single parameter, a boolean, which is true when the screensave goes active, false when the screensaver is deactivated.

Volume control on the command line is most easily done with aumix (though if you are on Ubuntu Gutsy you’ll have issues until you rebuild aumix yourself. Hopefully they’ll fix that bug soon.) A little regex fun captures the current levels to a package variable, and restores them back on unmute.

Now I’ve got global mute when the screensaver fires, restored when I return.

Better Away with Pidgin

My screensaver being locked is a pretty clear indication that I’m away, though it being unlocked isn’t a clear indication that I’m back. Especially on weekends, I pop back for a quick check of something, then the computer is put away again.

In order to set a status with a message, it has to be a saved status. To prevent growing that to infinity, I first look to see if it is defined, creating a new saved status if not. 5 is a magic number here meaning STATUS_AWAY (reference the pidgin status.h for more info). Then we set the message on that status, and activate it. A single line change on our screen saver signal adds this into play.

Keep on Hacking

One of the things I’m hoping to impress in these posts on dbus is that with a highly functional language like ruby, linking applications on a modern gnome desktop can be done even by mere mortals. Linking sound to your screen saver is something that would have required a reasonable chunk of c code. Now you can do it in 20 lines of ruby, thanks to dbus.

I’ve thought about creating some sort of extended control panel to enable the features I’ve hacked together, but the reality is the code is so small, and so simple, it seems like overkill. With code this easy, you should just jump in and hack it to your own needs.

As I keep playing with dbus, I’ll post more bits here. Twitter integration is still on my list of things to do, and maybe something I’ll even manage to get to this week.

Popularity: 18% [?]

A few weeks ago I was chatting with Dan about my one great annoyance with Pidgin: it takes up to 15 minutes to realize that it’s network connection isn’t valid any more and to automatically reconnect. With NetworkManager and dbus, this should be a reasonably simple feat. Dan got curious about this and wrote himself a python program that sets pidgin status in this way.

D-What?

A bit of background. One of the Freedesktop.org standards is d-bus:

D-Bus is a message bus system, a simple way for applications to talk to one another. In addition to interprocess communication, D-Bus helps coordinate process lifecycle; it makes it simple and reliable to code a “single instance” application or daemon, and to launch applications and daemons on demand when their services are needed.

The basic idea is that d-bus allows applications to publish interfaces that other applications on the desktop can interface with. Right now, only a few applications really support d-bus in a significant way. Fortunately, one of those is pidgin, which has an extremely rich d-bus interface.

Ruby & DBUS

Here’s a place where I know I’ll draw Dan’s ire. I’ve been on a Ruby kick recently, so my first reaction to Dan’s post was “great, I’ll have to figure out how to do this in ruby now.” It sounds like typical language bias, but I’ll try to justify it at least a little.

Right now I’m writing code on a weekly basis in 3 languages: C# (OpenSim), Java (Grad School Project), and Ruby (side web projects done in Rails). 3 languages is a lot to remain fluent in, and causes some interesting syntax errors when jumping back and forth between them. One less context switch seemed like a good idea. At some point I’ll bother writing up why ruby has seduced me, but that is for another day.

While finding python-dbus bindings are extremely straight forward, the ruby-dbus front is a little more of a wandering path. After finding a couple of abandoned efforts, I finally came to the active ruby-dbus project. Make sure to grab the latest and greatest code from there before proceeding.

Connecting to DBUS

Before I get into actually using dbus, I need to set up connections to it. There are 2 different buses, the system bus (which is shared for all users), and the session bus, which is unique per login session. Pidgin uses the session bus: it’s a user application. Network Manager uses the system bus: it’s events are system wide and affect all users.

#!/usr/bin/ruby

require "dbus"

bus = DBus::SystemBus.instance
session_bus = DBus::SessionBus.instance

# Get the Pidgin Service
pidgin_dbus = session_bus.service("im.pidgin.purple.PurpleService")

# Get the object from this service
pidgin = pidgin_dbus.object("/im/pidgin/purple/PurpleObject")

# Introspect it
pidgin.introspect
if pidgin.has_iface? "im.pidgin.purple.PurpleInterface"
    pidgin.default_iface = "im.pidgin.purple.PurpleInterface"
    puts "We have Pidgin interface"
end

n_dbus = bus.service("org.freedesktop.NetworkManager")
netman = n_dbus.object("/org/freedesktop/NetworkManager")
# Establish a proxy interface object for NetworkManager as it doesn't support introspection
poi = DBus::ProxyObjectInterface.new(netman, "org.freedesktop.NetworkManager")

Basically the pattern is clear. Get a service handle, get an object definition from that service handle, then get an interface from that object. There are 2 flavors for this, one where we’ve got introspection information, and one where we’ve got to go blind because introspection isn’t supported.

DBUS Introspection

DBUS interfaces come in 2 flavors, those that support introspection, and those that don’t. If an interface supports introspection you can get an interface definition off the dbus wire itself, otherwise you need to know the interface a priori. Pidgin supports introspection, Network Manager does not. The major short coming of the ars technica article on Pidgin and DBUS was the lack of information on using introspection to show all the other pidgin interface functions (of which there are > 600). Python dbus introspection throws an exception on my Ubuntu 7.10 environment, so it wasn’t helpful here. However, ruby, as usual, came to the rescue.

The ruby-dbus code contains an example application called gd-bus, which performs introspection on all dbus interfaces it can find, and prints them out nicely. The cheat sheet goes something like so:

M PurpleAccountsFind(in name:s, in protocol:s, out RESULT:i)

• M - DBUS Method, aka Function (might also be S meaning it’s a Signal you can register to listen to)
• PurpleAccountsFind - the method name
• in / out - whether this is an input or output parameter
• NAME:type - the name and type of the parameter. Types can be s - string, i - integer, u - unsigned int, ai - array of integers

Note: the ruby interface seems to always wrap arrays around the output parameters. I have no idea why, but it’s consistent, so a few extra “[0]“s get you a long ways. If you know why, please comment.

The translation of this interface specification into ruby gives you something like this:

def recycle_pidgin(pidgin)
    accounts = pidgin.PurpleAccountsGetAll
    for account in accounts[0]
        if pidgin.PurpleAccountIsConnected(account)[0] > 0
            pidgin.PurpleAccountDisconnect(account)
        end
        pidgin.PurpleAccountConnect(account)
    end
end

This cycles through all pidgin accounts, disconnects all the connected ones, and then attempts to connect all accounts. This is effectively what I end up doing by hand every time I switch networks with my laptop.

Bringing it all together

I now had connections to the 2 buses, and code to cycle the pidgin accounts. Last bit is actually watching for the Network Manager signal that I’ve got a new active network device.

poi.on_signal(bus, "DeviceNowActive") {
    recycle_pidgin(pidgin)
}

main = DBus::Main.new
main << bus
main.run

Network manager doesn’t support introspection. However, it does have pretty decent docs to figure out what the interface is. I’m still sad it doesn’t show up nicely in gd-bus though.

The proxy interface object sets up a signal using 2 parameters and a code block. Every time there is a DeviceNowActive signal on the system bus, I recycle pidgin. Pretty straight forward.

The last little bit is making this thing go into a loop. Ruby dbus contains it’s own main loop for just this task. I created a new main loop, tell it to watch the system bus, and then start it. And, we’re done.

The future’s so bright…

DBUS has been on my list of “I need to go figure this out” for a while. A morning of reading docs, hacking a bit, and crashing network manager a few times, and voila, you’ve got this blog post.

I’ve got lots of ideas floating around in my head now for other things that I can do with dbus to make my applications work better for me. As I bang a few of those out into code, expect to see more here about it. Pidgin is an especially target rich environment given how rich and interface they expose (nice job guys!).

Popularity: 24% [?]

@strips = @user.comics.collect {|c|    
    c.strips.find(:all, :conditions => ["date > ?", 7.days.ago])}.
    flatten.   
    delete_if {|c| c == nil}.   
    sort_by {|c| c.date}

The previous shows ruby in much of it’s glory, and is code that I needed this weekend for one of my side hacking projects.  I’ve colored it to match what xemacs shows me, just to make it a little more clear.

First off, it shows off the power of mixins. 7.days.ago does exactly what you would expect, providing you with a date object.

Ruby is such a fun language to program in.

Popularity: 10% [?]

My grad school class this semester is the Project Course, where the whole semester is spent on a group project.  No tests, no other grading besides the project, which is actually what I expected more of when starting grad work at Marist (that’s a different post though).  The project domain is search.

We have to build an application with an integrated search engine tuned for a specific problem.  Our group problem is an user driven online restaurant review site.  Our canonical example is searching for “boston seafood”, which should return all the posts that a human would, given the same tasks.  That means “the best lobster in bean town” counts as a hit that you’d want.  Guess what, SQL like clauses and regex’s aren’t going to cut it here.

But that’s ok, we don’t have to do everything from scratch.  We’re expected to base our solution on Lucene, which is a search SDK.  You build custom indexer, analyzer, and searcher classes from the Lucene base classes, and feed it documents.  Lucene does the heavily lifting of building the inverted index, and scoring the results based on the rules, weights, and policies you’ve given it.  A project like this is pretty open ended, as you can always make it better given more time, and more interesting analysis tricks.

The whole team is making nice progress, so for the last two weeks I’ve been able to focus squarely on Lucene integration code itself.  Pass one got some basic queries working in Lucene.  Pass two was earlier this week, when scoring started to be useful.  Pass three will be tonight, where I’ll start to integrate synonym support so that lobster is understood as a type of seafood, burgers are understood to be american food.  Though I’ll have to think about how to make sure crab cakes don’t show up in the desert category, though maybe we just need a hybrid seafood desert category.

A few interesting lessons have come out of the work so far.  First, search is way harder than most people think.  While Lucene gives you lots of nobs and levers to tweak how documents are ranked, the results of those tweaking aren’t always what you think.  It’s sort of like moving furniture by throwing bowling balls at it, you may get things close, but you do a lot of collatoral damage in the process.  Recently I was attempting to boost scores based on terms showing up in the subject of posts, which completely overwhelmed our post rating scoring, making low quality posts show up at the top of the list.

You also notice when people are using search badly, or more specifically using bad search.  Using SQL Like clauses is not search, it’s grep.  Unfortunately most php sites do that because they don’t have anything better (Lucene has been ported to a lot of language environments, php is not one of them).  The gentoo wikis fall into this category.

Finally, you realize that google’s scoring, while good in general, may not actually be what you want for your problem domain.  The fact that the word seafood shows up 3 times in a post doesn’t make it a better post, but default scoring gives it a boost based on the number of times relevant terms show up.  Badger, Badger, Badger, while being non kosher, shouldn’t be scored highly in our results, even if we had a category fully dedicated to badgers and mushrooms.

Popularity: 9% [?]

After years on Livejournal, I’ve decided to migrate off to my own wordpress installation. This will hopefully help consolidate some of my content which has been spread out on a number of different sites recently. Expect to see some more traffic here shortly once I get all the changes in place.

Popularity: 6% [?]

The good thing about changing technical focus is all the new exciting things to learn. The bad things is… all those new exciting things to learn mean your development output drops to the floor for some period of time. It’s always a frustrating window of time, be it a month or two, where you feel like an idiot. Having done these changes enough times in the past, I know this too will pass. That doesn’t change the fact that while you may have read 200 pages of developer documentation on a given day, your emacs buffer looks eerily similar at the end of the day as when the day kicked off.

Inevitably, you hit a break through, and now all that example test code that didn’t compile, and you didn’t know why, starts working, and patterns fall into place. Yesterday I had such a moment of clarity around C# and ADO.NET (which is MS’s db interface layer). It turns out that in the function “SqlConnection(string)”, Sql doesn’t mean “generic sql engine”. Sql actually means “MSSQL vendor extension”. Some set of compile errors yesterday got me to on a lark change that to SqliteConnection, and stuff worked. A lot of stuff worked, all at once.

I had to step back from the computer and make sure no evil spirits had come or gone in the process. Leave it to microsoft to very clearly muddle the difference between “something generic”, and “something only we have”, as to them the whole world looks like something only they have. Boo microsoft!

With that set of filters back in hand, the O’Reilly books around C# are now falling into place much more quickly. The persistence engine for OpenSim should have a good first pass by the end of the day, and I’m not feeling so stupid any more.

I also have to give MS some credit on ADO.NET. While C# looks a lot like Java, the patterns and objects they created for database interfacing looks way more like a dynamic language (be it php, perl, or ruby), especially on the read side. Read site, what will take 50 lines of code in C#, would probably be 200 lines in Java. So not boo to microsoft there.

Time to get back to that emacs buffer.

Popularity: 10% [?]

For those that had not previously heard, NYCCCP (aka “the space”) is coming to an end. The space was the idea of Porkchop and Mike, based on hacker spaces that existed in Boston and Phili. The idea is relatively simple. Rent a reasonable sized (in our case 50′ x 20′) location, set up desks for all people that are interested in joining. Build a server room, and get a pretty decent synchronous DSL line in. Cost for the space is distributed among it’s members, all of whom pay a monthly membership fee to keep the place running. I wasn’t originally part of the space, but did join up a year later, and it was a good place to host some Xen test servers.

The space has been running for 3 years, but over the last year people were going there less often, and interest had definitely waned a bit. A few members were lost as they moved a bit further away. Two weeks ago, the overall financials went from self sustaining, to dropping at a relatively sharp pace. In 3 years, the dynamics of the group changed. A lot of us met through the LUG, but became friends outside of it. Originally we only had computers in common, and the space was a good gathering point. But now we do scifi night every week at my house, see each other for lunch a couple days a week, and do plenty of things on the weekends (like biking and hiking).

So, Mike, Porkchop, and I agreed it was time to call the space a grand experiment, that was a good thing, but whose time had passed. The space will shutdown the end of September, and we’re in process of getting everything/one sorted out there and out of the building (there are a bunch of other folks with servers there that will need to move as well).

It’s a sad thing to see go, but times do change.

Popularity: 7% [?]

In an effort to wrap my head around some of the code for OpenSim, I took a detour and started adding C# support to autodia. Autodia was originally written as something to create dia UML diagrams from perl code, but extended from there to support many languages, and many output formats. Unfortunately, C# is not yet one of those, yet.

Right now I’ve got class and attribute parsing pretty well under control (except for generics). Autodia definitely evolved on less object oriented languages than C#, as one of the things I’m most interested in knowing is the contains relationships in the codebase, which isn’t supported in the current version (though I know know how to add it, just need a couple of hours). One of the things I’m trying to expose is one of the gotchas of object design: the inbreeding that can come from having parents and members all be the same base class. I’m sure there is some good banjo joke in there, but I’m a cup of coffee short of finding it.

The results, are quite pretty:

Once the work is in a more finished state, I’ll be pushing it back upstream, so others can benefit as well.

Popularity: 7% [?]