Sculptie Physics on OpenSim from Dahlia Trimble on Vimeo.

llTargetOmega didn’t work for us a week ago, which confused me, as I saw that in the LSL portion of our code it was doing exactly the right thing and setting the angular velocity correctly. I should work, but it didn’t. In the lack of it working people were setting fast timers that pushed out rotation updates. This caused a lot of extra load on the server, and was really the wrong approach for this.

Take 1: Terse Updates

OpenSim has 2 paths to sending information about Prims to the client (we’ll get to the first one later). Terse Updates are a small update packet that contains just a bit of information on updated textures and some of the vectors used to establish prim position, velocity, acceleration, rotation, and angular velocity. When a prim is updated in the environment, Terse Updates are used to tell all the other clients about that change. One of the heavy users of the Terse Update path is the physics engine, as all the vectors the physics engine changes are in there. We’ve seen a lot of work on the Terse Update path as physics have gotten more and more tested.

On Tuesday I finally dug in and traced our Terse Update path, and found an interesting thing. When the object was physical (i.e. movement coming out of the physics engine) we did the right thing for Terse Updates. When it wasn’t, we hard coded all the velocities to zero. So even if things were rotating, any time we sent an update we’d stop them.

Author: sdague
Date: 2008-05-06 15:17:00 -0700 (Tue, 06 May 2008)
New Revision: 4543

Modified:
trunk/OpenSim/Framework/IClientAPI.cs
trunk/OpenSim/Region/ClientStack/LindenUDP/LLClientView.cs
trunk/OpenSim/Region/Environment/Scenes/SceneObjectPart.cs
trunk/OpenSim/Region/Examples/SimpleModule/MyNpcCharacter.cs
Log:
send actual velocity and angular velocity in terse updates
instead of hardcoding to zero when the primitive is non physical.
llTargetOmega should work now.

Ok, so life is good, the issue is fixed, and we move on.

Except… it wasn’t.

CSI: OpenSim, getting to the bottom of this

At this point a whole bunch of people on the IBM side jumped in. Mike Osias had a build that was on it’s knees due to use of fake rotation, so he had all the good test cases, and opened mantis 1166. I’m not a scripter, so I needed some examples to know what should work. Alan Webb started to dive in and try to figure what was going on as well. I figured I’d spend an hour on it to try to figure out where things were at before getting back to avatar appearance bits.

After abount an hour Alan and I started comparing notes. The code in this area is extra confusing because we’ve got 2 vectors for angular velocity. An, no, they aren’t actually different in any real way. Lots of people have tried to rationalize that they do different things, but they don’t. This is cruft, and is part of what happens in an organically growing open project. The AngularVelocity / RotationalVelocity thing an opensim appendix, and should be surgically removed at some time in the near future.

But the behavior was even odder. I could set llTargetOmega on an object, and it wouldn’t move. Then I’d touch it, and it would. I got Mike into my test environment and was looking at a spinning cube.

“Ok, you see that cube spinning?”

“No”

I grab it and move it. “What about now?”

“Yes, spinning now.”

At this point I was confused a lot. Why would that be?

Take 2: Full Updates

I said there were 2 ways of a client finding out about prims, and this gets us back to the first one. In addition to Terse Update, there is what we call Full Updates, which are really just the full prim definition being sent down the wire. This is everything we know about the prim. This packet is also marked as reliable, to make sure the client doesn’t drop it (terse updates are droppable).

And now we get back to organic code bases. One of the big activities since October was working physics in opensim. Lots and lots of work were spent on Terse Updates. Very little work was spent on full updates. It turned out that Full Updates were always hardcoding all the motion vectors to zero. The SendPrimitiveToClient function predates both physics and scripting by months. In a pre-physical opensim world passing the motion vectors didn’t make any sense, as there wasn’t anyway to set those values. The code worked well, so no one was really looking at it again, at least not in this specific area.

TerseUpdates (sent on minor prim movement) would make things spin. Full Updates (sent on initial prim rez, or after calls to osSetDynamicTextureURL) stopped the spinning. My earth projector turned out to be the perfect test case for this once I added rotation to the globe.

Originally I was going to punt on this and leave it to someone else, but then the thrill of the chase got to be too much. But there was one problem. This information is sent to the client in a 60 byte array, with basically undocumented positions. It was easy to fix terse updates because someone had already sorted it out, and I just needed to copy the decoding pattern there. For FullUpdates, it was more of a trial and error approach, represented by a series of checkins, reverts, and new attempts.

You know what happens when you get that array wrong? Spectacular fail. 3/4 of prims aren’t in the right place, and touching an image board (user of osSetDynamicTextureURL) makes it fly away to some other part of your sim. Maybe in space. I eventually figured out a workable serialization:

Author: sdague
Date: 2008-05-07 12:44:22 -0700 (Wed, 07 May 2008)
New Revision: 4566

Modified:
trunk/OpenSim/Region/ClientStack/LindenUDP/LLClientView.cs
Log:
seriously hope this gives us rotation and rotational velocity

As you can see, I was getting a little punchy on changelog entries.

So we’re done and fixed, and back to work…. well not quite.

Take 3: Deselected Objects

When you edit an object the client stops it’s motion, as nothing would be more evil than trying to edit an object that is flying away from you at 60 m/s. When you’ve deselected the object it tells the server. But the object is stopped. The client needs to be told again that it is spinning. I got that critical information from melanie on IRC, which was enough to pass on the buck.

We had Mike almost working, and Mike is no slouch on our code base (he’s sent in a couple dozen patches in the past), so I flipped this one back to him with “we’re almost done, but you’ll need to find the right place in the deselect path to generate a Terse Update. Then I think we’ve got full llTargetOmega support.”

A day later Mike sent in this final patch:

Author: sdague
Date: 2008-05-08 05:48:29 -0700 (Thu, 08 May 2008)
New Revision: 4585

Modified:
trunk/OpenSim/Region/Environment/Scenes/Scene.PacketHandlers.cs
Log:
From: Michael Osias <mosias@us.ibm.com>

Patch to schedule terse update on deselect, specifically so llTargetOmega
sets rotational velocity on deselect.

This should complete our llTargetOmega support and fix:
http://opensimulator.org/mantis/view.php?id=1178

And now. For real. llTargetOmega works.

Final Thoughts

Avartar Appearance as a User Service isn’t coming this week, sorry folks. The above epic took much of my hacking time this week. It was a pretty solid educational experience for me in the way we actually communicate the contents of the Scene to the client, which was good to learn after a year on the project.

Something else to take away from this. Lot’s of focus is currently on the OpenSim scripting implementation, as it should be, as that’s a huge user visible portion of our function. llTargetOmega it self is < 6 lines of implementation. But our supporting scene model needed some work to actually get that info to the client.

I get asked all the time “how long until my favorite feature X is implemented”, and the answer is always an unsatisfying (to me and them), “I’m not sure”. Sometimes the plumbing is already there, and it’s quick. Other times we’re doing deep dives into our code base to implement what seems to the user to be a very simple function.

We’re making constant forward progress, I’d even say rapid constant forward progress, but patience is always a good thing. Also, if you want OpenSim to work for whatever you application is, you should be trying to use it now and filing bug reports. That’s how we function, personal itches, and knocking of mantis reports. Any ability that you have to narrow the bug to a specific section of code (even if you don’t have a fix), helps a lot as well, as it removes possibly hours of core developer time trying to track down where things fail.

Evite sucks. While it doesn’t force attendees to make accounts, it makes it look like it does. The evite.com emails tend to catch as spam. And the interface is now dubious under firefox. The idea is still good, but it hasn’t really ramped with the trends in the rest of the service web application space.

One of the key things I wanted in an evite replacement is getting rid of user logins. Given an event, and an email address, you can come up with a unique key that qualifies that person for that event. That means the user just follows a link, and they are in. Links are unique for people. If you make the key a hash of the person’s email and some secret seed key for the event, you’ve got something cryptographically strong as well. No one can modify another person’s entry because the key needs to match before you get any info.

Saturday was a rainy day, so I built this system. By Saturday night I had most of it working, and had rolled this out live by Sunday afternoon. This was my first rails 2.0 app, so I needed to catch up on a few things along the way. Things I learned:

• Rails 2 creates scaffolds in a slightly new way. That threw me for a bit, as I had already built models for most of my objects before creating scaffolds. The new way (putting attributes on the command line) looks like it is designed to make rails tooling easier.
• ActionMailer is crazy easy. It even does multipart mixed emails really easily. My mhvlug mailer script for month announcements is going to need to be converted to this at some point.
• Rails has a word_wrap function in the view context. Of course it does, why did I even doubt that.
• The google maps API is impressive. I had maps based on event location within 60 minutes of signing up for my Google Maps API key.
• The f= param on maps.google.com is which fuction to drive. q: location query, d: directions. That took a little bit of reading urls to realize.
• It’s pretty easy to integrate mercurial push to auto restart a rails app if it’s running under passenger.
• If you are running multiple versions of rails applicatoins under passenger, delete all the rails links in vendor/ so that it picks up the right rails environment.
• arround_filter in rails is really handy to catch generic exceptions and dump people off to an error page that isn’t the default rails one.

All in all, I was really happy how this turned out. As soon as I get some free time I’ll genericize the bits of the app that I coded just for our event, and get this out on rubyforge. I only wish there was a rails equiv of gems, as I’ve still found that it isn’t entirely clear how to best package a rails application as an easy to download open source component.

You’ve been warned.

A couple things changed in the last year. Linode went from UML to Xen, which definitely makes each linode more powerful. My shared hosting company stopped being helpful. I had a couple of small outages. They had moved from a knowledgable support staff, to a support pool that was clueless, and never seemed to understand the ways in which their system was broken. And, after hosting mhvlug email for a while with no issues, it seemed reasonable that dague.net email would be safe there as well.

Backups (thanks to backuppc) have been ramped up from every 24 hrs to every 6 hrs on the box, to narrow my window in which I can screw things up. Only one set of email delays so far, mostly because I set a wrong postfix param over the weekend, which may have been blocking mhvlug.org email as well. But that is resolved now. Dan will at least thing I’m a real man now.

Bugs are so much more amusing in 3D.

The upgrade, via update manager, only had one hitch, when the wireless card bonked out in the middle of it. I suppose it adds appropriate insult to injury, given how often the iwl4965 crashed on my over the last couple of months. Resuming the upgrade on wired ethernet, and all was well.

The Good

Upgrade went flawlessly; fonts look even better; wireless seems better; ssh-askpass now seems to actually trigger on login; firefox 3b5 is fast; liferea is much faster

The Bad

Pidgin 2.4’s usability improvements are anything but; A few of my firefox plugins (delicious links, firebug) don’t work with firefox 3b5 yet (as such, my daily links won’t be on the blog until delicious gets fixed).

The Amusing

During installation some 3rd party packages were removed, including Lotus Notes. While I appreciate Ubuntu’s attempt to make my life better, I sorted of need that for work. I’m pulling from our internal repos now.