computers should be less friendly

One of the common questions I was asked after demoing Majority Desk was how the heck is it built?  To explain, I will first break down the individual components and then talk about how the Wiihands move across the screen.  There is a lot more to the interactions in this system but, moving the Wiihands should give you a pretty good insight into how the whole thing is built.  Just to frame the following discussion Majority Desk is mostly an AIR application built using Flex Builder 3.

ODE2Paper

ODE2Paper is the main visualization and physics library I wrote that underpins the Majority Desk application.  It binds the Open Dynamics Engine (ODE for short), which is the underlying physics system to Papervision3d (PV3D for short), the 3d rendering system.  The foundation of this interaction is actually very simple: every time Flex fires an ENTER_FRAME event I issue a refresh command over a socket that is connected to the physics server.  The physics server is a python application that uses pyODE to wrap the ODE libraries.  The physics server “parses” the refresh command,

<refresh/>

iterates across all the objects in its world producing a new XML document with the position and transformation matrices of all the existing objects and sends that XML back to Flex. 

<world>
    <o1 x="0.0" y ="0.0" z="0.0" r0="0.0" r1="0.0" r2="0.0" r3="0.0" r4="0.0" r5="0.0" r6="0.0" r7="0.0" r8="0.0"/>
    <o2 x="10.0" y ="15.0" z="10.0" r0="0.0" r1="0.0" r2="0.0" r3="0.0" r4="0.0" r5="0.0" r6="0.0" r7="0.0" r8="0.0"/>
<collisions/>
</world>

When this data transfer is complete on the Flex side, I move all the objects in the PV3d scene and then call the camera render.  Simple right?

A Wii bit of background

The Wiimote is a blue tooth transmitter, it reports its position and state to the blue tooth receiver on the client computer.  It determines its position by “looking” through the IR port on the front of the Wiimote at the poorly named sensor bar.  The sensor bar in reality has no sensors in it — it has two IR emitters that allow the Wiimote to triangulate its X and Y position in space.  The Wiimote also has an accelerometer on board that allows it to measure movement in the X, Y, Z planes (pitch, yaw, and roll).  This page has a mountain of technical detail about the Wiimote.

WiiFlash

The information from the Wiimote is sent to the WiiFlash server.  WiiFlash removes all the complexity out of using Wiimotes to control any Flash application.  The API is pretty complete and we found it to be fairly stable.  This is the section Eddie dealt with most so, I can’t talk too intelligently about it except to say that we use the WiiFlash AS3 API to access all the position and button state information from the Wiimote.  The API handles all the calls through a local connection that streams the information from the projects WiiFlash Server, which for now only runs on windows, sorry Mac faithful. 

This pretty much rounds out the subsystems that lie underneath the application.  This doesn’t really explain how the thing works so, I will walk you through how the Wiihands move around the screen to hopefully give you a better idea of what is going on.

Moving Wiihands

The most common thing we do in Majority Desk is move our Wiihands to interact with the world.  To allow interactions with the state of the physical world maintained by the physics server we have a set of commands in ODE2Paper.  The ones that matter for the Wiihands are createSphere, setObjPos, createJoint and breakJoint.  These methods serve as proxy for the real commands flying over to the physics server.  The rationale for this is again the main underlying concept that the physics server knows where the objects are and PV3d simply renders them.  So all the interactions are really taking place on the physics server.  The Wiihands are represented in the physics system as sphere because they are in reality just a cool texture wrapped on a sphere in PV3d.

So as a Wiimote moves we get its position and call a setObjPos method in ODE2Paper library which moves the object in the physics server.  Again the physics server leads and PV3d simply renders a view of the world.

Grabbing stuff

If you got the chance during Hacker Night or our Meet the Jammers session to mess with Majority Desk you may remember that if you hold the trigger and bump into something with your Wiihands you can grab it.  Letting go of the trigger will release the object and let it float around again.  This is achieved using the collisions XML node that I didn’t talk about earlier.  If you look back at the XML doc that the physics server creates to represent the world you will see an empty node there for collisions.  ODE2Paper notifies our Flex application that two bodies are about to collide via this XML node.  This allows us to detect that the Wiihand is touching something and then build a joint, using the ODE2Paper method createJoint.  A joint is a type of attachment between two bodies in ODE, we use simple static joints but a whole slew of different joints types are supported by ODE.  This is why the widget seems to follow a Wiihand around as it moves.  The great thing is this affect is achieved by just calling setObjPos on the Wiihand, which is the normal behavior described previously.   To release the object we simply call breakJoint and the physics server breaks the joint allow the attached object to move fluidly again.

If you want a run down of links and a good video that shows Majority Desk in action, check out Eddie’s blog post about it.

SAP is really starting to push the Composition Environment at this TechEd.  Recently I’ve started to think a lot about this topic given that it could radically change the way enterprise developers do their jobs.  This environment must also come with a new and necessary workflow akin to the Design - Developer workflow that MS and Adobe are both talking about.  Adobe has recently released Thermo which tries to address this gap with software.  Ryan Stewart gives a great rundown of the Thermo and the importance of this interaction:

It’s all possible because Thermo is a jewel of the designer-developer workflow. We can rely on Adobe’s incredible experience in creating design tools with our growing and powerful development framework, Flex. A good designer-developer workflow is so important to creating great RIAs. You only get great experiences when you combine the right brain of a designer with the left brain of the developer and let them collaborate.

Although, the design-developer paradigm is a good starting point it isn’t nearly complex enough ( not to say this interplay isn’t complicated but, come on if you put the wrong color somewhere your enterprise doesn’t stop running ).  The design-developer workflow relies on something else too — that both parties have at least a tacit understanding of what the other does.  There has been a lot of talk on SDN/BPX about this but, I am unconvinced that a common “language” is all we need. 

Life Guards

Development-QA-Production, the 3 tiered architecture that helps stabilize SAP development.  How will this be implemented in this workflow?  Where do the we do business process re-engineering?  Who tests these changes?  How can we communicate the shared responsibility?

A common language doesn’t solve this problem — it would seem more like IT would have to transition resources to be the life guards at the pool.  Changing workflows and/or “document flows” can radically affect  the running or compliance of the enterprise.  Who serves as the gate keeper for these things?

Treading Water

Right now the average IT department is trying to do more with much less.  They are trying hard to build new functionality for their businesses in a choreographed manner, using all manner of controls from code reviews to OO design patterns.  One big question is how does the business person even ask for this new functionality?  What if there isn’t something in their enterprise service repository or what if a field they want isn’t there?  Do they create some vast wiki system?  Do they open an OSS message ( although we would have to make the GUI a bit easier to deal with! )?  Ah, I know Solution Manager — open a message there.  Great. 

We use certain programming techniques to do more with less, “Why not write this section of the project as a set of subsystems and a facade so we can use it again if we want.”  How can we see these interconnections if IT is just trying to tread water supporting these new “business process reengineering” tasks.

Leaky

I guess my main problem is that I know enough about programming to know that even when we have nice abstractions they never work exactly as planned.  “All non-trivial abstractions, to some degree, are leaky.”

…you can’t drive as fast when it’s raining, even though your car has windshield wipers and headlights and a roof and a heater, all of which protect you from caring about the fact that it’s raining (they abstract away the weather), but lo, you have to worry about hydroplaning (or aquaplaning in England) and sometimes the rain is so strong you can’t see very far ahead so you go slower in the rain, because the weather can never be completely abstracted away, because of the law of leaky abstractions.

When someone goes to change a business process they are using abstractions to deal with the complexity of business.  They are using objects created by programmers to represent the world and when we try and put them together there will be instances where they leak.  The biggest problem with this is that the person doing the re-engineering isn’t going to know.  They are just going to try and move blocks around till it works right and this is dangerous.  Adding more steps or trying to circumvent the abstractions inherent leakiness is bad and will just produce poorly designed processes. 

Mark Spitz

Success in this is possible — we just have to realize that the only way to do this is get people on both sides who have a shred of understanding about what the other half does.  The optimum process has to have a collaborative model at it’s core where the person doing the re-engineering is leading the way.  IT would continue to support this function with technical knowledge and broad enterprise experience.  Here is an over simplified version:

See even I can do swim lanes!

Big thanks to Marilyn Pratt for creating the “Community Mindspace.”  Which is a living graphical representation of the thoughts and spirit of TechEd.  Nancy Margulies, the artist for the project, has been taking people’s thoughts and ideas and drawing them onto 2 ~40 foot canvases in the clubhouse.  Here are some really great snippets:

 This is such a great representation of online communities everyone does this — everyone has multiple personas when they are online, at the same time a noob and expert.

“Get your Wiihands on!”

For more pictures of the Mindspace check out Marilyn’s flickr photo stream.