Rudi Grobler

Having a Wii bit of fun...

This week, I started playing with interfacing to the Wii remote. I have to give credit where credit is due:

Brian Peek wrote an excellent library to interface with the Wii remote

Johnny Chung Lee used the Wii's infrared camera to track objects (More about this later)

Rick Barraza from Cynergy started using the Wii and WPF together (Also look at Project Maestro)

Matthias Shapiro also did some excellent work getting the Wii remote and WPF to work together. He explored binding the Wii remotes status to the WPF applications data context!

Also check out the follow up article were I give the schematics of the IR lights and some code updates!

Background

The Wii remote uses standard Bluetooth (Broadcom BCM2042) to communicated with the host. It uses (abuses) the Bluetooth HID protocol. The WiimoteLib (by Brian Peek) handles all the reading and interpreting of the HID reports. All that we need to be concerned about is the WiimoteState. This contains all the statuses of the buttons, 3-axis accelerometer and the IR sensor information.

Matthias Shapiro has a excellent post on how to connect the Wii remote. Please also remember that not all Bluetooth adapters and stacks works, I will try and compile a list. I first tried to use my normal HP Bluetooth adapter but no success. Next, I installed the normal Bluetooth driver from Microsoft and it worked perfectly!

To test if everything works, try the test application that ship with the WiimoteLib (By Brian Peek) or the Wii Data Visualizer (By Matthias Shapiro).

For a IR source, I opened a normal infrared remote control and removed the IR led. I then connected the led to a AA battery holder (Available from Communica for R3). This allows me to create a infrared light source to track that runs from batteries!

Now that we have the Wii remote connected to our PC (Via Bluetooth) and we have a IR light source, we are ready to write some code.

Here is the code to setup WiimoteLib. First create a instance of Wiimote

Wiimote wm;

Next, initialise the Wiimote

wm = new Wiimote();
wm.WiimoteChanged += new WiimoteChangedEventHandler(wm_OnWiimoteChanged);
wm.Connect();
wm.SetReportType(Wiimote.InputReport.IRAccel, true);
wm.SetLEDs(false, false, false, false);

And now we are ready to look at what the Wii remote can do!

Every time something happens on the Wii remote, the wm_OnWiimoteChanged will be fired

void wm_OnWiimoteChanged(object sender, WiimoteChangedEventArgs args)
{
            WiimoteState ws = args.WiimoteState;

            //Here we can now inspect the WiimoteState object...
}

We can then look at the WiimoteState to see if buttons were pressed, the accelerometer's status or even what the IR camera "sees"!

BindableObject

Ok, the WPF geek inside me tells me that this might be simpler if I somehow make the state object inherit from something like a BindableObject (To implement the INotifyPropertyChanged stuff). This will have a huge performance boost because I will only update items which has changed and not everything each time the changed event fires! It will also simplify some UI stuff by allowing binding. Now I can bind to a buttons status and do animations based on its current pressed state all with the magic of dependency properties!

Instead of writing my own library, I decided to hack the WiimoteLib slightly to incorporate the BindableObject natively! I only did it for the IRState object and not even for all 4 points, I only did the first point (Assuming that the first IR light I see is the one I should use). If I have time I might just make my own WiimoteLibForWPF ;)

Now it is very simple to use the IRState object without even worrying about the WiimoteChanged event! I can now bind the WiimoteState object to my window DataContext. To simplify binding, I will only bind the IRState object and not the whole WiimoteState object!

DataContext = wm.WiimoteState.IRState;

And now its possible to bind using XAML!

<Label Content="{Binding Path=X1}" />
<Label Content="{Binding Path=Y1}" />

Very cool!!!

Next is the actual tracking. The Wii remote has a PixArt sensor in the front to assist with tracking infrared light sources. John Chung Lee had some excellent suggestions. The 1st option is to have a infrared emitting bar or array next to the Wii remote and then attaching reflective tape to your fingers to reflect the position of your hands or fingers or alternatively, have the infrared light source on your hands. Cynergy has a excellent video of how thy used this!

I opted for the more traditional way of using it. My IR source will be stationary and I am going to use the Wii remote as a "wireless mouse". This is very similar as to how it is actually used in the Wii console!

The idea is to have a very simple WPF application with 2 bars. One of the bars on the left hand side of the window and the other at the top. Each bar will indicate one axis of the "wireless mouse" with a simple line that is data bound to the status of the Wii remote.

The WiimoteLib's IRState object has a normalized X1 & Y1 that basically gives me a value of between 0-1.0 based on the current position of the IR objects. Next step is to write a very simple converter that takes this normalized value and the actual width or height to scale too and do the conversion.

public class NormalizedXYConverter : IMultiValueConverter
{
    public object Convert(object[] values, Type targetType, object param, CultureInfo culture)
    {
        try
        {
            float normalizedValue = (float)values[0];
            double scaleToValue = (double)values[1];

            return normalizedValue * scaleToValue;
        }
        catch (Exception)
        {
            return 0.00;
        }
    }
    public object[] ConvertBack(object value, Type[] targetTypes, object param, CultureInfo culture)
    {
        return null;
    }
}

NOTE: This can also be done using Marlon Grech's LambdaValueConverter 

And all that is now left is to bind my indicators position to the converted X1 and Y1 values! I just add two Border elements and bind the left hand side's height and the top one's width to the normalized X1 and Y1

Here is the left hand "bar"

<Border Width="20" Background="Black" VerticalAlignment="Top">
    <Border.Height>
        <MultiBinding Converter="{StaticResource normalizedXY}">
            <Binding Path="Y1" />
            <Binding ElementName="mainWindow" Path="ActualHeight" />
        </MultiBinding>
    </Border.Height>
</Border>

And here is the top "bar"

<Border Height="20" Background="Black" HorizontalAlignment="Left">
    <Border.Width>
        <MultiBinding Converter="{StaticResource normalizedXY}">
            <Binding Path="X1" />
            <Binding ElementName="mainWindow" Path="ActualWidth" />
        </MultiBinding>
    </Border.Width>            
</Border>

This is the end result

The bar's nicely indicate exactly where the Wii remote is pointing. It is also very easy to create a simple pointer on the screen.

<Ellipse x:Name="Pointer" Width="10" Height="10" Fill="Blue">
    <Canvas.Top>
        <MultiBinding Converter="{StaticResource normalizedXY}">
            <Binding Path="Y1" />
            <Binding ElementName="mainWindow" Path="ActualHeight" />
        </MultiBinding>
    </Canvas.Top>
    <Canvas.Left>
        <MultiBinding Converter="{StaticResource normalizedXY}">
            <Binding Path="X1" />
            <Binding ElementName="mainWindow" Path="ActualWidth" />
        </MultiBinding>
    </Canvas.Left>
</Ellipse>

Just remember that the root panel must be a canvas for this to work!

The infrared camera can track up to 4 infrared sources. this opens the door to multi-touch scenarios (Like Microsoft Surface)

Here I only investigated a single point touch solution. Multi-touch has it's own set of problems. If you want to learn more about multi touch, I suggest start looking at Microsoft Surface! 

Well, that is all for now... In the future I will try and get a multi-touch solution to work...