One mouse, one pointer, is sometimes not enough! It was my problem for one application, I needed to control 2 pointer with two wiimotes, so I found a trick only using Virtools’ native building blocks and 3D primitives resources.

The idea is similar to the accepted-fact that if someone point something with its finger, another person will not see the same thing pointed.

The system has a lot of common points with the human view and gestures, we can figuratively speaking, decompose it by three objects on the following figures:

-The eyes as the system’s camera.

-A finger (or more) in order to point an object which will be the pointer on the screen. A simple point, nevermind what kind of 3D object.

-A window. In this system I chose to attach the transparent plane, very close to the camera in front of it, which will perfectly fit to its field of view. Like a windows in front of us where we will put our fingers in order to point an object on the other side’s environment. This plane handles the pointers in order to easily define their position on the plane proportionally to the screen resolution.

-A light ray, an invisible ray launched that will pass by the eye then by the pointer (the “finger”) and finally reach the object wanted to be pointed. To make it easier you use a simple trick with simple 3D primitive (a tube for example) from the Virtool resources, and put it at the exact same position than the camera. The pivot point must be at 0,0,0 (regarding to the camera position). So with this object you can make it looking at the pointer on the” window” all the time and use it to control a “ray”.

To recreate this situation of pointing, we use one main tools integrated in Virtools, the Building Blocks“ Ray Intersection ”. This Building Block returns the first object, of a specific group, intersected.

The system uses it in order to launch a ray from the center position of the camera, which will pass through a pointer and return the first object intersected. Relatively simple, this system needs “3D entities” as pointers, placed in the virtual world at least at the beginning of the camera field of view, instead of a usual mouse pointer that can be considered as a layer on the screen.

So with two values X & Y we can control these pointers on the plane. You can use everything you want as value input (I use wiimote data), It can be an opportunity to use OSC/TUIO values to create this kind of pointer dynamically and have multitouch on virtools.

After making this tutorial I found some similar works, no especially for Virtools, and the most important the name of this method : the ray casting. So if you want more information It can be a good start.

Before, I want to say that their are others methods, but I use Vista X64 and the drivers must be signed (ok you can change that too ^^ ). For example you can use the balance-board with GlovePIE and PPJoy to create a joystick, but it’s a chore on x64 . So this example is not THE solution !

So, what do we need to do that ?
- a Wii Balance-Board & Virtools
- Something to connect your balance-board using bluetooth (I use BlueSoleil), If you had already try (and succeed ) to connect a wiimote it’s the same thing.
- GlovePIE 3.0 , for information Carl Kenner only give 2.9 link because 3.0 has serious bugs with the wiimote. But the 2.9 doesn’t have the Balance-Board support. This is the file found somewhere else.
- MidiYoke , in order to virtualize a MIDI Port and get informations into Virtools (or any other software using MIDI Input).

Actually the process is very easy. Even if I spent some times on this (in order to find the right solution), I’m sure you will spend few minutes !

Step 1. install MidiYoke
Installation is fast but you will need to reboot after.

Step 2. connect the Balance-Board
When you use a wiimote, you can synchronize it with the Wii (or your computer) by pressing 1+2. But the Wii-BB doesn’t have external button to synchronize. So in order to be detected as bluetooth device you will have to open the battery gate and push the red button (keep it pressed until the device is fully connected).

a - detect the balance board

b - get services (as Human Interface Device)

c - connect the device

d - now it’s ok you can release the red button !

Step 3. launch GlovePIE 3.0 and create a script

Launch GLovePIE and load this script. (it’s a mix between different scripts ^^). It is very easy ot understand (I think) and you’ll can change it easily after.

First we have to check if the board works. You can see that at the end of the script their is a debug line :

//GlovePIE Debug
debug = “X: “+ var.Xinit +” Y: “+ var.Yinit

When you will run the script the debug informations will appear on the upper-right corner of the GlovePIE window. So now run the script !
If everything is right you must have numbers on the debug window (different from 0.00).

(If it doesn’t work try to reconnect reload GlovePIE or retry from step 1, actually some times my balance board disconnect itself, don’t know why.)

Now you can climb proudly ( ) on the board and the numbers must change. In order to initialize the wiifit you have to stay on the board and use your mouse: left click (I chose the mouse because mine is wireless and it’s easier to stay static on the balance-board).
If you want to change that you just have to modify this lines :

line 10. If mouse.LeftButton = true
line 17. mouse.LeftButton = false

and put a keyboard action instead or anything else.
Move on your board and check the variations of values.

Step 4. send what you need
On this script you have two main values that I wanted to use (values between -1 and 1 just like any other joystick, for other reasons).
:

var.Xinit // left/right from -1 to 1.
var.Yinit // foward/backward from -1 to 1

this two values are calculate from the main properties of the board, so you can get all this informations about the board :

//sensors values
Wiimote.BalanceBoard.RawFrontLeft
Wiimote.BalanceBoard.RawFrontRight
Wiimote.BalanceBoard.RawBackLeft
Wiimote.BalanceBoard.RawBackRight

//weight
Wiimote.BalanceBoard.Weight
Wiimote.BalanceBoard.WeightBack
Wiimote.BalanceBoard.WeightFront
Wiimote.BalanceBoard.WeightLeft
Wiimote.BalanceBoard.WeightRight

//use as Human Joystick (doesn’t work very well on mine)
Wiimote.BalanceBoard.JoyX
Wiimote.BalanceBoard.JoyY

In order to send midi value you have to convert value in order to get variation between 0 and 127. Now let’s see how we can send informations.

midi2.Channel1.FirstNote = ((var.Xinit+1)*127)/2
midi2.Channel2.FirstNote = ((var.Yinit+1)*127)/2

Use the midi2, that correspond to the virtual midi-port, you can use up to 16 channel (if you don’t select channel it will be the channel 1), and use the First Note value (actually you can use 4 notes, but I didn’ find how to get them into Virtools, so I use different channels instead). Here I send the Left/Right value through Channel 1 and Foward/Backward through Channel 2. In this case 0 to 127 equal -1 to 1, so 63 is the value sent when nothing happen.

That’s all for GlovePIE !

Step 5. launch Virtools

Ok, now we only 2 building blocks and that’s all ! Actually I’m not very used to MIDI in Virtools (and MIDI at all ). Set the Midi input to 0 with the BB “Set Midi Input Port” , and after you just have to read datas with the BB “Read Midi” (both BB are in the category Controller > Midi). On the last BB edit the parameter Channel to 0 for the Channel 1 of GlovePIE and 1 for 2, 2 for 3 … And get the Output Note value. In this example I get the value from the two different channels using two BB Read Midi. (don’t forget to run the GlovePIE script )

That’s all ! As I said it was very easy, now do what you want to do with this values !

Soon I’ll show you what I made with the Balance-Board !