Short video of a telekinetic conductor, hand positions and orientation are tracked with a Kinect and 2 Wiimotes.
The conductor controls multiple orbs in 2D space using hand movements to attract the different orbs; thus using a virtual telekinetic force the user can control these orbs. The position of these orbs then gets converted into midi signals that can be used to control a synthesizer. In the video attached a midi track is being played into NI’s Massive synth and all 8 dials on the synthesizer are being controlled at once, using different controllers it is quite possible to control up to 20 parameters at once.
Being able to control hundreds of parameters at once is all well and good; however this device adds something extra. When someone uses this device they automatically find themselves dancing or conducting (dependent on the music type) in order to control the music properly, in fact if you don’t dance you don’t get the right timing.
The functionality is not limited to one machine either; each component connects to a server via TCP/IP so collaboration / jamming can take place over the internet.