SLEEPY REVIEW: NI Reaktor on Microsoft Surface Pro
Josh Morky – a Mac user for 10 years and also an iPad music app fan – was intrigued by the new Windows 8 Surface tablets
I’ve been a mac user for almost 10 years now and recently learned about the new Windows 8 taking the leap into touchscreen. More or less all new windows computers are touch. Its pretty obvious from the thousands of apps that have popped up on the iPad in recent years that performing electronic music on a touch screen is fun, expressive, and kind of what we’ve all been waiting for. It just makes sense. Ive used an iPad for playing music – which I love – but can’t help but feel that Apple is purposefully limiting its potential. File management is a nightmare, and multitasking just doesn’t work. The iPad is brilliant at making sure you get all your apps, music, movies through Apple. By controlling your work/data flow, they make more money – a good business model, but not good for the creative user.
Windows 8 seemed too good to be true, but nobody on the internet has been writing about it. I decided to take the leap and bought a Microsoft Surface. Best Buy has a great return policy, so if I didn’t like it, I could always return it….
Basically, I ended up loving this thing. The operating system looks great, works great, and is altogether fun to use. Multitasking and switching between programs is great, and its really fun to be able to use a pen/stylus in addition to touch (its super precise and pressure sensitive). It really feels like the future of computers. I use a new 15″ macbook pro in my studio and a new 27″ imac at work, and now it feels like a bummer every time use them. They feel outdated. Trying to use the iPad is even more depressing. There are a lot of new windows tablets/laptops out there – but there are two things that made the Surface appeal to me more: 1) it comes with a pen (which helps if you need precision – especially with such a small screen – and taking notes or drawing is a lot of fun) and 2) the kickstand. I didnt really realize how nice it would be to be able to prop up a tablet – the ipad can be annoying to use in a lot of situations because it doesnt have one. Other things that are awesome is that it supports flash (Hulu!) and has usb – so you can use external drives, midi controllers, audio interfaces, etc….
Using Reaktor is great. I personally hate having to use midi controllers whose interface bears no resemblance to the patch Im using, and touch OSC on an ipad can be buggy and you have to create templates for every patch you use and end up constantly changing your template if you change anything in your patch. I also dont want to have to take 2 computers to a show, I want a tablet up there with me and thats all. Surface does this. Reaktor is really responsive and expressive – but it doesnt support multitouch yet. I thought this would make it pointless to use on a touchscreen, but I barely notice it. There are times it would be great to use multitouch, and things could definitely get more interesting with multitouch, but even just touching one thing at a time completely blows away using a mouse/midicontroller/touchOSC.
I tried using Ableton and it was super buggy with touch. You’d need a mouse, but even then, the screen on the Surface is just a bit too small.
As much as I loved the Surface, I ended up returning it. My car required $1000 to fix the day after I bought this thing, and as close to perfect as it was, the screen size of the Surface is just a bit too small for performing. At 10.1 inches, it has the same problem as the iPad in that you can’t fit too much on a screen without making the controls too small. Its fine if you miss a knob when you’re working in your studio, but in a live setting, you’d want something you can reliably hit 100% of the time. Unfortunately, most of the windows tablets are around 10/11 inches – and for some reason the idea of getting a laptop sounds too old fashioned to me – I DONT want a keyboard. Im going to wait around a little bit till things like the Dell XPS 18 come out (an 18 inch tablet with a sweet kickstand thats even more versatile than the Surface’s – though doesnt have a pen) or Microsoft comes out with a second edition (rumors are that they’ll make a 14 inch version, which sounds like the perfect size for mobility, home use, and performance). In the next 6 months a ton of new computer sizes/formats will be coming out – and they’ll get cheaper. When the perfect one comes out, I’ll be ready for it.
What REALLY needs to happen is that software companies need get on board. Theres not much yet that takes full advantage of multitouch. It’ll come, but its not there yet. This is really exciting – its the obvious next step in the evolution of the computer. Windows made a great new operating system, now its the software companies turn to take advantage it.
Jordan Rudess and Morphwiz on the Lenovo A720 with Windows 8
Jordan wowing us in his piano room at his home in NY. I just set up the PC (Jordan’s first PC) and installed the apps before Jordan sat down to play. The PC is a Lenovo A720 running Windows 8 RTM, with 10 point multi-touch at 1920×1080. The app is MorphWiz by Wizdom Music.
Next iteration of OpenKinect Piano
Kinect is really getting hot, here´s just another example of this
http://www.youtube.com/watch?v=VgLp-KyK5g8
Demoing placing the keyboard wherever you like.
Also, attempting to play a duet (inspired by the movie “Big” – http://www.youtube.com/watch?v=rKrZid…
Made possible by libfreenect (http://openkinect.org) and coded in python
Works at 30fps with no lag!”
Air Guitar prototype with Kinect
British artist and designer Chris O’Shea created this Kinect Air Guitar prototype.
O’Shea explains how the Kinect Air Guitar works
Technical Details
Written in c++ using openFrameworks and openCV for image processing. Using the ofxKinect addon and the libfreenect driver on Mac. Thank you to the openframeworks and openkinect communities for enabling this to happen. A big thank you to Microsoft for bringing this technology to the mass market.
How it works
First it thresholds the scene to find a person, then uses a histogram to get the most likely depth of a person in the scene. Then any pixels closer than the person to the camera are possible hands. It also uses contour extremity finding on the person blob to look for hands in situations where your hand is at the same depth as your body. It only works if you are facing the camera front on. Then it uses one hand as the neck of the guitar, drawing a virtual line from the neck through the person centroid to create the guitar line. The other hand is tracked to see if it passes through this line, strumming the guitar. The neck hand position controls the chord.
Virtual Theremin for the Kinect controller.
It defines two virtual antenna points, which allow controlling the pitch and volume of a simple oscillator. The distance to these points can be controlled by freely moving the hand in three dimensions or by reshaping the hand, which allows gestures that are quite similar to playing an actual Theremin.
This musical instrument has been developed by Martin Kaltenbrunner at the Interface Culture Lab at the University of Art and Industrial Design in Linz, Austria. The software has been developed using the Open Frameworks and OpenKinect libraries.
Kinect MIDI controller (KMIDIC)
Microsoft’s Xbox 360 add-on Kinect has been a big hit in the videogame world, and its amazing technology has been exploited to create a movement-based MIDI controller by resourceful hacker Ben X Tan.
MIDI contoller using the Kinect.
Coded in C#.net using this: http://codelaboratories.com/nui
Very hacky ugly, yucky, alpha prototype, source code available here:
http://benxtan.com/temp/pmidickinect.zip
Next project is making a version of pmidic that uses Kinect. Then, you can control Ableton Live or any other MIDI software or hardware with you limbs. Isn’t that amazing!!!
If you are interested, you should also check out:
http://pmidic.sourceforge.net/
http://benxtan.com
If you are a musician or music business in Australia, check out and sign up to http://rockstarhookups.com.au
UPDATE:
Please watch the video for v0.2 here: http://www.youtube.com/watch?v=fCQwNL…
Source and binaries here: http://benxtan.com/temp/pmidickinect_…
MIDI via Microsoft Kinect
Cool hacks for Microsoft’s Kinect 3D camera system for Xbox to perform MIDI control. Result: depth-sensing, gestural musical manipulations
MIDI contoller using the Kinect.
Coded in C#.net using this: http://codelaboratories.com/nui
Very hacky ugly, yucky, alpha prototype, source code available here:
http://benxtan.com/temp/pmidickinect.zip
Next project is making a version of pmidic that uses Kinect. Then, you can control Ableton Live or any other MIDI software or hardware with you limbs. Isn’t that amazing!!!
If you are interested, you should also check out:
http://pmidic.sourceforge.net/
http://benxtan.com
























