Songfever, at it's heart, is a tangible front-end for iTunes. Album covers are projected onto four physical objects standing on a shelf. With a scroll wheel, attached to the shelf, you can scroll through your music library in a Cover Flow like fashion. The goal was to reintroduce the aesthetic quality of a physical music collection (CDs, LPs), while maintaining the comfort of a digital one.
Each of the four covers are tracked by a webcam mounted above the shelf. Their positions are translated in software in the same way the projector located relative to the webcam in the real world. When rendered in a 3D OpenGL view and projected back onto the shelf, the album covers line up with the physical objects again. Approximately, that is.
Songfever was our main project in the 4th semester for “Digital Media” at the Hochschule Darmstadt. I was responsible for the programming; the UVC Camera Control I posted some month ago, was a small part of it.
Visit the Songfever Website for some more photos and videos.
For a recent computer vision project I needed to pull images out of a Logitech QuickCam 9000 and track some markers with the help of the ARToolKitPlus library. I connected the camera to my Mac and was quite surprised to see that it just works. There was no need to install any drivers. As I learned later, that's because the QuickCam 9000 is a UVC webcam for which Mac OS X 10.4.9 already provides a driver. I was able to get to the raw camera images through the QTKit Framework in no time.
However, the QuickCam 9000 has its auto exposure enabled by default, which is absolutely deadly for stable tracking results. I thought I could just turn the auto exposure off and set it to a fixed value through some QTKit API – but no, there's no way to change the exposure of a UVC camera with QTKit. In fact, there's no way to change any parameters of your camera. No exposure values, no white balance, no gain, nothing. Apple just implemented the bare minimum in its UVC driver and the QTkit Framework.
Well, maybe I could get to these parameters through the older Sequence Grabber Framework then? After all, there's a VDIIDCSetFeatures function and a vdIIDCFeatureExposure key! But nope, as the name implies, this stuff only works for IIDC cameras. What's an IIDC camera? Wil Shipley asked the same questions almost 3 years ago - even back then, IIDC cameras were pretty much deprecated. Still, these cameras are the only devices that Apple provides an API for, if you want to change some esoteric parameters no one would ever need to change, like oh, the exposure time or white balance temperature for instance.
Apple is aware of the problem but hasn't done anything to solve it. And Logitech apparently doesn't see the need to provide a Mac driver for their cameras, since Mac OS X already ships with one. Great.
But wait, UVC is a standard, right? USB.org provides a documentation for all device classes, and the Video Class is no exception. So, I poked around in the documentation for a while, read some Linux UVC driver sourcecode and used Apple's USB Prober to see what's going on. After some more hours of playing around with the Kernel Framework's USB API, I was finally able to control some of the QuickCam's settings!
Read complete post »
After some hours of reading to figure out how exactly the memory management with alloc, dealloc, retain and release works with NSObjects in Objective-C (here’s a good article) and some more hours to get used to the funny syntax, I was finally able to do something with it.
To get my 3D model onto the iPhone, I had to find a data format that is easy to load and can be used directly in my Application without much modifications. I decided to export the model from Wings3D as Wavefront .OBJ file, which is a pretty straight forward ASCII format. I, however, didn’t want to go through the hassle of parsing ASCII data in C, so I wrote a small PHP script to transform this .obj file into a binary format that was ready to be used with OpenGL ES.
This is where the fun starts.
Read complete post »
