After the first successful test of my webcam marker tracking algorithm, it’s now time for the real deal.
The purpose of my experiment, is to be able to detect the position of my head in 3D space, by processing the webcam-captured frames, locating the 2 markers, and then performing an inverse projection from 2D space to 3D space. That information can be used to set the view-point of a 3D environment to follow the motions of the user’s head, thus increasing the user’s immersion in the 3D world considerably. Simple, natural motions of the user’s head, are carried along in the virtual world, making the screen act as a window into that 3D environment.
Of course the point tracking code from my previous test is the same. However, I modified my tracking program to accept local connections from client programs that need to use that tracking information (
Once again, you may watch the result at youtube. There’s still some way to go, and some details to be ironed out… I’ll keep you posted on anything new with this experiment :)
Oh, and of course, the code is always available at my subversion repository:
| webcam marker tracking program (server): | svn://mutantstargoat.com/nuclear/compvis/cam_test |
| 3D environment test (client): | svn://mutantstargoat.com/nuclear/compvis/vr_test |
| my webcam library (used by cam_test): | svn://mutantstargoat.com/nuclear/libwcam |
Code Laboratorium 
February 3, 2008 at 10:40 pm
I took a look at your YouTube video and found it very interesting. Have you tried Lee’s VR using head tracking with the Wiimote? I am wondering about how well is your system performing in term of accuracy of head tracking.
February 4, 2008 at 2:11 am
No, I haven’t. Lee’s wiimote VR tracking requires a wiimote (obviously), which I don’t have. It’s also written in C# and runs only on MS windows if I’m not grossly mistaken, so I couldn’t run it even if I had a wiimote.
In fact, Lee’s video was the inspiration behind this experiment. Of course I knew about the immersive effects of head tracking, in environments such as CAVE. But the thought had never occured to me before, that it’s possible to replicate the effect (to some extend) without expensive tracking sensors. So after watching Lee’s video I said to myself: “man, I really *have* to do that”. However, I decided it would be best to not use any obscure hardware, such as the wiimote if there is another way. And I ended up doing it with a simple webcam instead.
The tracking quality deteriorates with increasing distance from the camera currently, and some jittering is evident at some distance. This is unavoidable, I guess, due to the markers’ projected area becoming too small, and losing resolution. The problem was somehow mitigated by moving from 640×480 to 800×600 video capture, which also increases the processing overhead of course.
The good thing about the wiimote, if I’m not mistaken, is that it does the tracking itself, in hardware. Which makes the problem much simpler, and the processing much faster.
February 9, 2008 at 12:39 am
Very interesting! I wrote a post about it, please let me know if you are ok with it!
February 15, 2008 at 2:56 am
WTF, this last comment was marked as spam initially … Anyhow, thanks for your post, I accidentally found it by looking at referals the other day.
March 30, 2008 at 3:41 pm
i tried to link to your svn repository but it seems its down. is it down? if so, could i get your source from another link? much appreciated.
March 30, 2008 at 4:46 pm
It’s not down in general, however it’s at home, on my main computer, so there might be power or network outages for short amounts of time. If you can’t reach the repository just try again in 5 minutes or so.
April 1, 2008 at 6:08 pm
It seems that i still cannot access the svn repository. Could you put up a direct link to it on your website?
April 2, 2008 at 1:22 am
Alright, although I just tested the svn server and it works fine, but still here’s a tarball of all three of them.
April 3, 2008 at 6:31 am
thanks a lot!
June 19, 2008 at 8:30 am
Somehow i missed the point. Probably lost in translation :) Anyway … nice blog to visit.
cheers, Lucubrator.
February 27, 2011 at 6:06 am
[…] the first dissapointment, I obviously had to have that functionality, so I decided to start hacking my old 3dof headtracking experiment to make it 6dof and connect it somehow with games running through wine. However, while I was […]