After the first successful test of my webcam marker tracking algorithm, it’s now time for the real deal.

The purpose of my experiment, is to be able to detect the position of my head in 3D space, by processing the webcam-captured frames, locating the 2 markers, and then performing an inverse projection from 2D space to 3D space. That information can be used to set the view-point of a 3D environment to follow the motions of the user’s head, thus increasing the user’s immersion in the 3D world considerably. Simple, natural motions of the user’s head, are carried along in the virtual world, making the screen act as a window into that 3D environment.

Of course the point tracking code from my previous test is the same. However, I modified my tracking program to accept local connections from client programs that need to use that tracking information ($x, y$ normalized position of each marker). Then I wrote a test program, that renders a simple OpenGL “world” (a bunch of balls and a couple of coordinate grids), and uses the marker positions from the other program, to calculate the user’s head 3D position, and set up the virtual camera to coincide with that.

Once again, you may watch the result at youtube. There’s still some way to go, and some details to be ironed out… I’ll keep you posted on anything new with this experiment :)

Oh, and of course, the code is always available at my subversion repository:

 webcam marker tracking program (server): svn://mutantstargoat.com/nuclear/compvis/cam_test 3D environment test (client): svn://mutantstargoat.com/nuclear/compvis/vr_test my webcam library (used by cam_test): svn://mutantstargoat.com/nuclear/libwcam
Posted in hacks. 11 Comments »

### 11 Responses to “3D VR Headtracking test”

1. Dat Chu Says:

I took a look at your YouTube video and found it very interesting. Have you tried Lee’s VR using head tracking with the Wiimote? I am wondering about how well is your system performing in term of accuracy of head tracking.

2. Nuclear Says:

No, I haven’t. Lee’s wiimote VR tracking requires a wiimote (obviously), which I don’t have. It’s also written in C# and runs only on MS windows if I’m not grossly mistaken, so I couldn’t run it even if I had a wiimote.

In fact, Lee’s video was the inspiration behind this experiment. Of course I knew about the immersive effects of head tracking, in environments such as CAVE. But the thought had never occured to me before, that it’s possible to replicate the effect (to some extend) without expensive tracking sensors. So after watching Lee’s video I said to myself: “man, I really *have* to do that”. However, I decided it would be best to not use any obscure hardware, such as the wiimote if there is another way. And I ended up doing it with a simple webcam instead.

The tracking quality deteriorates with increasing distance from the camera currently, and some jittering is evident at some distance. This is unavoidable, I guess, due to the markers’ projected area becoming too small, and losing resolution. The problem was somehow mitigated by moving from 640×480 to 800×600 video capture, which also increases the processing overhead of course.

The good thing about the wiimote, if I’m not mistaken, is that it does the tracking itself, in hardware. Which makes the problem much simpler, and the processing much faster.

3. internetakias Says:

Very interesting! I wrote a post about it, please let me know if you are ok with it!

4. Nuclear Says:

WTF, this last comment was marked as spam initially … Anyhow, thanks for your post, I accidentally found it by looking at referals the other day.

5. Glycemia Says:

i tried to link to your svn repository but it seems its down. is it down? if so, could i get your source from another link? much appreciated.

6. Nuclear Says:

It’s not down in general, however it’s at home, on my main computer, so there might be power or network outages for short amounts of time. If you can’t reach the repository just try again in 5 minutes or so.

7. Glycemia Says:

It seems that i still cannot access the svn repository. Could you put up a direct link to it on your website?

8. Nuclear Says:

Alright, although I just tested the svn server and it works fine, but still here’s a tarball of all three of them.

9. Glycemia Says:

thanks a lot!

10. Lucubrator Says:

Somehow i missed the point. Probably lost in translation :) Anyway … nice blog to visit.

cheers, Lucubrator.

11. […] the first dissapointment, I obviously had to have that functionality, so I decided to start hacking my old 3dof headtracking experiment to make it 6dof and connect it somehow with games running through wine. However, while I was […]