Friday Video: Taking immersion to new levels

by Alisha Karabinus { Dec 21st 2007 at 9:00PM }

Johnny Lee, who we're now elevating to the status of superhero around here, is back with an all-new Wii-centric project: Wii in 3D, thanks to head tracking. What's head tracking? Here, it's centered around mounting a sensor bar on your head (which is much cooler than it sounds) so that the software (and a Wii remote) can track the position of your head and allow displays to take on a full three dimensional scope, thus turning a flat screen into something more like a window. The result? Why, it could mean games that look like full-on virtual reality. Check out the video above, which was too good not to be in our weekly spotlight, to see just what it all means.

[Thanks, Chad!]

Tags: 3d, head-tracking, johnny-lee, virtual-reality

Reader Comments (Page 1 of 1)

Drew @ Dec 21st 2007 9:22PM

Two words:

HOLY CRAP

maduin @ Dec 21st 2007 9:23PM

I was literally wowed by the effect. I didn't think that I would be that impressed at all, but when he actually demoed the head tracking it looked like some magical world of 3D technology that was sent from the future.

Robert @ Dec 21st 2007 9:33PM

This guy is... amazing

Gemski @ Dec 21st 2007 9:44PM

This is amazing! I find it interesting how this setup reverses the Wii as a party system though. As such my brief thoughts/hopes of a Wii game trying this or maybe Wii2 were quickly dismissed.

Maybe this is something PC games could implement?

upirons @ Dec 21st 2007 9:45PM

Wow....Just...Wow...

So when is Nintendo going to hire this guy?

ZapDash @ Dec 26th 2007 3:43PM

Probably when they hire the guy who made the Nintendo On and Mario Galaxy DS videos.

chad @ Dec 21st 2007 9:49PM

imagine this with the zcam (which u remember from dave's post). the zcam sees how you move to put you into the game. for example , move your fist in the air to, say....punch a boxer. now if u pair the zcam with johnny's work, if this boxer were to punch at you, you could move to the left and it would know(thx 2 the zcam). but at the same time you see the boxer's fist come into your living room as you dodge it, and it moves harmlessly out to the side(thanx to johnny's work). Pair that with ps3 graphics and you are in the game. It'd be every gamer's dream. This guy MUST be a developer!!!

Christian @ Dec 21st 2007 9:52PM

Nintendo, less Wii Fit peripherals, more Johnny Lee peripherals.

padgon @ Dec 21st 2007 10:49PM

Agree

spazzook @ Dec 21st 2007 9:55PM

OMFG

never before have i seen something so amazing

this is not only the future of video games but the future of movies and tv as we know it

HPFFE @ Dec 21st 2007 10:21PM

Oh.. My.. God.
That bloke is amazing, seriously.
Nintendo should hire him

Dig @ Dec 21st 2007 11:15PM

Nintendo, give this man a job. You can clearly tell that he's too modest to flat-out ask for one in this video, but he can give you guys a lot of excellent ideas.

Author X @ Dec 22nd 2007 12:31AM

I figured out what you were talking about from the post, but the video still blew me away. I'm going to quote exactly what I said out loud when he actually demonstrated it with the moving camera.

"That is. God. Damn. Awesome."

I seriously want Half Life 2 with this.

vidGuy @ Dec 22nd 2007 12:34AM

Count me impressed.

To all the haters who say the Wii isn't innovating, whatsup now? How much you wanna bet that Wii v2.0 uses this method for some games?

Ting Lee @ Dec 22nd 2007 1:12AM

Holy cow ! Nintendo should pay whatever it takes to hire this guy. I was pretty blown away with how realistic this 3D cam rock.

Go Johnny !

Max Headroom @ Dec 22nd 2007 4:57AM

That was stunning. I honestly didn't expect that much from it, but - wow - I was totally impressed.

Drahken @ Dec 22nd 2007 5:00AM

Holy hell. This man is my new god.

Jayenkai @ Dec 22nd 2007 5:57AM

I'm with everyone else on this. F'in amazing!

But..
Do remember that you have 2 eyes, where the camera only has 1, so the trick probably won't work quite as well when you try it on yourself.
Still.
Wowsa!

gevenstaines @ Dec 22nd 2007 10:27AM

But... two wiimotes have two eyes collectively.

Max Headroom @ Dec 22nd 2007 12:54PM

If we're required to wear glasses anyway for this setup, it would be cheap and easy to make them 3D glasses (like the kind used for the movie Beowulf in 3D). Then, you'd only need to figure out how to render a 3D image. I'm not sure if that is possible, or if it would require a more expensive setup.

Triptweeze @ Dec 22nd 2007 6:53AM

Its a shame that more people aren't really looking at the hardware and thinking... "Hrmmmm...How can I play around with this"

Gosh, the first thing that came to mind was an arcade game where you were a cop, and it would detect your movements on the screen as your character in the game.

Example, you're being shot at and you are standing behind a car, you have to duck down significantly behind the arcade machine and pretty much not even be in view of the screen in order to get out of the firefight.

Sure it is a rail shooter, BUT, its not your grandma's railshooter. Because if you weren't moving around, you were gonna die pretty quick..

Who knows, there could be products like this in Nintendo R&D that we don't know of. The technology is there, but I don't think the Big N is ready to release something like this until the Wii is in even more homes.

Pure Speculation.

Jeremy @ Dec 22nd 2007 8:31AM

I am not too sure what this guy makes as a living but I bet it isn't enough. This guy needs to be working with some top notch developers to improve on his idea and make a true 3D experience. I would love to see gaming go this route in the next few years but I am sure it needs more R&D for improving. Just imagine a Next Gen console that uses the Wii remote and with the sensor bar that has the IR camera in the center of it and you have this guys glasses. You could have the worlds first VR system for the home that would not cost that much and it would be better than any current VR system out (no large headset).

gevenstaines @ Dec 22nd 2007 10:35AM

according to his website:
Ph.D. Graduate Student
Human-Computer Interaction Institute
Carnegie Mellon University

check it out:
http://www.cs.cmu.edu/~johnny/

be sure to watch the video, "Foldable Interactive Displays"

Go1denchi1d @ Dec 22nd 2007 11:10AM

I have got to say that Johnny Lee is my Wii-ro!!

Nebinator @ Dec 22nd 2007 11:27AM

How do I connect my remote to my pc?

Andrew @ Dec 22nd 2007 7:59PM

You need a bluetooth adapter. you can get a pretty cheap adapter for like 15-25 bucks. i bought myself an Azio usb adapter for 16 or so and it works with the wiimote and vista 32-bit

John @ Dec 22nd 2007 2:02PM

Note that all software used in this, and the beyond awesome whiteboard demo were done using a PC, and not the Wii itself.

As impressive as the demo was, I'm skeptical on whether the Wii can pull off the effects without requiring a new Wii with more horsepower on the CPU and GPU. I've seen some nasty looking slowdowns in the Raving Rabbids arena due to the redraws of the crowd, and I'm sure this would mean environments would again need to be small enough (See Zelda and Metroid) to handle headtracking concepts.

Impressive, but I want something past SpaceHarrier 3D in the future.

Candace Savino @ Dec 22nd 2007 5:06PM

I love this guy. That was awesome.

will fn b @ Dec 22nd 2007 11:23PM

HE'S A WITCH!!!! BURN HIM!!!!

... jk

Xaijin @ Dec 22nd 2007 11:35PM

This guy must be getting loads of Nintendo fangirl pussy right about now.

PlutoClock @ Dec 23rd 2007 12:46PM

HAHAHAHA.

If anyone develops any games like this, 360 and PS3 fanboys will shit their pants at how much better the wii is than them.

ubiquityman @ Dec 23rd 2007 9:17PM

Joystiq, you guys have some pull don't you.
You should put Nintendo in touch with this guy.
This is AMAZING stuff.

Ando82 @ Dec 26th 2007 4:24AM

Holy shit - Nintendo, hire the man for the love of God.

non-41 @ Dec 28th 2007 6:14PM

You guys are making way to big of a deal out of this. All he did was mod a wii remote stick it to his head and make some homebrew demo where the third-person "camera" moves in relation to his head. It's not complicated whatsoever, anyone with some basic electronic skills could attempt this without much difficulty. Do you really think the major console developers haven't thought of this since the first 3D console? It's just not applicable. You can't look more then say 30 degrees without having to start using your peripheral vision to see wtf is going on, or run like a maniac from one side of the tv to the other, which is too rediculous for any console developer to put any serious consideration into. If the screen was built into a visor that would be a different story, but then you would be running into things and the visor would cost a rediculous amount. And a question, why is this limited to one player? Ever heard of split screen? That would be great, people would be constantly hitting eachother. I guess you could make it more sensitive so you didn't have to arc too far to make the third-person "camera" change angles, but then that would take away from the realism. The point is it would be cool for about 5 minutes and then you would say damn I am sick of running around my f*@king tv, and that's if you had the room to run around your f*@king tv.

CHAOSDRAGON88 @ Feb 22nd 2008 6:13PM

Wasnt a similar idea integrated in warioware for the wii when you had to put the remote on your head.

rnelson1 @ Feb 23rd 2008 12:36AM

Awesome concept, though not exactly new. This man is brilliant imo. Did he write the software himself or am I under the wrong impression? I really hope his ideas take off. Not just the head tracking but also using the Wii remote to make an interactive whiteboard like in the other video.

It's really incredible. So many people have a Wii remote and a bluetooth adapter around the house (even I do) that we could use for things like this and otherwise wouldn't even know.