Unity3D for Interactive Projection Mapping?

Hi folks,

Is Unity3D suitable for interactive projection mapping? Is there any examples|tutorials online showing how to setup something along these lines? An example of a finished project can be seen at http://www.youtube.com/watch?v=2ltp9nKKzsA .

What sort of hardware is needed to accomplish something like this? Would a Kinect suffice for tracking the people or would something more robust be required? Any help would be greatly appreciated.

Thanks in advance.

Glen

 

I would be very glad to see some answers to that question.
I am also very interested in it.

Or if someone could at least point us to some users (or companies) that have achieved something related.
Thank you.

 

I wonder if you set the input of a touch screen to world coordinates and go from there... hmm/

 

I just found out about The Elumenati http://www.elumenati.com/products/software/omnity/
Evidently, they make a package for Unity3D to work with their projectors and domes.

 

Thanks for that link Halestorm

 

This is definitely doable in unity!

For tracking a Kinect would work very well, but actually for something simple like that example you could also use a regular webcam and do some motion tracking. A kinect will however give you better image segmentation, depending on the intended environment.

If you want to use Kinect you can start by looking at the openNI unity wrapper.
https://github.com/OpenNI/UnityWrapper and http://www.openni.org/

Also if you want additional multi-touch like functionality there is an implementation you might want to look at here:
http://xtuio.com/ (its build on TUIO input, which you also can get via kinect, i think there are some implementations on the net)

 

Thank you for those links and informations! That will be helpful

 

Hey, I have done a few interactive projects in Unity using kinect, can see some over at XY01.net and https://vimeo.com/xy01/videos. Any q's just ask

 

Well I wouldn't call that projection mapping, that's simply a projection. Projection mapping more often refers to when you project a scene onto a non-uniform surface, such as a building. Its also not straightforward to accomplish.
e.g.

For the interaction part, generally, you can use touch-screens, web-cams or the kinect.

However in the specific case of the video you posted I don't see how the kinect could work. For starters it doesn't have the field of view to capture the entire length, secondly if place it next to the projection there isn't the depth range to capture the people putting there arms out, its also on the limit for capturing peoples bodies that close.

The obvious place for a camera would be in the ceiling, however once above 30-45 degrees the kinect will not recognise people, so again unlikely to work in this instance. You could place the kinect on the opposite wall to the projection, but then you wouldn't capture people touching the projection.

In this specific case it looks more like they probably used standard webcam or mini-cam, placed in the ceiling (maybe more than one) and simple blob tracking. Not to mention the video was uploaded in 2007 some time before the kinect was released

So overall is it possible to create something exactly like in your example video. Definitely. In terms of hardware, I'd either suggest a touch-screen (the easy method) or using one or more webcams in the ceiling and basic blob tracking. Do a google search and you'll find plenty of code examples of blob tracking, it isn't that difficult. However I don't think a kinect would work or be appropriate in this instance.

If you want to use the kinect for interactive experiences then you have two options.


1. OpenNI
OpenNI is an initiative started by Primesense, who created the kinect, or rather the tech behind the kinect. Its fully featured, providing a large number of interactive options, from full on skeleton tracking, to simple hand tracking and providing the camera streams (rgb, depth, IR).

2. MS Kinect SDK
Unfortunately the sdk uses .Net4 which will not work with Unity. I think someone has maybe written a wrapper plugin to get around this, but its not really a straightforward method and the last time I looked at the MS SDK it wasn't nearly as fully featured as openNI.


In both cases you'll find its quite a steep learning curve and although openNI can be harder its the one I'd recommend at the moment.

If you search the Unity forums you'll find a few threads based on getting openNI/kinect working in Unity, though these are all quite old now, but should give you some insight. Better yet join the UnityKinect google group and the openNI google group and maybe download 'Zigfu' which is a complete package based around openNI with a nice Unity package to get you started. Though it is aimed at getting kinect experiences in browsers and in my opinion may well become a licensed product at some point.

 

Sorry to ressurect, but : https://www.assetstore.unity3d.com/#/content/7415 - http://demos.elumenati.com/

 

>One option you guys might want to try is using LightAct media server with Unity.

You realise this thread originated in 2011 and hasnt been active since 2013, right? Those 'you guys' have been gone for most of a decade. And similarly several years with your other thread.

Please dont necro multiple threads just to advertise a product, its bad etiquette.

 

@sxa it is obvious necro spam as you noticed. Just use report.