Search Unity

Kinect plugin

Discussion in 'Made With Unity' started by bliprob, Nov 21, 2010.

  1. Mr. Me

    Mr. Me

    Joined:
    Apr 21, 2011
    Posts:
    11
    ive tried deleting the '_OpenGL_DataToTextures' but that didnt do anything. ive also tried the wrappers that are in NITE but they dont realy work either which seems weird.

    have you or has anyone else tried your framework in the indie version?!?
     
  2. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    1,528
    Not me.
    If the Unity wrapper provided by NITE doesn't work, then that is where the problem will be. Though quite what has changed to require using Pro i couldn't tell you.

    BTW when you say they don't work is it that you still get the same error as you posted above, about missing a license?

    You might want to join the UnityKinect google group, as i'm discussing my framework there andthe original author of the NITE wrapper (which my framework is heavily based) might have an answer for you.


    Edit:
    Apparently the Unity Wrapper has changed and hence why it no longer works in unity free.

    Posted on unityKinect google group by Shlomo

     
    Last edited: Jun 26, 2011
  3. xandeck

    xandeck

    Joined:
    Apr 2, 2009
    Posts:
    563
    Hey Noisecrime, I managed to install the OpenNI, PrimeSense and the Sensor, the Kinect keeps the led flashing and the drivers are good installed in my Windows (using W7 64bits, but installed everything acording your quickstart guide).

    But when I try to use the NiUserTracker sample, it says that the device is not connected. And so the Unity project does not work too.
    Do you have any tips for me?

    Thanks in advance
     
  4. xandeck

    xandeck

    Joined:
    Apr 2, 2009
    Posts:
    563
    Nevermind, I got it working in Windows XP... now trying in Win7
     
  5. xandeck

    xandeck

    Joined:
    Apr 2, 2009
    Posts:
    563
    Hello everyone... please, anybody solved the problem for the game closing itself after starting (already compiled)... I can only run compiled if I compile from Unity and run... if I hit the EXE file, it opens and then closes the game... o_O
     
  6. Lucas Shaffer

    Lucas Shaffer

    Joined:
    Jul 6, 2011
    Posts:
    2
    I am in a class and we are trying to get the kinect to work with Unity. We have a pre-built world with unity and I am trying to get the kinect to work with the character we have. I am new to Unity and have been working with the Kinect on the computer using OpenNI and FAAST. Thanks for any help
     
  7. tgraupmann

    tgraupmann

    Joined:
    Sep 14, 2007
    Posts:
    828
    When I was in school they barely started making 3d cards. You are really lucky. :p
     
  8. Todilo

    Todilo

    Joined:
    Feb 1, 2011
    Posts:
    88
    perhaps you are missing the xml file/dll files in the BUILD folder.
     
  9. Teddosh53

    Teddosh53

    Joined:
    Jul 11, 2011
    Posts:
    3
    Hi guys
    I have a problem using thiskinect plugin. In fact, i always have an error with the line:
    Code (csharp):
    1. image       = context.FindExistingNode(NodeType.Image) as ImageGenerator;
    I get a "No match found" error on this line, and a "NullReferenceException" in the nexts line i call image.
    Does someone have an idea to solve my problem?
    Thanks for your answers and sorry for my bad english.
     
  10. Rastafari

    Rastafari

    Joined:
    Jul 18, 2011
    Posts:
    196
    Epicsauce!
     
  11. rasq

    rasq

    Joined:
    May 25, 2009
    Posts:
    51
    Hi, unity3d 3.4 have problem with openni wrapper?

    I can`t run it on this version, any one have this same problem??
     
  12. brianchasalow

    brianchasalow

    Joined:
    Jun 3, 2010
    Posts:
    201
    Hey there Noisecrime- I've been working on a bunch of GL texturing projects in Unity as well. Why not use an FBO to render the texture in Unity, rather than setpixels/glTexImage2D etc?

    edit: nevermind, in porting libfreenect to unity myself, i now see what you're doing.
     
    Last edited: Sep 16, 2011
  13. Salvador

    Salvador

    Joined:
    Aug 10, 2011
    Posts:
    3
    I have the same problem than rasq, unfortunately I don't have the solution, but I think is cuestion of time, I guess.
     
  14. carllooper

    carllooper

    Joined:
    Aug 8, 2009
    Posts:
    64




    These are running at 25 fps through Indie Unity IDE. Standalone executables go a little faster.

    Uses a plugin (a windows DLL) written in C# but with a post compile IL rewrite (decompile/recompile) to operate as if it was an unmanaged DLL. The DLL is a wrapper around the Microsoft Kinect SDK. Can be used in both Pro and Indie versions of Unity. Indie usage just requires some extra management on where DLLs are located and how they are referenced from client scripts.
     
    Last edited: Aug 21, 2011
  15. rasq

    rasq

    Joined:
    May 25, 2009
    Posts:
    51
    Hi, i have one question:
    when i`am using NiteCursor&NiteCursorVisualizer - i hae good hand tracking, but when i also enable OpenNiUserTracker - i have ok skeletal tracking but i cant use hand tracking.

    Any one have some adwices???


    Thx
     
  16. cristian_ritter

    cristian_ritter

    Joined:
    Aug 11, 2011
    Posts:
    37
    Hi, about the KinectPlugin.
    Does anyone know how to implement a moviment script in KinectModelController.cs?

    I tried here, but not worked :(
     
  17. rasq

    rasq

    Joined:
    May 25, 2009
    Posts:
    51
    Ok, i have.

    If i need use skeletal controller and cursor, Cursor must be the first script.
     
  18. nikemerlino

    nikemerlino

    Joined:
    Jul 2, 2008
    Posts:
    66
    OpenNI dosen't work in Lion to me. Any suggestion?
     
  19. lazygunn

    lazygunn

    Joined:
    Jul 24, 2011
    Posts:
    2,749
    Zigfu is excellent i have to say, it only took a short while to redo the 'digital pinart' video on youtube with zigfu, unity free and a kinect
     
  20. alr1976

    alr1976

    Joined:
    Dec 23, 2010
    Posts:
    23
    Hi Guys!!

    This is difficult task for me.... i want change texture in Opengl in xcode and use it in Unity but doesn t works...
    Unity texture ->Change Texture in Opengl in xcode -> Use it In Unity

    This is my code... will somebody help me?

    Unity C:
    public GUITexture mio;
    public Texture2D m_Texture;

    void Awake () {
    m_Texture = new Texture2D( 640, 480, TextureFormat.ARGB32 , false);
    m_Texture.Apply();
    }

    void OnGUI () {


    if (GUI.Button(new Rect(10,80,90,30),"update_rgb_texture"))
    {

    Debug.Log("Bonjour.Buffer");
    Bonjour.pluto(m_Texture.GetNativeTextureID());
    m_Texture.Apply();
    GUI.DrawTexture(new Rect(0,0,Screen.width,Screen.height) , m_Texture);

    }

    public static void pluto(int number)
    {
    _updateTexture( number );
    }

    [DllImport ("__Internal")]
    private static extern void _updateTexture( int nTexId );

    Xcode:
    void _updateTexture( int nTexId )
    {


    CVImageBufferRef pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer);
    CVPixelBufferLockBaseAddress(pixelBuffer, 0);
    int bufferHeight = CVPixelBufferGetHeight(pixelBuffer);
    int bufferWidth = CVPixelBufferGetWidth(pixelBuffer);
    GLuint videoFrameTexture_;

    glClear(GL_COLOR_BUFFER_BIT);

    //glGenTextures(1, &videoFrameTexture_);
    glBindTexture(GL_TEXTURE_2D, myid+1);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
    glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);


    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, bufferWidth, bufferHeight, 0, GL_BGRA, GL_UNSIGNED_BYTE, CVPixelBufferGetBaseAddress(pixelBuffer)); //My Image
    GLenum err = glGetError();
    if (err != GL_NO_ERROR)
    NSLog(@"Error uploading texture. glError: 0x%04X", err);


    GLfloat quatVertices[8] = { 10.0f, 0.0f,
    10.0f, 10.0f,
    -10.0f, 10.0f,
    -10.0f, 0.0f};

    GLfloat textureCoord[8] = { 1.0f, 0.0f,
    1.0f, 1.0f,
    0.0f, 1.0f,
    0.0f, 0.0f};
    glEnable(GL_TEXTURE_2D);
    glEnableClientState(GL_TEXTURE_COORD_ARRAY);
    bool a = glIsTexture(myid+1);
    glBindTexture(GL_TEXTURE_2D, myid+1);
    GLfloat vertices[] = {1,0, 0,1, -1,0};

    //glEnableClientState(GL_VERTEX_ARRAY);

    glVertexPointer(2, GL_FLOAT, 0, vertices);

    glDrawArrays(GL_TRIANGLES, 0, 2);

    glDisableClientState(GL_VERTEX_ARRAY);
    //glDisableClientState(GL_TEXTURE_COORD_ARRAY);

    }
    Ios Device
     
  21. milali

    milali

    Joined:
    Nov 1, 2011
    Posts:
    16
    how is the progress on this plugin?
     
  22. milali

    milali

    Joined:
    Nov 1, 2011
    Posts:
    16
  23. Curious

    Curious

    Joined:
    Nov 19, 2009
    Posts:
    334
    Hey guys,
    I'm familiar with ZigFu, are there any other extensions like ZigFu for unity?

    Tnx
     
  24. Curious

    Curious

    Joined:
    Nov 19, 2009
    Posts:
    334
    Anyone?! I've found the OpenNI Samples as well, anything else?
     
  25. Timbecile76

    Timbecile76

    Joined:
    Jul 8, 2009
    Posts:
    238
    Hey Noise,

    Thanks for doing this. This is great!

    One question though....I'd like to color the registered users a different color based on user ids. Is there any way your stuff supports that?
     
  26. yuriythebest

    yuriythebest

    Joined:
    Nov 21, 2009
    Posts:
    1,011
    Hi! I am very interested in this- does this code already have the advanced stuff like, say swipe detection/ detecting if a person clicked on a button or something? or anything that actually uses the Kinect as a means of input/interaction and not just "cool visual stuff"
     
  27. fffMalzbier

    fffMalzbier

    Joined:
    Jun 14, 2011
    Posts:
    2,892
    The best commentated package i found so far was the package by Zigfu (http://zigfu.com/devtools.html)
    Zigfu has a lot of ready to use scene which are working withe the Kinect fine.
    But if it goes to skeleton tracking , OpenNI has some Sirius problem is detection speed and sability.

    So i searched for a package that uses microsoft sdk.
    I found the ETC Internal Unity3D Wiki , whitch have a package for skeleton tracking.
    http://wiki.etc.cmu.edu/unity3d/index.php/Microsoft_Kinect_-_Microsoft_SDK
    My problem is only how to attach the rigging points of the character molded to the NewModelController script.
    I know that i have to put the Gameobjects in the public variables of the script, but i tried 3 different models but every time it looks like scrambled eggs.

    Did anyone of you get the M$ version that i posted to work with a character ? (i do not mean the KinectPointController example).
     
    Last edited: Jan 17, 2012
  28. amir

    amir

    Joined:
    Oct 5, 2010
    Posts:
    75
    Hey! Amir @ ZigFu here. The latest version of OpenNI available in the latest ZigFu dev bundle has a calibration-free skeleton.

    We are adding support in the ZigFu unity package for calibration-free user selection we will also support the Microsoft Kinect SDK after Microsoft releases the commercial version. It should be simple to transition to whatever middleware you want. User selection in the calibration-free case still requires some way for a user to indicate that they are the player who should be tracked instead of other people who may be in the field of view.

    Also, if you want to use the current version of the KinectSDK beta through OpenNI you can try the OpenNI-MSSDK bridge from Tomoto: https://www.assembla.com/code/kinect-mssdk-openni-bridge/git/nodes/

    In addition to directly supporting the KinectSDK in unity, We will be able to support all OpenNI applications on the Kinect SDK this way too.



     
  29. silondi

    silondi

    Joined:
    Dec 12, 2009
    Posts:
    29
    Dear all,

    I'm using unity pro, 3.3.0f4. I installed the Zigfu's DevBundle and I try to run the Unity3d Bindings but I get some errors.

    Did some some of you already experienced it?
    Any idea how to fix it?

    Thanks a lot,
    Pascal
     

    Attached Files:

  30. fffMalzbier

    fffMalzbier

    Joined:
    Jun 14, 2011
    Posts:
    2,892
    Thanks amir for your replay, are your current Unity3d Bindings ready for calibration-free user selection or does it need to update?
    I have only very limited amount of time (just this week) to finish something that have calibration-free user selection and looks need.

    I would like to use the ZigFu package because i need a camera picture and the MS plug in does not give any camera data.
     
  31. amir

    amir

    Joined:
    Oct 5, 2010
    Posts:
    75
    @silondi: Color32 is part of Unity 3.4. Upgrade or remove the offending CS file from your project.

    @Malzbier: The current dev bundle provides the calibration-free skeleton. Unfortunately without User Selection, the calibration-free skeleton will track anyone who is in frame and not necessarily the person who is player 1. We will update our Unity3D wrapper with examples of how to use calibration-free skeleton and how to perform user selection as well as compile to WebPlayer using the ZigJS browser plugin to transmit skeleton data to Unity WebPlayer. We will also roll in support for the official commercial SDK. We hope to have this by the end of next week.
     
  32. silondi

    silondi

    Joined:
    Dec 12, 2009
    Posts:
    29
    @Amir:
    I upgraded to Unity 3.4 and it works perfectly.
    Thanks a lot,
    Pascal
     
  33. diese440

    diese440

    Joined:
    May 25, 2007
    Posts:
    105
    @Amir :
    The Unity-OpenNI bindings is still in v1.4. Do you plan to release soon ?

    Thanks for your amazing work !

    @all :
    If you want to try a Kinect plugin, have a look on Zigfu http://zigfu.com/.
     
  34. psydack

    psydack

    Joined:
    Sep 18, 2009
    Posts:
    8
    Im using zigfu package but I need OpenNI Motor.
    I can't figure out cause I'm using OSX. I know what have drivers for Win but for OSX?

    I tried with libfreekinect in processing but I don't know how I port the lib for Unity.

    Some tips?
     
  35. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    5,757
  36. Longman

    Longman

    Joined:
    Mar 19, 2012
    Posts:
    1
    The OpenGL method for displaying the maps as textures does not work for me anymore (since Unity 3.5 i think).
    Did they change something related to this?

    And: I have read they have made the Direct3D device pointer available. Does this mean that a performant display of the ImageMap is now possible with Direct3D? Is there an implementation yet?
     
  37. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    1,528
    The kind developers at Unity have suggested a workaround for this problem. The issue is that in 3.5 they implemented rendering on a separate thread for devices where it makes sense (i.e. PC Mac). I assume this is to allow it to run on a separate core? Unfortunately this means without access to the rendering thread the old opengl method no longer works.

    As you mention there is an alternative in that you can write a c/c++ plugin and use some new hooks in 3.5 to access the actual rendering device, including DirectX which is really nice, but a bit of a pain.

    So if you want to keep using the old opengl method, all you need to do (on Windows, not sure about Mac) is add the follow directive to the command line '-force-gfx-direct'.

    To be specific, on Windows you need a shortcut to either the Unity editor or your built project and in the shortcut 'target' field add the following directives after the path name. Make sure you include the spaces. This will run the editor or your project under opengl and force it to not use multi-threaded rendering.

    "C:\ .<snip>. \Unity.exe" -force-opengl -force-gfx-direct

    Hope this helps.


    EDIT: UPDATE

    Since posting this i've become aware of a few additional options, though not yet tested.

    1. PlayerSettings.MTRendering

    Apparently you can set this to false in an editor script before you build in order to
    force single-threaded rendering without any command line options. This should result in 3.5 working like previous versions and thus allow the opengl texture access to work again, though obviously you still have to add the -force -opengl parameter to the commandline.

    See this thread for more details and Kaspar Daugaard post which includes an editor script to use this setting

    2. -force-gfx-st
    According to this thread you can also use -force-gfx-st to force single-threaded rendering, just like the aforementioned -force-gfx-direct.

    For reference it appears that
    -force-gfx-direct will use the Unity 3.4 renderer in single threaded mode.
    -force-gfx-st wil use the Unity 3.5 renderer in single thread mode.
     
    Last edited: Apr 12, 2012
  38. DustInTheAir

    DustInTheAir

    Joined:
    Jun 8, 2012
    Posts:
    1
    Awesome job. I will keep an eye on this thread.
     
  39. jimgal

    jimgal

    Joined:
    Mar 6, 2012
    Posts:
    1
    Hi guys ,
    I am trying to run the Unity Binding from zigfu but i have some problems listed below on the image

    http://imageshack.us/photo/my-images/525/69672571.png/
    Unity version is 3.5.2f2 the free version
    i installed the nite-win64-1.5.2.21-dev,openni-win64-1.5.4.0-dev,SensorKinect093-Bin-Win64-v5.1.2.1.(64bit version)
    The Openni/primesense samples are working .

    Thank you...
     
    Last edited: Jun 14, 2012
  40. kinksid

    kinksid

    Joined:
    Mar 2, 2012
    Posts:
    3
    i have the same issue on unity 3.3.5 mountain lion 10.8 never installed Zigfu

     
  41. Tijszonderh

    Tijszonderh

    Joined:
    Oct 26, 2012
    Posts:
    1
    Hi

    I study Game digital design and for my project in my master year I want to use the kinect in unity.
    Unfortunately I don't know much about unity. I was wondering if anyone could explain to me how to instal the plugin and get the kinect running in unity.
    I'm running OS X 10.6.8 and unity 3.5.6.
    I'm a real noob so it would be great if anyone can explain it to me in very basic steps

    thanks a lot
     
  42. Papok

    Papok

    Joined:
    May 28, 2013
    Posts:
    5
    Hello kinect friends :)
    I'm a 3D animator with no experience in Unity (just Cinema 4D and stuff). Before I try to learn how to do it, I need some help to get started :)

    So I'm trying to make this kind of system with kinect, beamer and some stretched projection fabric:


    if user touches the stretched fabric -> animations start from this point(s).

    So, I understand that there's a kinect plugin and it retrieves somehow the depth map. My question is how to take the informations from the depth map to take and take the coordinates of the brightest points. Preferably vector from the darkest point to the brightest (it must have a limit - the stretched plane would be fixed grey value)

    Ok, hope I made it clear :/

    Thansks for the response and cheers :)
    johnalabama
     

    Attached Files:

  43. kockaart

    kockaart

    Joined:
    Dec 7, 2013
    Posts:
    3
    does anybody knows, how could I use the contour of the depth image?
    I am working on a kinect controlled pinball. I would use the kinect from above to track the person's leg when it reaches a certain area it would active the flipper arms.

    johnalabama: this solution could work for you too
    there is a similar installation at https://vimeo.com/54882144
     
unityunity