Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

Integrating Kinect Azure

Discussion in 'VR' started by Quique-Martinez, Jul 19, 2019.

  1. Quique-Martinez

    Quique-Martinez

    Joined:
    Oct 1, 2013
    Posts:
    141
    Hi there, I'm trying to make Kinect Azure work inside Unity as an unmanaged .dll without success. I can open the device, but not start it. Unfortunately, the device opens but it does not start, and the SDK is so closed that I cannot find what is going on.
    I would like to contact more people trying this.
    Cheers.
     
  2. chrisatwork

    chrisatwork

    Joined:
    May 25, 2018
    Posts:
    3
    I have been looking at integrating azure Kinect into a unity project for a hackathon. I observed that building my own plugin dll that referenced k4a.dll worked for obtaining the installed device count. However, when starting the camera it consistently failed. After debugging k4a.dll setup steps, I determined that the depth engine dll was dynamically loaded when starting a depth stream. This dll was not getting found by the unity editor. But the workaround I found was that adding the depthengine_1_0.dll to the same directory as Unity.exe resulted in the dll being found by the editor/the depth stream starting correctly.

    Fwiw, I haven't tested any of the other dlls in the sdk or standalone builds of my application. It may be that dlls in a assets/plugins directory get picked up correctly for a standalone build but aren't found by the editor. Also, most of the logic in k4a.dll is open source. The sdk also distributes s pdb sofor k4a.dll so you can obtain source for the appropriate release in the azure Kinect GitHub repo and obtain symbols from the sdk download to debug locally
     
  3. Quique-Martinez

    Quique-Martinez

    Joined:
    Oct 1, 2013
    Posts:
    141
    Thanks a lot, I had exactly the same problems that you describe. I have tried a lot of workarounds without success.
    I'm testing this right away al tell something.
    I'm afraid I'm no soo good programmed to dig inside this code ;)
     
  4. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,549
  5. Quique-Martinez

    Quique-Martinez

    Joined:
    Oct 1, 2013
    Posts:
    141
    I already use Kinect in Csharp, actually I am able to see and open the device from Unity, but cannot start it. The response form @crisatwork makes a lot of sense. I'll communicate my progress as soon as I test it.
    Thanks!
     
  6. Quique-Martinez

    Quique-Martinez

    Joined:
    Oct 1, 2013
    Posts:
    141
    Thanks @christianwork. I have run the editor version with your workaround as well as the binary without any special change.
     
  7. lucidtripper

    lucidtripper

    Joined:
    Aug 3, 2017
    Posts:
    28
    anyone know of a way to convert the kinect azure rgb-d .mkv recordings to a lightweight animated .ply file (gltf?) that could run on mobile devices without the azure sdk?
     
  8. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,549
    if you could convert it into webm (alpha channel for depth),
    then can extrude vertices by depth in custom shader. (so it looks like animated point cloud or mesh)
     
    lucidtripper likes this.
  9. lucidtripper

    lucidtripper

    Joined:
    Aug 3, 2017
    Posts:
    28
    i can supply a short mkv file if you wanna show an example?
     
  10. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,549
    not very clear example, but maybe can get some idea how it looks (video + high res plane with point rendering)
    would be nicer if add point size to the shader.
    depth-ezgif.com-optimize.gif
     
    lucidtripper likes this.
  11. lucidtripper

    lucidtripper

    Joined:
    Aug 3, 2017
    Posts:
    28
    cool! can you show the shader code?
     
  12. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,549
    this version is for separate depth, so can modify it to use alpha instead. (and doesnt have point size)
     

    Attached Files: