Kinect v2 with MS-SDK

Please read 'Howto-Use-KinectManager-Across-Multiple-Scenes.pdf' in the 'K2Examples/_Readme'-folder. Then look at the demo scenes in the 'K2Examples/KinectDemos/MultiSceneDemo'. You need add the 3 scenes to the 'Scenes in Build'-list of the Unity build settings, then open and start 'Scene0-StartupScene'.

As KinectManager persists across the scenes, in case of multiple scenes it should be added to the first scene only.

 

encounter the same problem. And the demo scenes do help. As for me, I find the LocateAvatarAndGestureListeners script in the demo scenes plays an important role.

 

LocateAvatarAndGestureListeners script is very important.

I have two scenes. The first one enables the kinect manager, after some seconds, it loads the second scene. But the avatar in the second scene just doesn't work. However, if I add the kinect manager to the second scene and run it alone, it works.

And then I figure it out that. I add kinect manager to the first scene, delete kinect manager in the scene and add LocateAvatarAndGestureListeners to the second scene. Luckily, it works.

 

Thanks!

Another Question: Is there a way to reset the sensor via a key input?

 

Open KinectManager.cs, make StartKinect() and OnDestroy()-methods public and, when a key is pressed, call KinectManager.Instance.OnDestroy(); and KinectManager.Instance.StartKinect();. This should stop and restart the sensor.

 

Hello

When I try to start the KinectSpeechRecognition demo, I get the following error : Error initializing Kinect/SAPI : 0x80040154.
Does anyone have any idea why this happens?
Thank you.

 

Please check, if you have followed the steps here: Kinect v2 Examples with MS-SDK | RF Solutions - Technology, Health and More (rfilkov.com)

 

So I wonder why SensorData.colorImageScale is a vector3 variable. Picture sizes are like 1920*1080 and two variables are enough.

 

Yes, you're right. Please consider the Z-part as "reserved for future use".

 

Hi, I'm using this script for a while now. But I've a problem. If in the scene there are a lot of people (more than 6) the system doesn't work anymore. Is it possible avoid this? I'm using "min user Distance", "max user distance" and max left right distance" and it seems to work well. But if there are a lot of people outside this area, the script doesn't work.

Sorry for my english, but someone else has this problem? there is a solution?
Thank you!

P.S. every 5 seconds I run "ClearKinectUsers();" but it doesn't seem to solve the problem

 

The Kinect-v2 sensors are limited to detecting max 6 people. In your case, I suppose it detects (some) 6 of these many people that are going around. In this regard, I think you would need to physically limit the space around the sensor, to prevent these random detections.

 

Hello,
is it possible to use these tools in Hdrp mode?
When I open the Kinect v2 example scenes, I get this message for all scripts:
"The associated script can not be loaded. Please fix any compile errors and assign a valid script."
Thank you very much.

 

I'm not sure about the K2-asset. But the K4A-asset (Azure Kinect Examples for Unity) works in HDRP & URP, and supports Kinect-v2 sensors too. If you'd like to try it out, please e-mail me and mention your invoice number.

 

Hello mr. roumenf
Thank you so much for making this awesome asset.
i am a student !
I was wondering is there any way to extract tracked animation motion to bvh file.
or print the anlge of joints.....;. i have been stuck at the point for a week
if you have any opinion, please kindly answer

thank you!

 

As far as I remember, the BVH file format was well documented. This together with the fact that you can get the positions and orientations of all tracked body joints at any time in the K2-asset, means you can save it in BVH format, as well. See this tip.

 

I'm able to successfully run the demo scenes in my project using the Kinect actor to override an animation controller using the avatarControllerClassic- if I run the demos in my project everything works perfectly.

When I try to do the same thing in the actual game scenes everything looks like its working properly- Waiting for users etc... the user is found but no skeleton is displayed or detected I assume as there is no animation. It acts as its finding and removing users- but there is no motion capture/animation.

I'm referencing the correct KinectManager instance- everything appears to be running properly but no motion capture animation is processed or displayed.

DisplayUserMap and DisplaySkeletonLines are set to true- in the demo scenes the skeleton is displayed- in game scenes they are not.

UpdateAvatar() is running on the AvatarController.

I looked at the Multiscene demo's and followed the recommendations but it still does not work.

Do you have any tips or information for me to follow? Thank you.

Kinect2, Windows 10, Unity 2019

EDIT: Got it working- since the AvatarController was settings the bones on Awake- there was an order of operations issues- I added manual calls to Init the AvatarController instead of using Awake and it works now-

Any tips for increasing performance and getting more fps out of a scene using this asset? Thanks

EDIT: Solved the performance issue as well. I had to use a different USB3 port.

 

Hi Mr.roumenf

First of all, thank you so much for providing an incredible asset. I successfully applied a new avatar and tracked the whole body in real-time via Kinect v2. in Unity!

By the way, I have two main questions. I hope you kindly answer the question.




I want to make a scoring system based on calculating each body joint based on global location

e.g.,
if 0 <= neck_angle - trunk_angle <= 20:
trunk_score +=2

However, as far as I know, the asset provides joint in local position, right?

1. Is there any way I could get a joint position/joint angle in Global?
2. To make a scoring system based on the angle after subtraction of each joint, I think I need to make a new script. Do you have any idea how to call global joint? Could you please provide me some examples of a script with at least a few lines?


Thank you so much!
I wish I could get an answer.

 

Please look at this tip: Kinect v2 Tips, Tricks and Examples | RF Solutions - Technology, Health and More (rfilkov.com)
The joint coordinates are in Unity global coordinates, given that the sensor is in position (0, sensor-height, 0) and looks towards the Z-axis. 'Sensor height' is a setting of the KinectManager-component in the scene.

To calculate an angle at the middle joint, you need 3 consecutive joints. If their positions are P1, P2 & P3, then B12 = P1 - P2 and B32 = P3 - P2 are the respective bones. Then, the angle in degrees at the middle joint is: Vector3.Angle(B12, B32). Hope this answers your questions.

 

hii

We're using this asset to display a kinect cutout of the user on a huge screen (using 3 projectors), using the background remover.

I have an issue, which is, the quality looks bad on the huge screen. I compared it to quality of the the kinect studio and it's obvious that in unity, the anti aliasing is not as well. I was wondering if there's any settings I can set to enable anti aliasing in unity?

I made a video to show the comparison :

The left one is the Kinect Studio and the right one is unity.
It might not be that obvious on the monitor but when it's displayed on the huge screen, it's very noticeable.

I wonder if there's any other settings I should be looking into.

update: I had some luck with changing the texture properties at runtime:

Code (CSharp):

1. RenderTexture renderTexture = (RenderTexture)image1.texture;
2. renderTexture.antiAliasing = 8;
3. renderTexture.width = 1920 * 2;
4. renderTexture.height = 1080 * 2;

 

Hello,
The plugin is very good but I have a problem with the KinectBackgroundRemoval demo. It works perfectly with the Kinect V2 but not with the Orbecc Astra. With the latter, the depth mask does not coincide with the real image and leaves some offset edges. Is there a way to control the position of the depth mask? Could it be a problem with my Orbecc camera that the optics are not aligned?
Thank you.

 

Hi, if you use Nuitrack SDK as sensor interface in the K2-asset, please find 'nuitrack.config' in the Nuitrack's data-subfolder, and make sure the 'Depth2ColorRegistration'-setting in it is set to 'true'.

 

How to use intel realsenes D438 in K2 asset

 

You would need the Nuitrack SDK. See this tip.

 

Sorry. I'm not sure I understand your question.

 

same here wirth gesture detection with only 1 user

 

hi, there are new sensor and allinone computer from orbbec named persee +(plus), is there any plan to support this device. I just asked Nuitrack recently, they are now current available supporting

 

already look but it still a singleplayer detection

 

http://ratemt.com/k2docs/ gives me
ERR_CONNECTION_TIMED_OUT

 

Thank you very much. For now, what I did was to directly access the

Windows.Kinect.KinectSensor

to check if the Kinect v2 is available or unavailable, in other words, to check if Kinect v2 it is connected or disconnected. I share my solution in case someone else finds it useful.

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4. using Windows.Kinect;
5.  
6. public class CheckKinectAvailability : MonoBehaviour
7. {
8.     /// <summary>
9.     /// Active Kinect sensor
10.     /// </summary>
11.     KinectSensor kinectSensor = null;
12.  
13.     void Start()
14.     {
15.         // get the kinectSensor object
16.         this.kinectSensor = KinectSensor.GetDefault();
17.         // open the sensor
18.         this.kinectSensor.Open();
19.     }
20.  
21.     void Update()
22.     {
23.         // get the kinectSensor object
24.         this.kinectSensor = KinectSensor.GetDefault();
25.         Debug.Log($"Is Kinect Sensor Available? {kinectSensor.IsAvailable}");
26.     }
27. }
28.  

 

Hi.

First of all, thanks for the fantastic tool.
Similar to MultiSceneDemo, I am using different scenes with different interaction options like grabdrop, gesture etc. Now I stumbled over the problem that I use a PlayerCalibrationPose in the KinectControllerSetup. But since the individual scenes are quite short, I don't want the user to have to do the Calibration Pose over and over again, but only, in the first scene and obviously if the active user was lost. Does anyone know a smart workaround for this issue?

 

Hey there,
I am trying to create dressings room with orbbec astra and using kinect ms sdk.
i want to place the cloth model to the user body on the screen.
for now it is instantiating at zero.
I have got the the code where the model is instantiated. But could not understand how to set its position so that it fits over the user body as shown in the title video.
and also want to know that is it possible to set the cloths model's size as per the user body size..
for example if the user have heavy body and muscles so the model size will change accordingly and fits to the user.
how to do that?
Does Orbbec work only with NUItrack here??

 

Hello!

I'm using the Orbbec Gemini2 device. Does this asset support that device?

- Orbbec Gemini2 Info Link:
https://shop.orbbec3d.com/Gem-Module