Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice

AnimationPoseConfig Falsely Returning "unset" pose state

Discussion in 'Computer Vision' started by jay_ee, Feb 16, 2022.

  1. jay_ee

    jay_ee

    Joined:
    Jan 24, 2022
    Posts:
    5
    Hi Everyone,

    I am working with poses in unity and am using the animationposeconfig to set pose states for my training labels (such as going up, going down, etc. etc.).

    I have added four animation pose configs for 4 animations to my keypoint labeller and came to understand that the dataset created still has many (50% +) "unset" pose states.

    I checked the corresponding RGB images and these are clearly pose states which correspond with animations that have an animation pose config and state labels for these poses.

    Also it does not only concern a single animation which could mean that I misconfigured. No, actually pose states are created for each individual pose state, but it appears that right after one animation is concluded (based on the frames per second for each iteration defined in the fixed length scenario) and the animation randomiser selects the next one, this next animation is not assigned the correct pose state and gets instead a "unset" label.

    I could use only the frames with labels, but it would be hugely inefficient so I am hopeful someone can help out.

    Here are some screenshots:

    1. Configuration of Keypoint labeller with all Animation Pose Configs
    2. Example of my JumpingJacks Animation pose config
    3. Proof that the image rgb 1101 shows a motion (going upward) but the json shows "pose": "unset"

    Thanks and best regards my fellow synthesisers :)

    Julian
     

    Attached Files:

    jay_ee, Feb 16, 2022
    #1

  2. MohsenK_Unity

    MohsenK_Unity

    Unity Technologies

    Joined:
    Jun 8, 2021
    Posts:
    10
    Hi Julian,

    Thank you for posting. I have done some tests but so far I have not been able to reproduce this issue. What I recommend is you either attach a debugger to the code or print out logs to see why the pose is being determined as "unset" in some frames. The parts of code you would need to check are the GetPose() function in KeypointLabeler.cs and the GetPoseAtTime() function in AnimationPoseConfig.cs (which is called by the former). The code here is pretty straightforward.

    I would be curious to know if you find what the cause is and whether it is a bug or unhandled edge case in the package.
     
    MohsenK_Unity, Mar 18, 2022
    #2

  3. jay_ee

    jay_ee

    Joined:
    Jan 24, 2022
    Posts:
    5
    Hi
    Thanks for the response. I ran some tests over different projects and Im quite confident that the issue can be reproduced (only) when using AnimationRandomizer with multiple animations/options. The AnimationPoseConfig writes an unset pose in the transition phase between the random poses triggered by the AnimationRandomizerTag of the Player GameObject.

    I deduce this because when i configure the pose execution "manually" through animator tab, there is no "unset" state in the data (see screenshot). Therefore some gap must exist between selecting random poses through AnimationRandomizer or AnimationRandomizerTag that the GetPoseAtTime() function interprets as a Null or !m_timestamp.any().

    Best
    Julian
     

    Attached Files:

    jay_ee, May 2, 2022
    #3

  4. MohsenK_Unity

    MohsenK_Unity

    Unity Technologies

    Joined:
    Jun 8, 2021
    Posts:
    10
    Thank you for the update. I have been trying to reproduce this using an animation randomizer with multiple clips but have unfortunately not been able to. There must be something in your setup that I am missing that causes this issue. At this point I think the best way to debug this would be to use breakpoints in the code to find out which line is returning the "unset" pose.

    Additionally, It would be great if you are able to share a sample project with us that demonstrates the issue so that we can dig deeper.
     
    MohsenK_Unity, May 9, 2022
    #4