Search Unity

Question Use webcam as control (pose estimation)

Discussion in 'VR' started by urimike1, Apr 10, 2023.

  1. urimike1

    urimike1

    Joined:
    Mar 23, 2023
    Posts:
    7
    Hi there!
    I want to develop a game having a character whose movements are controlled by the webcam.
    e.g. when the player raises left/right hand, the character acts in accordance.

    For that I need 2D human pose-estimation. What is the best way to do it in unity?
    Is there a package/asset/plugin that you can recommend? Or must I develop it by myself?

    Thanks in advance if you have any experience with such kind of project development.
    Uri
     
    urimike1, Apr 10, 2023
    #1

  2. DevDunk

    DevDunk

    Joined:
    Feb 13, 2020
    Posts:
    5,048
    Look around for VR body tracking using webcam. Then implement that.
    I don't suggest writing the full body tracking system urself
     
    DevDunk, Apr 10, 2023
    #2

  3. urimike1

    urimike1

    Joined:
    Mar 23, 2023
    Posts:
    7
    Thanks for the reply!
    I have a lot of experience in programming (computer vision), but I am very new to unity, would be glad if you could elaborate/explain.
     
    urimike1, Apr 11, 2023
    #3

  4. DevDunk

    DevDunk

    Joined:
    Feb 13, 2020
    Posts:
    5,048
    OpenCV probably is the most capable. Maybe Unity MARS has some tools?
    I'm far from an expert on this field
     
    DevDunk, Apr 11, 2023
    #4

  5. CiaranWills

    CiaranWills

    Unity Technologies

    Joined:
    Apr 24, 2020
    Posts:
    199
    AR Foundation is for AR on mobile devices, so it doesn't support the webcam. While Unity Mars does give you the ability to add your own providers it doesn't support your use case out of the box.

    As @DevDunk says you may find something on the Unity Asset Store.
     
    CiaranWills, Apr 13, 2023
    #5