Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Use webcam as control (pose estimation)

Discussion in 'VR' started by urimike1, Apr 10, 2023.

  1. urimike1

    urimike1

    Joined:
    Mar 23, 2023
    Posts:
    7
    Hi there!
    I want to develop a game having a character whose movements are controlled by the webcam.
    e.g. when the player raises left/right hand, the character acts in accordance.

    For that I need 2D human pose-estimation. What is the best way to do it in unity?
    Is there a package/asset/plugin that you can recommend? Or must I develop it by myself?

    Thanks in advance if you have any experience with such kind of project development.
    Uri
     
  2. DevDunk

    DevDunk

    Joined:
    Feb 13, 2020
    Posts:
    4,877
    Look around for VR body tracking using webcam. Then implement that.
    I don't suggest writing the full body tracking system urself
     
  3. urimike1

    urimike1

    Joined:
    Mar 23, 2023
    Posts:
    7
    Thanks for the reply!
    I have a lot of experience in programming (computer vision), but I am very new to unity, would be glad if you could elaborate/explain.
     
  4. DevDunk

    DevDunk

    Joined:
    Feb 13, 2020
    Posts:
    4,877
    OpenCV probably is the most capable. Maybe Unity MARS has some tools?
    I'm far from an expert on this field
     
  5. CiaranWills

    CiaranWills

    Unity Technologies

    Joined:
    Apr 24, 2020
    Posts:
    198
    AR Foundation is for AR on mobile devices, so it doesn't support the webcam. While Unity Mars does give you the ability to add your own providers it doesn't support your use case out of the box.

    As @DevDunk says you may find something on the Unity Asset Store.