Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Dismiss Notice

Question Interactive projections hardware

Discussion in 'AR' started by OmarOfMA, Feb 16, 2023.

  1. OmarOfMA


    Sep 6, 2022

    I'm creating an interactive projection. And I'd like some pointers regarding the hardware and SDKs I might use with Unity.

    • The projection will be flat on a flat surface, so I'm not worried about projection mapping currently.
    • The content might be projected on a wall or on the floor.
    • Players will control the game via body movement OR by touching the display surface (the floor or wall).
    What I know:
    I'm assuming that I need two pieces of hardware:
    • A short throw projector (to avoid shadows on the display surface)
    • A depth sensor
    Some projectors are interactive, so you can touch the display surface. But I assume that I only need a normal projector because I'm already using a depth sensor for body tracking. So I can also use it to detect surface touches.

    For the depth sensors, the options I found are:
    • Azure Kinect (the older Kinect seems to be discontinued)
    • Intel Realsense stereo camera

    The content I'm finding online is old. So I'm wondering if someone working on this currently (in the entertainment / installations space) can guide me towards hardware and software that integrates well with Unity.
  2. SF_FrankvHoof


    Apr 1, 2022
    For your depth-sensor, you could also look at using a smartphone, along with the XR systems (ARCore/ARKit).
    Unity - Manual: Getting started with AR development in Unity (
    Particularly a (new) iOS-device (one with LiDAR) would work well for this.
    AR Tank Exam using LiDAR & Unity ARFoundation - YouTube
    OmarOfMA likes this.
  3. andyb-unity


    Unity Technologies

    Feb 10, 2022
    Your mileage may vary. Using a depth sensor to try to capture surface touches will be a noisy signal with a likelihood of false positive and/or false negative depending on how you set up your system. If possible I would recommend a design that is more gesture-tracking based (playing to the strengths of a skeletal tracking system), rather than trying to use a skeletal tracking system to fulfill a surface-touch task. (Alternatively, consider using dedicated hardware to detect surface touch.)

    Azure Kinect or Intel Realsense are both potentially viable body tracking sensors. For an installation like this in 2023 I would also consider an AI pose detection solution using a standard webcam, but I can't recommend any particular solution.
    OmarOfMA likes this.