Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice

Importing or streaming phone movement to create a camera animation?

Discussion in 'General Discussion' started by DDAB, May 10, 2022.

  1. DDAB

    DDAB

    Joined:
    Aug 28, 2014
    Posts:
    2
    Apologies if this is covered elsewhere, I did a fairly thorough search beforehand and I would be grateful for a link to any existing resources.

    I'm awful at animating, and had the notion that by recording a video with a phone, the accelerometer/gyroscope data, or even the output of camera tracking software, could be used to import an animation that could be applied to a unity camera. It seems very doable and intuitive, similar to using mo-cap rather than keyframe animation for characters, but the only resources I could find were for mobile games looking to harness the phone movement in realtime. Some of those guides might be useful, but the workflow is completely different so I'd prefer something more relevant.

    I've never tried anything like this, and while I could bodge my way through it I'd like to ask for suggestions of approaches and tools, or any reasons why it isn't as feasible as it sounds? Thanks very much!
     
  2. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,019
  3. neginfinity

    neginfinity

    Joined:
    Jan 27, 2013
    Posts:
    13,337
    You can't use accelerometer data to reliably track position. It is very jittery, as a result errors accumulate in seconds. You SORT OF can track rotation, but gyros drift.

    To animate by yourself intuitively you'd need either VR headset, OR you'd need to be dealing with marker targets, OpenCV and pose estimations.
     
  4. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    2,005
    Sounds like you are looking for Unity’s virtual camera found in the Live Capture package here.
    Although currently its sadly limited to iOS devices only.

    I tried out the virtual camera with an iPad and it worked quite well but performance was a bit low and it’s designed for professional cinematographers.

    Not sure if there is a better alternative that’s as equally integrated into Unity.

    Alternatively you could simply record a video on your device, then import it into blender and run motion tracking on it. Obviously this is post production so you have to guess what your focusing on and moving around, but it’s viable for some situations.
     
  5. DDAB

    DDAB

    Joined:
    Aug 28, 2014
    Posts:
    2
    Thanks for the suggestions! I noticed the virtual camera originally and assumed it was VR related, but it sounds like exactly what I'm looking for if I can ignore 90% of the settings and keep just the transform keyframes.

    I'll need to find an iPhone to borrow and hope my machine can handle it~
     
  6. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    2,005
    One other thing to remember is generally we are terrible at holding a camera/ phone steady, so assuming Virtual Camera works for you, you may want to invest in a gimbal. I’m looking to pick up a DJI Osmo mobile 3 or 4 soon as I’ve heard good things about it. Ideally I think using an iPad for the larger screen would be better, though again you really need something to hold it and ideally be on a gimbal.

    One tip with virtual camera, since it seems to augment or maybe only use visual tracking via the device camera, pointing at a white wall means it loses tracking. So whatever area you are doing this in needs a good selection of objects to track, or maybe just put up tracking markers.

    anyway it’s a lot of fun, to mess with.
     
    Last edited: May 11, 2022
    stain2319 likes this.