Importing or streaming phone movement to create a camera animation?

Apologies if this is covered elsewhere, I did a fairly thorough search beforehand and I would be grateful for a link to any existing resources.

I'm awful at animating, and had the notion that by recording a video with a phone, the accelerometer/gyroscope data, or even the output of camera tracking software, could be used to import an animation that could be applied to a unity camera. It seems very doable and intuitive, similar to using mo-cap rather than keyframe animation for characters, but the only resources I could find were for mobile games looking to harness the phone movement in realtime. Some of those guides might be useful, but the workflow is completely different so I'd prefer something more relevant.

I've never tried anything like this, and while I could bodge my way through it I'd like to ask for suggestions of approaches and tools, or any reasons why it isn't as feasible as it sounds? Thanks very much!

 

check this forum, they have some virtual camera thing,
https://forum.unity.com/forums/virtual-production.466/

or there's unity remote app, which can stream your gyro data in editor also,
https://play.google.com/store/apps/details?id=com.unity3d.mobileremote&hl=en&gl=US

 

You can't use accelerometer data to reliably track position. It is very jittery, as a result errors accumulate in seconds. You SORT OF can track rotation, but gyros drift.

To animate by yourself intuitively you'd need either VR headset, OR you'd need to be dealing with marker targets, OpenCV and pose estimations.

 

Sounds like you are looking for Unity’s virtual camera found in the Live Capture package here.
Although currently its sadly limited to iOS devices only.

I tried out the virtual camera with an iPad and it worked quite well but performance was a bit low and it’s designed for professional cinematographers.

Not sure if there is a better alternative that’s as equally integrated into Unity.

Alternatively you could simply record a video on your device, then import it into blender and run motion tracking on it. Obviously this is post production so you have to guess what your focusing on and moving around, but it’s viable for some situations.

 

Thanks for the suggestions! I noticed the virtual camera originally and assumed it was VR related, but it sounds like exactly what I'm looking for if I can ignore 90% of the settings and keep just the transform keyframes.

I'll need to find an iPhone to borrow and hope my machine can handle it~

 

One other thing to remember is generally we are terrible at holding a camera/ phone steady, so assuming Virtual Camera works for you, you may want to invest in a gimbal. I’m looking to pick up a DJI Osmo mobile 3 or 4 soon as I’ve heard good things about it. Ideally I think using an iPad for the larger screen would be better, though again you really need something to hold it and ideally be on a gimbal.

One tip with virtual camera, since it seems to augment or maybe only use visual tracking via the device camera, pointing at a white wall means it loses tracking. So whatever area you are doing this in needs a good selection of objects to track, or maybe just put up tracking markers.

anyway it’s a lot of fun, to mess with.