What can Unity create with 360 vidoes?

Dear Unity Forum

(I am new to Unity and 360 videos.)

I plan to record 360 videos with the Instra 360 Pro 2 camera and wonder what Unity can create with the 360 files recorded.

As far as I understand Unity could be used as an alternative to Adobe Premier and other editing software e.g. the Insta360 STITCHER, however I am not sure what advantages Unity has.

Are there perhaps some tutorials you could recommend?

Many thanks, k

 

can use unity to at least for:
- view 360 images and videos
- create mobile/desktop/vr apps that have 360 views/features
- can overlay 3d models, UI and other interactive things, hotspots, over 360 images/videos
- can record videos inside 360 images
- can record 360 videos (asset store has plugins to easily do that)
- can play stereo video (display separate 360 images/videos for each eye)

cannot use unity for stitching images (unless you write/find script and algorithm that does it)

I wouldn't use unity for video editing.. although not sure what you would like to edit or record there.

 

thanks!

one goal is to capture/recreate internal and external environments (e.g. a room in a museum with images on the wall or a garden with statues) and embed something like a virtual tour in a website, something like this: https://casajacaranda.360hdtours.co.uk/

another goal is to create 360 videos which 'move' like this: https://storylabresearch.com/projects/respitefromreality/

also, is it possible to create a virtual reality (VR) based on (rendered from) a 360 video in which we could walk around with an avatar - similar to video games like Doom (first person perspective) or Tomb Raider (third person perspective)?

I would be particularly interested in creating environments with surround and binaural sounds for well-being (meditative) purposes.

 

How exactly do you expect to "walk around" a 360 video? Its a 360 video, its recorded around a single point (Even if that point is moving, thats the point and you cannot go away from it). You cannot move away from that point, its not a 3D rendered environment its simply a big video. So no you cannot walk around it, there is no real "3D" space at all, its just a spherical video in a sense. You can only look around in 360 video / images (view control), but cannot move (no locomotion). There is limited movement, but usually its only a couple meters at most. (in 360 vids that is)

But you can absolutely scan an environment using a 3D scanner or similar and model that as a 3D environment to walk around. Or just get a 3D artist to model it using photogrammetry or simply by hand.

So basically, if you want to actually control the movement in the environment, you do not want 360 video - you want a 3D environment.

Otherwise you have to have the video do the tour, as in the recording walks around but thats it, no control after that other than where you are looking it will always follow that path (because its literally just playing a video)

Source: I run my own company and we have done a ton of 360 projects, VR projects etc.

 

those are possible.

only issue with unity webgl builds is that it doesn't work properly in many mobile devices..
so if you target mobile and web, i'd use those virtual tour editors instead (which are meant for creating web based 360 tours)

if you are inside 360 image or video (practically inside big sphere), moving is possible,
but cannot move too far from center since the video is just a background..(might look distorted)

 

https://unity.com/solutions/360video
Lots of basics answers in the link.

There are also YouTube examples.

 

We have a WebGL product that works on mobiles etc. WebGL actually has worked on mobile for a while, you just needed to supress the warning via the html template. So this is no longer a limitation But it is a pain working with WebGL in general, its low powered and doesnt support lots of APIs.

 

No. Not automatically anyway.

360 VR video has your position locked in place. It also almost always lacks depth information, so it is not TOO immersive in VR, because most of the time it looks like you're either in a world of giants, or everything is rendered on a huge globe around you.

180 vr videos are more immersive, because they have strong depth information, and those could be awe-inspiring, especially if you use something like underwater sea life.

You can't move around in either of those, and you can't really turn head around in 180 videos.

--------------

Moving your head is not currently possible, but there's an experimental tech that allows limited head movement.
https://augmentedperception.github.io/deepviewvideo/

I've played such demos they pretty much make you feel like you're there, but distant objects can be blurry due to compression noise.
However this tech requires this kind of rig to shoot those videos:

Spoiler

Another option is photogrammetry. You'll go through the environment, shoot it with cameras and then perform 3d reconstruction with something like MeshRoom.

Doing that will result in an enviornment you can walk through, but it will looks worse than the real thing, due to lack of reflections and the like.

VRChat, has an environment like that so if you have a headset you could walk through and see what it looks like. It is called "The Hallwyl Museum"

Spoiler

 

thanks for your support!

I will do some more research starting with https://unity.com/solutions/360video

 

Good luck!

 

I would also watch this

and read this https://blog.unity.com/technology/how-to-integrate-360-video-with-unity

Good luck!