How to integrate 360 video with Unity

Hi touloused,

thanks for the heads up on how to integrate video in a unity project and it's very cool; I have it working for android and was wondering to get it work for webgl do I need to do something special?

I am trying to run it on my localhost, but the problem I have is that it is trying to load an sharedassets1.resource file instead of the video's that I provided. I am new in unity and have no clue how to tackle this. Does it have to do with buuildsettings compress mode or something?

thanks

 

Hi Folks!

Firstly, thank you Unity for this fantastic addition to Unity.

My question is this:

I'm building a simple little app for a client which needs to play 360 videos ( which they are providing to me ) in a youtube 360 format, as shown here. The top half of the video is the left/right panorama, and the bottom half is an up/down panorama. This, I am assuming, is so that the videos don't have stitching issues at the top and bottom of the sphere.

If I watch the video on youtube ( where the temporary, not final, videos are located ) they look fantastic, and I can look around in 3D and don't see any stitching issues anywhere.

However Unity's panoramic skybox shader doesn't seem to have support for this layout.

Can anyone help or suggest how I could get this to work?

Thanks!

Dave

 

I have the same question as zulaman! This shader is working great for me on the skybox but not when I apply it to an actual sphere. I can see it fine on the sphere in the scene view but when in the game view I don't see the sphere at all. Its not a normals issue either as its the same (in game) in or outside the sphere. Any ideas why this is happening? I'm hoping to combine 360 video with 3d sound so could really do with being able to teleport around to different "bubbles" rather than always being at the origin...

 

Hello!
We're using this to display stereo (upper/lower) panoramic images and are encountering a very visible seam. This doesn't happen when playing stereo video files.When playing, it looks great - the stereo effect is working perfectly, but there is a very visible seam.


When I uncheck "Generate MipMaps",as digableMatt,my texture changes on error color.
Texture MaxSize: 8192.
Dragged the image (7680x7680)
Oculus Go sdk.
But when I change texture's max size on 4096 - it's all ok. There is not a very visible seam without.

 

Hey Dave,

I also faced this issue and found a solution. It seems that since the beginning of this year youtube has a new encode format for 360 video's.
If you use the commands below you will get a normal 360 video from youtube as a "flat" video texture file.

Source:
https://github.com/rg3/youtube-dl/issues/15267

Example get video formats list from URL (with cmd windows)
youtube-dl --user-agent "" eyn9Bvr4UQ4 -F

Example: download fragments:
youtube-dl --user-agent "" eyn9Bvr4UQ4 -f 305+140 --merge-output-format mp4

 

I have a question and I hope someone has already solved it.
In my quest to get better quality 360 video's I encountered this:
https://developer.oculus.com/downloads/package/vr5kplayer/

John Carmack from Oculus seems to have created a way to playback 5k video's with 60frames per second for mobile and pc devices. At the moment (from what I can tell) it is only native supported and not by Unity.
Has anyone managed to get this working in Unity? Any help is appreciated.

Greetings,
Dan

 

This works for me, thank you DaneyT

 

Unfortunately after an android update i have the same problem on my Mi mix2

Regards,
Adrian Gates
Technical Consultant - Apps4Rent

 

Hi everyone!!

I just wanted to share my solution for playing 360 video content in Unity in case anyone else is working on the same problem.

I've created a dedicated thread so that future developers can reference the library easily.

In any case, I hope everyone can benefit from this!!
And if you liked it, please give a star on the Github repository!!

 

Hello everyone!

I'm new to unity VR and I am trying to create a scene in VR where a 360 video plays and then transitions to another one through a skybox transition.

Basically what I wanted to do is:

1. Start playing first 360 video with VideoPlayer in Skybox
2. Transition to video 2 by fading in another Skybox with video 2 attached to it

Is it possible to do this? Because from what I've found out you need a video player to play any 360 video regardless of the skybox. I've managed to change scenes with the SceneLoader() but that requires the scene to fade in and fade out.
What I want to do is blend the frames of the first video to the second video and then have the 2nd video play.

Any workaround this?

Thanks in advance!

 

I assume you're using Unity's Panoramic Skybox technique. The problem is that in Unity, there is only ever one skybox. In order to fade-transition from one thing to another, you need two things to exist at the same time. If you want to continue using the panoramic skybox technique, here's a suggestion off the top of my head:

1. Before you swap in video 2, render what is currently on the skybox into a texture. Render said texture onto a panoramic sphere or box around the camera, creating a fake skybox. Note that the texture is not a video, its a still.
2. Swap in video 2. It needs a moment to load up.
3. When video 2 is ready, fade out the fake skybox.

Of course, this means that Video 1 freezes while Video 2 loads and transits, but IMO unless you're doing low res and bitrate, attempting to play 2 panoramic videos is risky business at best.

 

Hey @cons_fx , I created this library to address just that issues.

https://forum.unity.com/threads/unity-360-video-player-library-implementation-solution.593386/

Check out the thread!

 

Hello everyone,
as i used SkyboxPanoramicShader to display panorama pictures as background, i have noticed that the alpha channel is not considered in the Shader. is there an easy way to add alpha channel to do some fadein-out effect ? thanks for any ideas.

 

Same issue here, anyone have found a solution?

 

Hey! I have 2 problems.
One is that when I view images (.jpg), over-under, stereo, 360 images, I get an ugly blueish line where the edges meet. I tried setting warp more to clamp, I basically tried changing every option on my image, nothing changed. See attached pic.

EDIT: Building errors have been solved by moving the script to Editor folder. I'm new obviously to this.
Now I'm getting grey screen even tho cam is on, working, and my scene is checked in build settings. I'll most likely figure it out as well, unlike the line in the pics which I can't seem to solve.

Why is this happening, how do I solve it?

 

I am having the same issue of Hunubul! it works but I have a bluish seam (video works in other player and unreal, so it is not the video file.. I guess).. anyone that can help with this issue?

 

News! The empty skybox material is also having the line after the unity update. Does anyone know how to make it disappear?

 

Hey Everyone! Seemingly simple question(maybe not so simple answer as I've searched quite a bit?[and I am sincerely a novice at this])….Is there a way to have multiple hotspots with videos attached automatically load at the same time at Scene Load? I'm trying to sync different camera angles that were recorded simultaneously.

 

Hi!

I've been following this google doc with some success (!) to but my 360 videos into Unity 2018.3.9f1 for an interactive project.

I have a dozen videos, and it's tedious to make a new material each time. I can't seem to duplicate materials... can't I prefab this process somehow? (I'm like an advanced Unity beginner, I suppose).

Also the doc says to go to Windows -> Lighting, but on this version of Unity on a Mac it's more like Windows -> Rendering -> Lighting Settings, which took some digging because you can't really search for that in the Help menu.

 

oh my god thanks for that!!!

 

nice things draw on 3d maya software i really admire from your work.

 

I know this thread is old, but I'm a non-technical 360 Live Streaming enthusiast and am dedicated on making a 360 vr solution. Will Unity ever natively support 360 video live streaming? Whether from AMS or AWS or other rtmp protocols.

 

Damm even I am having a similar kind of issue, I have searched all over the internet and even have posted on number of threads on different forum, no solution seems to work. I am really frustrated, can anyone of you here help me resolve this issue, I am very much tired now.

 

The 3D Layout does not appears with XR Plugin Management enabled. Which is what 2019.3 suggest to use.
You have to use the legacy "Virtual Reality Supported" in your project settings.

 

is there a similar way to view 3D 360° Pictures in unity as well ?

 

I just bought the Vuze + 360 VR camera. My question is, will I be able to take that video after stitching it and import it into Unity for the entire skybox + scene, so that I can then add the player so that they can look around as the video plays?

 

Is there a solution available using the new XR management? or a work around when using the XR system?

 

Has anyone figured out a way to enable vr for stereoscopic video when using the XR route? Really don't want to go back to using legacy Oculus utilities. Or is that the only way to use 3d video (side by side) as skybox content in unity 2019?
Hoping unity can answer this. Im finding a lot of questions on this but no answer from unity.
Failing an official route, maybe there is a kludgey option in using the old skybox shader and editing it?

 

I'm using the Unity VideoPlayer for playing 360 video content on Oculus Go/Quest. Having some issues with caching causing the system to crash at higher resolutions (>4k).
Best word I've got from Unity is that using the VideoPlayer with a URL is supposed to prevent it from loading the whole file into memory, though with older versions that wasn't how it was supposed to work. Whatever the case is supposed to be, that's not how it works now.
I thought about encoding the videos using h265 codec so file size is smaller so less memory is needed. But Unity player won't play them. Anyone have any export encoder settings for h265 (HEVC) that work with the Unity VideoPlayer? I'm using Adobe Premiere Pro...

 

That's a very detailed and quick response Dominique. Thank you.
To answer your Qs in order:
- Yes I was using h264. I followed that Oculus page guidelines for my encodes... Both h264/265 play fine with the standalone Gallery player on my Go in 6K. Issues come when playing videos using my Go app with Unity's VideoPlayer.
- I have since got some small 6K h265 video files (<10mb) to play on the Go through Unity's VideoPlayer but a larger file (450mb) failed to play.
- That Android Main profile Level is obviously much too low for 6k, which must be Main Profile Level 6.0 or higher.

I'm not actually a dev, I am managing this project and editing all the videos but not developing in Unity. My dev posted here about the problem if you wouldn't mind taking a look: https://answers.unity.com/questions/1755981/video-player-memory-usage.html

If that's not clear, I'll ask my dev to submit a bug report. Thanks again for your help.

 

My dev got it sorted with a bug report and some feedback. Thanks a lot.

 

Thanks for the clarification. get-mxplayer.in mxplayer

 

Hi all, I'm trying to use the skybox-panoramic shader to display equirectangular over/under stereo 360s on the quest, but I'm having an issue with distortion at the poles (definitely the south, not sure about the north as my test images don't have a lot of detail at the north)

This effect isnt mega visible on a flat screen but in VR when looking down creates quite a nauseating effect.
See image example from one of the blender open films, pole is circled in red:

Edit: Imgur is down so see attached image example 1 (PoleIssue1)

This is applied to a sphere in scene, I have tried different geometry / applying directly to skybox but the issue persists, I haven't been able to find a shader that solves this issue.

I tested a mono image with shape setting "cube", on a skybox cubemap material applied to the same geometry, aside from requiring rotation to match stereo shader orientation it displays mostly the same except that the pole looks much better:

Edit: Imgur is down so see attached image example 2 & 3 (PoleIssue 3 is using stereo pano skybox, PoleIssue4 is skybox&cube)

Is unity doing some seam fixing (without the option checked in my texture setup) here with mono images to clean the pole issue? Would this mean that the same fix would not work in VR as the guestimated fixed pixels wouldn't match stereo?

I have to assume there's a way around this issue but I haven't been able to find any posts about it online.
Could it also be render settings? I am using 3dsmax & vray for my own renders but they show the same issue as the blender example.

Is there a shader that fixes this? or a beta version of the stereo pano shader with this issue solved?

Or am I trying to solve this in the wrong way?
In the past I have used a 2 sphere setup with skybox cubemap materials for a cardboard app and selective camera rendering which didn't seem to have these issues.
But with the new oculus XR setup (specifically for quest) selective rendering per eye doesn't seem to work in the same way hence the need for the all-in-one shader.

Thanks for any help,
Joe.

 

I have also just tried the attached shader which seems to be similar to the standard built in cubemap shader, but supporting stereo images.
The effect is different and maybe slightly better but as I expected it seems like the fudging at the poles to clean up the seams still results in a nauseating viewing experience in VR.

 

Here is a modified panoramic 3d video shader that works with the new XR Plug-in Management System : t-o-f.info/public/Skybox-PanoramicBeta.shader.zip

 

Aaaand Dead link.

 

@Muskie Check this, much simpler solution, and works on all platforms.

http://www.andrewnoske.com/wiki/Unity_-_Panorama_Viewer

 

Bit late, but just in case anyone else has this issue (like I did) on your video player change the Aspect Ratio to Stretch. Then the black circles at the top and bottom will go away. I was using the correct resolution for the video file.

 

Hello guys,

I play a side-by-side 3D video (1920x1080 for each eye). I try to config this, but in top and button of my screen, the video are not good.

Can you help me? I will let a image.