Improve MovieTexture Playback on the Oculus (360 Degree Video)

Hello Everyone,

I have a situation where I am trying to get a high resolution video wrapped in a 360 degree sphere designed for the Oculus Rift to playback in unity. So the scene is setup like so:

A sphere gameobject is in the center. The sphere material has a video mapped to it. I have tested this with 2 videos. 1 has a resolution of 4096x2048 and the other has a resolution of 3840x1920. The goal of this project is to have a 360 degree video player. Here is where the issues begin. In order to have a good framerate to provide the best possible experience in the rift, the video quality needs to be dropped tremendously to run at >75fps. But in order to feel immersed and have the highest quality I need to leave the resolutions the same. Right now with those resolutions my frame rate is about 5-10 fps. It is very jitter and laggy.

I have tried the following to help improve framerate:
Disable Vsync, Run in DX9, Use a lower poly sphere, Lower bit-rates when converting (lose quality), having the video read directly from the hard drive (SSD), having it read directly from unity's built in "resources" folder (http://docs.unity3d.com/ScriptReference/Resources.html), and go as far as to remove the audio and try only video since some people said that it was the audio that was making it lag, and I'm sure I'm forgetting others. Another thing I did was use the profiler since it is now free in Unity 5. The profiler kept saying 90%+ of my CPU Usage is going into a property called: Gfx.WaitForPresent. After doing a very extensive amount of research I learned more about it, but I'm still not 100% sure I understand it. The best explanation I found was (https://forums.oculus.com/viewtopic.php?t=21833) and it is the post about halfway down by chrispruett » Wed Apr 08, 2015 2:40 am.

If it is worth mentioning, my unity version is: 5.0.1f1.

My question is, is there anyway to achieve a smooth video using MovieTexture? I have read that the OGG theora codec is not very good however I was able to keep the quality in the video after the conversion. It's just that MovieTexture is so resource hungry I'm not sure it is the best solution. Furthermore I have tried running this program on a very high end machine at work with 2 GTX Titan Z's SLI connected and it is still not going over 20 fps, if that. Is there any alternative to MovieTexture that I could use? I do not care if it is a free or paid asset.

One "exciting" post I can across someone mentioned that in Unity 5.1 they were going to update the OGG Codec to support 4k video and up to 3 1080p streams within the next 6 months and this was posted late January. Link can be found here and it is the very last 2 posts (https://forums.oculus.com/viewtopic.php?t=18185)

I am open to any suggestions. I can provide more information and pictures if it will help.

Thank you for your time.

 

I'm in the same boat, and I've tried many methods for smooth video playback in Unity (ranging from image sequences to EmguCV video streaming), but the only thing that's worked and worked well is the AVPro plug-in. You should get the trial of the Windows Media and/or Quicktime version and test it yourself. You should get the best results with a Hap encoded video running off of an SSD, on a system with plenty of memory.

 

Thank you! The trial plugin works great. Took some time to figure things out, but once I got every issue solved it worked great. Smooth 4k 360 playback. Thank you very much for your help!

 

Hi Vectorizing

Could you roughly explain how you used AVPro plug-in and still have the video mapped on the sphere? Did you do a video mapping inside the sphere, or is it outside? Because if it is inside then I am in the situation as you were before, and it will be great if you could share some helpful tips in your success with AVPro plug-in.

Thanks!

 

Sorry for the delay.

With the AVPro plug-in they should give you a sample scene that is called something like: MaterialMappingDemo.
Check out that scene and see if you can change it to your liking. I had to tweak a lot of little things to make it work. Another very important thing if you want success using the AVPro plugin with 4k video wrapped 360 degrees is: The video needs to be converted to the HAP video codec.

I've only been able to convert the videos on a mac using this:
http://vdmx.vidvox.net/tutorials/using-the-hap-video-codec

VVBatchExporter is the program you're looking for.

Then after that I copied it back to my windows computer. I was told that you can use premiere elements and convert to HAP but I never got it working.

The reason I use HAP is because it puts most of the processing of the video on the graphics card instead of the CPU.

A couple things to keep in mind while using the VVBatchExporter app: Your video NEEDS to be in a .mov format. (If it is in .mp4 you can rename it to .mov since it's using the same codec, otherwise you need to convert it to .mov). Make sure you enable sound if your video has it. Also make sure you select the highest quality settings for the best video.

Let me know if you get any better results with that or have any other questions.

 

Sorry for replying so late. I took a break from trying this approach after days of frustration and worked on something else.
But I came back after a while and tried it again, it is MAGICALLY working! Thanks for the tips! Also were you able to get rid of the moving green line and other green stats that are not part of the video?

 

The moving green line is the watermark of the trial version. You'll have to purchase the asset to get rid of it.