[RELEASED] VR Panorama - render 360 stereo videos

Be sure to use deferred rendering and set it on camera. Forward can give you a problems, but there is really no need to use it, as with VR panorama you can get nice antialiasing in deferred, and overall a better quality rendering with many shadows. If you are using forward rendering this should correct your problems. If you are using already deferred rendering, please attach a screenshot of your scene hierarchy (part with VR panorama camera) and VR camera attached scripts. Latest version of VR Panorama doesn't use depth buffer, because this gives you faster rendering and less VRAM usage. I will probably expose this value in a next version as an option, that will also support multiple layer rendering.

 

I was able to start a capture once, but I stopped it to fix the sky... I can no lobger get it to capture, get this error.
I've tried it with my rift off, and fully unplugged and pc rebooted. no luck.. vr is enabled in settings.
unity 5.2.1p3

Unable to render VR Panorama, please, TURN OFF your HDM device like Oculus Rift.
UnityEngine.Debug:LogError(Object)
VRPanorama.VRCapture:LateUpdate() (at Assets/VRPanorama/VRCapture.cs:720)

Update:
I was able to get it working again by rebooting twice without the rift attached. Then booted with it attached and on, until in Untiy where I turned it off. Not sure if that means there are bugs or it was a one off bad experience. I'll update if I have issues again. Thanks

Update2:
I'm back again with the error. I can repro it, if I try to render with the Rift DK2 off, I get the error. If I turn the DK2 on, try to render...get the error...turn it off and immediately run the game, it works. Seems to only trigger a render after a failed attempt not if its off to begin with.
Also, I tried using the 4k (milk setting) to create a GearVR video but it won't play in the gear past a few frames. It plays, then freezes. No audio added.
thanks!

 

It looks like bug on Unity 5.2. side, that has completely broken VR device detection (or on Oculus SDK 0.7 side). Results are inconsistent. I will have to remove detection part of a script for a future version. I'm sending you an updated VRCapture.cs that has to replace the old one (in a VRPanorama folder).

You can't play 4k stereo video on GearVR. This setting is for uploading videos on MilkVR service - and not to be played on GearVR. But if you have rendered already 4k video with that setting, you should use a button "Encode best quality video for Gear VR". Note that this works only with sequences at 25 - 30 fps. Gear VRs hardware can't handle higher framerate than this.

 

Great thanks! that fixed it. Yes, I was referring to that button, I'll try that! I am getting amazing results, your latest version is dramatically better than a month or so ago, thanks!
I'm wondering, how high can I take the bitrate before I hit the ceiling of video resolution versus the GearVR screen rez?
Raising it obviously makes a big difference, whats the trade off? just file size or can the gear only handle a certain max bitrate?
The 60fps GearVR created videos sure do seem 60fps, or were you just referring to MilkVR being max 30fps?

 

For a best compression bitrate you should stay at 20 000.. no more than that as any greater value won't give a better quality compression, it will only make a file larger. But it depends also on your content inside video. Gear VR recommended compression is also 20 000.
Gear vr has some limits, so you can have 2k video at 60fps. But when you go up to 4k x 2k (that is what a button does) it hits limits of gear VR, so you have to use a lower framerate.
BTW, Milk VR is a streaming service and they do a recompression from 4k x 4k videos. This should be a standard for uploading video on their service. And they will downsize that video. But I'm not sure, here in Europe, Milk VR isn't yet available so i've never used it.

 

Thank you! again, loving this.
Do we have the ability to move in this tool? My game's controls are all locked up.
Is it to avoid any ill feelings from controlling the camera?

 

Hi, I purchased this asset yesterday. It was awesome and really helpful!
But I met some problem with resolution.
1. Once I type a number of resolution like 1920, other than resolution presets, it doesn't work. There is only a estimated time in a small window, however, the blue bar doesn't move. Is that mean I can only render those three preset resolutions?
2. I am trying to make a short 360 animation with Unity and Maya and planing to use Full HD resolution, 1902*1080, as the final. The thing is I can only render 2048*1024, the ratio is 2:1 but not 16:9.

 

You can only render a power of two images (for now). But it can be surpassed with some coding, by resizing later. (or you can use video editing programs to resize it)
But I don't know why would you want to use HD resolution? Most players for VR panorama movies use a power of two sizes as a standard resolution.

 

Sorry, I didn't quite understand your question. Do you mean controls in unity or in panorama player that you are using?
You can't and should move with controllers in unity when rendering panorama (because panorama can't be rendered in realtime). But if you want to move a camera, you should make an animation and attach it to a camera.
On the other side, if you mean to move inside your player, you simply can't do that generally with panoramas. You can only rotate, but not move, as with any panorama you have always a one point of view.

 

Hi,

This looks like a fantastic asset but I'm having the same problem as HeadTrip on Saturday. My Oculus Rift isn't attached to my PC but I get the error:

Unable to render VR Panorama, please, TURN OFF your HDM device like Oculus Rift.
UnityEngine.Debug:LogError(Object)
VRPanorama.VRCapture:LateUpdate() (at Assets/VRPanorama/VRCapture.cs:720)

Could you please send me the updated copy of the VRCapture.cs script so I can see if that resolves the issue. Many thanks.

 

Hi! Awesome tool! Thanks so much for providing this for us. I am having a problem with the speed thought. For some reason, when I run my video after its compiled, it runs REALLY fast. Any ideas please?
Thanks

 

Ah ok, so in a FPS game, I cant control the camera unless I set up a fake path to simulate it...thats what I meant, similar to a path but I control it.
Is there a way to trigger the panoramic capture at any point? I'm a playmaker user and I'm able to trigger off the panoramic using a key, but it always starts capturing from the start of the level, not the point I'm currently at. Makes getting panoramics that are deep in a level impossible...
thanks!

 

For now, you can only render image sequences from certain frame (by setting Resume From Frame value or by activating a VR Capture script on that certain frame and setting a max frame number value - counted from start). But you can't encode this sequence to video 8you would have to use some other tool to make a video). A new version will have this option, but I'm not sure yet how to make it clear and easy to use without complicationg whole interface too much. If you have any ideas how it could work on a user side, you're welcome

 

No, because your scripts will probably go crazy, and anyway, you wouldn't be able to do it realtime without getting a jerky and unstable movement. For now, we are limited by horsepower of current graphics hardware and cpu-s and you have to treat VR panoramas as game cutscenes.

 

Ok, makes sense, thanks for the explanation! I have to say thank you for creating such an easy tool to use, really helps us spend more time being creative and no time stitching and encoding. I've been a stereoscopic photographer for years as well, and this is might be the most powerful creative tool for 3D Ive had yet. It is also going to be an unbelievable way to share my game environments. Sincerely, thank you.

 

I tried the "encode best quality for GearVR" button, using rendered frames from the Milk 4k setting.
The video it creates is 4096x4096 @ 30fps, but shouldn't it be creating a 4096x2048 video?
Otherwise, where would I set it to this rez? Or do I use the GearVR preset and it will upscale? The current resulting video wont play...how do I author a 2k x 4k stereo@30fps?

"For custom videos, the recommended resolutions and frame rates are either a 4096x2048 video @ 30 FPS or a 2048x2048 video @ 60 FPS. For bit rates, we recommend at least 20 Mb/s"
https://support.oculus.com/hc/en-us...ng-Your-Panoramic-Videos-in-Oculus-360-Videos

Update:
I was able to create a 4kx2k by using the GearVR preset then the convert to best 4k button, resulting in a 4kx2K video. Looks good but its showing stuttering artifacts in the oculus360 player, the 2kx2k 60fps video doesnt have this. Wouldn't we want to source a 4kx2k for this, instead of upscale a 2kx2k? excuse my noob questions...I'm sure there's reasons
thanks!

 

In reality, if you start with 4k rendered sequence, it will downscale that one - if a start file is 2k, it will upscale (but you don't want this).
You can use your custom resolution for gearvr by editing a VRPanoramaEditor.cs
Search for this string inside script:

Code (CSharp):

1.  -vf scale=4096:2048

and replace numbers with your custom resolution. Try something like -vf scale=3480:1920
Tha fact is that there are two versions of Samsung Note4, and 4kx2k is a best that they can decode but they probably hit a limit of hardware decoding - thiis can result in some artifacts. So you should try by lowering framerate to 25 or changing resolution to 3480:1920. This artifacts also depend on your video content, if there are more or less movement - there are many variables that can influence GearVR performance. I will probably insert a support for custom resolution resizing in a next version.

 

There are some goon news for all those that have video capture devices.
I'm working (right now testing) on a VR Panorama RT version (RT means Real Time). It will be able to render your panorama directly on screen in fullHD. This feature was requested by some of you, and with some good capture devices (like Blackmagic intensity pro ) or capture and streaming devices. Haven't tried it yet with Fraps or VLC. but it sould be possible to use them also.

Features:
-Capture your gameplay from your monitor directly in FullHD (2k) or dual FullHD - Full HD per eye (requires two capture cards)
-Preview your panoramas in realtime inside editor (and find best settings for Environment Distance)

Requirements:
-FULL HD video capture hardware (or screencapture software - performance depends on your computer configuration and capture software)
-Good graphics card (gtx980Ti, Titan or SLI configurations)
-For full screen streaming it has to be played from compiled player in fullscreen

 

Would a 180 degree stereo mode work well or better for higher quality scenes where you dont need the full 360? just curious...thanks

 

Hi,
I met some problem when rendering with VR Panorama. Every time trying to render from my own scene, I can only get images of my skybox. The only way I can get everything in the scene is rendering from the Demo Scene within the downloaded asset or dragging the Demo Scene's preset camera into my own scene. It's so weird. It can't render correctly by any other ways like 'right clicking and choosing VR Panorama Camera' or 'adding script to existing camera'. Only skybox in the images all the time. BTW, 'UnityEngine.Debug:Log(Object)' shows in Console. I have no idea with coding so that I can't fix it by myself.
This is what in script: Debug.Log (Time.realtimeSinceStartup)
Another thing is that why does it always start from the 4th frame. If I want to render the first 200 frames, the first images I got must be named by img00004.jpg.
I am looking forward to your reply. Thank you.

Alice.

 

Set your VR camera to deffered rendering. this should correct your problems with sky. New VR panorama doesn't use depth buffer for its rendering (this is due to using a depth buffer requires much VRAM memory, and lowers a performance). Next version will correct this "issue" by defaulting to deferred rendering. There is no feasible reason to use Forward rendering path: with VR panorama you can go crazy with quality settings and use them really at high levels and many shadows.
For a question of 4th frame rendering: unity requires 3 frames for full rendering pipeline initialisation. This means that unity will render a first black frame, a second with lights, and only a fourth with global illumination. So the first ones are a "garbage" that we don't save. This is unfortunately a Unity limit and there can't be done much.

 

No, 180 would be the same with some small performance improvements. But without any visual improvements.

 

Thanks for this plugin.

I'm however confused about its capabilities. In your description, you mention:

Does it mean that I cannot take a 360 .mp4 video and output it through Unity to a VR Headset?

I'm looking to see if it's possible to use your plugin as a playback solution for 360 videos, loaded dynamically into Unity, with Audio.

 

This plugin isn't a 360 player - but a renderer. You use it to MAKE 360 videos, and not to PLAY them. For playing, you have to use some existing 360 players or code one by yourself.
Making it simple: think about camera and TV. You can use camera to make a movie, but you need also a TV to play it. They are two different things.

 

I understand, thank you for the clarification. I was hopping it could do both.

 

I am making stereoscopic 3D panorama renders, can i use this asset to watch it ? or is this only for VR 360 video.

 

Hello. I just bought this Asset and I'm getting the "please turn off your HDM device" error code. Can I please get a new .cs file? Thanks!

 

Sent you in a PM.

 

I will be submitting a new VR Panorama 0.7b update to Assetstore. It will bring some new features like VR Panorama RT component, that lets you render 360 panoramas onscreen in realtime (*it requires a fast hardware)

Version 0.7b
-Added VR Panorama RT feature that lets you render 360 panoramas onscreen in realtime (requires a fast hardware)
-Added VR Compositor component for managing multiple layered cameras.
-VRPanorama now defaults to Deffered rendering path, as this gives a faster and more optimised rendering.

Bugfixes:
-Fixed RAM leaking when creating Mono Panoramas.
-added Workaround for incorrect VR device detection.
-other smaller bugfixes and optimisations.

After this one, we will exit a beta state of plugin.
TODO list before final 1.0 version:
-Support for Side by Side rendering (currently in works, VR Panorama RT already features SBS stereo mode)
-additional scripts for managing multilayered cameras that are placed in different locations (for now, all multilayered cams must use a same position)
-In-game single frame 360 panorama screencapture by using VR Panorama RT.

Also, from a VR Panorama 1.0, VR Panorama 360 will be dicided in 3 different packages: "VR Panorama 360 PRO", "VR Panorama 360 RT" and "VR Panorama 360"
VR Panorama 360 RT will feature only realtime onscreen rendering.
VR Panorama 360 will feature only capture/movie possibilities, without Realtime rendering.
VR Panorama 360 PRO will feature capture/movie and Realtime rendering.
All users that are currently using VR Panorama 360 will be automatically upgraded to VR Panorama 360 PRO.

 

I love this plugin. It is clearly made with love and a ton of hard work. Thank you!

Would it be possible to borrow the updated "VRCapture.cs"? I'm seeing the same 'unable to capture due to connected VR device' error issue.

One issue: we can get about 12 seconds captured before any of our computer's crash, likely hitting the max of the 16GB of RAM. We just bought a new computer just for your plugin in hopes to capture longer sequence, but wondering if this is normal or you have any tips. We're capturing at very heavy settings (4k @ 30fps, highest quality, and 20mbps).

Really excited about previewing and the RT functionality mentioned. Thank you for the auto-upgrade to Pro!

 

Are you using mono rendering? Mono rendering had one nasty bug (corrected in latest version that I'm sending you, the one that will hit asset store in few days) of RAM leaking (while stereo worked just fine).
Also, you don't have to use highest quality for 4K, for most of cases it's really ok to use that slider set to 16 (this will give you a nice 2X supersampled antialiasing).

 

Hi,
Any solutions for -> Unable to render VR Panorama, please, TURN OFF your HDM device like Oculus Rift ?

I have the latest version and I am not using oculus rift (because I am using my laptop not my desktop) and even removed processes (oculus rift configuration utilty). But I am still getting this error.
My setting:
-Mono
- png
- resolution:2048 (youtube 2k)
- encode video enabled

Thanks.

 

Activate "Virtual Reality Supported" checkbox in Player settings.

 

I tried both "Virtual Reality Supported" check box enabled and also disabled. In both cases I am getting -> Unable to render VR Panorama, please, TURN OFF your HDM device like Oculus Rift.
I am using the latest version of unity 5.2.2f1.
Just for your info: I didn' t have this issue in the previous release of Unity 5.2.1

 

I'm sorry. This is a unity issue, I'm also desperated for this problem, and completely unable to understand when this bug happens in a new version. I think there are some kind of interaction with OVR sdk and Unity.
New version (awaiting for Asset aprooval) doesn't do anymore this checking, so I'll send you this one right now in PM.

 

Thanks !
I will try with the new version you send me and let you know.

 

It works now. Thanks !

 

Hi,

before buying, I would like to know if this would work with this free asset (MK Glow System).

I'm asking this because there's a free alternative to this asset (360 Panorama Capture) which doesn't work with that one. I think it is because that asset relies on a script that you add to the main camera.

What it should show:


What it shows:


The problem is the text lacking the glow.

Thanks.

 

Hi Clisto, yes, it works with mostly all Image effects. Here is an example of a MK Glow demo scene rendered with my plugin:

 

Sold! Thank you very much for the quick and quality response.

 

Hi there.
Can your plugin output 60 fps videos even if the scene runs at 10 FPS during runtime.

 

Your plugin is amazing. Thank you for such a useful asset!
I am currently using it on a commercial project which would not have been possible without it.

 

Yes, it should output it without any problems. Nice thing about it is that you can use really high quality settings and render some things that are impossible to render in realtime (like using many lights, image effects and complex particle systems). You are safe as long as you have enough video ram. Rendering 4k panoramas can require as much as 2 giga of VRAM (VRAM usage depends on rendering quality). It's best to use gfx cards with 4giga of VRAM for 4k renderings. I did some really heavy renderings with 20 million triangles, more than 50 shadow caster lights and heavy image effects without problem.

 

Hi Olix, good to see you are so active with customers, when using VRCaptureRT the the top third of the left side of the screen is just showing up as pink if in SBS and the top third of the top image is pink if stereo. Any idea why?

 

Hey OliVR,

I have just bought this plugin and I am very exited to get started using it. I ran into a problem on my first render.

NullReferenceException: Object reference not set to an instance of an object
UnityEditor.DockArea.OnGUI () (at C:/buildslave/unity/build/Editor/Mono/GUI/DockArea.cs:264)
UnityEditor.EditorUtilityisplayCancelableProgressBar(String, String, Single)
VRPanorama.VRCapture:CounterPost() (at Assets/VRPanorama/VRCapture.cs:1103)
VRPanorama.VRCapture:LateUpdate() (at Assets/VRPanorama/VRCapture.cs:728)

I hope this is a known issue and can be solved easily. Thank you for your time.

elhongo

 

Hi troyfury, this is strange, looks like there are some problems with shaders (are we speaking of pink that happens with uncompiled shaders?) Could you attach a screenshot of your problem, and another one, of your camera setup? Also, does this happens in your scene, on in a demo scene?

 

Hi thanks for quick reply, it happens in both Demo and my scenes, the pink is indeed the uncompiled shader pink.

 

Strange. Never had this type of error with VRPanorama. What is your Unity version? Have you tried importing it into an empty project?

 

I am using version Unity 5.2.2f1 (64-bit). I am going to retry with a new project and if that doesn´t work I will try to reinstall unity. I will keep you informed.

Thanks!

 

Could you try importing into an empty project? For some reason, looks like one of materials used for rendering is broken. But I can't understand why. Also, does it render fine with VRCapture.cs?