[RELEASED] VR Panorama - render 360 stereo videos

EDIT - Solved, I'd neglected to turn off VR mode under the XR settings in 2017.2

Hi there, my renders are giving unusual results; creating a sort of overlapping double. What might I be doing wrong?

 

Probably your headset is trying to override settings of VR Capture. You have collapsed your camera settings, so I can't see there, but you probably should make sure that no script (or VR camera) is changing a a FOV of your camera)

 

can I use vr panorama with octane render?

 

Hmm.. no Octane has it's own 360 rendering system. Of course, it doesn't have nearly as good performance as VRPanorama. Problem with octane is that it doesn't support custom shaders, so it's most probably that any scene using custom shaders made for Unity won't render as expected.
One tip: Vr panorama doesn't need unity realtime performance - so you can always crank up all visual settings (like shadows resolution, realtime GI resolution, image effects) and render scenes that are photorealistic and dynamic. Even if your framerate goes down to 1 fps (because you are using hundreds of lights and tens of image effects), VR panorama can render it in a decent time.

 

I'm trying to create a routine which does a series of video captures with audio, by loading a desired scene, adding the VRCamera component to the camera with desired settings, once with audio capture enabled and once without. (Reloading the scene and resetting things between.) However, it seems Play mode is stopped on the Editor after each audio capture, preventing this from working.

Of course I can manually Play the project with different settings multiple times, but I'd really like to not have to do that. Is there something I'm missing here that would allow Play mode to continue after audio capture completes?

 

Before rendering, you should set a variable to desired mode
VRPanorama.VRCapture.captureAudio = false;

(where false is for Capturing Video, and True for capturing Audio)
Hope this helps.

 

Right. This is what I've been doing. However, the Unity Editor stops playing the game after each capture, making it hard to automate capturing audio and video for multiple scenes. If it just resumed playing normally after each capture I could do it.

 

For the record, I was able to achieve what I wanted by editing the code for VRCapture so it disables the component instead of stopping the game when finished.

 

Hello!

I'm pretty inexperienced with Unity but I really want to create an experimental experience for the Oculus Rift. I understand that the VR Panorama component is the best way to do it. I hope someone could answer some of my basic questions.

• As a student, can I use Panorama for free, or do I need to purchase it? Can I add it to 2017.3.0f3 student license without problems?
• I want to use Google Tilt Brush for the Oculus Rift to create everything except camera work. Then import it to Unity. I've found a Google Tilt Brush SDK on Github and was wondering whether anyone has experience with combining it with Panorama? (https://github.com/googlevr/tilt-brush-toolkit)
• My goal for Unity is only camera work, nothing else. I want to create something like a zoom in experience (going from human view of some natural area to an ant's perspective to blood cell level to molecular, etc. Allowed to look rather abstract/painterly.). In the end, I'd like to be able to start this 'VR video' and have it be viewable in the Oculus Rift, or Gear VR if need be. Is that doable?

I hope my questions don't sound too stupid, I have to start somewhere. Many thanks in advance to whoever is able to answer my questions!

 

Is it possible to get realtime gameplay? I am wanting to get 360 videos of my VR game.

 

We are currently using the asset for a project and getting weird artifacts with PNGs and JPGs, with or without alpha, all post-processing effects turned off, any pointers why this might be happening?

 

I suspect, if there is a black background some where, stitching marks become visible

 

What is your unity version? Are you using Write to depth buffer in VRPanorama?
This shouldn't happen, but it could if you are using some custom skybox.

 

It was explained many times here It can be done by using RT Module and rendering panorama to screen, and then capturing with external capturing hardware or with screen capture software.

 

VR Panorama is a separate plugin from unity. If you contact me with a description of your project, I can maybee send you a free version.

 

Hello,

Thanks for making this plugin. It's been amazing to use except for some issues I've been running into as I try and render out stereo panorama PNGs with alpha.

I'm noticing there is no alpha when I render a stereo panorama in HQ stereo stitch mode. When I render in non HQ mode the alpha is rendered however there are large stitch lines which cause unwanted alpha bleeding into my images. Is there a way to enable alpha w/ HQ Stereo Stitch mode?

Thank you!
Chris

 

Hi,

I am making a 360° equidistant mono video, but the top camera seems not been rendered and I have a black square on top instead.
Do you have any idea to solve this ?

 

Hi,
I want to take a 360° still image when I run the Unity project with HTC VIVE.
I know the VR mode has to be closed otherwise there will be some overlap.
But I still want to know if there is a solution for that... Thx!

p.s. I tried to control the position and rotation of the VR Panorama Camera but faied

 

You have to set "Target Eye" values (on VR Panorama Camera) to "None (Main display)"
Also, find prefabs (in VRPanorama/Resources) named "360UnwrappedRT2" and "360Unwrapped" and in their hierarchy PanoramaCamera, and set Target Eye values to None. This should force rendering without tracking.

 

I know this happened sometimes - usually it should be enough to reimport whole asset. Looks like unity looses references to objects.

 

Unfortunately no. Due to how unity handles alphas, aphas doesn't work in HQ mode. I'm trying to override this thing, but I wasn't able to find a performant soultion. Probably the best one would be to render it with a green screen.

 

Yes It works! Thank you!! XDD

 

Hello! I could capture the still image when I used htc vive with your help yesterday, and thank you for it again!
Another question is that I noticed there will be an exit the moment I pressed SPACE.
The interface in the HMD was like the image below. After a second it was normal and everything good.
So it this inevitable,or should I do something more?

 

I need some advice for a problem I am facing when trying to render a panorama sequence with the Panorama VR plugin. I hope it is possible to render a sequence without changing too much...

There is this scene here, where you are on board of an airplane. Normally it looks like this:



But the panorama shows this, where you see almost nothing of the interieur and parts of the wings in a weird reverse order of depth.



And another one with a shorter far clipping plane:



The shader I use for the plane (and most other objects in the game) is a custom shader I constructed in Shader Forge. It relies on Unlit/Custom Light Mode, which means I need the Forward Render Path.

I fiddled around with the camera settings, the settings of Panorama VR and the shader I use - and found the following facts:

- I need to disable "Use depth buffer", otherwise the picture is completely black
- the cameras Clipping Planes have some unexpected influence, where lower values for "far" seem to show more, but I always seem to get some weird slice of the scene
- playing with shader sorting order had some influence on what gets rendered, but the depth test seems to be messed up in all cases

This is the shader tree:

 

I’ve never had any problems with shaders made in SF. But I see that you are using some transparency so this could be a key for wrong rendering. I think that this comes from disabled use depth buffer that is, almost always, a must for transparent effects. Are you using some double sided shaders here? It shows black? Absolute black or something that is dark? Also, what is your unity version? What are you using for skybox. I outdoor try disabling a skybox and seeing if this changes something.
I have some thoughts on your cohort and far clipping plane issue: is your scene scaled in a world size? If it is smaller, then this could be an issue.
I would try to make it work with a use depth buffer.. and see what happens.

 

Hey there,

I am trying to capture a video playing on a monitor (using the Video Player component) in my scene while recording with VRPanorama. The issue I'm having is, the playback speed of the video in the scene seems to be playing back at 2x the speed.

any thoughts?

 

https://forum.unity.com/threads/rel...360-stereo-videos.336086/page-19#post-3181969

 

Hey there,
I would love to test before buying or at least see some recent examples of the stereo 360 rendering quality (TB pictures or highres videos) as this is my main usecase and the most important thing.
A also have a scene with a lot of custom unlit vertex shader and transparency and not sure where the limitations are or how easy it is to adapt a finished scene to work with the panorama rendering.

Best,
Phil

 

Thanks for this!

 

Hi! I am trying to capture a monoscopic 360 video, but VR Panorama Camera seems to ignore its components like post processing and more importantly HX volumetric light. Do I forget about something?

Thank you in advance for help!
Sarina

 

What are your capture settings (a screenshot would help)? I can confirm that VRPanorama works fine with post processing and HX so there shouldn't be any problems.

 

Hi @olix4242, thanks for your quick reply. Here is a screenshot, do you need anything more to see? Looking forward to your help, thanks so much!

Best,
Sarina

 

Hi! anybody know if is possible capture image sequence with transparency!? I mean the background is transparent.

Thanks

EDIT: I Found the problem, maybe a bug report!? If you active the Bloom post effect the transparent background get blocked and render with a solid color.

In order to make it works!? you should reactive the Bloom in your post effect.

 

Well, this isn't a bug actually. it is how certain (most) post processing effects work. They aren't made to maintain alpha information as they work on final image - so there is no way to make them work without modifying their code.
Also, for effects such as bloom, you would never get a good result - as a bloom uses adittive (or multiplicative) pixel layering - but any alpha work well only with linear interpolation (LERP) pixel layering.

 

Hi @olix4242 capturing now worked fine using the unity video solution. I stumble upon one very last thing though..

It seems to me the HX Volumetric Light is built like a box around the camera. I've attached two screenshots. Not even sure it's the volumetric light, but I don't get those artifacts when there is no light effect over the video playback.


Thanks so much!

 

Hi Olix4242: Thank you for your input! I tried another plugin and had no problems with rendering panoramas with it, But unfortunately it doesn't support real stereoscopic panoramas. I know yours does.

I went back to trying a few things based on your suggestions and it seems I found the source of my problem: The camera I used had its clear flag set to "Solid Color" but the color had an alpha of zero. Why? I don't know. I tried putting it to 100% and voila: works. Panoramas on my disk. I am happy now!

So, in case someone else has similar problems:

- I am working with Unity 2017 1.0f3
- "Use Depth buffer" has to be enabled
- When the camera clear flag is set to "Solid Color", the color actually has to be solid (alpha = 1)

 

So, I have another minor problem now: I added an "Amplify Color Effect" to the Panorama Camera and animated its properties using a Timeline / Playable Director. The problem is, that the animation is not effecting the camera effect when recording the panorama using your plugin.

Other Objects than the camera are animated using the same playable and while that works fine, control over the camera seems to be cut off for some reason. Any ideas?

 

Well, in case like this, you should know that VR panorama creates various copies of your camera, so that when you are animating post effect values, that animation won't work. Only way to amke this work is to make a script that can copy desired parameters from orignal camera animation (or a script that can broadcast parameters as global variables)

 

Well yes, this is something that works i little bit buggy in unity. I know that there were same problems with some shaders that didn't use actual transparency, but had their alpha value set to zero - and this resulte in strange issues. I think that this happens only after unity 5.5. Looks like unity gots confused, and does something when rendering to texture, that it shouldn't do (manages those elements at transparent, without using transparency).

 

I don't think that this comes directly from HX Volumetric light. It is most probable that it comes from flares or from particles (or some bloom effect if you are using it). Unity flares shouldn't be used in VR, and especially in 360 videos. They are a worst enemy of 360 videos (also when working with real cameras) and doesn't look good. And unity lense flares are even worse, as they aren't made to work well in VR in general.
Also, if you are using particles, be sure that you particles use (in project settings/ quality) "Billboards face camera position" option. This is a must for VR.

 

Hi there,
Is the current version working on Unity 2018.1 Beta?

 

Yes. It works if you aren't using SRP (scripted render pipeline - it isn't tested, so for that one I can't confirm if it works).

 

Hello! Two issues:

1) I'm getting some transparent doubling artifacts in my frames, and I'm wondering why this might be. It seems to happen more with objects that are closer to the camera (i.e.) the bed in the image on the bottom left.



2) I'm not able to successfully export an ambisonic track. I've followed the instructions in the manual and set Unity's audio output to quad, and I've created an aggregate audio device on my Mac that is 4 channels (and I set the audio output and input to that device). The issue is that an error gets thrown within a few seconds of clicking "Capture Audio", and I'm not quite sure what that line in VRCapture means. What's going on here?


Thanks in advance for your help!

-Jacob

 

Hi Jacob, I'm really not sure what could be your problem. It's difficult to understand from this scene, as it is quite "abstract". But, to me it looks like if you activated HQ stitching with wrong settings. Or that there is some kind of initialization bug (strange one), or a screenshot of VR Panorama settings isn't correct. Also, are you using some image effects? Or transparent shaders - decalls? It looks like a bed plane is pretty near to a camera, so this might generate some problems. But not as in your screenshot - this is a way to heavy and covers everything. Do you get any console errors? What is your unity version? Can you try to reimport VR Panorama from assetstore (after deleting it from your project)?

From a screenshot I see that you don't have activated Ambisonics output (but this shouldn't produce any errors - it could only export a normal quad channel file). Are you getting any WAv file for audio, or you simply don't get anything?
Important thing - Be sure that you have an audio listener attached to your camera - a same camera that has VR Capture component. Also, be sure to restart unity after changing audio settings.
p. s. I suggest you not to use quality vs. speed settings that are lower than 8. They can be used for fast tests, but in that case you don't want to use them with a 4k resolution and PNG images. PNG's are slow and are ment for files that need some adittional color correction or compositing. But saving PNGs is many times slower than saving JPG's. It is faster to do a 2k rendering with higher AA settings (speed vs. quality) than a 4k with lower speed vs. quality settings (as most of rendering time goes into encoding images) - but you actually can get a better image quality with a first option.

 

Hi @olix4242! Finally all the previous problems are solved. If anyone is using the same workflow (with HX Volumetric Light and a video sphere) I am happy to share my solutions, there were in the end small details in settings and not very obvious.

I am having one last problem now: VR Panorama does not seem to work with Visualizer Studio (audio reacting) or am I missing something?

Best wishes and thank you for your great support!

 

I don't use Visualizer Studio, but as I can immagine, it analyzes audio data spectrum in realtime. This means that it can work only in VR Panorama RT mode.. but not in a standard video capture mode where time behaves differently. And this is a pretty complex task to solve. You have few options:
-transform your realtime animation into a Mecanim Animation. You could use tools like this one that is free:
https://assetstore.unity.com/packages/tools/animation/scene-motion-capture-19622
But it captures only transform, and not things like colors or material properties.
-Ask a developer to make their plugin compatible with unity time.timescale. This means that it should play audio and visualize it by following a unity time scale - soif you set your scene time scale (in Project Settings/Time) to 0.5, it should be able to play back audio two times slower than normal, and act accordingly to that slow time.
-Use Unity Timeline Recorder Track feature - it can generate animation files from objects in your scene, by recording all their properties. This one is probably a best solution, but it requires some knowledge on using unity Timeline feature.

 

As I have explored a little bit unity Timeline, it should be able to convert your lipsync animation into a mecanim animation with Recorder track set to Animation.

 

Hello,
I have a problem rendering my scene with your asset. In my final render appears a shadowed box marking the cube of 360, like you can see in the attatched png file.
Please help me with this issue. Thanks

 

turn off Eye adaptation, that solved the problem for me.

 

"Hello,
I have a problem rendering my scene with your asset. In my final render appears a shadowed box marking the cube of 360, like you can see in the attatched png file.
Please help me with this issue. Thanks"
It looks also like as if you are using Vigneting. Vigneting doesn't work in 360 video, and shouldn't be used.