[RELEASED] VR Panorama - render 360 stereo videos

Hi, nice plugin!

I'm looking for a really easy drag n drop solution for importing 360 movies into Unity, for VR rendering. Is this possible with this plugin, or is it only for exporting movies?

Cheers,

 

@olix4242 Have you ever found a solution for the problem @vjyourself encountered?

The capturing didn't work and the console was flooded with this warning:

null texture passed to GUI.DrawTexture
UnityEngine.GUIrawTexture(Rect, Texture, ScaleMode, Boolean, Single)
VRPanorama.VRCapture:OnGUI() (at Assets/VRPanorama/VRCapture.cs:1370)

I am using Unity 5.4.4f1 and I have the SteamVR package imported if that matters. It's a fairly large scene that I can't share. I removed all SteamVR components from the scene for troubleshooting. Didn't help.

The capturing worked in a simple test scene with just a plane and an animated cube. So it must be something scene specific.

Sean

 

For console spam see this.
[RELEASED] VR Panorama - render 360 stereo videos
as for other problem, you should probably disconect your Vive and see if it works.

 

Sorry, could you explain a little bit more where you got stuck and what you are trying to do?

 

Really love this asset!! But I'm running OSX and can't get the video output :-/
Any idea when this functionality will be available to Mac users?

 

It works on OSx. Just be sure that your privilegies allow you to run external exacutables.

 

Sorry, I didn't see this one. No, VR Panorama is for rendering 360 movies, not for playing them.

 

I have successfully tracked and corrected this issue. Looks like garbage collector doesn't correctly clean up temporary render textures, so I've added some code to clean up those textures and free VRAM and RAM. Actualy, this was happening also when rendering long animations, but to a smaller extent. Now it should be optimized to a maximum.
I will submit a new version to assetstore tomorrow.

 

I have added this type of seamless mode to VR Panorama for anyone who want's to use it. It is still experimental feature.
From a version that will be released in few days, you will be able to use it by selecting HQ seamless stitching option. Notice about this feature:
-On my tested system, it is 5-10 times slower in rendering than usual VR Panorama mode.
-It workss mostly with all image effects (that were supported in VR Panorama) but there are some exceptions. For example, in this mode you can't use Motion Blur. There are probably some other third party effects that might won't work, but for a moment I didn't find them.
-Some effects like Bloom could have a better results in this new mode.
-It doesn't support Alpha Blending.
-For now, it works only in Top/Bottom stereo (SBS mode will come later).

 

that solution work best if the video start to playing from the start and unlooped, but sometimes dosen't work fine, the solution for me was this:

Code (CSharp):

1. using System.Collections;
2. using UnityEngine;
3. using UnityEngine.Video;
4.  
5. public class VideoPlayerScrub : MonoBehaviour
6. {
7.     private VideoPlayer videoPlayer;
8.  
9.     private void Awake()
10.     {
11.         videoPlayer = GetComponent<VideoPlayer>();
12.     }
13.  
14.     private void Start()
15.     {
16.         videoPlayer.Play();
17.         videoPlayer.Pause();
18.         StartCoroutine(SyncVideo());
19.     }
20.  
21.     IEnumerator SyncVideo()
22.     {
23.         while (true)
24.         {
25.             yield return new WaitForEndOfFrame();
26.             videoPlayer.StepForward();
27.             videoPlayer.Pause();
28.         }
29.     }
30. }

works fine regardless the time when the video start to play or is in loop.

 

Hi

I just downloaded your product, and I'm using it for unity version 5.6.0.f1 with daydream

So far I'm really impressed with how easy it has been to get it up and running. I only need it for stereoscopic 360° photosphere generation.

I managed to get it working somewhat, but for some reason I seem to have these odd lines on the outputed image. Do you know if there is a way to remove these? Or is it a case that this is required for software to read the image? I just need the images to be upload to Google play so if its part of the image then I guess thats fine. Otherwise could you tell me how to remove it?

Thank you. Example Image line: https://gyazo.com/9514f4376c4ff6630eb50142e8fdc652

-Sat

 

Hi,
why this lines are there was explained on a previous page. They are there becaise of alpha interpretation. You can get rid of them in few ways:
- by rendering into a JPG.
-by opening PNG file in photoshop, and converting transparency to layer mask. Then deleting alpha mask and resaving.
-by waiting next version of VR Panorama that introduces manual settings for alpha (in few days)
-by contacting me in private with your invoice number (I will send you a new version after that)
Hope this helps,
Oliver

 

New version is online!

 

Hi Oliver,

Thanks so much for the reply, I went with the JPG and it worked great, I appreciate the help.

-Sat

 

Try out a new version that went online today. Now you can use straight export n PNG
One suggestion:I'm always suggesting an export in JPG, as it is much faster than an export in PNG (different compressions require different exporting times). JPG is much faster and lighter (as it's fast to compress on CPU, and due to its size, its faster to write to HD).
PNG should be used only when you have to do some aditional color correction, and you don't want to loose color information.

 

@olix4242 The Asset Store didn't show an "Update" button for you asset. Why? I still went ahead and downloaded it and deleted the old version before. The console spamming still happens and capture doesn't work, too. I assume this must still be the old version I downloaded?
How can I clearly identify which version of your asset I am using? There is no readme.txt, just the PDF which states v1.9 which is probably the version of the PDF, but not of the package.

 

What version of unity are you using? To download update you have to use minimum 5.5 version.

 

Omg, sorry olix! I was so used to working with versions 5.5+ I totally missed this project was done in 5.4.
Seems to work now. I still don't have an understanding of the ugly seams, but I I remember you explained it elsewhere.

 

@olix4242 Ok, I looked through the docs and one of your answers in the review section of your asset.
You said the seams were inevitable for stereo 360 movies. Then why am I getting these horrendous, flickering seams when I try to render a Mono 360 movie (i.e. Equidistant Mono)? Capture mode is set to "Animation Capture".

My settings:


Sean

 

Could you send a screenshot of a rendering?

 

Turned out to be caused by "Virtual reality support" re-enabling itself all the time. I thought checking "Disable VR Tracking" in the VR Capture component would be all that's required to do, but the capture was obviously still captured in stereo even though I set it "Equidistant Mono".
Maybe it's a good idea to include a warning in the inspector when the option is set to "Equidistant Mono", but "Virtual Reality Supported" is enabled at the same time. The SteamVR SDK re-enables VR Supported automatically unless you manually disable it before hitting start or through a script.

 

One probable solution to try is to set your camera Target Eye value to "None (Main Display)". I didn't test it as right now I don't have Vive with me. If you can test it, and give me confirmation, I will add this feature into next version.

 

Hi

I'm working on creating a VR Panorama still of a fairly complex but stationary scene. It is linear/HDR/deferred/Englighten with multiple post-processing scripts attached to the camera.

I've attached the VRCapture script to the camera I want to capture and I'm getting blocks of dark pixels, looking a bit like shadow artefacts. See image here:

https://drive.google.com/open?id=0B0HJORhfMvneb1VYOFJjNFc0SUE

The black areas don't appear in the normal camera view. If I save the image as a PNG then the pixels have 0 in the alpha channel too.

Any suggestions on how to debug / fix this would be appreciated.

Another note, by default the camera does not fill the whole screen (viewport rectangle) but is cropped to a 16:8 window. If I don't fix this before sending to VR Panorama, then the capture goes wrong. Might be worth saving/restoring viewport rectangle after duplicating the cameras.

Finally, when I trigger the capture, I get the mesage "0 image files updated" "1 files weren't updated due to errors" but I don't know where these messages come from and I seem to get the panorama file saved anyway.

Thanks

Jules

 

With further investigation it seems to be related to the post processing scripts on my camera, I have:

• Anti-aliasing
• Bloom
• Tonemapping
• SSAO
• DoF
• Color Correction Lookup
• Noise And Grain
• Vignette and Chromatic Aberration

If I mess with the parameters of the Depth of Field (DoF) script then the artefacts appear to change. If I disable or delete the DoF script from the camera, then the capture goes black.

If I set the Depth of Field Focal Distance to 1 or less (my default is 3.56) then the black marks go away.

It also appears that turning on/off a number of these post-processing effects has no impact on the final render (chromatic aberration for example).

I'm running with Unity 5.6 and I believe the latest release of the plug-in (23/06 - though doc says 1.9).

Thanks

Jules

 

Hi Jules,
this is probably some problem with image effects.

What post processing are you using? From your description it looks like you are using old legacy image effects from standard assets? Is this true?
In that case i suggest you to switch to new Stack post processing - it is powerfull and optimized, and better to be used with VR Panorama (also because it supports grater resolutions without having significant impact on performance with VRPanorama) . It is also much better for Unity versions after 5.5 - because 5.6 introduced many changes to rendering flow, so this could be impacting performance.
Other than that, it looks like you are using tonemapping before DOF. This could lead to some visual errors too. tonemapping should be always rendered after SSAO and DOF.

Also, you don't want to use in any case effects like vignette and Chromatic abberation. Beecause of they're nature They don't have any meaning in 360 video. They are bounded to a screen FOV, but a 360 panorama doesn't have any FOV, as it is complete sphere that doesn't have start or end.

BTW, you did a great rendering (if we don't look at those artifacts)

 

Thank-you for the pointers about Stack post-processing. The asset does indeed use the old image effects. Unfortunately I'm working with someone else's asset and it's post-processing effects are already tuned. I am turning off some of the effects (such as chromatic aberration) to do the renders.

If I disable the effects one by one, when I disable DoF then VR Panorama captures black instead of the rendered image. After more investigation, it turns out that if I disable the anti-aliasing then disabling DoF doesn't break it. Any idea why antialiasng (FXAA1 Preset B) might have this affect?

Any idea why the DoF effect creates the artefacts when it is in use?

With disabling Antialiasing and DoF things are working.

Thanks

Jules

 

I think it could have something to do with loss of precision in HDRI buffers and clipping white pixel values. I had similar problems with it time ago (highlights turning black) because NaN values in pixels (pixels that are brighter than max value). This mostly wouldn't appear when rendering to screen, but it appears in high resolution rendering (as VR panorama renders much higher resolution image). Sometimes switching order of effects can solve the problem.
Btw, there isn't any need to use anitaliasing: vr panorama uses its own that is tuned for 360 and you can controll it by speed vs quality value. Also, a DOF isn't really an effect that should be used when making movies for Vr headsets and actually, it isnt even physically correct (as FOV increases, dof decreases (actually not dof, but a circle of confusion) and in theory, a perfect mono panoramic image should be sharp to infinity). Of course, if you make some artistic effects, than its another story.

 

Any updates in regards to compatibility with the Timeline editor?
Thanks so much. I'm totally a big fan of this product and have been for a while now and would love to see this compatibility with the newest Unity update. I'm currently animating all of my cameras in timeline and trying to export via VR Panorama to no avail. Cheers.

 

For now I still haven't used timeline for production, so I dont know all possible workflows. From what I know, it should work. If it doesn't, add VRPanorama cam as a child of main camera.
Also, I would like to see how do you use it, so that I can understand how to implement support for all possible usages.

 

Hey. We bought this plugin to capture the scene similar to your band video demo. We have a setup with video player but the problem we have is that that video playback is like 2-3 times faster than it should. What are we doing wrong?

Here's the video:

 

Onw of the first posts on this page features a script that sgould be used to sync unity video to vr panorama.

 

Thanks!!!

 

Olivr - thanks for the response. Here's how I'm animating my camera's now... Using the new unity Timeline and Cinemachine. Basicaly I'm animating the cameras using Cinemamachine via the Unity Timeline.

(Though Unity has also introduced a "Record" option for exporting video,) I would still prefer to use VR Panorama.

I tried to add VR Panorama as a child but it was like the camera animation was in the wrong space. Not translater correctly.

 

You should be sure that when a VR Panorama cam is a child (of a Main Camera), it should be set to position 0,0,0. I don't see any other possible error if it's set up like that.
This applies if you are using 360 rendering mode. If you are using standard video rendering, than you don't have to use VRCamera as a child, but you have to attach a VR Capture straight on Main Camera (so that it can take all FOV changes).

Is it available? I don't have it (or can't find it) in my unity version. 360 mode should also work with that one - by using VR Panorama VR mode.

 

Thank you, thank you, thank you. I got it working awesome now. For one scene I needed to add the VR camera as a child, and for another, resetting the main camera to 0,0,0 worked. Thanks.

Now.. The only thing I need to figure out is my movie texture.. I used your scrubber code which worked for other scenes when the movie textures started at the beginning. Now I have movies starting midway on my timeline and I need the videoPlayer.Frame to export relative to it's starting position on the timeline and export accordingly. Eg: if the VideoPlayer is activated at frame 221 then the export process would go something like this so that the video frames stay true to their relative position in the timeline....
Eg: videoPlayer.frame = Time.frameCount - 221;

Any tips about this?

Thanks again. This is one of the best plugins I purchased and my go to for exporting video and VR.

 

Hello,

We are using your plug-in at work and we ran into a weird problem with the Ambisonic audio capture : the audio was pitched twice as high as it should have been and lasted only half the time we asked for. So for example a mono voice supposed to last 24 sec would be really high pitched and last 12s.

Our test settings are : Animation Capture/Video Capture/JPG, 24 FPS, 1280/720, Speed vs Quality = 1, Export Ambi 4.0.

We do have the Default Speaker Mode as Quad in the AudioManager

Do you have any ideas where it could come from?

Thanks!

 

Unfortunately, for now this can't be done. This is a limit of unity player, that doesn't have any function that can be used to go to defined frame. Right now, I don't have any idea for some possible hack.

 

Hi, you probably didn't set a quad output in your system drivers (windows or OSX). It has to match setting in Unity. Otherwise you willget a pitched sound (and incorrect).

 

I use Final IK, Steam VR,VRTK and VR Panorama 360 PRO Renderer in my Project.
Export 360 images is ok,and I want to Export 2D Video images,So I change Capture Type to "Video CApture",BUT I got my model`s skeleton wrong and flew around.I guess It look like Render sequence error with Final IK and VR Panorama 360 PRO Renderer.so pls help me and thanks for you reading!

 

I think that you have problem with your skin renderers. I and some users had this happening when using characters with multiple meshes. You can correct it by controlling that all of your character meshes share a same Root Bone set in all skin renderers. This should force unity to update both meshes at a same time.

 

Thanks for your reply.I will try it.

 

Anybody managed to render out supersampled 8K images?
At 8192x4320, any value above Speed vs Quality '8' gives me artifacts on images or freezes up computer.

I have a Quadro K6000 with 12GB VRAM, 32GB RAM, i7-4790 system. GPU-Z reports that at SvsQ '8' around 6GB of VRAM are used, above '10' it uses all 12GB of it.

Windows 10 64bit
32GB RAM
Unity 5.6.1f1 (64-bit)
Driver ver. 385.41
Settings:

 

Hi,
I See that you are rendering a standard video. Standard video is managed differently than 360 video. 360 video uses some serious optimisations by rendering slices of image, and this requires less VRAM (it actually requires 5x less vram).
But this shouldn't be a problem unless you are using image effects. With image effects things get little bit complicated as many image effects are optimized for screen resolution and can use a lots of VRAM for their own buffers. When you are rendering a 8K with 16 supersampling) you are actually rendering a 16k and this means that all image effects will run at that resolution. This also means that if a single image effect is using lats say, 5 buffers at 16k.. you can get an extremely large VRAM usage. Also, some effects aren't made for this resolution, so they can behave strangely, or not as expected - so they have to be tweaked.
I am doing some tests on how to reduce this VRAM usage by applying a tech similar to one used for 360. But I have some problems in supporting all image effects (this would mean loosing support for vignetting, motion blur, and some other specific image effects) Also, this on the other side will slow down video rendering - so I really dont know if this is a good way to take.

 

VR panorama is on a discount of 25% THIS WEEK.

 

Some good news for those that are intended to use new Unity feature - integrated video capture. It integrates without problems with VR Panorama RT module and it can give you a speed boost in exporting videos directly into a video.

 

I wanted to explain a power of using new Unity Video Recorder feature with VR Panorama. Video recorder still isn't included into a unity, but it can be downloaded from Github:
https://github.com/Unity-Technologies/GenericRecorder-FrameCapturerPlugin
Once downloaded, it will give you a new menu item under Tools/Recorder/Video. When you open this menu, you will get a new window where you can set various setting. For this tutorial, let's say that we want to record 360 stereo SBS video in 4K.
-set Selected recorder to UTJ/MP4 (this says that you want to use mp4 output format)
-Set source to GameDisplay (this means that you want to render a screen output and not any specific camera - VR Panorama will automatically be loaded as only camera)
-Set resolution to X2160p_4k (this instructs Recorder to use vertical resolution of 2160pixels
-Set Aspect Ratio to X16_9 (this value will be calculated automatically by Unity - and it will default to standard 4K value from VR Panorama)
All other fields are relative to your project.

Now let's set VR Panorama to render correctly:
Add a VR Capture component to your camera (as you would normaly).
-Set VR Panorama Capture Mode to VR Panorama RT (this means that VR Panorama will render only to screen without using it's own video management).
-Set Capture type to Equidistant stereo (or any other mode that you want; Note that Video mode doesn't work in RT Mode)
-From VR Panorama presets, choose 4K preset (you have to match this preset with Unity Video recorder resolution).
-Set other options that you would set normaly in VR capture. like speed vs. quality and so on.

Now you can render directly to a video file. this new approach will give you some speed improvements as there is no need to save intermediary files and recompress them. And this unity solution is native.
I will be working on integrating it in a tight manner, so you can expect that in future this workflow can be made easy, and that all parameters gets synced automatically.

 

I'm getting odd ghosting on the top and bottom cameras. You can see it clearly around the edges of the stone at the bottom and on the fish at the top. What is causing this?

 

Here are my settings

 

Hi,
What is a distance between your camera and a plane below it? It looks like it's really near to your camera. You should always keep some safe distance in 360 stereoscopic video. Lets say that you should keep a distance atleast something like 50 cm when using new hq rendering. But you should always keep that distance anyway, if you don't want to make people feel a headache. Espetially in a scene that uses objects at large distance.
One way to avoid this artifact for near objects is to set render slicesto max. Now a max value is 18 and play with smoothing value.. but it's easy to make it larger with one script change (but you will have larger rendering times). But as said, if you gat this kind of problem, its likely that a stereoscopy will soffer, and with that one, also a final user.
Also, playing with environment distanece can help. But in your case its better to keep it high as your action isnt on a plane below camera.
Many users of 360 videos use a trick of putting a black shadow under a camera, to avoid any render and comfort issues.

 

After a bit of thinking, because you ahve also fishes that are blurry, set a environment distance to something smaller.