Search Unity

[RELEASED] VR Panorama - render 360 stereo videos

Discussion in 'Assets and Asset Store' started by olix4242, Jun 25, 2015.

?

What would you like to see added in a future version of VR Panorama?

Poll closed Mar 23, 2016.
  1. Audio Capture Support

    40.4%
  2. Side By Side Stereo rendering

    28.8%
  3. Automatic 360 panorama metatagging for Youtube H.264 Video Export

    30.8%
  1. DieterLaser

    DieterLaser

    Joined:
    Jul 1, 2017
    Posts:
    5
    I also have flickering and the light is totally different in the rendering. im using unity 2018.3.5f1 and the latest VR Panorama asset
     
    AlanUnity likes this.
  2. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    Please use normal cameras and not physical ones. It looks like Unity doesn't treat them correctly when converting values (that is probably a bug in Unity).
     
  3. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    VR PAnorama discards first 2 frames, as Unity actually outputs garbage for those frames = first frames are always missing GI, reflection probes and shadows. When you play your scene at high speed, you can't notice this, but it appears in a final file. So the only solution was to exclude those frames.
     
  4. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    As said here, turn off physical camera option. This should fix your issue.
     
    Waterlane likes this.
  5. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    I would need to know your render settings and your hardware. You coul be probably running out of VRAM.
     
  6. Waterlane

    Waterlane

    Joined:
    Mar 13, 2015
    Posts:
    149
    That's very interesting and I wasn't aware of it - Thanks for the info.
     
    olix4242 likes this.
  7. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    While working on my Deckard render, I've came across one hack to avoid this, so I will include it also in VR Panorama.
     
    Waterlane likes this.
  8. DieterLaser

    DieterLaser

    Joined:
    Jul 1, 2017
    Posts:
    5
    The Physical Camera option is turned off already, I never enabled that
     
  9. Waterlane

    Waterlane

    Joined:
    Mar 13, 2015
    Posts:
    149

    My system is:
    i9-9900K
    32GB RAM
    GTX2080

    The 4K render was set to 3840x2160 - 30 fps - 1830 frames
    The 5K render was set to 5120x2560 - 30fps - 1830 frames

    Both were Animation Capture/Equidistant Mono/360/jpg(quality 100)
    (FYI I seem to be getting a slight seem line with the 360, where the edges meet - not sure if it's the render of compression after I've recompressed the movie atm)


    If you can let me know what you think is going on that would be handy - If possible I'm wanting to render at 5K, so it would be useful.
     
  10. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    Speed vs. quality setting?
     
  11. Waterlane

    Waterlane

    Joined:
    Mar 13, 2015
    Posts:
    149
    16 :)

    On opening the project now VR Panorama states:
    "VRAM Usage 295MB
    VRAM available: 7715 from 8010"
     
  12. DrMario

    DrMario

    Joined:
    Jun 20, 2016
    Posts:
    6
    Assets/VRPanorama/Tools/HideRig.cs(24,25): error CS0122: `VRPanorama.VRCapture.renderPanorama' is inaccessible due to its protection level
     

    Attached Files:

  13. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    Be sure to import standard characters package.
     
  14. NetyaginSergey1

    NetyaginSergey1

    Joined:
    Jan 2, 2015
    Posts:
    3
  15. disny1234

    disny1234

    Joined:
    Sep 21, 2014
    Posts:
    20
    Hello
    I have just updated the package and capture the hdrp example scene , capture a 360 still image but output is not fine, see in the image
    Please reply soon
    Thank you
     

    Attached Files:

  16. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    Can you give me more info on Unity version, HDRP version, and if there were any opther scripts attached to your camera?
     
  17. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    Yes.
    Can you elaborate? What do you mean by using HTC vive?
     
  18. ShervinVR

    ShervinVR

    Joined:
    May 8, 2018
    Posts:
    7
    Hi @olix4242


    I've been trying to use this tool to capture equidistant, stereo 360 captures of a VR project. Currently the project is on Unity version 2017.3.1.

    The main issue we seem to have is that objects appear to pop, heavily distorting your perception of depth. To illustrate this, we've tried creating a new scene with a bunch of default cubes (all of the same size) placed at varying distances away. We recorded a capture with the following settings:
    Image Sequence Recorder
    • PNG
    • Equidistant Stereo
    • 30fps
    • W: 4096
    • H: 4096
    • Stereo Separation: 0.066
    • Default HQ Stereo stitch settings
    The result of the capture is really jarring. The objects that are further away (and thus appear smaller) almost feel like they're just as close or even closer to you than the objects that are very near you.

    Could this be a stitching issue? Any idea whats going on?

    Any help is greatly appreciated!

    Thanks,
    Shervin
     
  19. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    Can you proveide a screenshots? Are you using any image effects or scripts added to your camera? Is your camera scaled correctly (scale of 1, 1, 1)?
    If you are using HQ stereo stitching, be sure to set your Environment distance to 1000 or more.
    I personally don't suggest to use HQ rendering mode: it is somewhat a standard, and that's why I've included it into VR Panorama (it is otherwise called omnidirectional panorama) - but it has it's problems in displaying correct stereoscopy. It eliminates completely stitching artifacts, but it distorts perceived stereoscopy.
    VR Panorama is made about controlling stereoscopy instead about warying about stitching. A whole idea is NOT to use HQ mode if there is no need for it. You will get some stitch errors, that are mostly visible on 2D screen by pixel peepers, but it actually gives a best immersion.
    Manually tuning environment distance is like setting a focus on a video camera. It can be essential for storytelling and for immersion.
    BTW, there is no any need to render into PNG, if you daon't have any need to do post processing work. JPG will mostly do fine. It can compresso at 100% quality of JPG, and it can't be perceived as lesser quality from JPG if you don't have to boost your contrasts or illumination in video application.
     
  20. disny1234

    disny1234

    Joined:
    Sep 21, 2014
    Posts:
    20
    Hi @olix4242
    I am using unty version 2018.3.5f1, hdrp version 4.10.0 and nothing special i am doing, just importing and trying to capture 360 still image in hdrp, but render is not fine, please reply soon.
    thankyou
     
  21. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    I guess that a new HDRP version broke one of the shaders. Latest version that I've tested was 3.x and it workied fine.
    Unfortunately, HDRP is constantly changing and it continues to break shaders (even Unity's own demo of Book of Dead doesn't work with a new version of HDRP).
    For now, I am sorry to say that I don't have a solution - I will have to install new Unity and HDRP and see what happens - but this might take few days. I hope to have something for you on monday.
     
  22. brudolf

    brudolf

    Joined:
    Nov 15, 2016
    Posts:
    1
    Is it normal in Stereo equirectangular that you see 3D artifacts in the corners where the cubeMap corners were before morphing into equi.?
    When i change to HQ mode its gone, but than i have the problem that even with Env. distance set 10.000 the near stereo is broken.
     
  23. ShervinVR

    ShervinVR

    Joined:
    May 8, 2018
    Posts:
    7
    Hi @olix4242 ,

    Thanks so much for the quick reply! No scripts or effects, and scale is set to 1,1,1.. I've attached a capture of the simple scene here:

    EDIT: Fixed link so it's shareable!
    https://drive.google.com/open?id=13AgYNeMQUj6QmdENStbzy9y0GJ46eJOo

    You mention not to use HQ rendering mode, is that the same thing as HQ stereo stitching?
     
    Last edited: Mar 8, 2019
  24. disny1234

    disny1234

    Joined:
    Sep 21, 2014
    Posts:
    20
    Hi @olix4242 ,

    Thank you for quick reply!
    I'll be very thankful if u could provide some solution by Monday afternoon.

    Thanks
     
  25. VRotein

    VRotein

    Joined:
    May 16, 2016
    Posts:
    4
    I captured the video on a 360 stereo, and as you can see in the image, there's a twist. Is there a way to solve this?
     

    Attached Files:

    • 01.jpg
      01.jpg
      File size:
      240.8 KB
      Views:
      58
    • 02.jpg
      02.jpg
      File size:
      237.1 KB
      Views:
      74
    • 03.jpg
      03.jpg
      File size:
      134.1 KB
      Views:
      63
  26. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    This is how a stereoscopic 360 videos work. If you use objects that are close to a camera and placed on upper or lower zones of a panorama, you will always get nasty distorsions. And this is a reason why most of 360 creators add a blured black area below.
     
  27. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    It's pretty hard to tell from this example (if I don't know how it was setup in a unity scene). Can you render a default demo and see if problem persists?
    p. s. what 360 playerr are you using? Is it set to correct setting?
     
  28. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    I have checked and I can confirm that it works on latest HDRP 4.10.0 and Unity 2018.3.8f1.
    I can suggest you to try deleting VRPanorama folder, and reimporting it because at this poin't it mighgt be that it' wasn't imported correctly.
    If this doesn't work, try importing VR PAnorama into a fresh empty project, then copy its folder into your old project.
    Sometimes Unity serializer can corrupt prefabs or their connections resulting in a problem.
     
  29. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    Lately I was working on a new asset that might be handy for those that need only standard video rendering.
    If you are interested, here is some info about Deckard.
    Deckard is a high quality and cinematographic rendering tool for Unity, and it is available on Assetstore:
    https://www.assetstore.unity3d.com/#!/content/137778
    Features:
    -Real Depth of field
    -Soft Shadows
    -Real Motion Blur for a smooth motion even on cinema framerates
    -Simulation of film and industrial standard real camera sensors color response
    -Superior antialiasing
    -Renders up to 8K resolution but gives perfectly antialiased ond natural image even in low resolutions
    -Supports standard Unity workflow and shaders
     
    mblt2334 likes this.
  30. badradionz

    badradionz

    Joined:
    Feb 2, 2018
    Posts:
    19
    I've noticed after I render panorama that ffmpeg starts to stitch frames together but the video file is always broken. The only option that works is the "Encode best quality video for Gear VR from a 4k sequence"

    Does anyone else have this issue?
     
  31. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    It really depends of what do you mean by 'broken". Gear VR setting is using lower resolution, so it' might be that your file is encoded correctly, but isn't able to play on your configuration. Does your video have any size (when looked in explorer)? What are you using to play a video file? What are your export settings. Do you encode file with audio or without?
     
  32. badradionz

    badradionz

    Joined:
    Feb 2, 2018
    Posts:
    19
    Thanks for the reply, yes the file is in fact broken. I end up with a 2-3mb file usually. Which of course is far too small for a 4096x4096 video.

    The problem is easy to replicate.

    Unity 2018.3.11f1

    Encode H 264 video after rendering is ticked.
    I have tested the Encode H.246 button (this appears to be a typo in the Editor)

    Under the Audio panel, volume is set to 1, mute while rendering is ticked. However I see a box that says Ambisonics audio unavailable. Please, set default speaker mode to QUAD. (My scene doesn't have any audio)
     
    Last edited: Apr 8, 2019
  33. ivabibliocad

    ivabibliocad

    Joined:
    Jan 11, 2014
    Posts:
    71
    Hello, I bought your asset a year go or so but use it a little since then. Now I want to give it a more intense use for other stuff. So I bought new equipment to be more productive... but Panorama Pro doesn't seem to make use of it , so I ended up going slower than before. I have 24gb graphics card, 62gb of ram and 32 core processor. Whenever I render 360 videos I check the resource manager and find out only 1 processor is being used, 0% of the GPU, and 32gb of Ram.
    I know for a fact this is not a unity issue, because when I do the light bake all the processors work at 100%.

    ¿Is it possible to tell the plug via some scripting to take more advantage of system's resources? or is there some other stuff I could try??
    Thanks for the help!
    Image: https://ibb.co/KLw7pHK
     
    Last edited: Apr 11, 2019
  34. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    Well, actually Unity doesn't use multithreading. It can use it only for some functionality, but most things work on a single thread (like in most game engines). So, having 32 cores won't improove your performance much. CPU Clock speeds does improove performance, cores doesn't. Lightmap baking uses multi core but it is actually completely separated from Unity (enlighten). Lets say that a system with 32 cores running on 3 gHz will be always slower wit Unity than a system with 4 cores running at 4 gHz
    When rendering panoram there are few things that are happening, and you should understand where your bottleneck is.
    Step 1: GPU rendering and unity engine - scripts - if GPU is powerfull this will last less than other tasks
    Step 2: Image compression - image compression can be a slow task - it is closely realted to final resolution and export format choosen. JPG files get compressed much faster than PNG files. PNG file compression gets slow when dealing with large images like 4k x 4k or 8k. Most of a rendering time goes into this task and this can't be done faster. If you open a photoshop and try to save a same image, you will probably get even slower execution time on saving.
    Step 3: Writing images to disk. This can be also pretty slow if you aren't using SSD drives. Always try to use SSD drive for storing your images.I see that you are using your operating system HD, but it's much better to use another free disk instead.
     
  35. ivabibliocad

    ivabibliocad

    Joined:
    Jan 11, 2014
    Posts:
    71
    Man thanks for the response! Do you happen to know if unity has plans to take advantage of systems like mine in the future?
     
  36. amoeba_VR

    amoeba_VR

    Joined:
    Nov 28, 2015
    Posts:
    8
    hi im wondering if im trying to do the impossible here and asking too much.

    I have a unity scene in 2019.1.02. Deferred rendering using a 1080ti card i7 PC
    +SSD the scene has an a 3840 x 1920 25fps 360 h.264 video [1500 frames] as a skybox by using the video as a render texture to a skybox.

    In the scene there are 3 audio reactive bits of geometry using keijeros reaktion component.
    and a static camera

    The jobs brief changed from standalone EXE, to can we just do it as a 360 video which has thrown a spanner in the works.

    _______________________________________________________

    Everytime I try using VR Panorama in animation setting I get the full 1500 frames of video, but the skybox video stops at about 150 frames leaving a still frame as a background and the animations that were audio reactive just stop reacting at the same time {150frms}, they keep on animating on a rotate script I use but have lost the audio reactivity.

    [settings] USING ANIMATION CAPTURE/EQUIDISTANT MONO/PNG/3840x1920/1500 frames length+AUDIO SYNCH WITH VR CAPTURE (script)

    _______________________________________________________

    Ive also tried VR PANORAMA RT
    [settings] VR PANORAMA RT/360/RECORD RT TO VIDEO CHECKED/ 1500frms/3840x1920/DISABLE VR TRACKING CHECKED/SPEED16
    VRAM USEAGE 81mb with 6016 from 6097 available

    I am getting a 360 movie capture but instead of 1500 frames long
    the onscreen animation [with reactive elements] is over in 20 seconds instead of 1 minute and at a low bitrate.

    _______________________________________________________

    I understand im asking a lot here but just need to know if it IS possible for a decent bitrate render of the full 1500 frames/1 minute scene before telling the client the news.

    Many thanks
    /sct
     
    Last edited: Apr 16, 2019
  37. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    As for syncing video with VR Panorama, you have to use "AudioSyncWithVRCapture" script that is included in VR panorama. Just add this script to your video player and it will done everything. (note. it wasn't tested on 2019.1.02 but it works on previous versions, if you have problems let me know).
    As for third party scripts, I'm afraid that there isn't much that I can do,you should check if they are supporting unity timescale. (time.timescale).
    https://docs.unity3d.com/ScriptReference/Time-timeScale.html
     
  38. amoeba_VR

    amoeba_VR

    Joined:
    Nov 28, 2015
    Posts:
    8
    hi thanks for the reply, I have been using the AudioSynch with VR Capture script I wrote it in the setttings bit of my mail. I should have included in the post that I attatched it to my video player in the hierarchy that is the content for my skybox render texture. But when I capture using animation capture the resulting video I get is the full 1500 frames I require but both the skybox video and audio reactivity stop after 4 seconds on the render, leaving a still frame of the skybox animation and no audio reactivity of the animations.

    After about 1 minute of capturing using the audiosynch script/animation capture /equidistant mono once the audio stops playing whilst capturing, it keeps on capturing for a further 6 mins
    the resulting video I get has 4 seconds of Video as a skybox and audio reactivity then the skybox is a still and the animations stop reacting.

    I also just tried rendering the scene with no geometry just a video as a render texture/skybox that's all, and during the capture the capture I can see the skybox animation completing, it renders for a further 5 mins then the final output again is a video where the 360 video skybox stops after about 5 seconds and then renders a still. my skybox is 25fps would this be the issue, I am capturing at 25fp.


    many thanks
    sct
     
    Last edited: Apr 17, 2019
  39. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    Sorry, I wrote wrong. You should look into a folder called Tools, and there is a script called VideoSync. You should apply this script to your video.
     
  40. badradionz

    badradionz

    Joined:
    Feb 2, 2018
    Posts:
    19
    Is VRPanorama compatible with Unity 2019.1?
     
  41. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    Yes, it is. But if you intend to use it HDRP, you have to down load a small patch that fixes one issue with it.
    There were some changes in latest HDRP that bug VRPanorama. It worked in betas, but it doesn't anymore in final release.
    This patch should fix this issue.
    https://www.dropbox.com/s/cn0xd2c76b2jr2h/VRPanorama2019fix.unitypackage?dl=0
    Download it, import it to your project, and it should work.
    Be aware that you have to set up a few things on your camera before working with VR Panorama - turn off link FOV with Physical camera. Check that you post processing doesn't use automatic exposure and vignetting (this is turned on by default in Unity example scene).
     
  42. badradionz

    badradionz

    Joined:
    Feb 2, 2018
    Posts:
    19
    I have some issue with stitching, see in this video the zebra crossing lines on the road don't look good at all, same with the shadows from the characters.

     
  43. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    Can you share, please, a screenshot of VR Capture settings that you are using (and unity version number)?
    p. s. i see that you are using my CScape here, so I assume that you have some settings in VR Panorama set incorrectly. I will try to reproduce this.
     
  44. soleron

    soleron

    Joined:
    Apr 21, 2013
    Posts:
    106
    Does it support LWRP?
     
  45. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,452
    Yes, it does. If using 2019.1 you need a patch available here.
     
    soleron likes this.
  46. disny1234

    disny1234

    Joined:
    Sep 21, 2014
    Posts:
    20
    hey OliVR,

    It is not working with unity 2019.1.1f1 with hdrp 5.7.2 and nothing special i am doing, just importing and trying to capture 360 still image in hdrp, but render is not fine, please reply soon.
    You had replied a solution below.
    try importing VR PAnorama into a fresh empty project, then copy its folder into your old project.
    Sometimes Unity serializer can corrupt prefabs or their connections resulting in a problem.

    But it was working in unity 2018 with older version of hdrp.
    Now I am doing the same thing but 360 render output is not proper.

    thankyou
     
  47. Kareeem

    Kareeem

    Joined:
    Mar 1, 2013
    Posts:
    28
    Hey there!

    Any chance of getting rid of the black border when capturing 180° ?

    Thanks
     
  48. I33N

    I33N

    Joined:
    Jun 20, 2013
    Posts:
    4
    Hi!

    I have exactly the same issue.

    My setup is a bit complex, I am trying to do 3D post processing / VFX on 360 stereo footage.
    To do so I use the 360 stereo player to display my footage in VR (with fixed camera position) and I place my objects in my scene using real measured distance from the camera in the real scene. I then fine tune my object placement in VR using stereoscopy cues. This works really fine and when I end up being happy with the placement in VR it's most of the time quite coherent with the 3D scene.

    The main problem arises when I try to re-render my VFX layer in stereo 360. I use the same IPD then the one used for footage stitching but when I overlay my computed equirectangular image to the footage it looks like some objects appears way closer in terms of stereoscopy than what they are supposed to be, but their sizes are coherent with their position in the 3D scene (objects supposed to be closer are smaller than object supposed to be far in my 3D scene).

    It really looks like a FOV or focal issue but it makes no sense in 360 images? And playing with camera FOV doesn't change a thing...

    If I try to re-render the original image I have a strong offset in the final render only in one eye.

    I have been trying many things but no luck so far in rendering something coherent...

    Any clue?

    @ShervinVR Did you end up finding a solution?

    Thanks,

    Ben
     
  49. soleron

    soleron

    Joined:
    Apr 21, 2013
    Posts:
    106
    Thanks I bought it and I will give it a try.
    I am using 2018.3 but I may also use it on 2019.1 for a different project.
     
  50. PandaDump

    PandaDump

    Joined:
    Dec 11, 2016
    Posts:
    3
    Hey there @olix4242

    VR Panorama 360 is working great so far! Working on Audio now. The issue is I'm using a 3rd party plugin: WWise, for 360 audio--so your plugin does not capture the audio properly. My guess is the "AudioListener" is the issue. I'm trying to route the audio capture from the "AudioListener" to WWise's "AkAudioListener" by modifying the VRCapture.cs in order to use that component instead.
    Something like:
    [RequireComponent(typeof(AkAudioListener))]

    Any idea on this? Or if you have any other ideas/comments?

    Thank you,
    Paul