Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

[RELEASED] VR Panorama - render 360 stereo videos

Discussion in 'Assets and Asset Store' started by olix4242, Jun 25, 2015.

?

What would you like to see added in a future version of VR Panorama?

Poll closed Mar 23, 2016.
  1. Audio Capture Support

    40.4%
  2. Side By Side Stereo rendering

    28.8%
  3. Automatic 360 panorama metatagging for Youtube H.264 Video Export

    30.8%
  1. morquar

    morquar

    Joined:
    Apr 23, 2014
    Posts:
    2
    .... :(
    ok thanks...
     
  2. howardt12345

    howardt12345

    Joined:
    Jan 4, 2016
    Posts:
    31
    SAOTA likes this.
  3. julienrobert

    julienrobert

    Joined:
    Sep 15, 2014
    Posts:
    45
    I need to know before buying stuff...
    I'd like to do a real-time capture of a VR scene. I plan to buy 2 blackmagic intensity cards and using VR panorama RT. Did anyone try this and can confirm that's working well?
     
  4. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    Yes, this can be done. With VR Panorama RT you can unwrap your scene into a equilateral format. But you will have to use a good GFX hardware. Of course, all depends on complexity of your scene, your GFX hardware and what are your requirements (resolution, framerate, stereo or mono).
    Btw, You don't need 2 Blackmagic cards - one Intensity Pro 4k should be enough. (one is tested and I'm using them, but if you choose two - I don't know if you can even start two of them together). But a VR PanoramaRT is capable of outputting also to multiple screens. Yiou can see it here in action:

    Also, you will probably have to personalise VR Panorama a little bit for a resolution and your prefered layout - but this can be done without programming by simply editing a layout of a prefabs provided.
     
  5. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    You can record audio with spatializer plugins. But most of them will output only a stereo sound (spatialized). They ussually don't work as multichannel formats, as they are made to be used with headphones. This means that if you look your movie you won't get any rotation of your audio space.
     
  6. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
  7. howardt12345

    howardt12345

    Joined:
    Jan 4, 2016
    Posts:
    31
    Ah alright haha
    So then what about the default Unity plugin? would it work for that?
     
  8. GamerPET

    GamerPET

    Joined:
    Dec 25, 2013
    Posts:
    367
    Hey Oli!

    Any advice for this?

    I have some issues, not 100% because of the VR Panorama plugin. My main issue right now is that I have a QUAD that is playing a mp4 video on it using the AVPro Video plugin. In normal playmode is fine, but during the rendering the video get's bad.

    One problem is that the video get's played at the normal framerate but when I'm building the screenshots (frames), I'm building them much much slower, so the video get's repeated a couple of times + that it jinkers a bit.

    Any advice on how to deal with this?

    Second problem is with TENKOKU Water, during playmode I see some weird reflection in the water, but this is more related with the water plugin, but again, do you have some advice?

    Thanks Oli <3
     
  9. krroy03

    krroy03

    Joined:
    Mar 4, 2014
    Posts:
    2
    Hi Oli,
    New user of your awesome plugin here. Love it, and everything works great, just having some weird audio syncing issues when the ffmpeg adds the audio to the video file. The audio plays about 2 seconds slower than it should in the video. I could easily fix this in premiere pro or aftereffects myself if need be, since I have both the audio file and the video file, so it isn't a major problem. Still, it would be great if I could get the audio synced up automatically via the ffmpeg plugin to save time, as I would have to manually sync audio on every new render I want feedback on.

    I was wondering if this is an isolated issue on my end, or if you have seen it happen before? In either case, do you have any ideas on why this might happen?

    Thanks!
     
    Last edited: May 26, 2016
  10. GamerPET

    GamerPET

    Joined:
    Dec 25, 2013
    Posts:
    367
    I to had some problems but... At first I thought it was the audio but it was the scene itself. My game actors were moving a little bit slower, but it was a slowing over time and not a fix 2 second delay. I had some actors that I was triggering them with PlayMaker. Turns out that when I was building the video, the Audio Action was being ignored and my states were being played one after another.
     
  11. krroy03

    krroy03

    Joined:
    Mar 4, 2014
    Posts:
    2
    Yep, I figured it could be a problem with my scene too, so trying to debug through that as well. I should probably look into my transition timings and look into the audio transitions when the video is building too, thanks for your reply!
     
  12. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    Hi, this problem was discussed before wirth users having the same issue with AVPro. You should look on page 8 of this discussion and you will find a script i've posted, that (partially) solves this problem. unfortunately, this is a more a hack from my side than a real solution that should be done by AVPro team. So, try it, and please do as others did - write a request to AVpro to solve this problem.
     
  13. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    This is described in a manual. Apparently your audio starts on Awake (with StartOn Awake box ticked). This isn't a correct way to start audio because unity scene actually sttarts on Update and Awake starts before. So you will have to use a script provided with VRPanorama in a folder Assets/VRPanorama/Scripts/AudioSyncWithVRCapture.cs. Add this script to your audio clip, and deactivate Play On Awake on this audio clip. Note, that you have to do this only fro clips that start from a scene begining. Other audio clips that are triggered by script, should work just fine.
     
  14. julienrobert

    julienrobert

    Joined:
    Sep 15, 2014
    Posts:
    45
    Ok thanks! Just bought the plugin, will buy the intensity card soon. If I want to capture a stereo VR scene, I thought I needed two cards because I will have to capture 2x HD outputs at the same time. If I well understand, VR Panorama is capable to output the 2 outputs into one single 4k output through HDMI? That's why I wouldn't need to buy 2 cards...?
     
  15. jbeau3d

    jbeau3d

    Joined:
    Apr 20, 2016
    Posts:
    2
    Hello OliVR,

    I would like to use this tool to export an HDR sequence so I can use that to generate reflections and lighting for prerendering in VRAY. I only see support for PNG and JPG. Is there a way to add 16 bit float support?

    Thank you.
     
  16. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    Unfortunately, there is no support for HDR formats yet. Unity can natively export only 24 or 32 bit images (and also mp4 videos don't support higher depths).
    I'm currently testing a method to export also in 32 bit per channel EXR format - but this is something that can take some time as it requires a lot of work.
     
  17. metallizard

    metallizard

    Joined:
    Apr 10, 2013
    Posts:
    3
    Hi oli, i want to ask before buying your plugins. I want to create 360 video from my scene. Could your plugin do this in realtime / on .exe ?
     
    AndrewHeartnet likes this.
  18. AndrewHeartnet

    AndrewHeartnet

    Joined:
    Feb 7, 2014
    Posts:
    1
    Hi Oli,

    I am actually quite interesting with this post by metallizard.
    We want to build a desktop application (in *exe format) and that application can produce 360 Videos. Is it possible using this plugin without Unity Editor? We plan to preset all parameters for example the quality, and so on.
    Also, I concern about the audio, which is still experimental. Could you tell us more about the audio limitation (if any)? I plan the application produces the 360 video in 10-15 seconds only.

    Thanks
     
  19. rmuk

    rmuk

    Joined:
    Feb 18, 2015
    Posts:
    34
    Just had two quick questions oli:
    1) For some reason it seems like any object in my scene that is marked as "static" gets left out of the render. Is that intended behavior? EDIT: Actually I was wrong about this one, it's anything that is using a "mobile/Bumped Specular" shader. EDIT2: after removing and re-adding the camera object this issue has gone away. Not sure what caused that.
    2) For one scene i'm using different skyboxes for each eye and a two camera left/right eye rig during normal playback. Is there any way to capture that with VRPanorama?
     
    Last edited: Jun 3, 2016
  20. WillemKokke

    WillemKokke

    Joined:
    Sep 6, 2014
    Posts:
    31
    Hi, I've used your plugin successfully in the past, however for the current project I have a separate camera rendering some worldspace UI (to avoid it being affected by some postprocesing effects)

    I thought I could use VRCompositor for this, but it results in the world space UI being rendered into each cube map face, which obviously isn't right.

    What is the correct way of doing this?

     
  21. WillemKokke

    WillemKokke

    Joined:
    Sep 6, 2014
    Posts:
    31
    Oh fixed it by making the UI camera a child of the main camera, just like in the demo scene. Not sure how that changes things but I'll worry about that later ;)
     
    olix4242 likes this.
  22. WillemKokke

    WillemKokke

    Joined:
    Sep 6, 2014
    Posts:
    31
    Actually I do get it now, if it's a child of main camera it will get cloned as part of the setup, and the orientation of it will be adjusted correctly. Nothing to see here folks!
     
  23. julienrobert

    julienrobert

    Joined:
    Sep 15, 2014
    Posts:
    45
    Is there some documentation how to use the VRCompositor.cs script? I think it's what I need because I have one camera for each eye (I'm using layer to show different objects in each camera).
     
  24. WillemKokke

    WillemKokke

    Joined:
    Sep 6, 2014
    Posts:
    31
    Sorry to be the bearer of bad news, but I don't think this is easily possible.

    VRPanaroma creates the eye cameras internally when rendering in Equidistant Stereo mode. So it will basically duplicate the one camera you have the VRCapture script on.

    Having two active VRCapture scripts in the scene at the same time is not possible.

    However to answer your actual question this is how VRCompositor works for me (there is very little documentation about it, I've worked this out from the source code and the demo scene provided)
    • Make sure there is only one top level camera (i.e., move any additional cameras to be child)
    • Add VRCapture and VRCompositor to the top level camera.
    • Add a reference for each child level camera in the VRCompositor CameraLayers property. Don't include the top level camera itself.
    • Make sure the child camera's have the right clear settings (most likely none, or depth only, depending on what you want to achieve)
    Good luck!

    PS: What are you trying to achieve? I've not yet seen a VR experience that works well if the left and right eye are showing something different.
     
    Last edited: Jun 5, 2016
    olix4242 likes this.
  25. julienrobert

    julienrobert

    Joined:
    Sep 15, 2014
    Posts:
    45
    I want to use stereoscopic movie texture inside the scene to incorporate "real" stereo content inside the virtual scene. It works well with the oculus. The challenge now is to find a way to capture this in real-time to create a VR video...
    Will the VRCompositor understand that one camera is the right and the other the left?...

    hum, just realized that VRCompositor doesn't work with VR Panorama RT...

    Will try to use 2 instances of Panorama RT, one for each eye and assigning each to a different display... I'll let you know if it works... might be too expensive for a single computer...
     
    Last edited: Jun 6, 2016
  26. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    No, for now it isn't possible. But this is a feature on what I'm working. But for now it's difficult to make it behave in a stable manner as you can never know on what system it will be used - and you surely don't want to block a final user computer for a long time.
    Let me know your thougts about this, and maybe It can help me to decide what to do next.
     
  27. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    It is quite difficult to make any documentation about this one, as a multilayered setup combinations are really different based on cases.
     
  28. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    Audio support isn't experimental anymore. It should output what you hear. But you have to setup your audio hardware to a configuration that you want. This is due to a unity limitations.
    Be avare that (for now) I can't give any good support for spatialized audio for youtube (First Order Ambisonic (FOA) audio channel layouts)
    . It uses a coding that isn't documented very well, so I don't know from where to start. (it uses a Live recording techniques that are difficult to simulate in engine). But you can export multichannel audio that can be transformed with plugins like this one:
    http://www.matthiaskronlachner.com/?p=2015
    Be aware that this requires a good audio production knowledge.

    Also, any audio that isn't triggered by scripts (bacground music) shouldn't start on awake (default option for audio). For this task you should use a script that is provided with VRPanorama - AudioSyncWithVRCapture.cs
     
  29. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    Contact me in private with a invoice number, I can send you a new version that supports culling masks for both eyes. This should work for you.
     
  30. Realidade

    Realidade

    Joined:
    Mar 9, 2016
    Posts:
    24
    Work in Android?
     
  31. AgentJK

    AgentJK

    Joined:
    Apr 21, 2016
    Posts:
    1
    If the engine captures all frames, but fails on the ffmpeg rendering process, is there a way to restart the render without having to capture each frame again? I see all of them sitting in the folder.
     
  32. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    Of course. There is a button "Encode H.246 video from existing sequence" that does this ;)
     
  33. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    No, panorama rendering requires modern hardware and lots of Vram (1giga at least) to render hi resolution panoramas.
     
  34. unique-forms

    unique-forms

    Joined:
    Oct 4, 2015
    Posts:
    3
    Screenshot 2016-06-16.jpg I've just started using VR Panorama and it's great, useful in so many ways. I have noticed, that the video/frames exported are much lighter and less contrast than the game view. I'm sure there is a reason, is there a way to counter that or something I'm missing?
     
  35. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    Hi, you should use Linear Lighting color space, while you are probably using Gamma in . VR Panorama defaults to Linear lighting and deffered rendering and it's best when used with HDRI lighting. So, you should switch your color space to Linear in Player settings/other.
     
  36. pluMmet

    pluMmet

    Joined:
    Jul 18, 2015
    Posts:
    133
    So we can record from the Unity player without building out the game?
     
  37. Arkasis

    Arkasis

    Joined:
    Nov 22, 2010
    Posts:
    53
    Hi, I'm using VR Panorama and I actually got an issue. I will try to explain it.
    My project is divided on several scenes which are all loaded asynchronously and additively in a main scene.
    I have setup my VR Capture camera on my main scene.
    I put a DontDestroyOnLoad script on the VR Capture camera in order to keep it when I unload the main scene.
    I have modified the code in VRCapture.cs in order to always keep all instanciated used object between scenes (using dontDestroyOnLoad):
    renderHead = (GameObject)Instantiate(Resources.Load("360RenderHead"));
    renderHead.hideFlags = HideFlags.HideInHierarchy;
    **renderHead.AddComponent<DontDestroyOnLoad>();**
    renderPanorama = (GameObject)Instantiate(Resources.Load("360Unwrapped"));
    **renderPanorama.AddComponent<DontDestroyOnLoad>();**
    After this modifications, I don't have any errors but all the pictures taken look like this screenshot :(

    Have you any idea of what i'm doing wrong ?
    Is it possible to make it works with additive scenes ?
    Thanks
     
  38. martire

    martire

    Joined:
    Feb 27, 2015
    Posts:
    31
    Hi!

    When I upload the video to Youtube I'm getting this strange result:


    It's like only 180 degrees..

    Am I missing some configuration?

    Thanks in advance!
     
  39. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    It looks like you are rendering normal video capture. You should use Equidistant Capture (from a drop down menu in VR Capture component)
     
  40. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    Hi, vr panorama uses a a quite complex rig for its rendering, and it looks like some objects get deleted (probably temporary render rextures, not sure). Unfortunately, right now I'm unable to put my hands on a pc.
    My suggestion is to render scene by scene for a moment. Also, contact me to get a new version that has simplified code and should be simpler to implement what you want.
     
  41. martire

    martire

    Joined:
    Feb 27, 2015
    Posts:
    31
    Working now! Thanks!
     
  42. mtanu

    mtanu

    Joined:
    Mar 22, 2016
    Posts:
    15
    Hello olix4242,

    I've been using your VRPanoramaRT component for real-time rendering into equirectangular output into TouchDesigner for a 360 projection - it's working great, so thank you for your work on this.

    At the moment we have two separate workstations: one for rendering the Unity app, and the other for TouchDesigner to map the 360 video to our projectors. I'm using a Blackmagic 4K vidcap card in between to get the video stream from renderer to projector workstations, but we're hoping to bump up the resolution to 8K, and this rules out vidcap card for now.

    What we're trying to do now is to try using Spout2 so that we run both the Unity app and TouchDesigner on one workstation and they share a video texture/buffer between the two apps. There is a Unity4Spout plugin available, but since VRPanoramaRT creates a camera rig with six cameras, I think we're going to need to do some tweaking to make it work with VRPanoramaRT - and I was hoping you could help us figure out how to make the two work together.

    Both Spout2 and Spout4Unity are free to download:
    https://github.com/leadedge/Spout2
    https://github.com/sloopidoopi/Spout4Unity

    Here's a description from the readme file about how to set up Unity4Spout:
    Sending a Spout texture from Unity:
    Setup a camera in your Unity scene: Select a RenderTexture in the 'Target Texture' property to render into.
    Add the SpoutSender component to your Camera:
    Use a sharing name for your texture. (Has to be unique on your system so Spout can provide this texture to other clients under this name)
    Select a RenderTexture. (You have to select the RenderTexture that your Camera uses for rendering!)
    (RenderTexture settings: Color Format: ARGB32, Depth Buffer: No depth buffer)
    Depending on Unity's current DirectX mode and the graphics card of your system you have to change the Texture Format of your Spout Senders.
    DX9(Normal Unity rendering mode): Should work with all modes
    DX11: DXGI_FORMAT_R8G8B8A8_UNORM

    Any ideas or suggestions are most welcome! Thank you.
    Cheers,
    - mars -
     
    olix4242 likes this.
  43. tianya

    tianya

    Joined:
    Jul 19, 2012
    Posts:
    1
    Support mp4 player vr format it ?
     
  44. 3d_Artist1987

    3d_Artist1987

    Joined:
    Jul 9, 2012
    Posts:
    707
    Can record interactivity?
     
  45. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    Hi. No, it works as offline renderer (not realtime). So all movements and actions should be animated/scripted.
    But you could capture your interactive animations with this free tool: https://www.assetstore.unity3d.com/en/#!/content/19622
    And afterwards use that captured animation to render with VRPanorama.
    Or, you could use a VRPanorama RT component to unwarp panoramas in realtime, and use video hardware capture devices (like Blackmagic intensity PRO 4K) to create your interactive captures.
     
  46. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    Sorry, I don't quite understand your question :(
     
  47. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,430
    Thats quite interesting approach. :)
    But are you sure that an 8K texture could be easily transfered without hickupps? I think it would require a realy large bandwidth.
    As for implementing
    You don't want to use this rig for transfering RT texture. You would only have to render into a RT view and then transfer that RT view via spout. ou can easily change one of the "360UnwrappedRT" prefabs by changing target RT texture on their PanoramaCamera objects and send that texture via Spout.
    Let me know how it works and what performance are you getting.
     
    mtanu likes this.
  48. 3d_Artist1987

    3d_Artist1987

    Joined:
    Jul 9, 2012
    Posts:
    707
    Thanks for reply
     
    olix4242 likes this.
  49. unique-forms

    unique-forms

    Joined:
    Oct 4, 2015
    Posts:
    3
    Thank you that's great to know! I knew I was doing something wrong.
     
  50. mtanu

    mtanu

    Joined:
    Mar 22, 2016
    Posts:
    15
    Hi Olix242, I changed the target RT texture of the PanoramaCamera object of the "360UwrappedRTMono" prefab as you suggested, and that did the trick, however, the output is upside down - do you flip the cubemap in the code somewhere? Thanks! - mars -
     

    Attached Files: