Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

[RELEASED] VR Panorama - render 360 stereo videos

Discussion in 'Assets and Asset Store' started by olix4242, Jun 25, 2015.

?

What would you like to see added in a future version of VR Panorama?

Poll closed Mar 23, 2016.
  1. Audio Capture Support

    40.4%
  2. Side By Side Stereo rendering

    28.8%
  3. Automatic 360 panorama metatagging for Youtube H.264 Video Export

    30.8%
  1. kevincfy

    kevincfy

    Joined:
    Oct 12, 2012
    Posts:
    16
    @olix4242 hey there - been using your asset to compile some panoramas but I'm unable to adjust image effects through script. I have a blur image effect (standard blur optimized) that I enable part way through. But it never takes effect. If I enable it from the beginning in the editor it works, but I can't adjust its parameters.

    Is this by design? How can I enable image effects in such a way I can change their parameters through code as it's being recorded?

    EDIT: ok, going through the code I see that the camera is duplicated at the start. So if I want to adjust image fx I'm going to have to sync all the cameras in the rig

    EDIT2: Ok, it's hacky but it works. Have created a script that gets added to all the child cams so when a message is received from the main camera they sync the blur. Phew.
     
    Last edited: Apr 14, 2016
    olix4242 likes this.
  2. kevincfy

    kevincfy

    Joined:
    Oct 12, 2012
    Posts:
    16
    and also I'm getting this weird white artifact on higher resolutions (2k and above). It's right below the viewer when viewed in a headset. I tried it on an empty scene with a black background to better show this (look at the white ellipses bottom left):
     

    Attached Files:

  3. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    Hi, good to see that you've managet to solve your first question. Your approach is correct.
    Strange artifact, but I was able to reproduce and to solve it. It will happen alny if you don't have any objects on terrain. You should do this: go to a VRPanorama/Resources/360Unwrapped prefab. In a transform tab, select Y value, and change it to something like -2000. This should do a job.
     
    kevincfy likes this.
  4. reset

    reset

    Joined:
    May 22, 2009
    Posts:
    393
    Hi there

    when will this be released do you think?

    thanks
     
  5. reset

    reset

    Joined:
    May 22, 2009
    Posts:
    393
    Regarding the issue below:

    Seemed to be the lighting data. Once I deleted this and pre-comp'd the lighting again with VR Pana imported as an asset, it all worked fine!

    But had prob with audio. When I capture audio then hit play to render, VR Pana doesn't go into render mode - just plays scene normally. Had to reimport asset and render again.

    Also FFMPEG is kicking up issues about bands and exceeding them - an audio issue - once I deleted the audio file - it compiled the images fine.

    Do you know what this could mean?. I do have some mono audio in the scene.

    Cheers

    ...

    Hi

    your software has me beat ... or my GPU at least :)

    I am in the final stages of my animation and decided to do a test render and this is what I get - see image - this is a stripped back render - trees, gardens etc are missing.

    I have low poly houses and a road - if I turn them all on this is the result. If I turn off the road and keeps houses - all ok, if I turn off the houses and keep road - all ok ... but all together - disaster. Not to mention if I bring in the trees, gardens etc - then it all starts again.

    My scene runs at about 70 FPS on a 560 GTX card - and this is not optimized - but when I use your software it crashes my scene. There is very only one directional light and the shadows on this are at a close distance.

    I just assume my GPU simply cant handle the render - even on Google Cardboard settings.

    Any ideas cos I am at a loss.
     

    Attached Files:

    • VR.jpg
      VR.jpg
      File size:
      180.7 KB
      Views:
      839
    Last edited: Apr 17, 2016
  6. blindmansays

    blindmansays

    Joined:
    Oct 27, 2014
    Posts:
    3
  7. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    Hi, didn't tried it yet (this youtube functionality is few days old), but looking at a specifications page it should work without any problems if you are exporting 4.0 audio.
    How to export 4.0 audio from Unity?
    -First thing to do is to set your hardware audio configuration to 4.0 speakers. In windows you shoulkd choose audio device able to play 4.0 audio. Most audio cards support multispeaker audio, but not all does it when outputing via analog output. You should choose output that supports it. If you have HDMI output on your GFX card, it's best to use that one as it always supports multichannel audio. This is because Unity by default won't let you use multichannel if there isn't any hardware supporting it. Also, almost all audio configurations has to be done via aiudio drivers, and not all audio cards use the same approach.
    -once you have set up your system audio to be able to play 4.0 channels, you should set your unity to 4.0 audio (Under Project settings/audio Default speaker mode set to QUAD.
    -Just capture your audio via Capture audio button in VR Capture component. Afterwards capture your video. Once compressed, use a youtube 360 injection tool, and upload file to youtube.
     
  8. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    This is the default behaviour. If you start scene after capturing audio it will capture audio again when you press play (when using play button, it defaults to last action). No need to reimport asset. Just click on a button Render panorama and not on a unity play button.
     
  9. Hayato

    Hayato

    Joined:
    Jan 24, 2014
    Posts:
    1
    Hello I have tried and failed to making spatial audio to Youtube VR 360 video.
    My flow is :
    1.Set PC audio output to 4 speakers.
    2.Set Unity Project settings/audio Default speaker mode set to QUAD
    3.In every sounding objects set audio source/Spatial Bland to 3D (value=1)
    4.Capture Audio
    5.Render Panorama
    6.Use Youtube 360 injection tool (Spatial Media Metadata Injector V2), set Spherical , 3D Top-bottom , Spatial Audio
    7.Upload to Youtube and the spatial audio didn't work.

    I try to use mediainfo tool to check with those files ,
    The wav file do have 4 channels :

    But the mp4 file shows like below :


    And the MilkVR example MP4 files that support 5.1 spatial audio (You can download from https://milkvr.com/portal/content/content_specs )
    show as :


    So should it be a problem during encode to MP4 ?
     
  10. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    Hi, I've said too early that VRPanorama should support Youtube spatial audio, but unfortunately this isn't a case. Taht was a early statement, before I've done some serious research.
    -There isn't anything wrong with your settings and you have exported correctly your 4 channel audio. When you run Metadata injector, it should give you the info about 4 existing channels and it should run correctly (at least it does on my system).
    -MediaInfo gives strange/confusional result about channels as that file has 4 channels and it's not quite clear why it says 2 channels / original 4 channels.
    -You can check that your video actually has 4 channels by opening it in Adobe Premiere or some other video editing application that supports videos with 4 channels.
    -If you upload this video on Youtube, you will also notice that audio actually sounds differently (more spatial) than played from media player. That means that youtube recognised 4 channels, but, I wasn't able to notice any change when rotating view.
    Here comes a funny part that breaks everything:
    -After reading info on youtube site, it appears that Youtube requires Ambisonics audio (First Order Ambisonic (FAO) audio channel layouts ). Ambisonics is a complex encoding algorythm (signal encoding, not compression encoding) that is used when placing real physical microphones. As it seems, it can be created only by placing real physical microphones into a real world, and it is used for recorded sounds, and not for generated sounds. Also, some 360 cameras can record this sound type.
    Main problem is that Ambisonics specifications are a little bit confusional as there is more than one standard, so I'm not sure what specifications youtube uses.
    Possible solutions:
    -Create or use some existing spatialiser plugin that would let unity to record everything like in realworld. This is a difficult task as I can't find any usefull documentation.
    -Wait till Youtube implements standard surround layouts (I think it should done this, as this system, as it is, is too limiting).
    I will keep investigating and see if I can get some more information.
     
  11. mrabrehman

    mrabrehman

    Joined:
    Apr 29, 2016
    Posts:
    2
    Hello,

    I am really in need of the answer.
    I am looking to make an animation in 3D wherein I will make a construction activity (for e.g. showing brickwork: by placing bricks one by one on a wall to complete a brick wall).This will allow the audience to look how an activity is performed on site.

    Now i need to render this whole animation in 360 VR so that when a viewer sees this through google cardboard/Gear VR, he/she feels like standing on a virtual site environment where activity of brickwork is being performed LIVE ( so a movie in immersive 360 showing task being permed).

    Can i do it through VR Panorama Renderer for Unity?
    I would really appreciate a response on this. Any suggestions to make this work???

    Regards
    Abdur Nasir
     
  12. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    Hi, yes you can do it with VR Panorama, if you know how to use Unity and know how to make animations. If you are new to Unity, than it will be a really hard task.
     
    Last edited: Apr 29, 2016
  13. mrabrehman

    mrabrehman

    Joined:
    Apr 29, 2016
    Posts:
    2
    my idea is to import animation scene from maya in unity and then use ur add-in. Lets see what i get!
     
  14. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    Try to import it, and see if you can manage it to work before buying a VR PANORAMA. If you can render and lunch your animation as you want, than afterwards it should work also with VRPANORAMA.
     
  15. AdbC99

    AdbC99

    Joined:
    Jan 18, 2012
    Posts:
    53
    Hi, having problems with the latest version. It renders the models in a scene, but not the terrain. I've tried playing about with the environment distance; but it crashes when I set it to 6 or 7.
     
  16. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    It shouldn't ever crash by changing this value as this is only a simple transform.

    Also this looks strange. Are you using some custom shaders on terrain? Try by activating "Use Depth Buffer" and see if a problem persists.
    If it persists, let me know some more info. GFX card specs, screenshots of a VRCapture component and your camera, and Hierarchy view, attached scripts and enything that could be important.
     
  17. pangyarbo

    pangyarbo

    Joined:
    May 9, 2016
    Posts:
    3
    Hi,I want to output 180 degree VR video.
    It can be play on "180 X 180 3DV" type in Milk VR.
    Is it possible to do that?

    Another question,
    My GearVR can't read 4096 X 4096 60FPS video,so I want try 2160 X 2160 60FPS.
    But it can't work if I output 2160 X 2160 video,not use 2048 X 2048.
     
  18. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    Hi. No, for now you can't output 180 videos. You could use 360 video and crop it to a 180 video (but in some expetrnal program). But I'm not sure what are the real setting for Milk VR (MilkVR Isn't available here in europe, so I can't try it).


    You can't use 4096 60fps. For making a highest resolution for Gear, you should render it as a 4k video (deactivate "Encode h.264 video after rendering" button) and when you have your sequence rendered, you should click on a "Encode best quality video for GearVR from 4k sequence". It will resize to a best size (3840x2160). Be sure not to use any framerate that is larger than 30fps. (60fps 4k video can't be played by Gear VR hardware).

    Right now I'm working on a new version of VR Panorama that will be able to do other custom resolutions and it will have some serious performance gain in speed.
     
    marcipw likes this.
  19. SAOTA

    SAOTA

    Joined:
    Feb 9, 2015
    Posts:
    220
  20. Tubbritt

    Tubbritt

    Joined:
    Nov 30, 2015
    Posts:
    49
    Hello.

    I was wondering if you could help with me a problem I've run into. When I try to render a 360 video of my game, moive textures in my Game don't render normally at all. They play back at normal speed during rendering which means when rendering is finished, the movie texture frames were all captured wrong.

    Any idea's how to resolve this?

    Regards
    James.
     
  21. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    Hi James,
    I don't know what are you using for playing Movie Textures. There are some problems with some plugins, and also with Unity Movie Texture. If you are using Unity Moviee texture, if I remember well you should use it without audio. Right now I'm writing from mobile, and I can't do a search, but if you look here in forum, I think that you can find a solution (can't promise, because I don't clearly remember if this was discussed in private or here). Main problem is that MovieTexture has to be changed so that it respects unity Time.timeScale.
    http://docs.unity3d.com/ScriptReference/Time-timeScale.html
     
  22. Tubbritt

    Tubbritt

    Joined:
    Nov 30, 2015
    Posts:
    49
    Hello.
    Thank you for your reply.

    I'm using “AVPro Windows Media” but it seems it has the same problem as you described. I'll contact the developers to see if they can fix this.

    Thanks a million for your support.
    James.
     
    olix4242 likes this.
  23. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    I've already made a script that solves (hacks) AVPro problem. Let me search it and I will get back to you in minutes. Only problem with this script is that you have to use HAP codec (for some reason it doesn't work with other codecs).
     
  24. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    Here it goes. Just attach this script to your video, and set a framerate (it has to be same as in VRPanorama).
    BTW, I suggeest you to contact AVPro, as they should correct this issue (me and others have already sent a request, but for now there was no solution).
     

    Attached Files:

  25. Tubbritt

    Tubbritt

    Joined:
    Nov 30, 2015
    Posts:
    49
    Hello.
    I contacted AVPro and needless to say, they confirmed it doesn't support time scale. I replied asking for them to consider implementing this feature explaining why it's needed and so relevant to anyone wanting to render 360 video. Sadly I didn't get a reply to that email. I can only hope they are busy experimenting and are not just going to ignore my email.

    Thanks a million for the script you uploaded. I'll certainly give that a go. I don't mind using HAP if that's what's needed. If I hear anything back from AVPro, I'll post it here.

    Kind regards
    James.
     
    olix4242 likes this.
  26. MalboM

    MalboM

    Joined:
    Apr 29, 2013
    Posts:
    66
    Hello!
    Is it possible to have a version compatible with unity 4.6?

    thank you in advance
     
  27. Antidotenow

    Antidotenow

    Joined:
    Dec 23, 2012
    Posts:
    6
    Hi OliVR,
    Is Curved Wolrd compatible with Panorama VR ?
    Regards,
    Patrick
     
  28. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    Hi, unfortunately it can't work with unity 4.xx as it relies on functionalities that exist only in versions 5.xx. But you can always try to importa your project into 5 or later and be able to render. Sorry about that.

    I don't have an access to a curved world asset. But I've looked a little bit on it's description and I think that it shouldn't have any problems with VR panorama. If it supports rendering to multiple render targets (don't see any reason why it shouldn't) you should be fine. I will try to contact a developer and see if we can test compatibility.
     
  29. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    Some updates on future version of VR PANORAMA RT that will go online really soon.

    -speed improvements. On some systems up to 20% - 30% for VR capture (hard to go above this number as JPG or PNG compression requires CPU and HD performance that isn't possible to optimise more)
    -resolutions not tied anymore to power of two resolutions.
    -VR Panorama RT module that renders 50% faster (system dependent improvements). Right now I'm able to render Full HD at 60 - 120 fps on Titan x.
    -RT Panorama adds also a aditional speed optimisations by adding a possibility to render at lower resolution regions that aren't of interest (back, top, bottom or sides - can be configured). Thinking about making this option adaptive. This tech is somewhat similar to foveated rendering. Hope to have this feature as soon as Facebook implements LIVE 360 videos. Not sure yet what will be requirements from side of Facebook, so can't promise yet a really functional implementation for live streaming.
    -New interface improvements, no more popup window, but preview window in scene view with statistics interface. This is a step required for a non editor functionality. So yes, you understood well, VR Panorama goes from editor implementation to a full application/runtime implementation on desktop platforms.
    -aditional temppral dithering algorithm that can be used to avoid color banding artifacts in final movie.

    Experimenting for future:
    -realtime encoding to mp4 stream. It will have less quality than a standard encoding.
     
    SAOTA and reset like this.
  30. reset

    reset

    Joined:
    May 22, 2009
    Posts:
    393
    Wow cool.

    Will this go up this week?
     
  31. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    Hi, after testing I can confirm You that it works without problems with Curved World ;)
     
  32. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    Not sure that I can manage to finish everything this week. But I promise, it will be finished really soon.
     
  33. PROTOFACTOR_Inc

    PROTOFACTOR_Inc

    Joined:
    Nov 15, 2009
    Posts:
    4,052
    that's awesome!!
     
    olix4242 likes this.
  34. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    That looks cool but it looks like this has to be implemented into an engine.
    What is cooler is a new simultaneous multi-projection (looks like ansel is based on this one) that coild really boost a VR Panorama rendering performance. We will see when it becames available.
     
  35. pangyarbo

    pangyarbo

    Joined:
    May 9, 2016
    Posts:
    3
    Hi,
    I want to know how to set "Environment Distance" this value correctly more details.
    I have the outdoor scene,but forcus on some characters which close to camera.
    I set Environment Distance "3",when I look sky I feel not comfortable to my eyes.
    I also try to change "IP Distance",but I don't know which value I need.
    I want more comfortable to look but I don't know what Environment Distance really effect.

    sorry to my bad English,I hope you can see.
     
  36. Deleted User

    Deleted User

    Guest

    Hi, thanks for the awesome plugin!

    I just used the plugin in the demo project for generating a 360 pano video (Equidistant Mono Mode).
    But it is weird that the rendering result only contains one face of the cube map, as seen in the attachment.
    I read all the problems proposed here but it seems that no one met the same problem as me.

    Depth buffer is used, and deferred rendering path is applied.
    I am working in an old PC with a poor graphic card (GT420M). The version of Unity is 5.3.4f1 (64-bit).

    Look forward to your reply :D
     

    Attached Files:

  37. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    this value is more important for close objects and not for objects that far away like sky. What are you using for viewing? Gear VR, Oculus, Vive or Cardboard?
    3m as a environment distance with a IP distance of 0.061 - 0.066 should work perfectly.
     
  38. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    I'm afraid that your GFX card is simply too weak for rendering panorama. I don't know what is your VRAM amount, but probably it can't allocate necessary memory for its RT textures. You can try to set lower resolution and quality/performance value.
    But anyway, you should have a decent graphics card with at least 2 giga of VRAM to be able to render larger resolutions and better quality. I suggest you to try it on some more recent hardware.
    Please contact me with your invoice number, and I can send you a newer version of VR panorama, it features some VRAM optimisations and uses less memory.
     
  39. pangyarbo

    pangyarbo

    Joined:
    May 9, 2016
    Posts:
    3
    I use Gear VR.
    I think maybe it's more related to IP distance,I'm trying another value.
    I hope I can find the value that I can see character near me clearly,and look far sky comfortable.
     
  40. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    IPD value shouldn't be changed much. It should always be in range between 0.061 - 0.07 as this is a distance between your eyes and sholuld be a constant value.
    You should also check that your scene is properly scaled (1 unit = 1 meter in real life). But, without seeing a footage, it's hard to tell what is a problem.
     
  41. howardt12345

    howardt12345

    Joined:
    Jan 4, 2016
    Posts:
    31
    Hello!
    I am having trouble with the realtime rendered
    The RT doesn't seem to be saving any files...
    And I have this thing that reacts to live audio in my scene and it appears as if the renderer isn't rendering it...
     
  42. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    Panorama RT can't save any files. RT module is made for hardware encoding, so you should stream it from your graphics card to the input of a video capture card. RT Panorama does only a neccessary transformation to 360 panorama format. Fro now, it's impossible to render and save video or image files as this would require too much processing power. Future version will also feature this option, but it will have less quality than normal render quality with VR panorama.
    If you in need of rendering live musical visualisation, I suggest you to record all movements and transform them to Mecanim animation. You can do this easily with a Scene Motion Capture tool (free):
    https://www.assetstore.unity3d.com/en/#!/content/19622
    and than, once you've captured your scene, you can render it vith VR Panorama Capture.
     
  43. howardt12345

    howardt12345

    Joined:
    Jan 4, 2016
    Posts:
    31
    Thanks so much! :D
     
    olix4242 likes this.
  44. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    If there are some questions about workflow, please don't hesitate to ask. If I can understand your needs, maybe I can help you more.
     
  45. howardt12345

    howardt12345

    Joined:
    Jan 4, 2016
    Posts:
    31
    I'm wondering, how do you make spatial audio using the renderer? Is it possible with the audio settings on Stereo? My audio doesn't seem to work that well in Quad or Surround haha
     
  46. howardt12345

    howardt12345

    Joined:
    Jan 4, 2016
    Posts:
    31
    Hello there!
    I have ran into another problem with this plugin.
    upload_2016-5-24_22-40-11.png
    Why is it that in play mode (or rendering mode when I use the VR pano cam), the cubes that are clearly visible in the scene window aren't being shown in the play mode?
    Even in the camera previews of the independent cameras from the VR rig shows that it's seeing the cubes, so what's happening here?
    Edit: I animated the cubes as well, so...
     

    Attached Files:

  47. morquar

    morquar

    Joined:
    Apr 23, 2014
    Posts:
    2
    Hi, I have just bought the asset, you have some video tutorial for showing how to use it. I just want to show a 360 video and assign some key functions for replay and pause.
    Can you help me?
    thanks
     
  48. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    This is because RT Panorama doesn't use Depth buffer and this can create problems with some transparent shaders.
    There are two things that you can do:
    -Edit a VRCaptureRT.cs script: find these lines inside a script:

    Code (CSharp):
    1. flTex = RenderTexture.GetTemporary (qualityTemp, qualityTemp, 0);
    2.             llTex = RenderTexture.GetTemporary (qualityTemp, qualityTemp, 0);
    3.             rlTex = RenderTexture.GetTemporary (qualityTemp, qualityTemp, 0);
    4. ..... and so on
    There should be
    and change Zeros to 24 to meet this format:

    llTex = RenderTexture.GetTemporary (qualityTemp, qualityTemp, 24);
    This will enable depth buffer and your problem should be solved. But I'm not suggesting to use this tecnique as it slows down rendering (and this is why it is missing here while in VR Capture.cs you can set it to enable Depth buffer.

    Other solution is to use objects that write to Zbuffer in caskground. If you place a black cube with inverted faces around your scene, you will be able to render without problems. I'm suggesting you to use this fix, as it will keep a better performance and help you use less Vram.

    In a new version, I will enable a possibility to write to Zbuffer as anew version is by itself more performant.
     
  49. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    Sorry, didn't quite get your question. Could you explain more?
     
  50. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,962
    Play a video? VR Panorama is used to esport videos, and by itself it doesn't feature any Video player functionality.