Search Unity

[RELEASED] VR Panorama - render 360 stereo videos

Discussion in 'Assets and Asset Store' started by olix4242, Jun 25, 2015.

?

What would you like to see added in a future version of VR Panorama?

Poll closed Mar 23, 2016.
  1. Audio Capture Support

    40.4%
  2. Side By Side Stereo rendering

    28.8%
  3. Automatic 360 panorama metatagging for Youtube H.264 Video Export

    30.8%
  1. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    A new package was submitted today, and it fixes issues on a latest OSX.
     
  2. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    New updated package is now available on Assetstore.
    I'm currently working on a new version that will feature some Audio capture improvements with a support for surround (5.1) sound.
     
  3. ddf

    ddf

    Joined:
    Jul 9, 2011
    Posts:
    54
    The description says that you support Forward rendering but it appears to be forcing it to Deferred and I don't see any settings to indicate that it should use Forward instead.

    Also, the VRCapture component has public methods that make it appear as if capture could be triggered from another script, but the documentation does not indicate how these should be used. Is this actually support or do you intend to support it? I'd love to be able to play up to a certain point in my game and then trigger a short burst of capture.
     
  4. Aranda

    Aranda

    Joined:
    Jan 23, 2015
    Posts:
    16
    Hi, this is a great tool. Thanks for all your hard work!.

    How would you recommend I go about playing back the captured output in a Unity application for a VR headset? So far I'm imagining it involves:

    - Play video on MovieTexture
    - Set up left and right eye VR cameras with planes in front
    - A special shader that samples movie texture based on current HMD rotation value and source eye texture rect. I guess it would need to handle the pano remapping too.

    Does your VR Panorama asset include anything that would help with this?
     
  5. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    Hi, Your'e right. From one of the last versions VR Panorama defaults to Deferred rendering. This is because in deferred you can get better shading, shadowed lights, and overall better quality. Forward rendering doesn't have any pro's over Deffered (while in standard view it has multisample AA, you would have to turn it off while rendering panorama, becuse it uses unnecessary resources, and messes up a rendering, by doing AA many times). VR Panorama has it's own AA optimised for panorama (adjustable with quality/speed slider) so it doesn't really need forward AA.
    I've removed default support, as I didn't see the neccessity for it. But, I didn't think of users that maybee want to make a renderings that are 100% what you see in a game that is made for Forward rendering. If this is your case, you can simply remove it by deleting a line that changes camera rendering type:
    Code (CSharp):
    1. cam.GetComponent<Camera>().renderingPath = RenderingPath.DeferredShading;
    If there is a need, i can expose this as an option in a next version.

    It was WIP function. A new version that is already submitted to assetstore and should be available today or tomorrow, has this implemented. You should be able to start a recording by only activating a VRCapture script at a certain pont. This unfortunately doesn't work yet well with audio capture, but you should be able to make movie. (you can activate VRCapture script by starting with deactivated script, and than activating it via script made by you, or by activating it with a camera animation clip and it will start from that point).
     
  6. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    Best way is to map a movie on two spheres with inverted normals, with a self illumination material. Then position camera at center of those spheres, and set that one gets rendered by left eye only, and the other one with right eye only. Also, you should offset material tiling on each sphere.
    But the main problem is how to play 4k or 2k movie efficiently. As you know, unity doesn't have performant video player, so you hardly can get to reproduce even a resolutions as low as 720p.
     
  7. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    I've made a simple example of how this should be done. It doesn't contain any video playing functionality, but it has a simpe rig that let's you view a static stereo panorama inside unity (by using Oculus) with a unitys own Virtual reality support. For other headsets you should probably change some things (not sure how it's done with cardboard, as I've never used it). It's not a complete solution, but it should provide you with a good starting point.
    You can download it here:
    https://www.dropbox.com/s/3d2rb2lld7b1gr2/StereoPanoramaPlayerTemplate.unitypackage?dl=0
    Hope this helps.

    Usage:
    -import this package into your scene.
    -open a demo scene and look how it's made.
    -Activate Virtual Reality support in Unity. Attach oculus and hit play. you should be able to see a static stereo panorama.
    -If you want to add a movietexture, just add it to a StereoSphere material.
     
    Last edited: Jan 19, 2016
    billyjoecain likes this.
  8. Aranda

    Aranda

    Joined:
    Jan 23, 2015
    Posts:
    16
    Wow, thanks for the fast (and thorough) response!.

    I also found this more detailed description of playing stereo 360 movies in Unity, it takes exactly the same approach that you've described: http://bernieroehl.com/360stereoinunity/

    That's useful information. I didn't realise Unity's playback was so bad. This will probably be a deal breaker for us as we'd hoped to use fairly high resolution videos interspersed with our normal 3D VR content. I also tried the VR video player by http://www.whirligig.xyz/ (they also use Unity) to play our 4K video rendered by VR Panorama, and can confirm that frame rate is a pretty serious problem, even on a fairly high end desktop system!

    Thanks again.
     
  9. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    New version is available (1.1). It adds a possibility to capture and encode audio in surround configurations.
    USAGE:
    -Set a desired audio configuration in Unitys AudioManager (quad, surround, 5.1, 7.1).
    -Set a matching audio configuration in your OS audio section (it has to match the one set in Unity, because unity otherwise defaults to system audio settings). So, if you want to capture 5.1 audio, you have to configure your output speakers to 5.1 configuration in your audio driver section or utility (how it's done depends on your audio device drivers - check it in your manuals).

    TIP. if you don't have an audio device that is capable of playing back on multiple speakers, you can set your HDMI audio output to desired configuration, and set it as a default audio device.
     
    isidro02139 likes this.
  10. TLEGaming

    TLEGaming

    Joined:
    Jan 25, 2016
    Posts:
    12
    Hello, I just purchased the pro version. And its super cool, but I have some questions. How do you get google cardboard to work? I click carboard as the setting I want and I render it, on the screen the cardboard dimensions show up but its the lens are black for some reason. And in the background you can see the scene. Its like the opposite on how it should look. Also how do you get cardboard to work for youtube 360.
     
    Last edited: Jan 25, 2016
  11. TLEGaming

    TLEGaming

    Joined:
    Jan 25, 2016
    Posts:
    12
    Also do you have an tutorials for your product? thx
     
  12. TLEGaming

    TLEGaming

    Joined:
    Jan 25, 2016
    Posts:
    12
    This is what I get when I try to render panorama. img00001.jpg
     
  13. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    Sorry, I don't quite understand whats happening. Could you explain it? How do you use it and what are you trying to render? (some screenshots form editor would be helpfull).
    You can read a help file in Assets/VRPanorama/VRPanorama_ReadMe.txt, it explains a correct usage of this plugin.
     
  14. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    Only player that is performant on Windows system is a AVPro Windows Media and it can handle 30fps 4k videos. I'm sugggesting you to use lower framerate, ussually it's enough and will give you a better performance. Don't go to FPS higher of 25-30. This apparently seems like a thing that you would not want to do, but in reality, as long as your main application framerate has high refresh rate (used for tracking) and video has a lower framerate, you will get a better experience because a GPU and CPU has to work less. Think of 360 movie like watching a normal movie at cinema (at 24 fps).
     
  15. TLEGaming

    TLEGaming

    Joined:
    Jan 25, 2016
    Posts:
    12
    Thank you for your quick reply. I will send some shots shortly. What I want to do is do a youtube 360 but with google Cardboard. Is that possible with your plugin?
     
  16. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    Yes. Siple way is to export a stereo panorama for youtube. Then read this page how to upload that file on youtube.
    https://support.google.com/youtube/answer/6178631?hl=en
    You will have to use 360 Video Metadata app and inject metatags required by youtube. It's a simple process. Unfortunately, matatagging isn't yet supported in VR Panorama. But I'm working right now on it and it will be available in a next version (so there will be no need to use external tools).
     
  17. TLEGaming

    TLEGaming

    Joined:
    Jan 25, 2016
    Posts:
    12
    Ok great! I got it to work for YouTube 360. Thank you. But my next question is. How can I export to YouTube 360 but with Google Cardboard. This is what happens every time I try to export for any VR Headset, I made a quick video explaining the issues and showing the settings I have too. Thank you so much here is the link.
     
    olix4242 likes this.
  18. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    Hi, I think that your video is encoded just fine. But you probably have some mp4 codec issues on your PC. Let me explain - there are many mp4 codecs, and some of them doesn't support high resolutions. If you are using a capture card (just guessing) your capture card hardware codec is probably made for standard HD video resolutions, and anything beyond that resolution simply doesn't get drawn on a screen, while the data inside video is correct. You can check it by uploading that same video on youtube - it should work there.
    -The first frame in unity is always garbage (actually, first 3 frames) because unity takes some time to initialise itself (render textures, GI, shadows). That's why you getting that strange effect at the begining.
    -For Youtube, it's enough to export stereo video(and set stereo in a metatag app). Suggested settings are: render at 4K resolution (MilkVR standard) and deactivate "Encode after rendering". Then afterwards use a button "Encode best quality video from 4k sequence". This should give you a better result for youtube. Sorry that these thing are right now a little bit confusing, but there were lot of changes on Youtube side and I will probably have to update some functions and button names to reflect those changes. But for now I wanted to concentrate more on some new functionalities as automatic tagging.
    -To activate Cardboard viewing in youtube, just be sure to use latest version of thye app. It features an icon (eyeglasses) for enabling it for Cardboard. I'm not sure right now if you before have to install a Google Cardboard app for this setting to appear.
    BTW, when animating camera, use Animator component instead of Animation component. This will setup correctly your camera. Also, in this kind of animations, I suggest you to activate "Align with horizont" option. This is a cool feature that will keep your camera in a correct orientation for a vr experience even if your camera has wrong orientations looking down or up. It's an anti-puke option :)
     
  19. TLEGaming

    TLEGaming

    Joined:
    Jan 25, 2016
    Posts:
    12
    Thats awesome I am going to try this right now! As for the Encoding is it my graphics card thats the issue is what your saying? I have dual AMD Radeon HD 7900 cards and 32gb Ram. Is it that? Or is it cuz Iam running Bootcamp. Its a Macpro I have and I like using windows a lot for most of my developing and the mac side for app creating and such. So I do a lot on the windows bootcamp side do I need to download something for it. Or is it just the video player I am using lol.. Thanks again so much for the quick responds btw.
     
  20. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    Well, I'm suggesting you to try VLC player. It uses it's own codecs, so you don't have to install or mess with system codecs that can always be a "risky business". Not sure if it's your bootcamp, it should be compatible. I know that on some OSX systems you can't play 2048x2048 videos (but I thought that this is a QuickTime problem).
    I can send you another video that I know works on a PC, so you can check if it works on your system.
    But before, let me know if it works on youtube, because we don't know yet if there is some bug in encoding (I don't think, but we can never know).
     
  21. TLEGaming

    TLEGaming

    Joined:
    Jan 25, 2016
    Posts:
    12
    Ok I am actually rendering out a new one since I deleted the old one. I will check it right now. Thx
     
  22. TLEGaming

    TLEGaming

    Joined:
    Jan 25, 2016
    Posts:
    12
    Ok this is what I got. after doing everything. That was said.
     
  23. TLEGaming

    TLEGaming

    Joined:
    Jan 25, 2016
    Posts:
    12
    Also you where right. the video plays in youtube but not on the video player I have.
     
  24. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    You have to set stereo (Top/bottom) in a metatag app. Also, you have to set on Youtube in advanced tab:
    3D video: Top-bottom/ left on top.
     
  25. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    Also, you don't have to re-upload your video. Just set that value.
     
  26. johnluxford

    johnluxford

    Joined:
    Oct 14, 2015
    Posts:
    17
    Got it working right away, but it looks like a UI mask I'm using isn't masking in the captures. I've tried with the Show Mask Graphic setting checked and unchecked, but same result.

    Here's a link to the output of the Cardboard cameras in Unity, and another of the captures from VR Panorama, for comparison:

    http://imgur.com/a/IOlCO

    Let me know what I can try to fix it. Thanks!
     
  27. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    Not sure how that UI was done. But to me it looks like depth sorting issue, you should check "Use Depth Buffer" checkbox in VR Capture component.
    VR Capture by default doesn't use Depth buffer (in this way it optimises VRAM usage) and this in some cases can lead to problems with transparent shaders, Skyboxes or other cases that rely on depth buffer. By checking "Use Depth Buffer" you can avoid this problems.
    If this doesn't help, than a closer inspection of a GUI scripts or shaders is neccessary.
     
  28. feel__more

    feel__more

    Joined:
    Sep 13, 2015
    Posts:
    1
    Is there a way to invert the 360 video so that you can pan around a central point of focus while the scene is playing? So that when you're clicking around you're watching the scene except the rotation is around this central point of focus?
     

    Attached Files:

  29. TLEGaming

    TLEGaming

    Joined:
    Jan 25, 2016
    Posts:
    12
    Ok so I tried it. Doing everything you said. And I still get the same results. How exactly does one get the Google Cardboard or Gear VR Lens GUI on the video its seems like nothing works it just has a top to bottom split screen.
     
  30. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    Could you pass me your video in a PM? Or put it on WeTransfer?
     
  31. johnluxford

    johnluxford

    Joined:
    Oct 14, 2015
    Posts:
    17
    RE Mask issue: Depth buffer totally fixed it. Thanks!
     
  32. ddfire

    ddfire

    Joined:
    Dec 22, 2010
    Posts:
    53
    does this support linux?
    Thanks
     
  33. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
  34. TLEGaming

    TLEGaming

    Joined:
    Jan 25, 2016
    Posts:
    12
    Yes. Here is the link to the video. http://we.tl/rFVQa1ws20 Thanks so much.
     
  35. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    Hi, I've managed to upload it correctly. This one is in 3D and can be viewed on Cardboard.


    Things to do:
    -inject metatags (stereo on, spherical on)
    -upload your file to youtube.
    -wait till it becomes available as spherical video (I had to wait something like 20 minutes). When you are able to rotate it do and modify it on youtube. If you try to do it before Youtube makes it available as spherical video, it won't work.
    Untortunately, here in italy I have only italian youtube, so I don't know correct names of menu options. You have to go to advanced settings tab, and there is a section for setting 3D at the end of a page. It will show TOP/BOTTOM setting. Enable this one.
    -Your video will be now available as 3D spherical video.

    Try to do this on your uploaded video, and it should work. You don't have to re-upload it.
     
  36. TLEGaming

    TLEGaming

    Joined:
    Jan 25, 2016
    Posts:
    12
    Awesome!! Thank you so much. I will try the same steps on my end. Also will this work with iOS too? Or just Android. Because I notice that on my iPhone 6 plus the cardboard icon does not show up. If thats the case. Then do you have a method or way to export the Youtube 360 Video with the Cardboard Lens GUI already added to the video? That way the users can avoid the no cardboard support for iOS?
     
  37. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    I'm not sure, I think Youtube Stereo still doesn't work on IOS. At least, the last time I've tried, it didn't work. I think they are still developing a IOS version for a player. I'm not sure about other players on IOS, I'm using only Ipad so I can't try them. I know that Kolor should be good, but I'm not sure if it supports 3D 360 videos.
    Bdw, if you like (or don't like) my product, please leave a review on assetstore :)
     
    Last edited: Jan 30, 2016
  38. yojerry

    yojerry

    Joined:
    Nov 17, 2015
    Posts:
    1
    Good day,

    I had an issue with the 360 Video build for Youtube where the top and bottom stitches are reverse. Anyway to solve this?

    Thank you.
     

    Attached Files:

    • bug2.png
      bug2.png
      File size:
      610.8 KB
      Views:
      430
  39. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    You have probably attached some scripts that control camera to your VRCamera (like mouse look) or you have your Oculus activated. Be sure that you don't have any components that modify camera rotations (other than Animator) attached to VR Camera. And your Oculus (if you using it) has to be turned off.
     
  40. billyjoecain

    billyjoecain

    Joined:
    Jul 10, 2015
    Posts:
    9
    I am just hearing about your creation and I want to get started. I'm not a programmer and I don't even play one on TV.

    Where should I start reading?
     
  41. billyjoecain

    billyjoecain

    Joined:
    Jul 10, 2015
    Posts:
    9
    Okay. I bought it and dropped a camera in my scene. I left the settings as they were upon loading it. The words "Render Panorama" are in red, which makes me concerned that I haven't set something correctly. But I pressed it anyway and it looks like it's trying to render. I get the "Rendering" window to open, but I have -35791394 min. -8 sec. to wait.

    What am I doing wrong? It's gotta be something simple.
     
  42. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    It seams that you are using it with some other cameras and scripts for google cardboard (but I can't see what other scripts you have attached to your camera). This probably creates a mess. Look into a demo scene, and how it's done. Try first with that one.
     
  43. treytech

    treytech

    Joined:
    Sep 16, 2012
    Posts:
    1
    Forgive me if this has been asked before, but I have a project that is synced to and basically controlled by an external mp4 video file, being played on various textures throughout my experience with AVProWindowsMedia. From what I can tell, this tool takes much longer than realtime to render each frame, so I could only guess that my video would begin playing when the first frame is rendered, but then would be far ahead of where it should be for each subsequent frame.

    Has anyone tried to render an app that contains an externally embedded/playing video? Off the top of my head, the only solution I can think of would be to add a script that manually advances the external video frame position when this 360 rendering plugin begins to capture the next frame of my app.

    I also saw you have a realtime capture tool, but since this video is the only realtime aspect I need to be captured, I would much rather go with the non-realtime capture method, assuming we can solve this external video problem.

    Any quick thoughts on how to handle this? Thanks for any help you can provide.
     
    olix4242 likes this.
  44. billyjoecain

    billyjoecain

    Joined:
    Jul 10, 2015
    Posts:
    9
    Thank you for the suggestion! I am going to remove all the camera stuff and try again. If that doesn't work, I'll try a demo scene.
     
  45. billyjoecain

    billyjoecain

    Joined:
    Jul 10, 2015
    Posts:
    9
    I removed the camera altogether and had the same problem. Although it does show a new thing in the game view.

    Any ideas why the Remaining Time is negative? Also, I cannot Cancel the render, so I just close Unity every time.
     
  46. billyjoecain

    billyjoecain

    Joined:
    Jul 10, 2015
    Posts:
    9
    I haven't loaded a standard Unity demo yet. I loaded another of my projects and now I can render no problem.

    The only issue is that it's only rendering the skybox. ?

    Before rendering

    During rendering
     
  47. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    Check "Use Depth Buffer".
     
    billyjoecain likes this.
  48. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    This is a known issue with AVpro that doesn't support unity time.timescale.
    Here is a script that lets you do what you want (it works only with a video encoded as HAP. Check AVPro manual for that).
    Put this script to your AVpro wm object and set a correct framerate (it has to be equal to your AVpro video and to your export settings).
    For this feature to be implemented better, you should contant AVpro evelopers, as there is no much that I can do. They have to implement time.timeScale into their workflow.
    >For now, you can use my workaround script.
     

    Attached Files:

  49. billyjoecain

    billyjoecain

    Joined:
    Jul 10, 2015
    Posts:
    9
    YES!

    Okay, now it's time to figure out how to get it so it works in stereo so I can put it on YouTube and see it in stereo! Here are my current results!



    Maybe I'm confused, but I want to view this in stereo on Google Cardboard, so won't it only work if it's side by side, instead of top / bottom?


    *** I see your example above, and it shows that Top / Bottom is good for YouTube upload ***

    Let's see how that works! :)
     
  50. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,513
    Nope, Youtube Side by side is default for normal videos, while top/bottom is default for 360 videos ;)
     
    billyjoecain likes this.
unityunity