Search Unity

[RELEASED] VR Panorama - render 360 stereo videos

Discussion in 'Assets and Asset Store' started by olix4242, Jun 25, 2015.

?

What would you like to see added in a future version of VR Panorama?

Poll closed Mar 23, 2016.
  1. Audio Capture Support

    40.4%
  2. Side By Side Stereo rendering

    28.8%
  3. Automatic 360 panorama metatagging for Youtube H.264 Video Export

    30.8%
  1. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
  2. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
  3. nicolasNablaware

    nicolasNablaware

    Joined:
    Apr 4, 2017
    Posts:
    19
    Could you explain a little more what you are trying to achieve ? I don't understand where I should use StepForward() and Pause() ? Maybe you know some videoPlayer working well with your plugin ? Free would be good but I can buy it too if needed.
     
  4. PatrickFaith

    PatrickFaith

    Joined:
    Jun 28, 2016
    Posts:
    8
    This is a new error, popped up on me when I was testing 5.6f03 with the most recent build of VRpanorama.
    I have tested the files with nuke, and it has nothing to do with the alpha channel on the PNG file. I have been trying to figure out what triggers it but haven't nailed it down yet.

    [most of the time ... the png file has the alpha channel, but sometimes when I bring it into nuke it's pre-multiplied. I'm also getting "grey" screens sometimes when I'm in 8kx8k. this probably has to do with some weird changes in 5.6 and not anything to do with vr panorama. I do have some settings at 4kx4k that seem very stable. my main system is staying 5.5, and that is working perfectly]
     
    Last edited: May 8, 2017
  5. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    I've made asmall script that should work with Unity Video player (5.6).
    Be sure to disable audio on your video when rendering video. Also, be sure that your video has a same framerate as VRCapture..
     

    Attached Files:

    nicolasNablaware likes this.
  6. nicolasNablaware

    nicolasNablaware

    Joined:
    Apr 4, 2017
    Posts:
    19
    Got it working with your help ! Really nice job here, thank you.
     
  7. mikriloulou

    mikriloulou

    Joined:
    Nov 30, 2010
    Posts:
    7
    Hi, I am considering buying the plugin! People in the forum seem very enthusiastic about it, so congratulations!
    I mainly need this for stereoscopic images, but before proceeding with the purchase I would like to ask if I will need to use the cameras prefab included in the package, or could I replicate the settings on my own cameras? I am actually using leap motion and their camera setup is obligatory for VR. So will it be enough to include a script on the leap cameras and adjust the settings in order to achieve the same stereoscopic effect as with your cameras??
    Thanks a lot in advance and sorry if the question has been asked before...this thread is quite long :)
     
  8. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    In some cases (that probably isn't yours) it's enough to add VRCapture component to any camera. But, this applies only to cameras that doesn't have attached any scripts that change transform of a camera. In a case that it has, then it's enough to add a new camera as a child of your scripted camera (or some other object). It is long time that I didn't use Leap Motion, so I don't know how it works nowdays. Also, while this can be done, I don't suggest that you use it this way, as any fast head movements will be reflected also to a VR Panorama camera, and this isn't somethong that you would like to do as it is a sure way to make unconfortable 360 video. In this cases, it is much better to use a 360 camera from a fixed position, or with some scripts with easing (position and rotation lerping).

    VR Panorama has two main wworking modes.
    -offline rendering - and offline rendering means that it doesn't render in realtime. This also means that you have to use animations for moving any objects, as you can't do any interactions with objects (like in your case with Leap Motion). This method produces high quality videos at any desired framerate up to 8K * 8K resolution. But it can take some time as a 4k or higher resolution video compression requires much resources from PC (it mosly depends on a strenght of your PC configuration). Offfline rendeing is also good if you want to make 360 videos on a computer that isn't powerfull.
    -Realtime rendering. This method renders spherical output directly to a screen and it's perfect for realtime applications. But it also has it's downsides: you wll need a powerfull graphics card - something like GTX1080Ti can do it most of a time at 4K resolution. You have to use some third party screen capturer if you are planing to make full HD videos most video capturers like Fraps or OBS do fine with a modern CPU. If you want a high resolution 4k video, you would have to use 4k hardware capture cards like Blackmagic Intensity Pro 4K, as this can't be done with a CPU encoding.
     
  9. mattbuehler

    mattbuehler

    Joined:
    Dec 30, 2016
    Posts:
    21
    hi OliVR,


    I have an issue with the plugin, something that did work properly 2 months ago, but now I'm not sure what has changed.

    I'm rendering a 360 movie and the images have a scaled time: Playing the video shows the animation about 6 times too fast and then all the rest of the video is black frames.
    The animation is 10'500 frames long, but in the 2'000th jpg, the animation is visually finished.
    [The audio timing is correct.]

    What could this be?
    Any input welcome!
    matt

    ps. attached is a screenshot when a sequence is processed.
     

    Attached Files:

    Last edited: May 31, 2017
  10. mikriloulou

    mikriloulou

    Joined:
    Nov 30, 2010
    Posts:
    7
    Thank you for your prompt answer!

    I was probably not clear with my question. I was mainly referring to the process of playing/rendering the 360 videos and images. We don't really need any special cameras' setup such as leap motion's for capturing the videos/images. We can simply use your demo setup.

    So my question is whether your solution includes a special camera setup and a projection environment for applying the captured images for each eye in order to achieve the stereoscopic effect, and whether we will be able to replicate this setup with our own custom (leap motion) camera setup.

    And yes we are mostly interested on a real-time rendering. We are currently including interactive elements in front of a 360 video that is projected in the inside of a sphere, but videos are on a mono format.
    Thanks a again in advance!
     
  11. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    Hi Matt,
    It's really hard to tell as I don't know anything about your project. Are you rendering something with a video texture inside? If you are, and you are using Unity 5.6 video player, then you should look few posts before - there is a script that should be used with that player. If it's something else, please explain how are you animating things in unity (Animatior or procedural-scripted animation?)
    Nothing has changed from that stanpoint in VRPanorama, so it's probably something inside your project that doesn't respect unity timescale.
     
  12. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    Hmm.. VR Panorama isn't made for playing those videos. VR Panorama is for making videos.
     
  13. mattbuehler

    mattbuehler

    Joined:
    Dec 30, 2016
    Posts:
    21
    hi,

    sorry for the lack of info. I'm tracking down one issue that could actually solve it, but I'm not sure if this will be the solution. will let you know!

    [ and no, there's no video-textures, but I'm using Pegasus]

    matt
     
  14. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    Ok, this is probably a source of timing issues. I don't have Pegasus so I can't try, but you should look if there is a function that syncs it to a unity time.timescale. Now it is probably synced to realtime.
     
  15. tgentry

    tgentry

    Joined:
    Feb 11, 2013
    Posts:
    27
    I have an issue which you said was normal.?. Like on the previous page I have black lines. I do not wish to use alpha and I need my skybox because it is Tenkoku. This is in both png and jpg. I guess these are occurring because people wanted alpha masks? How do I get rid of these so I do not have to take it to Adobe. Your plugin use to be all in one and there was no need to go external. I read the other posts about this but I guess I am a little dense and do not understand what I would do if I went into AE. "you have to set Interpret mask as premultiplied with black, and this border will disappear." What is that? How about a check box that would allow the user to choose if he wanted alpha or not.

    Thanks
     

    Attached Files:

  16. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    This hasn't anything to do with alpha channel (if those lines appear also in JPG). alpha channel "problem" appears only in static PNG-s (and not in any video). It is only a preview issue with some image viewing applications - it's not something that is written into the image

    I think that this is a problem of Tenkoku sky system and it's fog/atmospheric scattering that supports only one camera and is made like a screenspace effect - so it generates this shading problem. Unfortunately, there is no way to go around this - If I remember well, you can avoid it only if you deactivate one of those screenspace effects (sorry, but I don't remember what it was - fog or scattering or something like that). I can't give you any better solution, as tenkoku uses rendertexture prefabs instead of dynamic render textures, and it supports only one camera at a time - so, this is something that should be adressed by Tenkoku developer.
    there is no any need to go external. You have to go external only if you have to do compositing with alpha.
     
  17. mattbuehler

    mattbuehler

    Joined:
    Dec 30, 2016
    Posts:
    21
    hey!

    yes, I believe this was the issue. I'm pretty positive it works now.

    in conjunction with your plugin, pegasus must be set to:
    record sound:
    pegasus fps setting must be MAX FPS.
    record panorama:
    pegasus fps setting must be EQUAL to capture output fps in the VR-panorama render settings.


    cheers!
    matt

    ps. I noted that rendering in 'video capture' (UHD4K 3840 x 2160) mode is much slower (> 2x) than equidistant mono (4320 x 2160).
     
    olix4242 likes this.
  18. mattbuehler

    mattbuehler

    Joined:
    Dec 30, 2016
    Posts:
    21
    oh .. and for me, if I have an image sequence, re-record the audio again and then press the 'Encode H264 video from existing sequence', this doesn't actually start the task of creating the video.
     
  19. Jesterofthesky

    Jesterofthesky

    Joined:
    Jun 26, 2014
    Posts:
    1
    Hey, it's been mentioned once before in this forum but is there any possibility of support for stereo cubemap capture, in the style of OTOY's 18432*1536 ORBX images?
     
  20. seanmiller17

    seanmiller17

    Joined:
    Jun 1, 2017
    Posts:
    2
    Hope it will work on my ip camera. These are absolutely great features to attache.
     
    olix4242 likes this.
  21. Unityaware

    Unityaware

    Joined:
    Feb 14, 2014
    Posts:
    86
    I reported this problem before (on the 29th of December 16). Somehow it went away, but now it reappeared although I had not altered anything except of saving and opening the file: Terrain texture is not seen by VR Panorama Camera and appears black but it is seen by the Unity and Oculus VR cameras.

    I had a console error in one, but not in other scenes, which showed the same problems (see image).

    I deleted VR Panorama and the folders and re imported, but the problem persisted.

    mistake.jpg

    Thanks for your help.
     
  22. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    No, and probably it won't be possible anytime soon. For now Unity supports a max texture resolution of 8K. Also, textures as large as that would require extremely large amount of VRAM.
     
  23. tgentry

    tgentry

    Joined:
    Feb 11, 2013
    Posts:
    27
    I was thinking about how jpgs do not have alpha and was scratching my head as well. The lines did look a lot like what 3d_artist1987 posted and I use png for quality. What I found was that Tenkoku works as long as you do not use sunshafts. This was what was causing the weird haze that looked like fog. The black lines were from the post processing stack, Vignette. Guess I will have to figure out a different way of doing that. Thanks for your help and I really like your plugin.
     

    Attached Files:

  24. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    This shoukldn't be related in any way with VRPanorama - as I see that you here have a black, but VR Panorama isn't even activated. What I mean by VR Panorama isn't activated? Well, I mean that VR Panorama works exactly like any other camera (untill it's activated), so If other cameras see those objects, than they should be seen also from VRPanorama. .
    If you have any camera that works, it's enough to attach VRCapture component to it, and it will work as VRPanorama camera.
    I'm guessing that you are having problems with deferred cameras. Check if other cameras are defered or forward?
    If it doesn't work in defered, then you should check your material shaders - as you probably have some shaders that are compiled only for forward rendering (or they have some errors like too many textures or instructions).
     
  25. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    Yeah, that is possible in some cases. But it mostly depends on your VRAM amount, hardware and render settings. Standard video rendering uses normal approach that requires rendering to single texture. 360 approach uses multiple temporary textures and spòlits rendering to multiple smaller frames - and this helps to reduce VRAM usage.Reducing VRAm usage means that some things can work better. This task is higly optimized for 360 videos.
    I was also thinking about implementing it for standard video captures - but I still didn't find an approach that could be compatible with all image effects. I'm still investigating this possibility.
     
  26. JohannesR

    JohannesR

    Joined:
    Jun 7, 2017
    Posts:
    1
  27. mattbuehler

    mattbuehler

    Joined:
    Dec 30, 2016
    Posts:
    21
    Ok, sounds great! Thanks for the explanation!
     
  28. mattbuehler

    mattbuehler

    Joined:
    Dec 30, 2016
    Posts:
    21
    I have an other issue.

    In my animation of 10500 frames (25fps, 7mins), when recording a video (not panorama), at random locations, I get 2 frames that are 'stuck' and flipped vertically and repeated until the end of the animation, completely breaking the recording.
    e.g.
    frame 6900 ok
    frame 6901 = ok
    frame 6902 = 6901 but flipped vertically
    frame 6903 = 6901
    frame 6904 = 6901 but flipped vertically
    ..
    [ See also screenshot of the files. ]


    what could cause this?
    Any input welcome!

    Thanks,
    matt
     

    Attached Files:

  29. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    This looks like a bug of flipped render textures. This can happen in unity from time to time. But it's quite hard to track this issue as this can be a Unity bug, or image effects bug. I had this bug wwith some image effects, but never with VR Panorama, so I think that it would be impossible to reproduce this error. Does it happen always at the same point? If it doesn't I'm suggesting you to try another (updated) unity version and/or deactivate image effects and see if it still persists.
    https://forum.unity3d.com/threads/c...pped-upside-down-in-forward-rendering.415922/

    Another reason for this could be that you have some scripting issues (transforms and rotation rounding problems). What are you using to animate your camera?
     
  30. mattbuehler

    mattbuehler

    Joined:
    Dec 30, 2016
    Posts:
    21
    Hi,

    Thanks for the reply! I'm using Pegasus.

    After rendering the animation out about 8 times, the last time it worked without error. It indeed seems very erratic.. Must be exceptionally hard to track down..
     
  31. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    I have tried to render this water, and was able to reproduce this problem. It looks like water shader (that is quite old) has some problems with handling transparencies (this affect unity 5.6). You should change a water color field and set it's alpha to max - now it has alpha value set to 0).
     
  32. mikenowlin

    mikenowlin

    Joined:
    Jan 11, 2017
    Posts:
    3
    OliVR,

    I'm attempting to record a 360 video using a 2 PC setup, external capture hardware, Vive and VRPanoramaRT capture mode. Everything seems to be working fine except when I try to run the scene, my Vive display exits the game (and enters unresponsive mode) and "Display 1" becomes the 360 view; when I pick up the HMD, it's movement is unresponsive in-game.

    Is something in the plugin conflicting with the way SteamVR runs?

    I appreciate your help, thanks!

    UPDATE: I changed the "360UnwrappedRT2" prefab's camera to have a higher Depth number than my Camera(eye) and changed the Target Eye to "None(Main Display)" and Target Display to "Display 1" [same as Camera(eye)]. This is working how I want it EXCEPT that when I launch the build, SteamVR still goes into unresponsive mode (three dots animating as if something is loading). When I uncheck my gameObject with the "VR Capture" script attached, everything goes back to normal and runs perfectly.
     
    Last edited: Jun 21, 2017
    olix4242 likes this.
  33. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    Hi,
    to make it work with a headset, you should remove the following line in VRCapture.cs script:
    Code (CSharp):
    1. #if !UNITY_5_0
    2.             UnityEngine.VR.VRSettings.enabled = false;
    3. #endif
    This line stops VR usage, and is there from old days to avoid any strange behaviour while capturing to video file - as only lately it became easy to do this type of captures also with a consumer hardware (GTX 1070 -1080 ti). I plan to add an option to expose this value in interface - and a new version of VR Panorama (that will be out after Unite Europe 2017) will have some neccesary scripting tools and features for this kind of task.
     
  34. mikenowlin

    mikenowlin

    Joined:
    Jan 11, 2017
    Posts:
    3
    This worked! Thank you so much. Great product
     
    olix4242 likes this.
  35. MattMurphy

    MattMurphy

    Joined:
    Dec 24, 2013
    Posts:
    110
    Just come across this problem too- doesn't happen when doing stereo, but it does when just doing a normal capture. It's happened in the same place twice now. Unfortunately upgrading to a newer Unity version is not a option at the moment as project is currently locked down to 5.5.2f1

    One other thing I've noticed is suddenly my speedtrees (part of terrain) are no longer affected by wind. If I create a new camera, still, capture a short sequence- I can see them moving like they should but with my main camera that's animated to move all the terrain trees are just completely still, but it used to work fine.
     
  36. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    a problem described abowe was due to a Pegasus system, adn wasn't realted to VRPanorama. How do you animate your camera?
    This is strange, and shouldn't happen. But it could be probably some king of a bug with Speedtree and timescale functionality in unity. I will see if I can track this one -but it should be impossible for me to correct this issue if it is like I think (bug in unity or speedtree).
     
  37. Justin-Wasilenko

    Justin-Wasilenko

    Joined:
    Mar 10, 2015
    Posts:
    81
    Is there a reason or something I need to look out for why videos stop getting encoded after capture? Yes encode video is still checked.

    It was working this afternoon, and now it has stopped encoding.

    Also with YouTube's 180 support, can we also get a 180 capture option?
     
    Last edited: Jun 25, 2017
  38. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    Only things that comes in mind: be surr rhat you dont have any numbers in a sequence name. Other than that, there shouldnt be anything else..
    Gret news for you: I have implemented this feature, and submitted it to assetstore. It will be available as soon as possible (probably monday).
     
  39. MattMurphy

    MattMurphy

    Joined:
    Dec 24, 2013
    Posts:
    110
    Animated using Unity's built in animation system, nothing fancy.

    Is it any help that, speedtree's placed by hand (and not part of terrain) react to wind just fine?
     
  40. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    I have tested right now, and it works just fine in both cases. I'm completely unable to reproduce this problem.
     
  41. Peter705411

    Peter705411

    Joined:
    Apr 8, 2017
    Posts:
    7
    Hello there
    I use "Equidistant Stereo", the picture is not normal as shown below
    Unity5.5.2
    Unity5.5.4
    VR Panorama 360 PRO 2.2
    VR Panorama 360 PRO 2.3
     

    Attached Files:

    Last edited: Jul 3, 2017
  42. paxilon

    paxilon

    Joined:
    Aug 9, 2016
    Posts:
    25
    Hi! Is it possible to have a PNG with alpha so that the skybox/solid color will be transparent? Thanks :)
     
  43. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    This is a normal behaviour for close objects and stereo panorama - this is due to a parallax difference from back and front views with near planes, and there is no way to make it disappear. Some sort of this problem will appear with all techniques of rendering in stereoscopy.
     
  44. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    It already renders with a transparent background. Just be sure to set transparent color as background fro your camera (change color to black with alpha set to 0).
     
  45. paxilon

    paxilon

    Joined:
    Aug 9, 2016
    Posts:
    25
    Hi thanks first it dind't work out. I saw other guys who also had the problem with black borders from the cams. I checked it in After Efeects and set the alpha to premultiplied but it doesn't work.
    Note that I have now other transparent elements stored in the alpha which are overlapped by the cam borders.

    By the way if you use SEGI or the Post processing stack it turned out that that kills the alpha. I have to figure out why it is like this.
     

    Attached Files:

  46. olix4242

    olix4242

    Joined:
    Jul 21, 2013
    Posts:
    1,432
    I think it comes from a fact that those effects doesn't consider alpha - they probably don't use any alpha buffer, and they just simply overwrite it. There isn't much to do about that. One of the ideas could be to render without post processing effects - then render again with them, and than use an alpha from first one anto second render.
     
  47. mblt2334

    mblt2334

    Joined:
    Jul 8, 2017
    Posts:
    3
    Hello
    @
    olix4242.

    Justin-Wasilenko recently mentioned how captures are not being encoded into a video after rendering. I am having the same issue. you mentioned "that be sure that you don't have any numbers in a sequence name" I'm fairly new to unity... my project is creating stop motion animation in 360. My animation files are being played as image sequences because I am using the free unity version without full video texture support. these Images are saved with numbers in the file name and played as a slideshow. does this mean that VR panorama does not encode the video because of the sequence of images are numerical? I've developed a work around by importing the images back into my stop motion program, but it takes 2x as long. just curious if there's any way around that and getting the video to encode?
     
  48. truebototics

    truebototics

    Joined:
    Jan 2, 2016
    Posts:
    21
  49. Peter705411

    Peter705411

    Joined:
    Apr 8, 2017
    Posts:
    7
    Hello there
    I still do not understand
    Stereo 360 vr will be such a problem
    But it is normal in Mono
     

    Attached Files:

    • 01.JPG
      01.JPG
      File size:
      220.2 KB
      Views:
      339
    • 02.JPG
      02.JPG
      File size:
      177.8 KB
      Views:
      256
  50. vjyourself

    vjyourself

    Joined:
    Dec 6, 2016
    Posts:
    6
    Hi there,
    I'm using your VR Panarama for 8 month now, it worked amazingly for my 360 videos, so first, thanks for the great work!
    Recently I tried to capture still images for Facebook, and the plugin repeatedly froze the Unity Editor and my whole computer. Considering, I have a pretty stable Alienware PC, something has to go pretty wrong.

    Even if it not freezing, after doing a single capture, my console id flooded with warnings, what are keep on coming after stopping my game.

    Is anybody experienced something similar ?

    ( video / animation capture still works perfectly)

    Thanks,
    VJ Yourself