[RELEASED] VR Panorama - render 360 stereo videos

https://www.assetstore.unity3d.com/en/#!/content/35102

VRPanorama is a fast and easy way to make fully functional stereoscopic panorama movies and standard screen capture movies for VR Headsets like Oculus or Gear VR.

FEATURES
-Output Stereoscopic (and monoscopic) 360 panorama image sequence and movies directly from Unity to VR HUDs or video streaming services.
-PNG or JPG sequence output.
-H.246 (MP4) VIDEO output * (available on PC platform, in development for OSX)
-Support for deferred and forward rendering with screenspace image effects.
-All video sequences use 2x supersampling for better antialiasing. This slows rendering a little bit, but it gives a superior quality CG without jagged edges.
-Render standard HD video captures.

V 2.0
-Export Ambisonics Spatialized Audio for Youtube
-Added new Realtime modules
- VR Panorama RT and Still Image capture (with a click of a button)
-Export 360 images for Facebook (only PC - OSX support coming soon)
-Improved performance (in some cases even 50% speed gain)
-Added color conversion dithering (from HDR to 24bit) for better image quality
-Corrected issues with wrong colors when using Gamma colorspace
-Integrated 360 injection for Youtube
-Fixed Unity 5.4 error
-Updated documentation
-Added updated resolution settings for various platforms.
-Added SBS stereo mode (Side by side stereo).
-Added ability to render any resolution and aspect ratio panoramas up to 8k X 8k
-Lower video memory usage.


USAGE
-Right click in Hierarchy View and select VR Panorama Camera, or in menu Game Object/VR Panorama Camera. This action will add VR Panorama Camera into your scene. You can also add a VRCapture script to your existing camera.

Set rendering settings for VRPanorama by modifying VR Capture Component (attached to VR Panorama Camera object). Palce your favourite image effects on camera (like ambient occlusion, ssr, tonemapping, bloom). Note that some Screen Space image effects (like adaptive Tonemapping) could introduce artifacts, but majority of them will work just fine.

• Capture Type: Can be Equidistant stereo, Equidistant Mono or Video Capture. Use stereo for VR HUDs, and Mono for 360 video streaming services like Youtube. Video Capture is a standard camera HI-quality capture with configurable antialiasing.

• Sequence Format: Can be JPG or PNG. JPG is faster to render, but it introduces some compression artifacts. PNG is loosless but requires more space and is slower to render.

• Save to Folder: Folder under Your project folder, where your image sequence will be stored. If folder doesn't exist, script will create it for you.

• Open Destination Folder: When rendering finishes, script will open a destination folder for you.

• Resolution: For most devices (Gear VR) it should be 2048 for stereoscopic VR panorama movie. For some services like Samsung Milk 360 it should be 4096). Here you can set a custom resolution, but it's better to shoose from a presets below.

• Resolution presets: Choose one of them. 1024 gives you a poor quality, but its good for making a first preview.

• FPS: Image sequence framerate. Can be any, set it in base of your final device. Note that if you don't have fast moving objects, it can be as low as 25 fps. You can choose from presets instead of typing.

• Number of frames to render: Your final sequence lenght. A scene will stop playing after this number of frames.

• Start at frame: Time in frame domain, from where a rendering will start. Usefull if you have to change only one portion of animation.

PANORAMA SETTINGS:

• IP Distance: Interpupillary distance. Default for average human is 0.064, but I suggest using 0.066 as for people with larger IP distance, 0.064 can be really uncomfortable.

• Environment Distance: Wery importan feature. Setting this value correctly makes a lot of difference. It compensates for stitching artifacts. This is a value of a nearest distance where your stitching has to be perfect. Don't set it too close, and don't set it too far. Best values are from 2 to 7. If your scene is large (like outdoors) go for larger values, if you are rendering a small corridor, make it smaller.

• Align Horizont: Keeps animated camera leveled with horizont. You can use it you already made an animation that rotates cameras on other axes than Y.

OPTIMIZATIONS:

• Speed VS. Quality: Supersample Antialiasing and subsample quality. Value of 16 is pixel perfect AA, but without antialiasing (but you could use Image Antialiasing screen space effect). Smaller values are good for speed preview, they will export into a desired format, but with pixalisation. Value as 32 will give you a best smoothing that is great for transparent texures.

• Encode To MP4: Encode a video file for GearVR, Youtube or fast preview after rendering finish.

• MP4 bitrate: Encoding bitrate for videos.

• Encode H.264 Video from Existing Sequence: Encodes to video an existing image sequence. Usefull if you want to re-encode at a different bitrate.

• Notify by mail: You have to render a long sequence, and don't want to stay in front of your computer? Check this box and fill your email settings, and you will receive an Email when rendering finishes. You have to use credentials of an existing Gmail account.

• Change Image Name: Here you can change a prefix of a numbered image sequence.

• Render Panorama (button): Render your panorama with a click. But, you can also render it by playing a scene.

GENERAL INFORMATION ON STEREOSCOPIC 360 PANORAMA MOVIES

Be aware that VR 360 panorama suffer some stitching artifacts.
Unfortunately, these artifacts can't be avoided for stereocopic VR panoramas (in general, it applyes to all tecniques). It will happen when objects are near camera (they are bigger as object gets closer to camera, but they will get smaller or eventually dissapear for objects that are far away). Due to a paralax difference between eyes, you can't correct them in any way without breaking stereoscopic illusion. There are different techiques for minimizing (one of them is usage of vertical scanline rendering technique, but it works only with static stereoscopic panoramas, as for animation, it would introduce heavy wobbling and spatial distortion (closer objects will be vertically stretched resulting in a wrong dimensional perception).

Here are some tips on how to minimise stitching artifacts:

-use correctly a Environment Distance value.
-place camera in a way that those stitching edges look at objects that are far from camera.
-Project your animations in a way that they capture viewers attention at zones that are artifact free.
-Avoid using thin vertical objects.
-On near planes like floor use textures that don't have large checker or stripe patterns. High frequency textures or uniform colors work better.

VRPanorama asset is made to use psico-optical perception illusion: these artifacts are more visible in unwrapped image than in VR.

TROUBLESHOOTING NOTICE
-DO NOT change a default installation directory (Assets/VRPanorama) as this will break correct functioning of a script!
-BE SURE THAT YOU AREN'T USING A WEB PALTFORM (under build settings). WEB PLATFORM doesn't support writing to disk, and using it will break the script.
-Unity has a shadowing system that is limited on available VRAM. If you notice that in your scene a realtime shadows deissapear, lower your "Speed VS. Quality" settings or a final resolution. This cuold happen if there are many ScreenSpace effects that use RT's, rendering to 4K stereo panorama output, especcialy on GFX cards that have 2 gb or less VRAM. A future version of VR Panorama will be more optimised on VRAM usage.
-Don't forget to feed your animals

NOTE. VR Panorama isn't a realtime renderer and doesn't support any type of video streaming. However, it is quite fast - on a modern computer it will render 1 minute 360 sequence for Gear VR in 10 minutes.
-------------------------------------

New version of VR Panorama (V0.5beta) features advanced stitching that is almost invisible, while maintainingt fast rendering performance, without introducing spatial distortion common for 360 stereo panoramas. Now you can have artifact free stitching without vertical streching for close objects, that can be watched without eyestrain. This example scene is a Unity Labs asset and uses image effects like Candela SSRR, SE SAO, Tonemapper Deluxe. It also uses VR Panorama supersampled antialiasing with a HDRI and Deffered workflow.
It took 15 minutes to render this 30 seconds video with an ASUS STRIX GTX 970.

Some Videos made with VR Panorama:

 

New version update: 0.11 alpha
Perfect pixel stretching for mono panoramas and small bugfixes.

 

What did you use to author your video after creating the frames? Vital info that's tough to find...
Thanks!

Update: found it on your video comments, how do you author your video if not using urecord? http://www.magadan.com.br/2015/06/1...0-virtual-reallity-4k-videos-with-unity-3d-5/

 

There are many softwares that can make a video (mp4) from image sequence files. Some more advanced (like premiere pro, after effects) and some as simple as command line tools. I personally use FFMPEG, that is really fast and efficient.
https://www.ffmpeg.org/
Install it and insert it's location in Windows PATH (If you are using windows). Then lounch it with command line like this:

For PNG:
ffmpeg -f image2 -i "shot %04d.png" -r 25 -vcodec libx264 -pix_fmt yuv420p -b:v 9600k "FeatureTour.mp4"


For JPG:

ffmpeg -f image2 -i "shot %04d.jpg" -r 25 -vcodec libx264 -pix_fmt yuv420p -b:v 9600k "FeatureTour.mp4"


Afterwards, if you want to put your movie on Youtube like 360 movie, you should use 360 metadata tool, found here:
https://support.google.com/youtube/answer/6178631?hl=en

 

Thanks! Wow, I thought there was much more to it than creating a video, thats great! I appreciate the help.

 

Right now I'm working on hi-quality antialiasing that will improve image quality for small and large resolutions. It removes efficiently all jagged edges and texture. It should hit the store very soon.


invia immagini

 

Awesome asset. Will review soon.

 

Great news

 

There are some other even greater news
A new version will expand it's possibilities: you will be able to export also hi-quality antialiased (2D) camera capture renderings. Also, from now on you will be able to export not only jpg or png image sequences, but also mp4 encoded videos ready for GearVR or Youtube, directly from VRPanorama.

 

Hi. When importing your asset I get this


I was in WebPlayer mode. So I guess this doesn't work in WebPlayer

 

Yes, it doesn't work in WebPlayer, because web player, for security reasons, isn't able to write to a file system. Also, VRPanorama is an Editor extension and isn't mean't to be used at runtime (for now).

 

New 0.20alpha version was submitted yesterday, I think it will be online monday.
It features some important changes, so I suggest to try it in a fresh project (or if you use it in your old project: delete all previous files and backup old files).

Most important changes are:

Stereoscopic viewing:
-Correction for back-top and back-bottom stereoscopic eye inversion problem zones.
-Better stereoscopic stitching and overall stereo effect

HI-Quality rendering:
-2X Supersampled antialliasing for better quality renderings vs. more rendering time per frame. (first version for supersampling. It will be developed more in near future, for various AA modes vs. Performance.

File export:
-H.264/MPEG-4 Video file export. From now on, you can export video file ready to be loaded on your GearVR or streaming services.

Export standard screencaptures:
-From now on, you can use VRPanorama renderer also for rendering HI Quality standard camera captures. This mode is in early betatest, so I'm not sure if it works on every platform. (right now I'm on vacation, and I'm using only one laptop that had some inconsistency problems with scripts on various Unity versions).

 

Hey saw the new update super excited to get my hands on it, wanted to let you know that using this asset with my AMD gpu causes the driver to crash after about 700 frames of pngs. jpeg seems fine.

 

very useful update

 

Glenn, could You give me more info (GFX card model, RAM, VRAM and if it happens only in one project-scene, or in all scenes projects? Does it happen all the time or sometimes? Are there any scripts inside your scene that could cause it?) so I can investigate further?

 

Here you download a video with results rendered inside new version.
https://www.dropbox.com/s/l3lg8epcy0bh3x7/FeatureTour.mp4?dl=0

 

Intel Quad Core(8 Threads) i7 4790k, 16G Corsair DDR3 2666 RAM, and 500GB 850 EVO SSD. My ATI AMD 5870HD(1Gig VRAM) It happens between 700-1500 frames of .pngs only, .jpegs seem to work fine. It happens every time. in the project we tested it on.

We tried it in a (mostly)empty project and it worked for 15k frames. So maybe it's related to VRAM? Might be something worth letting people know

 

Really strange. I think the problem lies in Unity as the only thing that really changes between JPG and PNG, is one line of code. It uses EncodeToPng class. But I don't understand why it crashes your graphics driver, as I don't think that EncodeToPng even uses GFX card. But ReadPixels use, so I presume it must be a bug on unity or ATI drivers side.
Does Unity gives you some info log on crash?
Did you try some other unity version?
Can you test this project on other PC?

 

Hi Olix4242,

I can get screenshots however for mp4 I guess, I am getting this error:

Win32Exception: Sistem belirtilen dosyayı bulamıyor. (which means system cannot find file)

System.Diagnostics.Process.Start_shell (System.Diagnostics.ProcessStartInfo startInfo, System.Diagnostics.Process process)
System.Diagnostics.Process.Start_common (System.Diagnostics.ProcessStartInfo startInfo, System.Diagnostics.Process process)
System.Diagnostics.Process.Start (System.Diagnostics.ProcessStartInfo startInfo)
System.Diagnostics.Process.Start (System.String fileName, System.String arguments)
VRPanorama.VRCapture.LateUpdate () (at Assets/VRPanorama/Scripts/VRCapture.cs:307)

 

Hi Olix4242
I am looking for a plugin like yours. Nice work. I just need to understand a couple of things before buying it
When you say stereo and mono, do you mean stereo as in rendering 2 images one for left eye and one for right eye? Is it two images or one side by side image? I guess mono means only one eye / image

When you mention stitching artifacts do you mean that you get the info/individual images from different cameras and you blend them to create one big image? Thanks a lot

 

I'm guessing you are using Osx? Note that mp4 isn't yet supported on OSx, but only on PC. I will add this functionality as soon as I get to put my hands on a Osx computer. If you need it soon, contact me in private, and I can try to send you a code that you could test and see if it works.

 

For now, stereo encoding is done in top-down format to one image. You can download an example here: https://www.dropbox.com/s/l3lg8epcy0bh3x7/FeatureTour.mp4?dl=0
As for mono, they are one single image.

Yes. This gives a fast rendering, and a possibility to use mostly all screenspace image effects. You can examine a video file, and see the result.

 

I'm proud to announce that a new stitching stereoscopic algorithm is almost finished: it will give you a possibility to tweak stitching parameters for your scene and make almost artifact free stereo panoramas.
-One other important upadate is an adaptive performance mode where you can choose with a slider between rendering speed or rendering quality. Last version introduced high quality antialiasing, but added on rendering time. From next version you will be able to choose your quality.
With this modifications VRPANORAMA will exit its alpha state.

 

Hi,

Sorry for lack of information . I am on PC and using unity 5.1.2.

 

Great!! Cant wait for this.

 

One another handy new function is an ability to send Email when VR Panorama finishes it's rendering. Very useful when you are rendering large sequences that take much time to render.
Also, a whole UI is rewritten, and easier to use. It will also feature some statistics info.

 

Sorry, I've found an error on submitted scripts (I don't know how it happened, but it looks that I've submitted wrong scripts).
You have to find a VRCapture.cs script, and edit line 307 from

Code (CSharp):

1. System.Diagnostics.Process.Start( "ffmpeg"

to:

Code (CSharp):

1. System.Diagnostics.Process.Start(ffmpegPath+ "ffmpeg"

Let me know if it works (it should work).
Anyway, I'm sorry about this error, and I will submit a correction as soon as possible.

 

New interface:

 

When do you expect to write this for Mac OS? Have you made all the barrel lens distortions and color correction as in the DK-2? This is definitely a tool I need. Thank you for this work.
Nick

 

Hi Nick, this already works for OSX, but with some limitations: only thing that doesn't work is an encoding to video file (A new feature added one week ago). You can export only image sequences for now. Support for video will be added as soon as possible, when I get a possibility to test in on a Mac.
Barrel lens distortion and color correction hasn't to do anything with 360 panorama movies. You can't barrel distort panoramas in render. It is always done on a panorama player side, and depends only of what player and HMD you are using to play 360 movies. There are many different panorama players for DK2, Gear VR, Google Cardboard etc. As long if a player you are using supports barrel distortion correction, it will work with 360 movies made with VR Panorama

 

Hi OliVR

Your plugin is pretty awesome, but i am running into a issue. There is a black border on the right side of every generated image. Have a look:

It's a 2 pixel wide black bar that runs across the entire image.
The Compensate Driver fix does not work and only widens the black bar with an additional pixel.
If you need any additional information just reply with what you need and ill provide it to you asap.

Best,
Mike

 

Great I'm sorry to hear this. It looks that there are some differently 'broken drivers'. For my curiosity, are you using a mobile graphics card?
A new VR Panorama version (soon in store) eliminates completely this problem by using different approach. Please contact me on PM with your Invoice number, and I will send it to you ASAP.

 

I thought this was part of the stitching methods. It does not show when applied as a texture.

if not, I am having the same issue. Excited for your updates.

 

Yeah it's kinda sad, but i have send you a PM with my invoice number. I am looking forward to the new version of VR Panorama I hope this fixes the problem.

Btw, i am not using a mobile graphics card, see systeem specs below.

OS: Windows 8.1 64bit
CPU: Intel I7 4930K
Memory: 64GB
Graphics card: Nvidia GTX 580
Unity version: 5.1.2p3

 

I've sent you a link.

If someone else is interested to test the newest version before it hits Assetstore, just send me your Invoice number and I will send you a link for a new version.

 

So i have been testing the new beta. It works perfectly, not stitching errors at all. I am extremely impressed. Thanks for the new version. I did however notice that if i animate the vrcamera, the camera's seem to mis align. But by putting the vrcamera in a empty game object, and animate that in stead, fixes the problem.

Overall nice work. Looking forward to more new idea's that you come up with

 

I'm working in Unity 5.1.2f1 on Mac OS 10.10.5.
Does your plugin generate top/bottom image sequences?
How are these rendered to Left Right eye with barrel distortion, chromatic aberration corrections for the Gear Vr or DK-2 etc. etc?

 

How can i render for services such as VRideo?
http://www.vrideo.com/
I talked with them at Siggraph LA last week. They need TB or over/under format, but their content does not play well on the DK-2.
Thanks,
-Nick

 

Yes, for now it generates only top/bottom (future versions will support also side to side).
Barrel distortion ha to be done on player side. thi plugin generates only a content video, that is later passed on a player or player service. It's impossible to do barrel distortion or chromatic aberration correction directly on a 360 movie.

I'm not familiar with this service, but I'm quite sure it can output a sequence that can be loaded correctly, if they are using standard coding for videos.

 

Olix424,
Thank you. The publishing site VRideo will accept equirectangular 360 stereoscopic for immersive (streamed) playback.
They use Cromium/webGL and deliver to the Oculus DK-2.
I'm excited to try your plugin. This may solve my problem.
Thank you,
Nick

 

Olix424,
Can your VR panorama work with google cardboard Unity sdk? I am developing a 360 shooting game. It would be wonderful if the background can be a stereoscopic panorama video.
Thank you!
jamesjk2000

 

New version of VR Panorama (V0.5beta) features advanced stitching that is almost invisible, while maintainingt fast rendering performance, without introducing spatial distortion common for 360 stereo panoramas. Now you can have artifact free stitching without vertical streching for close objects, that can be watched without eyestrain. This example scene is a Unity Labs asset and uses image effects like Candela SSRR, SE SAO, Tonemapper Deluxe. It also uses VR Panorama supersampled antialiasing with a HDRI and Deffered workflow.
It took 15 minutes to render this 30 seconds video with an ASUS STRIX GTX 970.

 

With VR Panorma you can create a texture that can be used as a background. But, VR Panorama is a content generation system, not a player for such content. So, you would have to implement a movie playing by yourself. It isn't that hard, it would require to put two spheres with inverted normals, map a video made by VR Panorama on them, and set/exclude one or the other for left or right eye.

 

Hi,
I've purchased this plugin in order to create TB/upper-lower renders for a site called VRideo. The first renders all have a stitching artifact slicing the image vertically. I'm using Unity 5.1.2, Mac OS 10.10.5. I've been working with Unity for two years and developing for the Oculus. I need this (potentially) wonderful plugin to work. There are no updates on the asset store. Please help.
Nick

 

Thank you for this work! With VRpanorama I'm able to create image sequences which I import into Premiere. The TB video is created there. Am I missing an update? I've disabled AA and tried numerous combinations of Quality settings, forward/deferred lighting, etc, etc. The artifact does not go away. I'm stumped.
Nick

 

Nick, new update isn't available yet on asset store, it was resubmitted yesterday and should be available in few days.
Send me your invoice number in a private message or on a support mail, and I will send you a newest version immediatley.

 

Hi, I purchased your product few days ago but I cant' find a way to export H.264/MPEG-4 Video file as described in the product.
I only have an option to export PNG or JPG. Am I missing something ?
Dan

 

What is your email? Your profile page has no information.
Should I leave it here on the Forum? No a good idea?

 

.....I'm working in Mac OS.

 

Thanks for the new version. It's working now !!