[RELEASED] Deckard Render - Cinematographic Renderer for Unity

-Screen space reflections are supported for both of PPS stacks. PPS1 SSR has a better quality but it is less performant as it does some things that pps2 SSR doesn't - all this results in having more precise/correct reflections with pps1. So, it's more a suggestion to use ppsv1 than a real limit. Also, some of the effects from PPS you will not use and you should turn them off - as Deckard does them better, and you want to avoid accumulation of same effects (applies to motion blur and DOF - effects that Deckard adds natively). As I'm working now on adding also natural bloom, in a future also a bloom could be avoided).
note: there are some problems currently with a latest PPS included in Unity 2019.1 package manager where some of the effects doesn't work. I hope that this will be sorted out, as it could be a simple bug in PPS.

-Setup is simple as adding any image post processing effects. Just add a script to your camera, add soft lights scripts to your lights and you are ready to go. It doesn't require any significant change in workflow. (if you plan to use HDRP, setup is right now a little bit more complicated - as it requires adding two adittional effects to post processing stack - but even like that it is quite simple).

-New version, that should hit assetstore in few days adds screencapture button - with the ability to export even 3D images for facebook (it exports image plus alpha)

-Most of the image effects work - but this can depend pretty on how some effects are implemented in Unity. I know that it doesn't work with Aura (free version) as it wasn't properly a native post processing effect. I don't have Aura 2 to test it. I'm using HX Volumetric lighting for this task and it works perfectly. I will see if Screen Space snow works or doesn't (I'm in contact with developer, so I can try it out - and if it doesn't we can add a support for it).

 

Yes, it supports HDRP, but still not latest version included in 2019.1. Unfortunately, we still lack documentation on new integrated post processing for HDRP, so for now, you have to sctic to older versions of HDRP. I will add a support for 2019.1 as soon as I get informations that I need.
-It doesn't require special shaders but it can do some things that you can't do easily with special shaders. It comes with Amplify shaders for those specific cases that can be examined and modified to liking.

In theory no - most characters work fine. But while testing, I came across one model that required some small fixes - but they weren't animated with Mecanim and had pretty complex structure with physics. I didn't had time to examine that model more closely. (With a new version od Deckard that is available on request and will hit store in few days).

I have included an autofix - it will fix timing by itself. But as this problem is actually a bug in unity, it might break in future versions if unity fixes issue If this happens, I will release a patch.

Don't know, I don't own OBI assets. But, if they support unity time.scale, they should work. (you can try by yourself if this is true - change time scale in unity Time settings and see if OBI respects it like real physics in Unity). If you change time scale to 0.5 a simulation should work 2X slower. If this happens, than this means that it will work with Deckard.

Yes, as said above. I'm currently testing it and will submit it soon to assetstore.

 

Excellent. Thanks for the info! Purchases made.

We'll share out some tests:
- OBI Products
- Various Terrains
- Various Rigs and Animation
- Cinemachine/Timeline
- Particles, Volumetrics
- Alembic Sequences

More to come. Exciting stuff. Looking forward to the new release.

 

Same here. And every time I try to render it crashes with no error.

 

I have had a few different experiences since posting that… A few scenes rendered surprisingly fast (relatively), while some others prevent anything functioning on my mac (and even some crashes). One time I had to remove Deckard component from the camera to regain control (that's not normal).

Whatever changed that made it work faster, was not obvious to me at the time, so it could be a bug or some un-identified Unity element within my scene, or a mac thing…. Who knows. If I track it down I will post, but tbh my scenes were not that different from each other….

One thought, are you on Unity 2018.3 or 2018.2…? I *think* I'm getting better stability on 2018.3.

 

Can you give me some more info? It crashes immediately? What hardware configuration are you using? What Deckard settings?

 

I think that it might be some post processing effects - if you are using any. It could be that some of those effects have some kind of memory leakage. Sometimes memory leakages are difficult to notice when using image effects alone in Unity as they could require some time to fill up your memory - but with Deckard they might arrise sooner (as deckard is more intensive on rendering). Best way to know if you are dealing with memory leakage is to activate Deckard and look at a memory usage in your system. BTW, I know that there were some Unity versions that had a bug with releasing Render texture, and this leads to memory leakage. I can't tell what version was that.
Other than that, as deckard uses pretty much all of juice of a graphics card - you should also check if temperatures aren't too high (GPU and CPU) - as this could lead to throttling, crash or even restart. If this is a case, then you should provide better ventilation. New mac's are known for pretty unefficient ventilation and extremely high temperatures under load.

 

Now that you say that, I think I just realized exactly what was different: a 3rd-party volumetric light asset (I won't say which until I get a chance to test. I don't want to spread bad information.)

 

I little comparsion on how new version of Deckard handles characters and motion blur.

 

Is that comparing PPS2 or PPS3 from 2019?

 

This uses PPS2.
PPS3 should have a same quality motion blur - there shouldn't be any significant changes on that side. PPS3 uses motion vectors, and motion vectors have this quality limit that can't be overridden.
One note: at this point pps3 doesn't work with deckard - in this stage of developement of pps3, unity doesn't provide any way to use custom effects. It is closed system wothout any API. We have to wait 2019.2 for API implementation.

 

Edit: Updating this after lots of testing

Having issues with PPS2 (2.1.6), Cinemachine (2.2.8), and Deckard. Unity 2018.3.14f1

I've tested all of the below with components placed between Deckard Preprocess and Render.

- PPS2 works properly on a newly created camera with Deckard applied. But as soon as you add a vcam component to the camera, PPS effects get removed (stops working) but show in the editor. Placed properly between Deckard Preprocess and Render.

- Choosing PNG format for image sequences results in Black frames. JPG works. Testing EXR next

- There seems to be a disconnect from the Main Camera and what Deckard is rendering. I tried adding a look constraint (for composition and focus of main camera) and then animating the Focus Object and the Camera in Timeline. When Deckard is OFF, the camera renders properly - panning and tracking across a room. When Deckard is ON, it ignores the look constraint, even though the Scene view shows the camera properly looking at the target.

Also, there's a script(s) missing from the demo and when creating a Deckard Physical Camera object (2 missing). The Physical Camera has the script order flipped (PreProcess is last). Not sure if that matters?

All of that aside, the 4K test renders are excellent. So far, our volumetrics tests have worked. Getting this to play well with Cinemachine would be a game changer.

 

Those should be from PPS1 - so no problem there.

PNG will show black frames when viewed from broser. But they contain all data needed for image processing (or video compression). This is due to alpha being written like transparent. I will see to solve this issue if it's needed to be solved - unfortunately, it will have some unneccesary performance impact (and this is why I left it out).

This can be tricky for a way how Deckard renders. Best way would be to add scripts to a parent object, and then add a camera as a child of that object. Deckard has to do some transformations on camera in LateUpdate, but some scripts might block or interfer with those transformations. Anyway, this won't be needed with a new Deckard version - you will be able to safely use timeline without any worry about what happens under a hood.

I'm testing Cinemachine with new Deckard and it should work fine - but I have one problem that I'm unable to correct (I can't find any solution in Cinemachine Api): cinemachine works well, but as it is now, it doesn't support previewing of Deckard DOF while in editor. It will render out perfectly into a file - but if Unity isn't in play mode, Cinemachine takes all controll of deckard, and is unable to do realtime preview. Looks like there is some inconsistecy in Cinemachine, and it's behaviour when using play or just editor. I'm still searching for any solutions on how to hack this behaviour.
Here is a simple example that I did with timeline and Cinemachine.

 

I had this behaviour with a new PPS2.. not as in your case but similar, where some effects didn't work and some did... You should go to Post Process Layer script and uncheck Directly To Camera Target checkbox. This should propbably fix your issues.

 

Back up and running now.

- PNG works as you described. Once imported into a compositor, just choose "Ignore" for the Alpha. May need to notate that in the PDF.
- The "Look At" was due to parenting like you said. Pulled the Target Object out and Timeline now works correctly with Deckard.
- I was unable to find "Directly to Camera Target" checkbox in the PPS script, but VCams work now (minus DOF preview as you mentioned). Not sure what changed otherwise. Currently, Deckard is applied to Main Camera, with only PPS applied to VCams. Main Camera has PPS Layer as usual.

 

Wanted to share some findings. All tests performed on 8 Core 5960x, 64GB Ram, and GTX Titan-X (Maxwell):

- Render Quality does not seem to improve significantly beyond 75-80. Tested up to 4K at 90 setting and saw no major improvement after 400x frame analysis under various lighting conditions.
- Render time difference between 75 vs 90 Render Quality setting was 16min vs 18min per 100 Frames in 4K. Likely about 15% increase in render time.
- Render time for 1080 vs 4K was 4min vs 16min for 100 frames. There is a quality improvement, but Deckard renders better Anti-Aliasing, Color, and DOF at 1080 than anything we've tested in 4K standard Unity.
- The default images are softer in Deckard, so we recommend rendering 4K and applying post-sharpening OR adjusting Deckard's AA Bias and Supersampling. Need to test these further.
- No render time difference for JPG vs PNG. PNG has slightly cleaner noise (as usual).

The recommendation is to build out your entire scene as usual in Cinemachine/Timeline and then flip on Deckard LAST. Do a spot check throughout your timeline/animation and then hit Render.

After a half-day experimenting and learning the quirks of Deckard, this is some really exciting stuff. Some of these renders are DCC quality at roughly 1/100th the render time on a high-end GTX that's now 2 generations old. Will have to test this on our 16+ Core machines with RTX 2080Ti cards to see how well it scales.

Really looking forward to some power user controls and features such as AOV's, additional focus options (diopter/tilt shift would be fantastic), and more file structuring for renders to allow image naming from wildcards and folder sorting.

Keep up the great work!

 

@josterday Thanks for sharing your Cinemachine Deckard tests. This will be really helpful.

@olix4242 The Deckard fight video is a nice demo. It's great to see Deckard integrated with the timeline and cinemachine.

I noticed that Deckard is improving everyday and you are probably really busy but do you know if and when the hair tutorial ( amplify shader) will be released. There is a big quality gap in unity for hair textures and I believe Deckard could be the solution to fill this gap.

 

Yes, for most scenes it should be enough to use default quality of 70. You should eventually crank this value to something higher only when you use extreme defocus or soft lights with big radius.

Yes, this should be true Actually, Deckard is much better at rendering FullHD than real digital cameras (and even some famous offline renderers) for various reasons (if anyone is interested, I can write something about why it is like that - but it's quite a complex question). You should get always pretty high detail out of it without aliasing issues even on lower resolutions. A difference between HD and 4K shouldn't be as large as one would assume that it should be.
-ONE TIP: Deckard has this nice feature where you can render actually in HD while outputting 4K image. I suggest you to try this feature as it renders more or less at speed of HD rendering - while upscaling to 4k image and video file. This works really well with encoder codecs, as you can use higher resolution to eliminate some compression problems. It should be also used for uploading videos on Youtube.

Deckard has also a sharpen functionality that can be used for this purpose.
But here I would like to say few things about sharpening that I think are really important.
It is true that Deckard has some softer output - but this is due to it's physical nature - is simulates blurring that happens when light passes thru a lense. It looks soft because this is how optical systems work - we always have finite depth of field and actually only an infinitely small part of a depth on image will be at perfect focus - everything else gets blurred. (only a theoretical pinhole camera can have infinite DOF - but a perfect pinhole camera doesn't exist. And standard CG renderers are actually perfect pinhole cameras )
Some images might look more sharp than others - but this is only due to electronic sharpening in camera (or renderer) or in post.
Sharpening (or Unsharpen mask) are coll when used on prints (they were made for needs of printing industry) and for static images. But for a cinematic video, they are a killer and mostly are assotiated with a classic 'video or tv' look. They artificially add a contrast on edges of an image - by adding artificcialy blurring and layering it on top (yes, sharpening is actually blur that is overlayed on top of an image). But this results in added spatial and temporal aliasing. This type of sharpening destroys correct motion blur and correct depth of field - and should be avoided by all means (if you aren't trying to simulate 'TV' look). You can evetually use it in static scenes where your background doesn't move and you have objects that move slowly (like a person speaking). But a good way is to capture always without sharpening and use sharpening in post process only in mastering phase (in video editor) - where you know exactly what will be your final output format - like for an example - when you are mastering for Netflix or for Cinema.
Due to long rendering times, we, persons from CG industry, tend to focus too much on static image and try to enhance it to make it look better and more detailed (of course that we want to see all the detail in meticulosly made textures or highly detailed models), but a REALITY of physics works differently than we would like.
Here is a nice video that explains and shows some of this differences in Arri Alexa (industry standard camera) and Iphone camera. You will see how Arri has much softer image than Iphone. This is mostly because professional cameras doesn't apply any image sharpening.

I'm also curious to see how it works with RTX cards. I still don't own a RTX card - but from what I know, it could probably help to render faster, if unity supports some of the enhancements made in a new architecture. For example, what I know is that GTX 1080ti renders somewhat faster than Titan X (Maxwell). I hope that RTX 2080 improoves onto this even more.
Also, RTX tech should also be able to work with Deckard - and bring it to better quality level than using only RTX.
I really have to buy me a RTX card and start testing with it.

 

It should be included in latest version of PPS2, I think one dedicated to 2019.1

 

Tilt shift is already implemented in new Deckard.

 

Under H264 movie export options.

 

Any updates on compatibility with Cinemachine VCams for DOF and Motion Blur?

Our current workaround is to child a new camera to the VCam and add Deckard to that, but we're sometimes experiencing missing DOF or PPS looks in the final render.

Also curious if there's minimum focus incorporated with certain sensor sizes and lenses?

 

I'm in uploading process on a new update. But had to fix some issues across different Unity versions.
If you wan't to test new version, send me a PM with your invoice number.
Cinemachine should support correct motion blur (and fix issues with characters). As for dof, it should work correctly in render, but it doesn't work correctly in edit mode - due to how cinemachine handles refreshes. I'm still trying a colution - but only solution is to temporary deactivate Vcam so that it leaves controll to Deckard.

PPS could behave strangely with latest PPS stack. Solution is to toggle Direct to camera option (not sure about correct name - I'm writing from a phone right now).

There is no any limit on minimal focus distance as it can varry between lenses. Not sure if this was a question (about minimal focus distance). If it was on maximum blurring, then it is related to sensor size and lenses.

 

Good news - I finally got my RTX 2080, and will be working on supporting Unity DXR raytrace for Deckard.
Just did some tests on performance difference without raytracing, and as assumed, because of a new graphics card architecture, rendering speeds are hugely improoved just by using new card.
Here are results of rendering a pretty complex 1 minute animation with tons of lights and effects with a GTX 1080TI vs. RTX 2080.

GTX1080ti rendering time: 19 minutes
RTX 2080 rendering time: 9 minutes

This is a really significant improvement of more than double speed with RTX2080 over GTX1080ti. This is far better result than in standard gaming tests that usually give only 0-10% boost in performance on RTX 2080 over GTX 1080ti!!
I'm so happy right now

 

Did you test with the RTX2080 or RTX20180 TI? I am happy with my GTX 1080 for now, but I would be curious to know how much it would cost to do a swap.

 

I have tested on RTX2080 and I didn't test on RTX2080TI (I'm pretty curious to know how it works on TI version - as it should add further boost in a performance). Right now, main performance problem is on a pretty high CPU usage (on my pretty old i7-4790K CPU @ 4.00GHz, 4001 Mhz, 4 Core) so my CPU is bottlenecking. Performance could be probably better with a new CPU.

 

I apologize for another "When When When" post but I'm hoping for the Deckard update. The features I need for my project (which I think have all been mentioned) are snapshot and alpha-channel…. Other new features sound fun, but those are the 2 that (I hope) to use in production… Do you have a rough estimate?

 

I'm sending you a PM.

 

First test of a RTX raytrace combined with Deckard.
I caould make it work - but as Raytrace Unity Build is quite in initial stages, for some unknown reason, unity crashes when trying to export images - there is probably some bug not realted directly to RTX or Deckard. Important is that it works in viewport.

 

I resolved it by reducing the render quality.

 

Interesting development with Raytracing. Curious to see this implemented.

Continued testing with Deckard has been pretty positive. Obi fluids and cloth work, but need further testing with motion blur. It looks like they will need shader tweaks for Deckard/Unity to recognize motion.

One thing that’s evident in DOF tests is not quite finding the right look for extreme foreground objects when out of focus. As the focal point moves toward camera, the background bokeh finds an accurate creaminess, but the foreground tends to become a collection of overlapping circle textures, like a Venn diagram. This somewhat depends on the texture/mesh of the foreground object, but it can be difficult to get a very soft, subtle out of focus subject near camera.

 

Cool. I will try to contact OBI developer and see what can be done. Does it use custom shaders?

It can be somewhat complicated to acheive good blur on near objects, if those obeujcts have high frequency details or are thin or small and really close to a camera. It is possible, but it would require setting render quality to really high value. But, actually, in real life, those objects would actually become so blurry that they wouldn't almost be percieved.
One trick: you can add a subtle post processing DOF and set it so that it influences only a near plane. But it's always better to compose your shot by avoiding having those objects that near. As this would help in a rendering performance.

I am also working on some new noise distribution algorithms, that will probably produce less of a Ven Diagram shape (or atleast - give more controll over a shape needed for defocusing).

 

Interesting. We're currently testing this with CTAA (Cine Temporal AA) and other AO solutions (HBAO and GTAO from Amplify). Just adding CTAA gives Deckard a fantastic motion blur quality and an unrivaled anti-aliasing. Still need to test PPS1 Screen Space vs PPS2.

Throwing all of this together with custom Amplify Shaders (like Hair) is giving amazing results. Really excited.

Spoke with the Obi developers. Their fluids use custom shaders, but are screen-space based, so they can't quite process properly in the render path for motion blur. Maybe there's a way around it?

We're using Obi for smaller fluid sims in scenes for realtime interactions with animation and objects - Alembic (x-particles) for large renders. The constraints are different with Cinematics, since we don't care about performance. Whatever gets us the shot.

 

It doesn't mean anything bad if their fluids are screen space based. Well, it could be a big problem with any post processing motion blur - but when dealing with Deckard, motion blur could eventually be pretty easy to implement. But I'm not sure if OBI is based on Unity particles or their own system. Later is probably better as Unity particles have one bug (but I should check if it was resolved in latest Unity). If OBI uses it's own system - and has some way of scrubbing simulation thru time (advancing in desired steps via scripting), then it should be pretty easy to sync them for use with deckard motion blur.

 

@olix4242 any update regarding the ability to use this for screenshots in standalone builds?

 

Wow, I am in love with the motion blur!
Wish we could have something similar for that motion blur for real-time render.
Would totally buy that specific motion blur post process if it were on a separate package and works in real-time
(Probably overstretching it, but hey I can't help but wish with how beautiful it is! )

 

I just got a RTX2080TI a couple of days ago and just Bought Deckard last night. And I am buying Aura 2 now. SO I will be happy to test them. I am really hoping Aura 2 works with this because it is just so pretty.

Quick Questions:
1 What version of Unity for HDRP is currently Supported?
2 Do I need to download the HDRP file you posted on Dropbox to use it or is it already in your most recent build?
3 What build should I be using for Non HDRP?

 

I'ts pretty difficult to say what version of HDRP is supported as there are many versions and all are different. But, as of today, I plan to support only latest HDRP version - so, I can say safely that it supports HDRP that comes with Unity 2019.1. This was pretty hard to implement, as latest HDRP doesn't support any custom image effects (we don't have any public API for a new post processing stack). But, today I found one workaround to be able to visualize Deckard render buffer without post processing stack. It requires to open a new custom preview window for Deckard. More or less like in any offline rendering application where you have viewports and render windows. This solution should be pretty future proof - and easy to maintain untill HDRP doesn't come into it's final stage.
This solution is in it's initial testing - and if you need it, send me a private message with your invoice number.
That said, there are some Deckard features that doesn't work with HDRP.
- Feature that isn't supported in HDRI mode is rendering 3d images for Facebook.
-Another one is Deckard outdoor ambien lighting - HDRP supports shadowing for only one single directional light, but Deckard requires atleast two shadowed directional lights for this feature. This is pretty nasty limit of HDRP that I hope will be removed in a future versions of HDRP. .
- there are some other minor Deckard features that doesn't work but they can be easily replaced with HDRP built in post processing (like exposure, or sharpening).
But, overall, for now I don't suggest to use HDRP for production purposes, as HDRP lacks some of the important features (like SSR) while most of HDRP features can be efficiently simulated with Deckard without using HDRP.

 

Sorry, I still don't have any ETA on this I am doing some modifications to be able to make this (and some other features).

 

Quality settings shouldn't influence any stability. Unless you are having some thermal issues on your hardware. What is your PC configuration?

 

So using Unity 2019.1.4f1Post Process V2 I get this warning opening the demo scene and When I go to the soft Light script, For some reason the Add component button is over the fields you input. This is Standard Pipeline Project (NON HD)


Update: after I saved everything and reopened the project all of this vanished.

 

Aura 2 works for us with Deckard. We also tested Fog Volume 3, Volumetric Light Beam, and a custom Fog script we have.

As mentioned, HDRP starts to have fewer benefits when comparing an offline render such as Deckard. Adding other post FX such as CTAA and HBAO can really compliment the package.

 

So my first request is a higher range for F-Stops. I have lenses that go up to 32.

 

It rendered 10 4K frames in under 3 Seconds! I am loving this!!!

 

It looks like you are using Unity 2019.3 alpha version. This probably comes from some bug in Unity with a new interface design. Soft Light script uses standard editor GUI, so it shouldn't give you this problem with editor layouts.

 

Your right, I didn't even notice it created a project for the Alpha Build I had installed instead. Still getting used to being forced to use Unity Hub. I'll try it again on the current Unity.

 

I truly understand you It happened also to me few times in last few days that I created wrong project version with a wrong Unity version. Old habits are hard to die.

 

I am hoping this will work with Alembic Files, USD Files and Hair Composer.
https://forum.unity.com/threads/released-hair-composer-nvidia-hairworks.453136/
https://blogs.unity3d.com/2019/03/28/pixars-universal-scene-description-for-unity-out-in-preview/

I already mentioned that a higher range of F-Stops to match all my real world lenses would be a huge help so I can use this to combine with live action footage.

And Render Passes would be a nice touch:
Diffuse, Specular, Emission/Lighting, Reflections, Depth, Velocity, Normals, AO, Motion Vectors, ObjectID, and Cryptomatte

https://github.com/Psyop/Cryptomatte

 

Small and useless animation that I've made with Deckard and HDRP-

 

USD should be supported via Deckard Timeline support, unless there are some bugs in timeline support: "USD animation caches and skeletal animation can be streamed into Unity using the Timeline. Each frame of animation is streamed from disk on demand, enabling efficient playback of massive animation caches."
As for Hair composer, I don't have any idea - it could work if developer implemented or implements Timeline support. Actually, it should work, but I don't know if it works with Deckard motion blur (that is probably a function that we want)

I can assure you that this isn't needed at all. 22f in deckard should match correctly with 32f. 22f almost doesn't have any blurring and in deckard should be even sharper that a footage from a camera. Higher f-stops from 22 are more about luminosity, and not about depth of field.
Anyway, if you need it, you can easily edit DeckardRenderEditor.cs script, and shange a line:

Code (CSharp):

1. VRP.fStop = EditorGUILayout.Slider("Aperture: " + "f/" + (VRP.focalmm / (VRP.irisRadius * 2f * 1000f)), VRP.fStop, 0.7f, 22f);

to:

Code (CSharp):

1. VRP.fStop = EditorGUILayout.Slider("Aperture: " + "f/" + (VRP.focalmm / (VRP.irisRadius * 2f * 1000f)), VRP.fStop, 0.7f, 32f);

This simply can't be done. That workflow is meant to be used with flat CG graphics that has to be post processed for natural looks elsewhere. Due to a nature of Deckard, that already has a natural output look, and behaves like physics camera, you can't have a usefull output from those channels. For exapmle, you can't get a correct depth, as deckard renders with a lens simulation blur. This means that blurred parts will have a incorrect depth. Same appplies to Normals, Object ID etc.. When you use multilayered export, you are always exporting raw and not blended data (and not even antialiased). Otherwise it doesn't work. Take for example a Material ID: with a lens blur, if you have two objects that are overlapping, and one uses flat green color, other uses flat red, blurred region will have yellow color - and yellow is another Material ID. Same applies to depth. As for AO, it could be done, but I don't see any point, as even modifying AO after deckard would result in unnatural effects.
You should look at deckard as a pre-final stage for compositing. You can import your videos, compose them, apply blurring, chroma keying, and then export final video that you might pass onto color correction software (if there is a need - but actually, there shouldn't be any).