[RELEASED] Deckard Render - Cinematographic Renderer for Unity

I'm working on a new addon for Deckard (that I think to include in a next release).
I was working on some compositing stuff that requires to use background and foreground videos mixed with deckard renderings and started implementing some video functionalities just to be able to scrub video in timeline even in edit mode. But then I've discovered that it is impossible to scroll videos with a perfect and correct timing. Unity will always skip some frames here and there or advance faster than video source.
So I came to a solution of using image sequences to correctly sync video with timeline and to be able to have fast previews while working in timeline.
Workflow is more or less this:
-You open a video file (from provided script) that you want to use as texture.
-Script generates two image sequences from this file: one is high resolution (for final rendering) and another is a proxy sequence (lower resolution sequence that can be used in timeline in edit mode - it makes scrolling much faster and more responsive).
-From there, you can just assign timelin to this component, and it will scroll your video correctly.

Is this something that is generally needed and if it is, what features would you like for me to include?
p. s. I think that this could be even a small pass into making some simple video editor for deckard sequences

 

Hi!
Please tell me how i can make the steadicam effect together with the Deckard Render.
We use the Slate Cinematic Sequencer as a timeline editor. (Built-in steadicam effect does not work)

 

Hi. I don't own Slate and actually don't know how it works. I assume that even if you get it to work with it's steadycam, it will not support correct motion blur. I have contacted developer of Slate, and I will se what he says.
For now, if you need to copy movement of Slate camera, you can do so by setting deckard camera as a child of slate steadycam. Or make a script that will copy transforms (in update) from slate to Deckard camera.

 

In general, now we choose which timeline we will use (slate, cinema pro director, build in) so using a slate is not important .. Maybe you advise a specific script to achieve steadicam/swim effect that would work with the decard. I do not understand programming in c# at all. I searched a lot on the Internet for camera shake, noise, stadicam, etc., but found nothing. For almost a week now I’ve been trying to resolve this issue and to no avail. Help me please.

By the way, slate works great with a decard, with the exception of the steadicam effect.

 

I heavily anticipate this feature. Right now I have a 2400 frame animation and I need to render a shorter corrected frame sequence after the 2000th frame....

Could you simply deactivate Deckard in the starting frames (with an option to increase timescale until the set frame to make this time shorter) and reactivate it several frames earlier and start rendering and saving the images from there?
I wrote several frames earlier so that Deckard could have enough time to warm up motion blurs and else it needs.

 

I tried to use the Deckard Skylight and I was puzzled with what parameters it was created with from the menu:

Is there any reason for it isn't exactly vertical? I supposed it was tricky workaround to turn the light a bit to overcome some glitch otherwise, but I found that making the light vertical resulted in totally clean shadows, the existing glitches went away. This might be a "bug" then that the prefab isn't vertical.

And why is the Light Size 75.67x80 instead of a nice round number, like 80x80? Probably it won't matter, but I couldn't help thinking that there must a be a reason for this.

This skylight reminds me of how we faked ambient environment lighting for archviz in the old days before GI, placing spotlights around the scene in a hemispherical array. The base of this skylight is a directional light, and I thought making it an area light with Deckard would just render the same light multiple times from the area of the rectangle, but I see that the direction of the "sublights" converge, they are not parralel, this behaves much like the hemisperical light I described. What is happening behind the scene, how do you make the directional "sublights" converge?

 

I was experimenting with the Deckard soft light component to understand how it works and a strange thing happened. I added it to a narrow spot light and set a relatively large soft area.
I knew that Deckard somehow makes an array of the light and even turns the arrayed lights (at least for directional lights) but it seems that with the spot light this didn't happen. I see that the spot light is just arrayed in the square, perhaps not rotated, but as the lighted circles on the floor are different size I suppose that the spot is not arrayed on a plane perpendicular to the light axis, but gets moved in the direction of the axis too. I don't see why this would be needed, not just using the perpendicular plane.
But maybe I am wrong with my observation, and of course, this exact setup is quite nonsensical, a larger spot angle would be more reasonable for an area spot light.

 

I wonder is it possible to use Deckard Renderer with Visual effect graph effects?
I want to offline render real time effects.
Thanks for your effort by the way. This asset is amazing!

 

I ran into some trouble with object motion blur and I am looking for a solution.

In my animation scene objects are positioned in their default position and they are deactivated in the first animation frame by a script. I need them visible when editing the scene because I don't want to have to switch them always on and off.
Later at certain frames, using a script, each object is positioned to another place, activated to become visible and its position interpolated to the default position through a number of frames. This is like objects falling down from the sky.

The trouble is that when the displaced objects appear they are motion-blurred from the original location. What can be done to avoid this one frame unwanted motion blur?

 

Hi OliVR,
Would please for the sake of cleanliness, organize all the Deckard materials and post processing effects (alpha, depth) under the Deckard category?

 

You can do this by setting your frames in Unity timeline. Deckard will just follow any starting point.

 

Like you said, this exact setup is nonsensical. So, when setting area, you should also set your spot angle accordingly. I cousl fix this issue also in a script, but I won't do so, as this gives more flexibility for simulating some effects (like Led strips).

 

I havent tested Visual Graph, but I don't see any reason why it shouldn't work.

 

Good point, I will do this for a next update.

 

Unfortunately nothing can be done (other than activating and deactivating objects or animating with timeline by using curves set to FLAT). Deckard respects physics of a real world and it has to assume that those objects are moving from one point to another).

 

But the issue is not primarily about spot angles, it is about moving the sub-lights in the direction of the light axis. If there is a lighting fixture model or a shadow casting surface right behind the spot it can happen that some sub-lights will get behind this surface and be "blocked".

 

I'm thinking about deactivating just the mesh renderer instead of deactivating the whole object. This way the Deckard Object Motion Blur script should be able to run every frame and visibility would be controlled with the mesh renderer component.

Or have the Deckard Object Motion Blur script deactivated on the object and activate it only when it is already displaced from original location. I'm not sure this would work since I found that regardless if this script is enabled or not, Deckard renderer will find the object and force motion blur on it.
Update: In fact, if I just deactivate the Deckard Object Motion Blur script then the object will stay in the same place even if it is transform-animated.

 

Sure I can do that but Deckard will not take this into account and keep numbering the saved frames from 00000. (At least using SLATE timeline editor.) So this hardly can be used for re-rendering a bad subsequence because the rendered images need to be manually renumbered.

What could be done to make Deckard offset the starting frame number either automatically or manually?

 

Right now there is no other way. But I will look into implementing this one today.

 

Ok, I've made a fix for this issue. Now you can set Resume from frame value, and it will fast forward and render from a desired frame.
https://www.dropbox.com/s/2br0wtel8rt6k01/deckardrenderresumefix.unitypackage?dl=0

 

Thanks for this fix, it seems to work in a simple test scene. I'll try it with the production scene...

 

I tried it and it works well, but the remaining time calculation is broken, shows a very small value.

I'd like to add a wish to this feature, the option to define not only the starting but the ending frame number, too, so that the sub-sequence renders would stop after the specified frame.
I know that the "number of frames to render" field could be used for that, but it would be more convenient for this subsequence render option to have a separate ending frame so that the main render length could be kept permanent.

 

I noticed that Deckard Render settings are not affected by undo. I change values and settings and undo doesn't restore anything.

 

I have contacted Slade developer, and he was kind to give me acess to his Slate Editor.
A solution for using Slate with Deckard was pretty simple (but I hadn't had a time to try all possible use cases - mostly because I'm still not familiar with user interface).
This simple script should be enough for Slate to work.
-Attach DectkardSyncToSlate script to Render Camera (in director camera root).
-Set a Cutscene to your desired Slate Cutscene.
And you should be done.

p. s. Please test this workflow, and let me know if it works.

 

Hey there, I just bought this but am having a heck of a time trying to get it to work. Are there any known compatability issues with Linux? It causes my editor to become barely responsive, and when I try to play to render, it gets to about 2 frames and then just closes Unity.

I don't think the massive editor slow down is my PC but you never know. I am using Ubuntu 19.04, 2019.3.0b6, standard renderer using Vulkan. My pc is a R5 2600, 32gb ram, RTX 2070, NVMe SSD.

 

Deckard wasn't tested on Linux, but it by itself shouldn't give any compatibility issues as it doesn't do anything else than just rendering.
On the other hand, as it is pretty graphics intensive, it can stress your hardware and software. Let's say, if there are memory leaks in Unity (or bugs in drivers and Unity), with Deckard they will fill memory much faster leading to a crash. Also, if hardware doesn't have good ventilation, your PC might get hot and overheat faster. First thing that I would check is GFX card temperatures.
Also, I would try to switch to OGL and see if a problem persists.
p. s. your hardware should be able to work really fine with Deckard (it is more or less similar to mine). But you are probably dealing with some issues like drivers, bugs in Unity or thermal problems. I also had some problems in some unity beta versions - when Deckard crashed Unity - and just changing Unity version fixed those issues.

 

I am super careful when it comes to thermals. My components are watercooled and I keep high flow dust filtration on all parts of my case. I have a hardware stats monitor opened at all times on one of my monitors and made sure to watch them as I gave it a go. Nothing out of the ordinary happened. I would venture a guess it may be with the editor as I make sure to keep my system up to date and am using official Nvidia drivers.

At least I hope that is the reason. I was quite excited to use it as I have been working on a cinematic the last few days and I feel Deckard would really give it a nice touch, but as it stands I, unfortunately, can't really do anything with it. : (

 

Yes.

Something in your scene (post-processing? volumetric effect?) is burning memory as a result of Deckard manipulating Unity's time to re-render frames from slightly different angles. You are able to render a few frames before the memory as all used up, then Unity crashes. I see similar behavior on my Mac, except after a few speedy Deckard frames Unity starts using the SSD for RAM. I don't crash but all computer processes slow waaaaay down and I have to restart before the memory is completely released.

I'd start by removing all post-processing fx, if you have any…. If that does not work try a bare bones scene and add your elements to it, until you find the process(es) that cause the crash. Once you know what the problem is, you might be able to work around it, or switch up the script order, or do post-processing in a literal post-process (in a video editor).

good luck!

 

------ EDIT - - It seems *not* having the basic post processing layer on the camera enabled is what was causing the camera to not work for some reason. I enabled it but left the post processing volume in the scene turned off and now the camera is working again.

Good call on removing of the Post-Processing. I disabled that, as well as Enviro's weather and it's post-processing and it seems I am able to get it functional enough to begin rendering. I can't seem to get the camera to work properly though with Cinemachine. I noticed that placement plays a role on the camera gameobject. Where exactly do I need to have the Cinemachine brain gameobject? I have the Deckard component on the main camera only, between the Preprocess and Deckard render component I have the (now disabled) post-processing items, then above that I had the Brain and Animator gameobjects, I also made sure to add the Timeline director component into the Animation/Timeline settings menu category, but when I scrub or play the scene, the camera no longer functions properly. It does indeed begin rendering, but the camera is not animating or following the paths in which I have setup within Timeline.

Also, am I just going to have to deal with not being able to use the post-processing? It played a pretty significant role in the overall mood of my scene I am trying to render. I hope it is not going to be a "damned if you do, damned if you don't" type of situation, where I can't render the scene with the post-processing, but without it, the scene isn't worth rendering anyways.

This is the scene I am working on that I was trying to render to start out as a trial. It would be nowhere near the same without the processing, unfortunately. : (

 

It's a cool-looking scene.

Unfortunately, I hit similar convenience/workflow decisions over Deckard and Unity's PP (PP2 specifically). There are 3rd-party alternatives for most post filters – I bought plenty before PP was a thing.

I was also told I might have "dirty fans" haha.
At least we have confirmed what the guilty culprit is.

 

Sorry, two last things. For the "Number of Frames", if I am using Timeline, is there a way to have it render just the length of the actual timeline, whatever that happens to be for the current settings? If not, how exactly am I to know how many frames it is? My timeline says 1570, but when rendered, it ended up being only 627. I am guessing a discrepancy in the framerate set, but is there a quick and easy way to know how many frames it is going to be for a given timeline sequence with the current settings?

Also, I am not sure why, but my animation of the second undead is acting really... weird? If you see the first video I posted, then take a look at this one below. I don't understand why the animation is like, hopping? It's definitely not what it is supposed to be doing, lol.

 

This looks interesting and I'd like to test it but I'd need to know what exactly this synchronization scripts does, what should I expect it to do?
I see that the script updates the scene to Slate's timeline for the specific sub-frame time on PreRender, so even CPU based geometry deforms will have correct motion blur?

As I wrote previously here I was able to make the two tools work together through some specific settings already, seemingly working well. At least the Deckard features I am using. No need for those settings if I use this script?

(By the way, I am not using Slate's render camera just a normal animated camera, should I add this script then to my own render camera? The scripts looks to me like it could be on any gameobject.)

 

You should set timeline to use correct FPS. (in timelines right upper corner) - set it to match a FPS taht you want to use with deckard.

 

Well if using Slate or any editor, I'm suggesting to go with consistency, and use that editor for everything that it is ment to be used. Otherwise, you could potentially have various problems that could be really difficult to pinpoint. Slate is made also for camera work.

 

I really can't give you an answer without testing a scene From what I see, it doesn't seem like any issue that I know or have seen. Does this happen only with deckard or also when playing normally? Are your characters controlled with timeline?

 

I understand your recommendation, thanks for that, but your response dealt only with the topic in parenthesis in my post. Could you please also take a look at what I wrote as the major body of my post:

 

CPU based deformation is a really broad concept. In theory, it should be taken into account if it's controlled by Slate.
But this also pretty depends on if it's controlled by some custom scripts that work only in update.
If this is a case, then instead of running scripts in Update, they should eventually run in OnPreRender() and adjust their time as it is done in a Slate script provided.

 

Was is the current state regarding multi-pass rendering? It wouldn't be much useful for me if it only supports a beauty pass and nothing else.

 

We just purchased this plugin.
We are glad to say this plugin can achieve amazing results. We just have one problem we would like to ask you.
Is there any chance to render with alpha??

 

Unfortunately, due to how Deckard renders (by simulating real optics) and how Unity works it is almost impossible to make it to support efficiently an alpha channel without hugely impacting performance and VRAM consumption. I'm trying to find a way around this problem, but unfortunately I'm almost left without any options. I will work further on this issue, but I don't have any ETA on this. This is a same reason why Unity most post processing effects don't work with alpha.
As a workaround, I've made a script that can be added to post processing stack and used to render separately only the alpha channel (it requires rendering a scene two times, once with a normal rendering, and once with an alpha).
I understand that this can be a priority for some users, so I really would like to find some better solution.
https://www.dropbox.com/s/868y2oeltnxq53p/DeckardPPSv2_channels.unitypackage?dl=0

Answer above explains a little bit a situation: As for now, there is no way to render more passes at once, as this would hugely impact overall performance. Only solution is rendering separate passes as explained above.
BTW, I really don't know what is Beauty Pass. Different softwares use this terms with different meaning. Usually it should be a final render but sometimes this term is used for only texture objects.

 

Update on Deckard Developement:
-I was able to add aplha channel export option. For now, it works only with objects that write in Zbuffer (this means, no support for semitransparent objects).
-I am adding some new features: one of them is a bloom effect optimized for Deckard. It is more similar to an actual lens response to bloom and lense flares.

p. s. I'm posting one new animation that I did today.

 

Your animations are looking great! I just purchased this today. I'm hoping it gets continually developed (even if slowly) and not deprecated like most new assets on the store these days.

Aside from that, can I ask about how the DOF is calculated? I know it's a camera style simulation so not everything is 1:1 with how a game engine would react, but I feel like I'm missing something big here. I took the demo scene and just plopped a car down in it to test stuff out and look at how everything works with the renderer. I followed your above statement to animate the DOF focus point instead of making it static (like the car for example) and the actual focus point seems to be working fine, but the final result still seems a bit blurry. Maybe I'm not doing something right with the camera settings?

When I view the DOF in debug mode, everything looks fine, the lines are way behind the wheel. However it looks like there is some type of "near" blur going on that isn't being shown in the debug view. Is this the case? Or am I being dumb and missing something obvious here? Thanks for the help in advanced.

 

All the blur that is there in debug view should be there in final render. There isn't anything addittional going on.
I think that your main problem could be a resolution - you are rendering in 720p and this resolution is pretty low to get any sharper image.

 

That may be true, I haven't tried a higher resolution yet. I'll try it out and get back to you. Thanks

 

Seems to have solved my user error! Rendered it out in 4k and it's looking very crisp. I feel like an idiot

 

Good Morning. Just a question. Is it possible to render 6k ( 6144x6144) output video ( of frames)?

 

Hi, it really depends on how much VRam do you have and what image effects are you using. In some cases rendering a resolution that high isn't supported by image effects, and you might run out of VRam that will lead to crash of Unity.
May I ask why you need that resolution?

 

I'm working on some new exotic features for Deckard Render.
-Light Fields rendering option. Lightfields are pretty exciting tech that can 'describe' a 3d scene in 3 dimensions. It can be used in VR or with holographic devices like Looking Glass.
-Off plane projection. This is a type of custom projections mostly used in fields of screen mapping or architecture visualization.
Light Fields in action with Looking Glass:

 

Lightfields, yay! This is a very nice demo you showed. I am missing lightfield support from Unity since long ago.

But how is it related to Deckard Renderer? Deckard is meant to render out videos but the essence of lightfields is interactivity, isn't it? Wait, I think I get it, Deckard will not input lightfileds but output them instead of animation frames, right? I thought first that this video was rendered with Deckard with a lightfiled asset imported into Unity. Didn't know the Looking Glass device before.

By the way, do you have a recommendation for displaying lightfields with Unity in VR, like in the Welcome to Light Fields app?

 

Nope, essence of lightfield is a possibility to have correct parallax effect on screen X or Y axis. This means that you can view correct auto stereoscopic image from any distance (in VR or with holographic display like Looking Glass), to move camera on Z and Y axis or to put things in focus after the rendering. So now Deckard can output this type of images and videos. Usage? VR comes first in mind. Backdrops for 2D scroll games. Backdrops for 3d games.

No.. I didn't do any research if there is something. But I will be putting my hands on making some shader that can be used for this purpose.
This is how a LightField image looks (it is called Quilt) it contains small images of a 'scene' and that image afterwards gets reconstructed into a large image.