Utilities [RELEASED] Deckard ChromaKey System (20%OFF)

I can announce that my hi-quality ChromaKey system that is targeted at live production workflow (for film and TV).
It has some advanced features that can be found mostly only on high end hardware solutions, but it is pretty performant and works in realtime.
On top, it will have also some robust and stable body motion tracking abilities for usage with cinemachine, or with any advanced scripting. I'm still working on this body motion tracking system, and on other features dedicated for broadcast.
You can watch few of my livestreams here (I'm always doing livestreams, just to show it's real life performance and problems):

In this one I'm talking about new tracking features.

Deckard Chroma Key Indie is made to solve needs of high quality chroma keying for Virtual Production purposes, Live events, prerendered videos, live streaming, AR/VR applications for broadcast.*
Deckard Chroma Key uses new paradigms in a field of chroma keying related to virtual production (TV, moviemaking and streaming).
Chromakeying is a pretty complex work, and Deckard Chroma Key manages to meet all needs of professional chromakeying by using some features not available even on industry standard hardware solutions.
Features:
-Chromakeying that can treat transparent objects.
-Chromakeying can use custom shaders and color correctors.
-Chromakeying supports shadows, SSR, volumetric lights, DOF and raytracing.
-high performance and stable person Tracking system that can be used with Unity Cinemachine or custom scripting for various effects.
-System for avoiding (or supporting) shadows on chroma keyed surfaces.
-Support for easy Garbage matte keying**.
Garbage matte is a process where you sample green screen with it's imperfections and achieve a perfect key even without sampling colors.
-Supports inputs from a webcam drivers, video Input or Texture (for simplified usage with third party plugins).
-Uses video relighting tech that automatically generates normal maps for usage with virtual lights in Unity.
-Deckard Chroma keyer is supported by Deckard Render for superior chromakeying for film.
-Supports HDRP and Standard pipelines.
-It features simple but powerful system for stacking denoisers, sharpeners, color correctors and other creative effects (with included holographic screen and comics effect). Authoring of those effects can be pretty simple with Amplify Shader or Shader Graph.

Indie version has all features as an Enterprise version but is meant for single developers/artists that earn less than 100.000$/Year. With Indie version you get a forum support.
If you are user that doesn't belong into this category, you should buy a Deckard Chroma Key Enterprise version, that gives you 4 hours of live training, consultancy and support via video meetings.
If you are a part of an educational institution, please contact me.

*due to it's nature it isn't intended or tested for mobile phone applications or low performance systems.
** Garbage matte can be applied only on static camera

DECKARD CHROMAKEY INDIE

DECKARD CHROMAKEY ENTERPRISE

 

First pages of manual that I'm working on.

Good chromakey requres good source footage. There are few guidelines hoiw to obtain good source images out of your camera.
-Choosing a right camera
Greenscreen requires a good camera and you will never get perfect results with a simple webcam. Webcams have a lot of noise, low dynamic range, do some image pre-processing and auto adjust their features and this can and will affect overall quality of your keying. Best way to obtain good input video is to use DSLR or dedicated Video Camera for this task. There are some pretty affordable solutions available, and even a cheap second hand DSLR will do a great job. I'm personally using a Blackmagic Pocket Cinema 4k camera. It is incredibly versatile budget camera that has some important features required for good keying: Manual control, high dynamic range and low latency. Unfortunately, even if it captures at 4k, it doesn't output 4k HDMI signal (it can output fullHD resolution - that should be enough in most cases). Tip: when Using BMPCC4K set its profile to Extended Video - this will give you an optimal dynamic range for best keying.
-Capturing with video capture card
Your camera or DSLR probably doesn't have a webcamera interface, so you will also need a video capture device that has HDMI (or SDI) input and can act as a webcam device. I'm using mostly Blackmagic Desing Intensity Pro 4k card (or Decklink) for this task, but there are many other available solutions on a market.
-Well lit greenscreen
Greenscreen should always be lit evenly as possible. Avoid having any wrinkles on your greenscren material. UniKeyer works pretty well even with unevenly lit screens, but this works mostly for cases when you are using a static camera. When you are using moving camera, well lit screen can be really important. Deckard UniKeyer is made to deal pretty well with all extreme cases and features many tools to do some good keying even with a problematic background.
Getting Started with UniKeyer
Once imported into your project you can add it to your scene by Right Clicking on hierarchy and going to Deckard UniKeyer menu and selecting Add ChromaKey Rig. This will add a special chromakey rig that contains all neccessary scripts and objects.
Next steps will be to configure your chromakeyer.
-Keying Profile
Here you can create or assign existing Chroma Keying profile. Setting a profile is somewhat similar to setting post processing profiles in Unity. But in this case, a "profile" is in a form of a material with it's shader. Once you create or assign an existing profile, you can then edit material properties - keying properties. But before you start doing so, you will want to set up a source of your video stream.
-Source Type
-Webcam: it will use webcam device or capture card
-Render Texture: it will use a render texture - this is usefull when you are using third party video capture plugins like AV Pro Live,AV Pro Decklink or when you are using Video player with render texture as a target.
-Video: you should use this if you are using pre-recorded video file.
-Key Type:
We have two key types, one is a standard keying type where you input a color for keying out background(for moving camera), and another is a keying with a Clean Mate Source (when your camera doesn't move).
Capture Clean mate
What is Clean mate source? It is a simple image of a background without any objects, for example: just a unevenly lit green screen. This type of keying can produce best results in a non perfecly lit environment. You can sample this image and save it as a texture in Unity by clicking on Capture Clean Mate.
Note: it will always save image with a same name: ultimatte.png If you are using more than one profile in your unity project, be sure to rename it after capture. This behaviour is here so that you can update clean mate image across many Unity scenes in one single run. But there might be some other cases when you are using multiple cameras, where you probably want to rename those sample mate images.
Custom Mask
Sometimes you probably need to mask some zones of your screen so here you can add a texture with masking zones. Black color is a masked color.
Setting a Key Profile (keying):
When using ost important values are Key Colors (there are two of them for better flexibility of keying if a), Threshold and ThresholdLow. This is where you are setting a range of color similarity to key out. Once you have those values perfectly set, you can tune other parameters. When keying a full figure, you want to have a full control on how you want to treat your keying. UniKeyer is capable of doing soft keys (where you want to preserve shadows and preserve real shadows or semi transparent objects. But sometimes you actually want to preserve smooth transition behaviour on upper part of a body (where your actor may use glasses, hold bottles or transparent products) but you want to eliminate any shadows that are casted on the ground. So here comes a need for separate Threshold parameters, one that works for the upper part of body, and one for lower. In this way you can tune differenlty both parts of an image.
Despill Chroma and Despill luma defines how the borders will behave and how a despilling (removage of green cast from a screen behaves). Despil Chroma should mostly be set to 1 but a despil Luma should be set on a base of your virtual background. White and bright backgrounds require positive values, dark baskgrounds darker values.
Border: defines how mush smoothing has to be applied to borders of a mask. This can help to have a natural defocused borders, and it can clean up borders.
Additional passes:
This is one of the interesting features of UniKeyer. Additional passes give you an ability to add multiple layers of chromaKeying (for other colors) or to add some special filters ie. It can be used for generating some effects like holographic effects, edge refinement shaders, beauty filters and so on. NOTE: additional apsses can have impact on performance as any image effects - they should be used with care and mostly for high quality applications.
Image Tracking:
One of the features of Deckard UniKeyer is it's ability to track figures. It uses a simple enough tracking to be able to run in realtime without big performance overhead, while giving an incredibly stable figure tracking. It can track extents of a figure, head Top and shoulders position. This tracking data can then be easily used with Cinemachine for virtual camera guidance, with robotic systems for automated camera movement, for alerting system when a figure is aproaching borders of a camera capture area, for activating behaviours (a virtual touch screen), or for example, positioning feets on a floor level and generating correct contact feet points for placing shadows. This system is in developement, and there may be many future uses for it.

 

Tuned in, very excited for this! Works great in arfoundation with shadergraph!

 

Hi, I saw Antonia's tweet, and the note above, is the asset available?

 

Not yet, but it is available for betatesters. If you are interested, send me a PM.

 

Watching your workshop with Antonia now. Any idea when this might be available? Looks amazing!!

 

Hi, I think that I should finish it this week. Right now I'm in final phase of betatesting and polishing interface. Lost probably too much time into implementing new features, and for this reason I have decided to stop working on new features before first release.
I hope to be able to submit it to assetstore this weekend.
As for pricing, I will have two versions of plugin:
-INDIE that will cost 39us$ and will have all features, but will be targeted to single team developers that earn less than 100.000/ year, with suppurt only on unity forums.
-ENTERPRISE, targeted at broadcast, filmmaking, youtubers and companies earning more than 100.000/year. It's licence should be 599us$ and will give support/consultancy of 4 hours via video conferencing.

 

Ah that's great! Thanks!

 

Oliver, this is a fantastic component and I look forward to supporting it via Asset store purchase!. Per your comments, possibly something for a future release?? Its regarding Shadows.
I don't know if its possible or not but I'm curious if there's a way to provide the ChromaKey component a primary shadow casting light source, whereby a duplicate of the content of the ChromaKey (after key removal), could be rotated/aligned using an angle somewhere between the direction of the camera and a designated light source?) such that the shadow cast, is more full. i.e. Keep the 'visible' chromakey content facing the camera, but somehow have the 'shadow' generated using an invisible, but rotated normal or source of the same. I used the demo scene and a video with some cubes. The directional sun sees the video as thin, but I'm thinking, perhaps if the normal or texture was angled between the camera and sun, the shadow cast would appear more full. Just an idea, and again, don't know if it makes sense or not.. Overall though, seriously, a very nice component so far with excellent control over chroma keying, and enhancing output colors.

 

Out of interest, does your chroma key system use an unlit shader? I was interested in why you chose to use DMX to the real world to match lighting rather than in-game lighting? Was it an aesthetic choice or a technical one? Just because presumably there will be a slight lag with going out to DMX

 

Looking forward to trying this. Will it be released soon or is there an early version we can try?

 

Hi everybody, I have a good announcement for all of you interested into virtual production.
My Deckard Chromakey is available on Assetstore. It is a high quality chromakeyer made for virtual production needs, and is tightly integrated with Deckard Render.

Deckard Chroma Key Indie is made to solve needs of high quality chroma keying for Virtual Production purposes, Live events, prerendered videos, live streaming, AR/VR applications for broadcast.*
Deckard Chroma Key uses new paradigms in a field of chroma keying related to virtual production (TV, moviemaking and streaming).
Chromakeying is a pretty complex work, and Deckard Chroma Key manages to meet all needs of professional chromakeying by using some features not available even on industry standard hardware solutions.
Features:
-Chromakeying that can treat transparent objects.
-Chromakeying can use custom shaders and color correctors.
-Chromakeying supports shadows, SSR, volumetric lights, DOF and raytracing.
-high performance and stable person Tracking system that can be used with Unity Cinemachine or custom scripting for various effects.
-System for avoiding (or supporting) shadows on chroma keyed surfaces.
-Support for easy Garbage matte keying**.
Garbage matte is a process where you sample green screen with it's imperfections and achieve a perfect key even without sampling colors.
-Supports inputs from a webcam drivers, video Input or Texture (for simplified usage with third party plugins).
-Uses video relighting tech that automatically generates normal maps for usage with virtual lights in Unity.
-Deckard Chroma keyer is supported by Deckard Render for superior chromakeying for film.
-Supports HDRP and Standard pipelines.
-It features simple but powerful system for stacking denoisers, sharpeners, color correctors and other creative effects (with included holographic screen and comics effect). Authoring of those effects can be pretty simple with Amplify Shader or Shader Graph.

Indie version has all features as an Enterprise version but is meant for single developers/artists that earn less than 100.000$/Year. With Indie version you get a forum support.
If you are user that doesn't belong into this category, you should buy a Deckard Chroma Key Enterprise version, that gives you 4 hours of live training, consultancy and support via video meetings.
If you are a part of an educational institution, please contact me.

*due to it's nature it isn't intended or tested for mobile phone applications or low performance systems.
** Garbage matte can be applied only on static camera

DECKARD CHROMAKEY INDIE

DECKARD CHROMAKEY ENTERPRISE

 

Thanks for sharing this! Looking great from what I've delved into so far. Obviously there's a lot in there!

I'm just having a couple of issues that I think might be related to the fact my project uses the new Unity input system. I think it might be breaking a couple of key areas.

1. Tracking. I'm using a portrait webcam video. It tracks my sides fine, but the shoulder and bottom trackers are just at the edge of the video, so it looks like I'm hovering in mid-air. I was wondering if this could be because of the GetKeyDown call in the Update function in debugSwitcher? Just wanted to check before I go tinkering with it.

2. Garbage keying just results in a totally transparent image. I'm not sure if this is related? Or maybe I'm just doing something (or more than one thing) wrong?

Despite not managing to get garbage keying (or the histogram) to work [I'm told the histogramMaterial hasn't been assigned, but I can't see anywhere to assign it], I've still managed to get a very clean image with a not-very-amazing green screen, so that's already great!! Thanks!!!

I'm using Unity 2020.2.6.f1, HDRP 10.3.1, Mac OS 11.2.3

 

Ok, so I've gone back to your live stream for Unity where you use your keyer, and I think the garbage keyer wasn't working because I wasn't using it right, but there definitely seems to be an issue with head, shoulder and bottom trackers. They're just not moving at all.

 

You probably have to tune in better keying, or change threshold values to meet your scene. It doesn't have anything with input system (it is only used for showing extents). BTW, do you see extents of tracking?

 

Ah, or probably, this could be more possible: you have to be sure to place a pivot point of a chroma plane on a floor level.

 

Haha, ok maybe I just don't know what I'm doing, sorry. I think there's probably some combination of settings that I'm doing wrong.

 

For those interested for some Q&A live, I will be doing a Youtube Live stream this friday at 22.00pm CET.

 

I'm getting somewhere vaguely. I'll keep messing around and if I can't figure it out I'm guessing you'll be covering some of this in your live stream on Friday anyway. Thanks for getting back to me!

 

For those needing a URP support, here is a patch that adds it.
https://www.dropbox.com/s/lj0e0hklast6mfy/urp_fix.unitypackage?dl=0

 

First live Q&A stream was made today where I went into describing main features of Chromakeyer.

 

Cool! A question, is it possible to receive a video feed from a remote location and input that source into Deckard Chromakey?
For example, I receive a quality video of performer in front of greenscreen from a performer in the USA, and in Canada I have the system set up for Deckard Chromakey and the virtual set for streaming onto the internet?

Second question, Deckard Render is not real time (or is it) and is Render used with ChromaKey?
Thank you.

 

For streaming, deckard chromakeyer doesn't have internaly any features for this. But, this thing can be done with proper hardware. I did some tests with Blackmagic Atem Mini Pro and Blackmagic Streaming Bridge. Atem is able to send a hi quality HD stream over internet, and streaming bridge can receive it and output it over a HDMI cable. It works pretty well with deckard chromakey.
Second question: deckard render is separate pack, and it is dedicated to offline rendering.
Chromakeyer can work with or without deckard render as it works in realtime.

 

Hi,
I just love this asset and it might bring me a new job.
But I upgraded to 1.5, when compiling I get :
Assets\DeckardChromaKey\DeckardChromaKey.cs(127,35): error CS0103: The name 'EditorApplication' does not exist in the current context

I tried did on your demo, only the scene in hierarchy and had the same message
Unity 2020.3.2 (PC)
Thanks for your help

 

I've sent you an update with this issue fixed.

 

Thanks so much, works perfect !
I have another issue :
in edit mode I keep having this errors :

UnassignedReferenceException: The variable materials of DeckardChromaKey has not been assigned.
You probably need to assign the materials variable of the DeckardChromaKey script in the inspector.
UnityEngine.Material.SetTexture (System.String name, UnityEngine.Texture value) (at <10564ed154d647e194bef4aef8878649>:0)
DeckardChromaKey.UpdateMaterials () (at Assets/DeckardChromaKey/DeckardChromaKey.cs:260)
DeckardChromaKey.Update () (at Assets/DeckardChromaKey/DeckardChromaKey.cs:176)

And I get it a lot, most of the times (999+). Some times only 50 times.
I did not find to what this refers to.
Thanks for your help.

 

Hi,The 1080p sony real camera is turned off. In an empty unity scene, there is only one Deckard chromakey prefab. Every few seconds, the CPU latency is very high, and the minimum FPS is about 7.source type = webcam.
Thank you.

Deckard Chroma Key Indie version:1.6
unity version: 2019.4.27f1c1

 

The 1080p sony real camera is on. In an empty unity scene, there is only one Deckard chromakey prefab. The CPU latency is always the lowest, and the FPS is about 7. source type = webcam.
Thank you.

Deckard Chroma Key Indie version:1.6
unity version: 2019.4.27f1c1

 

Hi,
(1) Now I use avpro live camera and Deckard chromakey. How can I set the parameters of Deckard chromakey prefab? For example - sourcetype.

(2) How can I use the Deckard material apply component? How can this component work with the av pro live camera component? Because avprolivecamera has meshapply, materialapply and so on, do you have any related tutorials?

(3)Look at the picture below, I tested it, but the green screen is still there.
Thank you.

 

You have to remove/deactivate MeshApply and MaterialApply scripts and Use just DeckardMaterialApply one. Otherwise those scripts will overwrite materials set by Deckard.

 

This is probably problem with combination of webcam drivers and unity Webcam built-in system. What graphics card are you using?

 

I removed MeshApply and MaterialApply scripts,but there is no video on the model.How Is the setting of sourcetype and material of Deckard material apply correct?

 

PC: Window10-64
Graphic Card: NVIDIA Geforce RTX 2070 SUPER
Capture Card:TC-4000HDMI PRO Card
Real Camera: Sony HDR-pj350

 

It's incorrect from what I see here. You are setting your own material with your own mesh - while you actually should use deckard chromakey plane for this (with it's setup of meshes and materials). This is mandatory. Replace that mesh with deckard setup.

 

Hi , I made a green screen mixed reality in unity .Test and use SonyPJ350e camera, its sensor size is 1/5.8 inch.
I want to simulate Unity physical camera to Sony PJ350e camera.How do I set the sensor size of the physical camera in unity? What makes their perspective the same?
Should the focal length be set the same?
How should focal length and sensor size be set in unity physical camera?
Thank you.

 

I just purchased this, and I can't get it to work with URP. I installed the patch, but I still can't find the right prefabs and am getting a bunch of errors. Any ideas would be appreciated!

 

What URP version? (You should use a prefab for HDRP)

 

V11. The HDRP prefab appears pink, even though there is a correct video assigned to the scripts.

 

Hi @olix4242 , The HDRP prefab appears pink,two materials(ProjectionPlane and ProjectionPlaneMaskMaterial) are missed too.

HDRP version:7.6.0
Unity version:2019.4.27f1c1

 

Hi, Have you solved this error now?

 

Nope!

 

Smallest HDRP version supported is HDRP10 (and bigger) - (as it uses some shader graph functionality not available on 7).

 

@olix4242 Hi, I wanted to follow up about the URP issue?
I can't get it working at all. Do you have an example project, or maybe an updated package?
Thanks!

 

What unity version are you using?
Can you open material from a shader graph?

 

Hi, thank you for the reply!
I am using 2021.1.12, URP v11, and yes, I am able to use Shader Graph, though I am not super well versed in it.

 

You just have to try to open it and see if it opens. As I'm suspecting that you are using an older version of shader graph and that this is a reason why it doesn't compile.

 

Hello there:
1. I used your software package in Unity2020.3.12, but many errors and warnings appeared in the console. What version of Unity can run correctly?
2. In addition to the green background, can it work correctly under the blue background? The many parameters on the material bother me.
Thank you

 

What errors do you get? (BTW unity 2020.3 should work fine).
As for blue background - currently it doesn't support blue background and is optimized for greenscreen. It could be added, but I sincerely don't suggest using it ever in digital domain as almost all cameras (all those that use CMOS sensors) are capturing blue channel at half of the resolution of a green channel resulting in pretty lousy quality keying.