Anti-Aliased Alpha Test: The Esoteric Alpha To Coverage

https://medium.com/@bgolus/anti-aliased-alpha-test-the-esoteric-alpha-to-coverage-8b177335ae4f

An article I posted about using alpha to coverage in Unity3D. In some ways a continuation of an old thread of mine here:
https://forum.unity3d.com/threads/fixing-screen-space-directional-shadows-and-anti-aliasing.379902/

 

Nice article Bgolus

 

Hi @bgolus,

Thanks for writing that post, it has really helped a lot with my understanding of Alpha To Coverage. I am having an issue using it that hopefully you can help with.

When rendering the objects to a Render texture and then to the screen (via the UI system) the overlapping A2C object edges become oddly transparent and you can see through to the background, this does NOT happen when rendering the same objects directly to the screen.

I think this could be a bug in Unity as the outcome when using the RenderTexture is different depending on the build target, windows standalone looks ALMOST correct with just a few artefacts, but both Android and iOS have many many artefacts.

RenderTexture -> Screen (transparent black camera clear colour)
Note: Using a solid clear colour does not help, you still end up with transparent areas where the objects overlap

Android & iOS


Windows (Subtle artefacts, arrows added to highlight areas)


Direct To Screen - Expected Result (solid red camera clear colour)


I have attached a Unity 2017.4.9 test project showing the issues. I am also seeing this in in the latest LTS release 2018.4.9f1. The editor Game View shows the same artefacts as an actual on-device build. Make sure to switch to Android/iOS build target to see the worst of the issues.

Any ideas of what the problem could be?

Thanks a lot!

Niall


Note: My actual use case for rendering to a RenderTexture is to create icons with transparent backgrounds that can be used in the game's UI. Normally I would use pre-multiplied alpha for "dirty edge" issues when doing normal alpha-blending, but this appears to be a very different issue.

 

This is an interesting use case, because it’s showing off a subtle difference between normal alpha blended output and A2C in terms of what gets written to the target buffer.

Your comment about normally using premultiplied alpha when using traditional alpha blending is an important part of this story. Indeed when you render using traditional alpha blending the color values that end up in the render texture are premultiplied by the alpha. Also important is if you just do the usual Blend SrcAlpha OneMinusSrcAlpha the alpha values stored in the render texture will be wrong since the value will be the alpha^2 rather than the alpha. So the correct blend mode when outputting traditional alpha blended for use as a premultiplied render texture is:
Blend SrcAlpha OneMinusSrcAlpha, One OneMinusSrcAlpha

That means the RGB channels are multiplied by the alpha, and the alpha is not.

So why am I explaining how to properly handle alpha blending from a render target? Because I suspect this is fundamentally the same problem. However the solution is a bit more elusive.

When using A2C, the alpha value is being used to determine the coverage mask, but the alpha value is still there in the color value that the MSAA color samples have. This is a problem because it means the resolved color isn’t correct. Ideally you want each of the color samples to have an alpha of 1.0 since each coverage sample is actually fully opaque.

I can think of two solutions.

The first one is to not use A2C. “But that’s not a solution!” I hear you thinking, the idea would be to use coverage samples still, just not the automatic conversion of alpha to coverage. This would be done using the SV_Coverage semantic to manually set the mask. This means you could output the color value with an alpha of 1.0 and still get the benefits of using MSAA coverage samples like you got from A2C.

The venerable Inigo Quilez has an excellent example of some code to do this, with the addition of some noise for mimicking a greater range of alpha. If you use a constant “noise” value of 0.0 this will replicate what A2C does on modern GPUs.
https://twitter.com/iquilezles/status/947717661496041472?s=20

There is one big downside though. This requires DirectX 11, OpenGL 4.0, or OpenGLES 3.2, so it’s not a great option in this case since you want this to work on mobile.


This brings me to option #2. Inverted alpha. This is theoretical, so I’m not sure it’ll work properly. But the idea is this:
Clear the background to black with an alpha of 1.0.
Render using A2C, but use a special blend mode at the same time:
Blend One Zero, Zero Zero
Then when rendering you need to use a shader that uses a modified premultiplied alpha blend that expects an inverted alpha
Blend One SrcAlpha

Alternatively you could run this through a Blit that inverts the alpha value so you don’t have to use the special blend mode for the last part. However I’m not entirely sure this will work properly.


One last option would be to render to two render targets with one being color and one being just the alpha (output to the color value) and then merging the two into another render texture.

 

Oh, one last thing. The difference between mobile and windows is likely because you have your project set to linear color space, but currently mobile is always defaults to gamma color space. Alpha blending (and A2C!) behave differently as the blend is being done in that color space. In linear space the white subsamples overpower the red background making them far less obvious.

 

BTW, I tested out the inverted alpha "solution #2", and it does work.
I took the above project and modified the CameraRT to use a (0.0, 0.0, 0.0, 1.0) clear color.
Changed the A2C shader to use Blend One Zero, Zero Zero
Duplicated the built-in UI-Default shader, renamed it UI-DefaultPremultipliedInvertedAlpha and changed the blend mode to Blend One SrcAlpha, made a material using that, and assigned it on the RawImage.