White rims around transparent textures - an analysis

Hiho.

The often experienced and discussed problem that under certain circumstances, alpha textures often have white or incorrectly colored pixels along the rim is influenced by several factors. I did some exhaustive testing to determine, which factors in which combination are involved.

Some introduction, details, examples, as well as viable workarounds to actually prevent the problem (which lies in the generation of the texture, not with Unity) can be found here: http://answers.unity3d.com/questions/10302/messy-alpha-problem-white-around-edges.html

As my extension to Eric's answer is too large to fit in the "comments" section, I'll post it here for reference.

-----snip!

As the shown examples are incomplete and there are additional factors to be considered, I did some detailed tests for all possible combinations.

For better understanding, let me first clarify a few things:

• A "texel" is a pixel of a texture, a "pixel" is a pixel on the screen
• Mipmapping in Unity *exclusively* influences the look if the texture is displayed at smaller sizes (=several texels per pixel), never for pixel perfect or enlarged display. This corresponds to OpenGLs/DirectX's "minfilter".
• Mip levels are always generated using linear interpolation, even if filter mode is "Point" (wrap modes and filter modes never modify the texture).
• The following examples all show 3D objects - neither OnGUI() elements, nor GUITextures
• The textures used are 64x64, the NPOT variants are 65x63
• Trilinear filtering does the same as bilinear filtering, except that it also interpolates between mipmap levels, if mipmapping is enabled. So when zooming to/from an object with a mipmapped texture, with bilinear filtering you will see "flipping" or seams when the engine switches to the next mip level, while trilinear filtering will be smooth.
• In most cases, mipmapping does *not* influence the visibility of the white edges - but see below.
• The "Multi Sampling" anti-aliasing in the Quality settings does *not* influence visibility of the white edges. It never changes/improves texture samling, it only affects triangle boundaries and so on.

In the following images, the 6 columns contain 6 variants of an alpha texture:

• white transparent: a "standard" texture, where fully transparent pixels have a white RGB value encoded
• black transparent: a clean background that has the same RGB values as the visible pixels (=black)
• white transparent, NPOT: a "standard" texture, but "non-power-of-two" scaling is disabled, which makes Unity resample the texture, although the Inspector preview claims it has NPOT dimensions (but which is true only for GUI textures).

"AA slope" means there are interpolated/anti-aliased pixels present in the texture along the slanted edge - otherwise the original texture only contains either 100% opaque or 100% transparent pixels.

Note: You should generally ignore the NPOT images, they are only included to demonstrate "yes, all NPOT textures *always* show white rims, regardless of any mipmap-, filter-, or NPOT settings, or zoom level".

The 5 rows contain different combinations of mipmapping and filter modes. Note there is no row "no mipmap, trilinear", as it is always identical to "no mipmap, bilinear".

NOTE: The images are too wide, so the forum downscales them. The link to each original image is given below.

First image: pixel-perfect (=exactly 1 texel per pixel) screenspace aligned objects.


First image, original size here

Observations:

• all rows are 100% subpixel identical, except the NPOT variants in the first row (as a side note, with mipmap/point the NPOT setting "None" is *much* worse than any of the other NPOT settings)
• ignoring the internally rescaled NPOT variants, **no** white edges are visible, except for the anti-aliased slope, which @Eric5h5 explained

Second image: pixel-perfect screenspace aligned objects, but with a 0.5 pixel offset in both X and Y. This can easily happen if your screenspace alignment method does not take even/odd screen resolutions into account, or computes the wrong offset.


Second image, original size here

Observations:

• mipmapping is irrelevant in this situation. The two "point" rows are subpixel identical, as well as the three "bi-/trilinear" rows
• all linear interpolation methods produce white rims for all "standard" texture variants

Third image: Objects scaled to 70% (=2 texels per pixel). This triggers the next smaller mip level for point+bilinear (while trilinear interpolates as soon as the scale is <100%)


Third image, original size here

Observations:

• now even the "mipmap, point" textures exhibit white edges!
• the only "standard" texture that has no white artifacts is the "no mipmap, point" variant
• note that white edges immediately appear at *any* scale that is not exactly pixel-perfect
• note that only at scales <100%, mipmapping does anything at all
• at 70% (i.e., 1/sqrt(2)), bilinear+point mipmap suddenly switches to the next mip level, and again at 50%, and so on. trilinear always interpolates

Last image: Objects scaled to 200% (=one texel per 4 pixels).


Last image, original size here

Observations:

• again, mipmapping is irrelephant. All point (resp. linear) rows are identical.

I attached a snapshot of the Inspector view of the texture.

Conclusion: Never ignore the RGB values of 100% transparent pixels. Don't leave them unmodified (=white), don't simply set them to black. They should have the same, "extended" color of your pixels along the rim.