TextMesh Pro How to get consistent small text?

I'm using HintedSmooth / HintedRaster modes, but there are multiple problems that I think should be fixed.

The font asset that gets generated has to be changed to be usable. As it is, directly after font generation, the text looks horrible.

This is Libration Sans, Size 11, Padding 5, Hinted Smooth (I am aware that padding is mostly irrelevant here).
By default TMP sets the shader to TextMeshPro/Bitmap, which seems to enforce pixel-perfect display.
I'm saying that because changing OffsetX/Y in the debug settings only moves the text by whole pixels, which is actually great(!) if it weren't for whatever offset that gets applied here.


Canvas: ScreenSpace and PixelPerfect.
I've made sure that there are no fractional values in the hierarchy anywhere.
Obviously that's not even close to the preview of the texture. (which does look ok-ish... take a look at the lowercase 'i' or '1', 'T', all those have some sort of misalignment even on the texture itself.)

Can it be fixed?
Yes, switching the shader to TextMeshPro/Distance Field (yes, even with a raster font asset!), and then setting GradientScale back to 1, will get us partially there!

Not good, but better.

How to improve it?
This is the tricky part that I don't understand since there's no reason why the text should still be misaligned.
Now we can play with OffsetX/OffsetY to align it as best as possible.




Now the obvious questions:
- Why is all this even neccesary? Is it the bitmap shader that is broken, or something else? Am I doing something wrong?
- Can the text rendering for small fonts be improved so the base texture of the generated font asset looks as good as it does in Photoshop?
- What is the best way to get fonts rendered as good as it gets - like this with TMP? (This is 11px HintedSmooth Unity standard text, not tmp)

 

Havent tried textmesh pro, but using built in textmesh, I can get sharp text by assigning high textmesh.fontsize and rescalling the textmesh transform... it might look like stupid workaround but it works...