Small fonts have uneven baseline

Hi there, I have a small 8px pixel font that I'm using. I've set it to Hinted Raster and Point filtering.
My game has a small design resolution of 320 x 180, and the issue only occurs at this resolution.



On the left is how it should look at all resolutions, on the right is how it actually does look.
I am using a Screen Space - Camera canvas, with Pixel Perfect turned off (doesn't seem to help either way), and I'm using a Canvas Scaler set to Constant Pixel Size. I set the Canvas Scaler's scale factor based on the target resolution, as an integer multiple (2x, 3x, 4x) to scale up from the design resolution as needed. Interestingly enough, when the scale factor is set to 1, the text baseline is jagged. When set to 1.0001 or some integer multiple the issue goes away. I've also tried various small fonts under 10px and recreated the issue, so it's probably not specific to this font.