Problem with content size fitter

I'm trying to get an image to change it's size according to it's child, which is a TextmeshPro UI text, but I don't understand the content size fitter behavior.
On the parent I have a Vertical layout Group (because apparently I need that even if I have only 1 object) and one Content Size Fitter, as the picture. And on the child I have one Layout Element, setting the preferred size to 350, but for some reason it sets the size to 841 instead (which is the native size of the sprite).

I looked this up and tried many different settings and nothing works. When trying with a default UI image instead of my Sprite, it works as expected.

 

I stumbled upon this with a similar problem, however not exactly the same.

To solve your issue, you need to ensure "Child Controls Size: Width" is checked on the parent Vertical Layout Group.

However my issue is I don't want to set a PreferredWidth, I want the width to be calculated automatically based on the height of the image (using Preserve Aspect Ratio). While this works, it leaves a huge empty space around the native image's size. Where I want this area to all be cut back. I don't suppose anyone knows how to achieve this?