Aspect ratio fitter children positions

I have a map that has an aspect ratio fitter with mode set to "envelope parent". It has children that are positioned to specific locations on the map. The problem I'm having is when the map's aspect ratio fitter resizes to fit the device screen size, the children's positions are off.

How to I setup the children of an aspect ratio fitter to maintain position when it resizes? So its based on percentage of the parent.

Thanks.

 

The only fix for this has been to switch my Canvas Scaler back to "Scale with Screen Size" rather than "Constant Pixel Size".

 

Old thread, but did you find a solution for this? Your suggestion does not work.