Unity UI Changing interactable flag renders a frame

I'm disabling a button in the Awake function of a script in a scene. When I load the scene the button is shown enabled (as part of the scene) for a single frame and then disabled afterwards. This causes a flicker on the button each time the scene is loaded which doesn't look very nice.

Is there a way to fix that?