Question XR grab interactible does not update bounds when child(ren) are added

Heya! I currently am having an issue with the XRGrabInteractable. In my game you can "merge" 2 objects together (essentially moving the second object to be a child of the first one). Both are Interactables. The second one gets its individual one remove, and only the parent has the XRGrabInteractable component on it.

The problem is however that it does not seem to allow the child to be hovered for the parent to also be targetted. Example:

2 Cubes are joined together. The red cube is the parent, the blue one is the child. If I were to point at the red one I can grab it. If I point at the blue one it does nothing.

I think it has to do with the child colliders not being used properly, but I have yet to find a way to update the colliders to work with that.