4.7 Editor: 3-5 second HITCH when selecting prefab instances with colliders (outside play mode)

We've had this problem for a while, certainly in 4.6 and earlier clients.

In our large project we have some complex scenes. When selecting prefab instances that possess (including children) colliders, a lengthy hitch takes place. If the colliders are all disabled or deactive, the hitch is gone. The scenes play fine, build fine, etc. This is an editor issue.

I tried profiling this with the 'Profile Editor' and 'Deep Profile' togglers and all that is printed is:

GameObject.Deactivate - 100 calls - 1264 ms (delay is definitely longer)

What can I do about this? Any ideas aside from writing logic that manages the enabled state of all my colliders and the disabling them all be default (ugh)? Does Unity know about this issue? Why does a non-playing editor care about enabled colliders?