First and Third person rigs?

I have a game that allows you to swap between third and first person. Each perspective has it's own rig and camera.

So when I swap, I simply disable the current rig and enable the other.

However, this causes noticeable issues - since the rigs have different animator controllers, the states don't exactly carry over, and as I'm sure you all know, disabling and enabling a controller causes some wonky problems.

My current thought is to simply always animate and update both rigs, and just disable the SkinnedMeshRenderer for the rig that is supposed to be disabled. And I would also need to make it so it doesn't trigger any animation events, somehow. :^)

Any other ideas?