Mixing iOS native ViewController and Unity makes Unity unable to rotate correctly

I'm having an issue trying to integrate a native UIViewController and Unity together, where the app starts off as a regular native app, and later on switches to unity and back again.

I do this by setting the main window.rootViewController to nil, and making Unity's window.makeKeyAndVisible(). And just reverse that when switching back to native.

The problem I'm having is that Unity is set to autorotate, and the first time unity is shown, it works just fine and auto rotates to the devices orientation.
However on the second time and beyond unity is shown, when you rotate the screen, the screen itself rotates, but Unity becomes black and doesn't render anything. When you rotate back again, it stays black and is irrecoverable at this point.
There are no errors, and Unity itself is not dead. It's responding to touches correctly, and the window/view are all the correct sizes when looking in the debugger. Unity is just showing a black screen.

Anybody have any idea what would cause unity to black screen when doing a rotation?