Unity Multiplayer [Mirror] Cant change scene (isServer false...)

So I am trying to change my scene for all users but I have some problems with it.

First I have a Game Manager (not Network) that has a Timer etc. and checks if all players are dead.
If yes it will change the scene.

My code for changing scene:

Code (CSharp):

1.  
2. public void NextMap()
3.         {
4.  
5.             if (isServer)
6.             {
7.                 RpcNextMap();
8.             }
9.             else
10.             {
11.                 CmdNextMap();
12.             }
13.  
14.         }
15.      
16.         [Command]
17.         public void CmdNextMap()
18.         {
19.             RpcNextMap();
20.         }
21.         [ClientRpc]
22.         public void RpcNextMap()
23.         {
24.             manager.ServerChangeScene("newscene2");
25.         }
26.  

Okey, my problem is that I cant get Server Authority for the GameManager somehow. The scene is switched by clients but the server outputs
"There is already a player for this connection."

On the clients there is a NullReference to the NetworkIdentity

What am I suppose to do? Do I need to make the GameManager spawn by the server with Server.Spawn to have isServer true?

How to transfer players from one scene to another?

It works sometimes if a Server is also a Client. It still gives me the "already connected" error but the scene switches

 

For some reason you're trying to call ServerChangeScene only on the clients, when as its name implies you only call it on the server.

 

For anyone who's having this problem where ServerChangeScene doesn't seem to work, here's what I did wrong:
I had my server build uploaded into Playfab, where the scene that I was asking for wasn't in the build yet, haha