Networkmanager is called from old scene

My Problem: I have a Lobby, where players can connect to a host, then the host clicks play and a new scene is loaded.
My Networkmanager calls a RPC Loadlevel function and has a OnLevelWasLoaded function, which instantiates a player.
What I noticed is, that the OnLevelWasLoaded gets called at least twice, if not a third time. I partly solved this problem, by only allowing a Spawn if it's at the same time, however it's still sometimes spawning about 2-3 Player at the client or the server.

So my scripts look like that:
(Note, these scripts are on a Networkmanager, which has to persist between scenes)
Levelloading:

Code (csharp):

1.     void LevelGate(int LevelN){
2.         if(Network.isServer){
3.             networkView.RPC("LoadLevel",RPCMode.All, LevelN);
4.         }
5.     }
6.  

and

Code (csharp):

1.     [RPC] void LoadLevel(int LNumber){
2.         //Debug.Log (playerInst);
3.         //Debug.Log(playerInst.networkView.viewID);
4.         Network.Destroy(playerInst);
5.         DontDestroyOnLoad(this.gameObject);
6.         Application.LoadLevel(LNumber);
7.     }
8.  

then, the everyone instantiates the playerprefab it needs:

Code (csharp):

1.     void OnLevelWasLoaded(int LN){
2.         Debug.Log("Level:" + LN + "Time:" + Time.time);
3.  
4.         if(LN != 0){
5.             if(Network.isServer) {
6.                 if(inTime == Time.time  playerInst == null){
7.                     SpawnPlayer(1);
8.                     Debug.Log("- Server with id "+ playerInst.networkView.viewID);
9.                 }
10.             }
11.             if(Network.isClient){
12.                 if(inTime == Time.time  playerInst == null){
13.                     SpawnPlayer(2);
14.                     Debug.Log("- Client with id "+ playerInst.networkView.viewID);
15.                 }
16.             }
17.             if(!Network.isClient  !Network.isServer) {
18.                 Debug.Log ("-Single");
19.                 SpawnPlayer(3);
20.             }
21.         }
22.         inTime = Time.time;
23.     }
24.  

As you see, the inTime variable checks, if the script gets called twice, which still happens...
With that method I then instatiate my players:

Code (csharp):

1.     private void SpawnPlayer(int PN){
2.         switch(PN){
3.             case 1: playerInst = Network.Instantiate(playerPrefab[0],new Vector3(0,0,0),Quaternion.identity,0) as GameObject; break;
4.             case 2: playerInst = Network.Instantiate(playerPrefab[1],new Vector3(0,0,0),Quaternion.identity,0) as GameObject; break;
5.             case 3: playerInst = Instantiate(playerPrefab[2], new Vector3(0,0,0), Quaternion.identity) as GameObject; break;
6.         }
7.     }
8.  

I then get atleast 3 TimeLogs as Debug in my Console, as well as the correct Object to spawn, but way to often.
This basically means, that the OnLevelWasLoaded function is called 2-3 times.

I have no Idea, where this behavior is coming from, but it surely it shouldn't do that.
Thanks for helping me.