Spawn failing to find target for sync message when client connects

My game currently consists of a large town with roads and houses randomly generated by the server on start up. When a client connects to the game, the houses and roads will begin spawning like normal. At some town sizes (more houses and roads), the client does not spawn all of the objects and receives the error

"Did not find target for sync message for (some integer)".

I know this error arises when an object on the client no longer exists yet the server tries to upkeep the SyncVars. I've narrowed the problem down to one script used to apply a rotation to the houses so they face roads correctly:

Code (CSharp):

1. using UnityEngine;
2. using UnityEngine.Networking;
3.  
4. public class LotTransformSync : NetworkBehaviour
5. {
6.     [SyncVar] public Quaternion syncRot;
7.  
8.     public void SetRotation()
9.     {
10.         syncRot = transform.rotation;
11.     }
12.  
13.     public override void OnStartClient()
14.     {
15.         transform.rotation = syncRot;
16.     }
17. }
18.  

When I omit this script from the house game object, no error occurs. Also, when i reduce the town size to something smaller than what I was hoping to have, there is no error. I don't want to limit my game's town size to fix the issue, because that might cause the error to randomly pop up when I add more objects to the game world.

I have channel 0 on my NetworkManger set to Reliable Sequenced, which other answers have said should fix things, but it still does not solve the issue.

Any ideas on how to fix this? Thanks in advance for the help.

 

I should also mention that if I omit this script and add a NetworkTransform to the game object, the same error occurs.