Objects with network identity behaving odd

I'm currently adding multiplayer to my single player game and in the single player game the player controls a squad of 4 soldiers. You can customize them before the match and as it loads a level it will load what you've chosen into these preexisting character skeletons on the map. Now that I've added the network identity component to them the game still runs fine without crashing but its just not loading their equipment in and isn't updating their animation controllers even on the host. The information is loaded in during the start() function using the binary formatter to pull data from a save file if that helps. Taking the network identity component off immediately fixes the issue but I obviously need it for multiplayer.

The weird part is about half the stuff will load correctly and the other half does not, and its the same stuff each time but as far as I can tell there should be no difference as two of the characters 100% load in fine while the other two are missing pieces.

Any ideas?