Multiplayer and Mods

Hello, I'm trying to develop a game where the player can make their own mods and play with them. This works perfect in the client/singleplayer way. But I'm trying to make the users play with each other's mods across a server/host/matchmaking type of system.

So far, each client has a custom mod system, which basically loads assetbundles with code (this works perfect in singleplayer).

Each client has two mods, one containing a Map(scene with info), and a Vehicle(a prefab with a vehicle controller script and some other components).

When you start a matchmaker game, you will spawn in on nothing (at least at the moment) Until you chose a map, then you spawn in and can move around.
Then, a client can connect to your game, and they will be able to walk around if they spawn in on a map. (Their player position might be different if their on a different type of map) but you still can play with them
If you both select the same map, then you both can see the same things nicely.

The problem comes in when a player wants to spawn in a custom vehicle mod. This works fine, but only for the client. Nobody else on the server can see it.

I need to figure out a way to instantiate the vehicle into the server so everyone can interact with it, see it, and control it.

Side note: This also has to work with different types from different mods. I plan on making a Prop type or Plane type eventually. They need to be compatible with the network server, multiplayer system.

If you need any code, screenshots, or videos. Just ask for what you need and I'll be glad to send it.