Third Party [Mirror] Looking for help in understanding what I am doing wrong with dealing with Server only

I'm working on a project, and in my head I want the project to play similar to The Forest, but I want the server to run similar to Rust, meaning the server is not a player, and persists all buildings, tree states, etc.

With the below code, I want to be able to chop down a tree > tree enables physics and falls > everyone sees this happen, and importantly, this will have happened for a new joiner.

Now, if I run the server as "Server+client" this all works as intended. However I don't want that, I want the server to not be a player and if I run server only, then try the below on a client, it doesn't work, the tree falls, but new joiners don't have it synced.
I tried getting some help on the Mirror discord server the other day, but it just seemed to complicate things and made it so new joiners don't see all the changes that have been made so i reverted back to what i had.

CameraScript>

Code (CSharp):

1. public void Update()
2.         {
3.             Ray ray;
4.             RaycastHit hit;
5.             ray = cam.ScreenPointToRay(Input.mousePosition);
6.             if(Physics.Raycast(ray,out hit, 2.0f))
7.             {
8.                 if(Input.GetMouseButtonDown(0))
9.                 {
10.                     if (hit.transform.CompareTag("Tree"))
11.                     {
12.                         hit.transform.gameObject.GetComponent<TreeScript>().CmdTakeDamage(20);
13.                     }
14.                 }
15.             }
16.         }

TreeScript>

Code (CSharp):

1.         [Command(requiresAuthority = false)]
2.         public void CmdTakeDamage(int damage)
3.         {
4.             Debug.Log("Hit");
5.             if (health > 0)
6.             {
7.                 health -= damage;
8.             }
9.             if (health == 0)
10.             {
11.                 DropTree();
12.             }
13.         }
14.         [ClientRpc]
15.         public void DropTree()
16.         {
17.             rb.isKinematic = false;
18.             rb.AddTorque(Vector3.left * 100f);
19.         }

As mentioned above, if the server is running as a host(server+client) it works perfectly, however, I want to run it as a server only and have all this work on clients.

I have read over Mirror docs several times, but just cannot seem to get my head around what I need to use, as soon as I can get it figured out on one example I can use that as a reference for future features.
Any help would be so greatly appreciated, If I'm in over my head please let me know

 

Rpc is sent one time to whoever is currently around.
SyncVars are synced state that is also synced to new players.

Make health a syncvar and perhaps make tree rigid body kinematic on clients. use it only on sever and sync position and rotation to clients via NetworkTransform.

so new players would see the fallen tree