Unity server and golang

Hello guys,
I have a new proyect where im currently working and implies a custom networking(cant use photon or any other current solution).

I have done before a similar thing in golang for other projects. I could use it with few modifications, but i need physics simulation.

My idea here is to use an instance of unity as a "simulation" server and connect to my server in go and use it as a bridge with my users.

My question is: do you thing that the overhead of unity will be too much? The project needs "realtime" performance.

Thank you very much and sorry for my generic question, but i really cant decice between do it all in c# or reuse my old things.