Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.

Headless server

Discussion in 'General Discussion' started by Labewo, Nov 14, 2019.

  1. Labewo

    Labewo

    Joined:
    Apr 12, 2019
    Posts:
    21
    Hi,

    Mi Linux server is consuming 800 MB of RAM, is there any way of not loading the map and the terrain to reduce it? I tried with the batchmode and nographic options and building as headless but it only reduces 100 MB of RAM.

    Thanks
     
  2. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    18,360
    Yes, but you would have to implement it yourself. Basically you have your code check if it's in client or server mode before it starts loading data and then change which data loads based on that. That said just unloading the map and terrain isn't a good idea if you need the data for some purpose like server-side collision detection.
     
  3. AndersMalmgren

    AndersMalmgren

    Joined:
    Aug 31, 2014
    Posts:
    5,358
    How would server side raycasting be done without colliders?
     
  4. Labewo

    Labewo

    Joined:
    Apr 12, 2019
    Posts:
    21
    Okey, so I have to keep map and terrain. Any suggestion on what I can do to reduce the RAM only in the server build? I can not have a server that is using 800 MB of RAM per game room.

    Thanks
     
  5. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    18,360
    Have a less complex world. Have less of it stored in memory at a time.
     
  6. Joe-Censored

    Joe-Censored

    Joined:
    Mar 26, 2013
    Posts:
    11,825
    You could check out the Headless Builder asset on the store. It automatically replaces textures and audio assets with small dummy assets for your headless server build, which saves RAM with those two. As far as things like character meshes, you could switch to no longer referencing them directly in the prefabs or the scene, and instead get them via resources.load, and just don't load them on the server.

    If you're using Unet, it is notorious for high memory use, since it preallocates all possible memory it might ever use ahead of time. If so, switch to another network API.
     
    angrypenguin likes this.
  7. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,569
    Have you tried using the Memory Profiler to get more information about what that 800MB is made up of?
     
    Labewo, Joe-Censored and angrypenguin like this.
  8. Labewo

    Labewo

    Joined:
    Apr 12, 2019
    Posts:
    21
    Hi, I used it and the Texture2D is using 700 MB. But how is that possible in a headless build? I also tried the Headless Builder asset to change the textures to dummies but it still consuming that much RAM

    Also, the textures loaded in RAM are not used by the server scene (it is the first and only scene loaded). Is there any way to do not allow to load that textures in RAM if they are used only in another scene?

    Thanks
     
    Last edited: Nov 23, 2019
  9. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,569
    When you run with the null graphics device (you should see it mentioned near the start of your player log file) then texture data shouldn't be taking up memory like that, unless you have them marked as 'Read/Write Enabled.' Is that turned on for your textures, by any chance?
     
    Singtaa likes this.
  10. Labewo

    Labewo

    Joined:
    Apr 12, 2019
    Posts:
    21
    I disabled that and I reduced the RAM used. Now it uses 400 MB. 280 are from ShaderLab, but if it's a headless build how can I disable all shaders?

    In Linux it consumes almost a 60% more that in Windows. Is that normal?

    Thanks
     
  11. Opeth001

    Opeth001

    Joined:
    Jan 28, 2017
    Posts:
    986
    i hope this can help you.
    in my case i have 3 namespaces,
    Client: Contains all Scripts, structs, ECS systems related to client only (Camera System ....)
    Server: Contains all Scripts, structs, ECS systems related to Server only (Health management System ....)
    Hybred: Contains all Scripts, structs, ECS systems related to both Client & Server (Movement System ....)

    these namespaces are used to prevent scripts from accessing any forbiden data.
    eg: if a script is in Client namespace it can only access data from the same namespace or Hybred. (vice-versa)

    each script using Client or Server namespace have to be contained in a #if Client_Build or #if Server_Build symbol.
    also i have 2 different scenes one for Client and one for Server.

    this way by simply removing/adding the right symbols and Selecting the needed Scene, the Player will automaticaly include/exclude code, references and assets.

    Good Luck!
     
  12. Labewo

    Labewo

    Joined:
    Apr 12, 2019
    Posts:
    21
    Thank you so much, I will try that!!

    And how can I disable the ShaderLab rendering in a headless building? I removed lots of shaders an still consuming 200MB of ShaderLab rendering in a batchmode and no graphics (and in server mode).

    Shaders are only consuming 70KB of RAM, that's the reason I can't understand that ShaderLab takes 200MB of RAM.

    Thanks
     
    Opeth001 likes this.
  13. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,569
    So, investigating this a little, I think that there is actually an optimisation we can do in the engine that will bring this down a fair bit. I'll explore this with the graphics team.

    However, like with the textures, what will happen is that we'll discard the data needed for actual rendering, but we'll still keep around the 'object' itself. Compared to the rendering data this is pretty small - it's things like the width/height information for textures - but we can't do it automatically remove it because even in a headless build you might still have code that accesses things like Texture2D.width, or even just basic stuff like comparing whether two variables are referencing the same texture.

    If you know that your code isn't doing anything like that, and that you really do not need the objects at all - then what I think you should be able to do is one of two things:
    1. After loading your scene, use Object.FindObjectsOfType to find all objects of type Shader, Texture2D, etc, and just loop through and delete them all. This should bring your memory usage back down, though you will still have spiked to high usage during the load (and wasted a bit of time loading them). You might also need to call Resources.UnloadUnusedAssets() after you're done (and yield on its return value until it completes).
    2. I think you should be able to use AssetBundles to package up your stuff - you'd put your scene into one AssetBundle, and all the shaders and textures into another. Then at runtime, on the client you load the shader/texture bundle followed by the scene bundle, while on the server you skip loading the shader/texture bundle and just load only the scene bundle. Loading an AssetBundle does not automatically load any other AssetBundles that contain dependent resources, so even though the scene is using the textures/shaders/etc in the other bundle, if you don't load it then the scene will just fail to find those objects and will effectively get 'null' references for them.
    It's probably worth trying out approach #1 first regardless, just to validate that the memory usage does come down like you expect. This is also something you can do today, without waiting for any changes on the engine side.
     
    Ryiah likes this.
  14. snacktime

    snacktime

    Joined:
    Apr 15, 2013
    Posts:
    3,351
    I think a lot of people load stuff completely unnecessarily to start with. Like if you just want physics you don't need the terrain, just the collider. Lots of things like that which might not be readily apparent if you are new to Unity. What will surface those is to just take the approach of start with nothing and build it up piece by piece verifying what you actually need. Vs starting with a client scene and then trying to remove stuff for the server.

    A minimal Unity instance configured for server use took around 10mb ram last I checked a few months back. Mesh colliders and navmesh can eat up a bit, but really what is actually needed on a server shouldn't ever put you over say 100mb. Maybe spiking above that for scene loads but it will go back down eventually.
     
  15. Labewo

    Labewo

    Joined:
    Apr 12, 2019
    Posts:
    21
    I did both things and reduced a little the RAM but it stills consumes 200 MB because of the ShaderLab component... It didn't happen in the editor only in player build.

    All textures and shaders together only consumes 10KB so that rendering of ShaderLab it is not necessary but I don't know how to stop it.
     
    Last edited: Nov 27, 2019
  16. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,569
    Hmm, if you're sure that you are no longer loading your shaders but you're still seeing 200MB of Shaderlab allocations, then I think we will need a bug report about this in order to investigate further.