Unity 2021.2: Dedicated Server target and stripping optimizations now live! Please share feedback!

Last year, we shared our goals for 2021. Three of our key focus areas have been improving workflows and usability, improving performance and iteration speed, and focusing on multiplayer networking. As a part of this, we’ve developed a new way to allow developers to create Unity dedicated server builds regardless of the netcode you choose to use for your application.

What is Dedicated Server target?
Our new Dedicated Server is a desktop sub-target that improves server build efficiency by automatically stripping assets and code that are not necessary for server builds each time a project is built, providing increased space and processing efficiency. As a result, Dedicated Server builds also make it more economically beneficial for server operators to run these more-efficient server applications created with Unity.

Why did Unity create a Dedicated Server target?
Traditional Unity headless server builds often contain assets and compiled code which are not accessed by headless server processes. This might include artifacts such as audio files, textures, meshes, and shaders. In a multiplayer context, rendering and asset management processes occur unnecessarily when server runtimes are being built and executed. This even happens with traditional headless Unity servers since the rendering operations are still executed, though to a null-display target. When doing builds with all this additional overhead, longer build times occur and iteration speed suffers, too.

How do I use this new feature?
Starting in Unity 2021.2, when setting your platform target, you now have the ability to specify Dedicated Server as a target type with an option to choose for which Desktop platform. This not only makes it more clear what kind of build you are creating, but also allows Unity to know to apply these optimizations during the build process. Another advantage is you get to programmatically query if the target is set to Dedicated Server in order to program Dedicated-Server-specific behaviors.

How can I access this new feature?
Dedicated Server platform support is now available inside the Unity Editor as of Unity 2021.2.0b1. Before 2021.3 LTS, Unity will move the feature from the default Unity Editor installation and into a separate support installer via Unity Hub in a future release.

What feedback is Unity looking for?
We’ve reached the stage in our development where we’d like to get your feedback on this new functionality and how it performs when tested on real game projects. These changes are focused on the way we build dedicated server projects and the final content it is packed in it, therefore the types of feedback we’re looking for are:

• Data statistics on the projects you build with this version versus the results from build with previous versions of Unity. More specifically:
  • Project size on disk
  • Memory usage
  • CPU usage
  • Build time
    
• How this new Dedicated Server target feature impacts your workflow and iteration times
  
  
• If changes made by the Dedicated Server target could break or conflict with any previous approaches or features of your project

We welcome your feedback on ways we can improve this tool in the future, too! Please feel free to reply with feedback on the features that you would like to see in upcoming versions. If you would like to share your experience or provide any feedback or suggestions for us, please leave us a comment in this thread.

What about bug reporting?
Ideally we'd like any bugs reported through the built in bug reporter tool, as that will automatically provide us with some relevant context. For instructions on how to file bugs through the Unity editor, please go here.

Once you have submitted a bug report through the bug reporter, please feel free to refer us to it in your replies or new posts.


Changelog

• [UI] Dedicated Server target is now listed in the Build Settings window.
  
  
  
• [Memory Optimization] Pixel data for textures that are not marked as read/write is not included in Server builds.
• [Memory Optimization] Vertex data for meshes that are not marked as read/write and not used for internal systems, is not included in Server builds.
• [CPU Optimization] Audio system disabled in Server builds.
• [Scripting] UNITY_SERVER defined for Assemblies in Server builds.
  

Currently Known issues

• Error message “shader is not supported on this GPU” in Server Builds
  

 

What code is stripped exactly?

If you don't mind me asking, did you do any long running dedicated server tests?
For example, what about performance & memory leaks after several days / weeks?
Is there any data you could share with us?

Given all of Unity's stability issues, people are often concerned about launching servers into the Unity black box.

 

No user code is affected. We follow the same stripping rules as the regular player. For now and in this first release, we're only stripping and optimizing engine code around instance rendering and audio.

Unfortunately, we don't have long duration test outcomes, or performance improvement comparisons. This is the information we hope that you and other users can share back with us.

In terms of stability, there should be no crashes or memory leaks. As with any feature coming from Unity, we have a litmus of unit testing that we do on small samples and test suites. However, if any are discovered by the community, we do appreciate your bugs filed using the Bug Reporting tool.

 

Thanks for the reply.

Your community is more than willing to use Unity for their game servers.
Sometimes it's not obvious if Unity is ready though.

For example, mention using Unity for a game server anywhere on the internet.
People just laugh at you. They laughed at us for years, still do.

Point being that nobody knows what's going on inside the black box.

• Getting regular bugs fixed is virtually impossible. We've tried, a lot.
• If a production game encounters a leak, we can't expect the poor QA guy to run it for a week with 100 CCU to reproduce it

Imho for Unity to become a serious option for game servers:

• Either the community needs to see inside the black box.
  
• Or the people that can see inside the black box need to show believable commitment to game servers.

Long running tests would be great.
Open sourcing more parts would be great.
Unity making their own multiplayer game would be even better.

 

Why is this called dedicated server build?
This would indicate it is building a server, but in fact it just builds a headless unity build.
So calling it Headless Build would fit much better. For example in network tests we also use headless clients that we can spawn, this means our headless clients would require a dedicacted server build?

 

Yeah might be time for name change, i build headless clients, and have had to explain in the past that headless server is for that too.

 

May be you need to provide a way to keep audio files that can be read from cpu for audio centric game servers

 

About time. No more having to deal with separate projects or juggling asset bundles to make builds without shaders, sounds, and textures.

Now find a way for us to test multiple clients on the same project without having to make builds or trick Unity with symbolic links and working with multiplayer in Unity can start looking less like a hack.

 

You right
but there are games on your Engine (not my games) where audio is main part of game everything else is just environment around sound.
Such games can not be online with your new Server Target because you dont provide option to keep actual audio data

So building generic tech you must think about all usages not only most used ones.

 

May be you right

But the same can be sad about textures or meshes that they dont need on server.

 

That's great, and why not take the idea a step further by developing a server engine isolated from the game engine. With an integrated and modular internet transport system, from there we could build all the multiplayer possibilities.

It would be very fast for multiplayer games. Allowing developers to use a server base with the whole structure of Unity is a big advantage. A market for high value AAA games.

Developing a server from scratch requires network developers, but starting from an empty solid foundation can be developed by game programmers, with less server training.

 

Looks like we can all agree, that customisable options is best way for the new headless feature, if for some strange reason people need to keep audio, or textures, shaders etc

Good luck andrews_unity , UnityGio and the rest of your squad!

 

Publish to Win And Linux, Crash during runtime.

Crash Report configuration:

 * App Name: Island

 * App Version: Unity 2021.2.0b6_2055a14f48ed

 * Mono DLL: G:\Development\GameServWin\MonoBleedingEdge\EmbedRuntime\mono-2.0-bdwgc.dll

 * Bug Reporter App Path: 

 * Crash Report Path: C:\Users\JAGER\AppData\Local\Temp\Crashes

 * Is Editor: false

 

Thanks @JagerZero . Because we make use of other metadata needed, and also leverage the system's triage capabilities, may you please file a bug into our Bug Reporting system for this crash?

Instructions on how to do so are here: https://unity3d.com/unity/qa/bug-reporting

You can then follow-up with via direct-message to me with a link to the Fogbugz.

 

This sounds interesting. I'm currently using Unity for a thin client for input + rendering that interacts with a authoritative C# server. It would be nice to take advantage of Unity code, like physics, to ensure the client + server simulation are the same, but I always thought the overhead of a headless client made it not worth it. This might change things for sure.

 

@UnityGio Is there a document that fully details what's stripped? Lights aren't mentioned above, and wouldn't mesh colliders need the mesh? What can be configured?

 

hihi
Took me a few hours to figure out the problem, but got there eventually.

Mac building to Mac, gave this error, wouldn't even run a blank headless scene.


Mac building to windows, gave this error, cannot build windows IL2CPP, even though it was set to MONO.


Removing Unitys ProjectSettings, close/reopen Unity, fixed both the above problems.

So something in 2021.2 or the headless feature doesn't like Unity 2019 (and below) project settings files.

The following Audio error, when an AudioSource is on a prefab in a scene, removing the audio source from prefab makes the error go away.



And then i get spammed with .dll's not found
(stripping is set to disabled, and allow unsafe code is true)
Might have to leave that part for another day, tired eyes.

(P.S everything worked and looked fine running in Editor)
Hope thats enough info to investigate with

Compression sizes:

2021.2, Non optimised headless build size = 212mb
^ Texture override to lowest "Max 64" = 105mb

 

@JagerZero - we appreciate what you were able to send in. The team was able to track down the source of the error and has a fix for the bug committed. It should be available in the near future on one of our next updates.

@jesusluvsyooh - Thanks for this information on the two issues you have observed, and your compressions experienced. May you please also submit a Fogbugz for both issues observed and then message me the issue numbers? Instructions on how to do so are here: https://unity3d.com/unity/qa/bug-reporting

@MrG - investigating and will reply with more info asap. Thank you for your patience ahead.

 

Hi. How about assetbundles? Can I build assetbundles for dedicated server which has no rendering/audio data?

 

That is good trend, people want running many unity programs as room server on a bare metal machine.
so let unity build as much as possible small is very important.

 

Hi @MrG

Here are some answers to your questions. Please let us know if you have any others.

No, for now we have is what is described in the changelog:

• [Memory Optimization] Pixel data for textures that are not marked as read/write is not included in Server builds.
• [Memory Optimization] Vertex data for meshes that are not marked as read/write and not used for internal systems, is not included in Server builds.

Lights are not modified. Mesh colliders require the read/write parameter to be enabled, therefore they are not stripped. The read/write parameter for mesh colliders is the property that allows accessing it from CPU, otherwise it's GPU only. This is what we check to confirm that we can remove the mesh and texture data.

For now it is not possible to configure further how striping works. We started with textures and meshes data of non read/write assets (note that we strip the pixel and vertex data, not the assets themselves). This is because we can be certain that the data is not accessible by the CPU, and without a GPU output, not accessible at all.

 

Seems like real-time lighting / Enlighten / loading baked lighting should be stripped. I recall seeing in a headless console that it was starting threads for Enlighten, which always seemed rather silly.

 

Currently, when we run our game server as Vanilla C# in Mono we use about 15MB of memory, and running in Unity as headless "out of the box" it uses close to 600MB of memory. There is an optimization tool in the asset store that sounds like it could get down to the 150MB range, but that's still too high for our needs.

Are you able to give any indication of generally where on the scale of 15 - 600MB I would expect to land with these additional optimizations?

 

Hello, when will it be possible to install it via "Add Module" in the unity 2020?

 

Hi @JarviTeam . Dedicated Server Target with asset stripping won't be backported to Unity 2020 as that is already a stable LTS editor.

 

Thank you for feature! Now my server use only 100-200MB ram. (was 300-500).
I have few crashes (related to ntdll.dll). My editor 2021.2.0b13 (also tried b10-b13). Windows dedicated server build always crash. I fixed it by disabling Unity Audio Plugin in preferences.
Second crash in editor while opening scene, open prefab, press play etc. Fixed by disabling Asset Chache server.

Waiting for next updates. Would be nice to fix/remove these errors/warning.

...

WARNING: Shader Unsupported: 'TextMeshPro/Distance Field Overlay' - All subshaders removed

WARNING: Shader Did you use #pragma only_renderers and omit this platform?

WARNING: Shader If subshaders removal was intentional, you may have forgotten turning Fallback off?

ERROR: Shader TextMeshPro/Mobile/Distance Field - Masking shader is not supported on this GPU (none of subshaders/fallbacks are suitable)

ERROR: Shader TextMeshPro/Mobile/Distance Field Overlay shader is not supported on this GPU (none of subshaders/fallbacks are suitable)

...

 

@UnityGio How can I build dedicated server from script? BuildPipeline.BuildPlayer() not support this target yet?

EDIT:
I found solution:

var buildPlayerOptions = new BuildPlayerOptions()

{

    subtarget = (int) StandaloneBuildSubtarget.Server,

    target = BuildTarget.StandaloneWindows64,

    options = BuildOptions.Development

};

 

Hi,

i tryed to build a server for a game as a dedicated server.
Everytime i try to start it, it crashes. The crashlog says

UnityPlayer.dll caused an Access Violation (0xc0000005)
in module UnityPlayer.dll at da22e59a.

when i build it as headless in an older unity version, i didnt had any problems.
I also tried to start the server with "-nographics -batchmode"


Any idea how to fix this?

 

Hello, I would like to ask if this is going to the 2019.2 version and if It will finnaly be able to use Input and Output in the Headless server, since in my project there is a huge necessity of Headless user input for admin messages and commands and also message output soo I can get a proper feedback.

 

I met this too, dose anyone in unity team track this issue?

 

Sure I will try it.
Actually in my project with the plugin "TopdownEngine", which uses many audio features, hope it will be fixed. Many thanks

 

This dedicated server I click on.... is it for Xbox consoles/Windows? Or should I still use URP for Xbox if I'm in the Creators Program/Partner Center on Microsoft.

 

Hi @UnityGio. Have u start battle testing this Dedicated Server target for dots netcode and make sure it's always working properly for dots netcode?

 

Just tested building to dedicated server in 2021.2.4f1, taking longer than expected to build because it's going through and compiling and stripping all the shaders. ( currently in 2021.2.4f1, it takes a super long time to do the shader stripping of shaders with any build target ) Curious why it even needs to take the time to compile shaders at all as headless?

At least it doesn't need to swap the art asset files though like the headless server builder 3rd party asset.

 

Just verified in 2021.2.4f1, It wont crash anymore, thanks.

 

Hi. How about asset bundles? Can I build asset bundles for a dedicated server that has no rendering/audio data?

 

From someone starting from scratch... Is there a page in the documentation or tutorial out there explaining how to create a Dedicated Server with Unity Netcode, how to host it and try it with friends ? Been looking around for a while and the Discord channel point me to this subforum. Thank you

 

did this get added in 2021.2? it's not in 2020.3?

 

I got

when running my game server in virtual box VM. Isn't the dedicated server build have visual stuff stripped? Should I just ignore these error message?

 

Hi, nice option.
However, is there a command line argument to set the subtarget ? It seems there is only -buildtarget which doesn't include dedicatedserver (https://docs.unity3d.com/2021.2/Documentation/ScriptReference/RuntimePlatform.html).