Search Unity

[In Development] NImGui, a 1 Draw Call Immediate Mode GUI for Unity

Discussion in 'Assets and Asset Store' started by psuong, Sep 18, 2021.

  1. psuong

    psuong

    Joined:
    Jun 11, 2014
    Posts:
    78
    NImGui stands for Nimble Immediate Mode General User Interface and is designed to be a replacement library to Unity's runtime ImGui.

    I have always found Unity’s runtime ImGui solution to be lackluster and displaying any kind of widgets on runtime is just too annoying to get it setup easily. Similarly, with Unity’s UGUI solution, there’s too much set up required in order to display some information or set up widgets to tweak values in your game. While Debug.Log is extremely useful, when debugging many pieces of information it can be difficult to search through and contextualize information.

    This framework was designed with performance and simplicity in mind, so you can focus on drawing widgets to interact/display/debug information while developing your game.

    Features
    • Supports builtin render pipeline and URP
      • Built for Windows, Mac, Linux, & WebGL
    • Simple immediate static API to draw widgets
      • Draw Scopes via using pattern to begin and end areas (see the API Example below!)
      • Write GUI naturally in MonoBehaviour.Update() and DOTS’ SystemBase.OnUpdate()
    • High performance
      • Utilizes C# Job System & Burst Compiler
    • A library of built widgets
      • Box
      • Button
      • Collapsible Area
      • Dropdown Menu
      • Label
      • Line
      • Pane
      • Progress Bar
      • Toggle Button
      • Scroll Area
      • Slider (float & int)
    Limitations
    • Static APIs cannot be called from separate threads simultaneously
    • Static APIs cannot be called from LateUpdate as this will throw a warning in the rendered UI
    • Static APIs can be called from FixedUpdate but will only appear for a frame as UI gets updated every frame and FixedUpdate does not
    • Does not support the new InputSystem yet
    Performance

    NImGui can process 250 widgets in 1 millisecond on the main thread in the Editor. On a build, you can expect it to take 0.4 millisecond instead. This is achieved via multithreading using Unity’s Job System and Burst Compiler.

    These stats were recorded on an Intel Core i7-7700HQ @ 2.8 GHz. More stats will be collected across varying hardware and operating systems.

    Versions Supported
    • Unity 2020.3 LTS
    • Unity 2021.1
    • Unity 2021.2
    Dependencies
    • Unity Collections
    • Unity Jobs
    API Style

    The API follows an immediate mode style. Here is an example of the API you would be writing:

    Code (csharp):
    1.  
    2. using (ImPane pane = new ImPane("Sample Pane", new float2(500), new float2(500))) {
    3.    if (pane.IsVisible) {
    4.        var t = Mathf.PingPong(Time.time * 0.5f, 1f);
    5.        ImGui.Label("Here's an example of drawing panes and widgets");
    6.        ImGui.ProgressBar(t);
    7.        ImGui.Dropdown("Dropdown", options);
    8.  
    9.        if (ImGui.Button("Click me to toggle")) {
    10.            showMsg = !showMsg;
    11.        }
    12.  
    13.        if (showMsg) {
    14.            ImGui.Label("Message shown!");
    15.        }
    16.  
    17.        ImGui.Slider("Int Slider", 0, 10);
    18.  
    19.        using (new ImScrollArea("Scroll Area")) {
    20.            using (ImCollapsibleArea group = new ImCollapsibleArea("Group")) {
    21.                if (group.IsVisible) {
    22.                    if (ImGui.Toggle("Show Box")) {
    23.                        ImGui.Box(300, Color.red, true);
    24.                    }
    25.                }
    26.            }
    27.        }
    28.    }
    29. }
    30.  
    This would produce an output like so:


    Demo

    The demo can be tested at the following link on itch.io:
    https://initialprefabs.itch.io/nimgui

    Immediate Roadmap

    The first version of NImGui is expected to be released at the end of 2021/early 2022 as there are few things that I need to finish such as:
    • escape characters support in the Label API (for new line and tabs)
    • Textfield API
    • Support Unity’s new Input System
    • a browsable API / docs website
    • LateUpdate support for drawing widgets
    • One draw call UI (right now each widget drawn will issue between 1 to a few draw calls)
      • This may also allow me to shift away from using TextMeshPro materials as it is a current dependency
    Longer Term Goals
    • RTL language support
    • Complex text layout support (e.g. support for Arabic, Khmer, Thai, etc)
    • Command Buffer like API for dynamic widgets and panes & multithreading support
    • Stateful FixedUpdate support (e.g if you need to debug netcode which will run on a Fixed Timestep, it will make sense to store and display the information until the next Fixed Timestep)
    Initial feedback and questions are welcomed! Feel free to comment in this thread below.
     
    Last edited: Oct 30, 2021
    mgear and Lukas_Kastern like this.
  2. psuong

    psuong

    Joined:
    Jun 11, 2014
    Posts:
    78
    Thought I'd post a new update since it's been two weeks since the last time I made this post (I can't guarantee that I'll be posting an update every 2 weeks but I'll try).

    I decided to remove my dependency on TextMeshPro. TextMeshPro is great but there were some problems I ran into:
    • TextMeshPro materials had to be dilated in order to remove jaggedness (there's some setting that I haven't figured out and going through TextMeshPro source code is a pain).
    • TextMeshPro did not generate glyphs for certain characters despite the Font actually supporting the character
      • This made it almost impossible for me to aim for a 1 draw call UI.
    So what replaces TextMeshPro?

    NimGui has its own SDF Font Texture generator and is based off of the AntiAliased Euclidean Distance formula by Stefan Gustavson: https://weber.itn.liu.se/~stegu/edtaa/ (MIT license will be included in the source file where it's used).

    See the sdf_texture.jpg attachment below for an example.

    With the underlining font rendering system replaced, what are the limitations that come with it?
    • Unfortunately, this means that you cannot load new fonts and render multiple fonts. You can only have 1 font loaded in at a time as the shader is extremely simple and only takes in a single texture.
    • The SDF texture generated by NimGui takes up more space than TextMeshPro. There is no texture packer to make the texture more compressed.
    • Loading in custom images will no longer be supported.
    However, despite these downsides, I am now able to render everything in 1 draw call, which reduces the complexity in building the mesh as well the command buffer that will render the content.

    1 Draw Call UI


    However, with these changes, I cannot take advantage of hardware scissoring so scroll views will need to be reworked such that content is interpolated and clipped if it is out of bounds. This will be my immediate step after finishing converting widgets such as Sliders.
     

    Attached Files:

    Last edited: Oct 2, 2021
  3. psuong

    psuong

    Joined:
    Jun 11, 2014
    Posts:
    78
    Hey all, another update!

    With the change to a 1 draw call UI, hardware scissoring was no longer supported. This meant that I had to write my own "software" scissor technique which allowed me to cull content that is out of view (which is very useful for scroll areas).

    Right now the implementation is based off interpolating UVs to the correct value when vertices are culled out of view. This ensures that glyphs that are rendered are not compressed in a limited space which would cause each glyph to look distorted.



    For performance stats, I'll need to optimize software based scissoring, right now for 6k vertices, software scissoring takes 0.5ms on a Core i3-8100 @3.6GHz. Without software scissoring, the job takes 0.15ms to run, so we can see that software scissoring makes the job run 3x longer.

    As for next steps, I'll continue and move onto implementing Textfield.
     
    Last edited: Oct 11, 2021
    exiguous likes this.
  4. psuong

    psuong

    Joined:
    Jun 11, 2014
    Posts:
    78
    Hey all,

    Here are some updates with ImGui. First and foremost:

    Optimization
    For 6k vertices, software scissoring originally took 0.5ms to run. Without software scissoring, the job originally takes 0.15ms to run. This is pretty much a 3x performance lost. To improve the performance, after some optimizations on the Bursted Job, it now takes approximately 0.35ms to perform software scissoring on all vertices.

    Some further optimizations I can do include:
    • Only scissor vertices when scissor rects are introduced
    • Use an approximation of the sqrt to produce fewer assembly instructions when interpolating
    If software scissoring is not an option, I'll consider offloading it the GPU, however I would like to keep the SDF shader extremely simple.

    Textfield
    The Textfield API is pretty much functional with western based languages. RTL and complex text layout is not supported yet and is on my todo list in the near future.

    You can input text via keyboard and press or hold backspace to delete text. See the gif below:



    Next Steps
    The Textfield API needs some cleanup and additional features such that you can move the cursor and insert characters in the middle of the textfield and be able to highlight. I'll also explore complex text layout such that I can plan for additional changes to the logic / API.

    After that the base API is pretty much done, so I'll be updating the documentation and demo.
     
  5. psuong

    psuong

    Joined:
    Jun 11, 2014
    Posts:
    78
    New updates from the recent week.

    TextField API

    TextFields are primarily done and backspace detection is done using Unity's Input.inputString. (Instead of using Input.GeyKey(KeyCode.Backspace) variants, I check for a \b to see if the user has pressed backspace.) To help GC load with string generation, I'm considering that instead of returning a string type, the TextField will write to an allocated StringBuilder. This will have a lighter load to C#'s garbage collector.

    Project Cleanup

    TextMeshPro is finally removed from the library! This means NimGui officially uses its own SDF texture generation & SDF shader. The SDF shader is kept as simple as possible therefore it does not implement all features presently found in TextMeshPro.

    Documentation

    Documentation is updated to reflect an additional step in the installation process. This involves opening the Setup Wizard and clicking the Add Shader button, which will add the SDF shader to your graphics settings' Always Included Shaders.

    The Setup Wizard attached below, although extremely simple for now, will also include a hyperlink to the documentation.

    Next Steps
    • Switch TextField to take in StringBuilder that the developer allocates themselves to lighten the GC Load
    • Update the Setup wizard to include a link to the documentation
    • Allow the Setup wizard to appear on project open & provide users with a way to disable the auto popup after start.
    • Update documentation on workflow when introducing new fonts & the limitations that it has.
    • Begin researching into complex text layout
     

    Attached Files:

    Last edited: Oct 30, 2021
    exiguous likes this.
  6. psuong

    psuong

    Joined:
    Jun 11, 2014
    Posts:
    78
    Hi all another update.

    I've been busy doing some clean up to the amount of padding widgets have and migrating the documentation from originally, mdbook to DocFX because generating the XML API documentation to work with mdbook wasn't a good idea.

    The link to where the documentation currently lives here.

    Inline code documents will be picked up in your IDE of choice, although inline code examples vary in how they are rendered per IDE. (Please see the attached image below.)

    inline-docstrings.png

    The screenshot above is based on the following snippet

    Code (csharp):
    1. /// <summary>
    2. /// Returns the current state of the Toggle box. If the box is checked, then
    3. /// the state is true, otherwise returns false.
    4. /// <remarks>
    5. /// The control ID is useful for pruning cached global states. You only need to
    6. /// prune when you are no longer using the Toggle functionality.
    7. /// </remarks>
    8. /// <code>
    9. /// class UIBehaviour {
    10. ///     uint toggleID;
    11. ///     // Gets called once per frame.
    12. ///     void Update() {
    13. ///         if (ImGui.Toggle(out toggleID)) {
    14. ///             ...
    15. ///         }
    16. ///     }
    17. ///
    18. ///     ~UIBehaviour() {
    19. ///         // Internally this will remove the cached Toggle state from
    20. ///         // it's internal hashmap, when this Object is destroyed.
    21. ///         ImGui.PruneToggle(toggleID);
    22. ///     }
    23. /// }
    24. /// </code>
    25. /// </summary>
    26.  
    Next Steps

    Over the next few weeks, I will be continuing updating the documentation by swapping some still images with gifs (as I think gifs will get the purpose across better with interactive widgets), updating docStrings with code examples, and cleaning out the API.

    I'll also need to start prepping for asset store and itch.io release.
     
    Last edited: Nov 13, 2021
    exiguous likes this.
unityunity