Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct here to familiarize yourself with the rules and how to post constructively.

  2. Unity 2022.1 is now available as the latest Tech release.
    Dismiss Notice

Unity Introducing System Metrics Mali package

Discussion in 'Profiler Previews' started by antonk-unity, Jun 15, 2021.

  1. antonk-unity

    antonk-unity

    Unity Technologies

    Joined:
    Oct 4, 2016
    Posts:
    27
    System Metrics Mali Package (SMM)

    Welcome to the System Metrics Mali discussion thread. You can use this thread to ask for help, share feedback, and have discussions about the experimental System Metrics Mali package.

    Introduction
    System Metrics Mali provides a simple and extensible interface for reading GPU hardware counters on Arm GPUs. Hardware counters provide a low-level view of application performance. These metrics can provide insight into what impact your changes produce on the hardware level. System Metrics Mali package exposes a set of hardware metrics via Unity Profiler window and Unity ProfilerRecorder API.

    Disclaimer: This package is still under development and anything could change on the API side.

    High-level features summary
    It allows access to and monitoring of GPU low-level counters. Sampling of the counters is performed on a per-frame level, so you can monitor changes in hardware system metrics along with other Unity Profiler frame stats.

    Resources
    For more information, please have a look at:
    Requirements
    • The minimum required Unity version is 2021.2
    • Supports Samsung Exynos, HiSilicon Kirin, or MediaTek CPUs (as these use Mali GPU's)
    • Doesn’t support Qualcomm Snapdragon CPUs (as these use Adreno GPU's)
    Getting started
    • Download Unity 2021.2.0a17 or newer through the Hub.
    • Create or open a Unity project.
    • Go to Window > Package Manager.
    • Follow “Adding a registry package by name” instructions and add “com.unity.profiling.systemmetrics.mali
    • Build and run your application
    • Go to Window > Analysis > Profiler
    • Connect to a device and start capturing
    • Click on Profiler Module > Gear Icon

    • Click Add icon > Add counters from User group
    Current state
    The experimental System Metrics Mali Package is still under development and only a limited number of counters are currently available. When using this package you can expect that:
    • It should have zero CPU overhead if no hardware counters are being read through ProfilerRecorder API and/or Profiler isn’t attached
    • It has minimal fixed memory overhead of around 5-10Kb

    Hardware counters are sampled on a per-frame basis. Currently, whenever an application has CPU spikes that might cause System Metrics Mali to sample a portion of the next frame that looks like a spike up followed by a spike down. This is caused by captured frame interval and GPU frame interval misalignment.

    Most values are provided as absolute GPU cycles. This is due to the fact that GPU frequency is unavailable on most phones and we can’t measure relative performance.

    Older phones might have outdated implementations, but generally, the plugin should work on any phone released after 2010.

    What's next
    • We plan to significantly improve UI usability by introducing of custom Profiler Module which would allow providing pre-configured Profiler Module and have custom visualization for the counters
    • Replace absolute value counters with relative counters related to specific GPU pipeline stage
    • Add more hardware counters for different GPU pipeline stages to have high-level for each stage which might be an application bottleneck
    • Add support for non-mobile platforms and CPUs
    Feedback
    In terms of feedback, we're especially looking for:
    • Are there any workflows that are unclear or missing?
    • Any hardware compatibility or stability issues?
    • Are there any issues or unclear parts in the documentation?

    Please feel free to post any feedback in this sub-forum.

    How to report bugs
    Ideally, we'd like any bugs reported through the built-in bug reporter tool, as that will automatically provide us with some relevant context. When reporting bugs, please:
    • Report exact mobile phone model name and number
    • Attach logcat logs files if available

    Once you have submitted a bug report through the bug reporter, please feel free to start a discussion about it in this thread.

    Thank you for your interest, we're looking forward to your feedback!
     
    Last edited: Jun 16, 2021
    codestage, Jes28 and alexeyzakharov like this.
  2. antonk-unity

    antonk-unity

    Unity Technologies

    Joined:
    Oct 4, 2016
    Posts:
    27
    0.2.0 update is available

    Changes:

    • Updated HWCP library to the latest version.
    • Added custom Profiler Module "Mali System Metrics" with an improved detailed view,
    • Added more hardware and derivated counters. See "Available metrics" for more details.
    • Improved metrics description documentation

    Newly added metrics:
    • GpuShaderCoreUtilization - % Shader Core Utilization
    • GpuNonFragmentUtilization - % Vertex And Compute Utilization
    • GpuFragmentUtilization - % Fragment Utilization
    • GpuShaderFragmentActive - Shader Fragment Cycles
    • GpuShaderComputeActive - Shader Compute Cycles
    • GpuShaderTripipeActive - Shader Tri-Pipe Cycles
    • GpuShaderFragmentUtilization - % Shader Fragment Utilization
    • GpuShaderComputeUtilization - % Shader Compute Utilization
    • GpuShaderTripipeUtilization - % Shader Tri-Pipe Utilization
    • GpuShaderArithmeticUtilization - % Shader Arithmetic Utilization
    • GpuShaderLoadStoreUtilization - % Shader Load/Store Utilization
    • GpuShaderTextureUtilization - % Shader Texture Unit Utilization
    • GpuReadStallRate - % Memory Read Stalled
    • GpuWriteStallRate - % Memory Write Stalled
    • GpuInputPrimitives - Input Primitives
    • GpuCulledPrimitives - Culled Primitives
    • GpuClippedPrimitives - Clipped Primitives
    • GpuVisiblePrimitives - Visible Primitives
    • GpuCulledPrimitivesPercentage - % Culled Primitives
    • GpuClippedPrimitivesPercentage - % Clipped Primitives
    • GpuVisiblePrimitivesPercentage - % Visible Primitives
    New custom profiler detailed view:

    image (7).png
     
  3. Jes28

    Jes28

    Joined:
    Sep 3, 2012
    Posts:
    872
    Thanks very useful package :)

    Adding GPU Utilisation metrics on graphs and in detailed view will be very appreciated :) as much as adding description in tooltip for every metric in detailed view :)
     
    alexeyzakharov likes this.
  4. Kleptine

    Kleptine

    Joined:
    Dec 23, 2013
    Posts:
    179
    Wish we could use this! Do you know of any way to get these counters from Adreno GPUs?
     
  5. antonk-unity

    antonk-unity

    Unity Technologies

    Joined:
    Oct 4, 2016
    Posts:
    27
    Unfortunately, Qualcomm doesn't provide API or documentation on how to access Adreno GPU hardware counters.

    Although it might be possible through kernel IOCTL. Here is an article on the topic.
    Sampling Performance Counters from Mobile GPU Drivers | Lei.Chat()
    Haven't tried it myself yet, but it seems it's possible while not easy though.
     
  6. antonk-unity

    antonk-unity

    Unity Technologies

    Joined:
    Oct 4, 2016
    Posts:
    27
    New 1.0.0-pre.2 version is now available

    The package is Release Candidate now and can be searched and added via Unity Package Manager now.

    Changed

    • Significantly reduced plugin binary size.
    • HWCP library is updated to the latest version.
    • Counters were renamed to be named uniformly.
    Added
    • Exposed additional counters for relative load measurement.
    • Exposed additional counters for primitives count and GPU shader core activity.
    • Added API documentation.
    • Added tooltips to Profiler Module UI.
    • Added examples and usage guide into documentation.
    • Added detailed description for each hardware counter.
    • Added new play mode verification tests.
    Fixed
    • Fixed issue when Pixels counter might have shown large negative value.
    • Fixed issue that it was impossible to add counters before connecting Profiler to an Android device.
    • Fixed crash on some Mali-T880 devices caused by zero data in counters
     
    codestage likes this.
  7. antonk-unity

    antonk-unity

    Unity Technologies

    Joined:
    Oct 4, 2016
    Posts:
    27
    New 1.0.0-pre.3 version is now available

    Changed

    • Reduced plugin binary size by ~10%
    • HWCP library is updated to the latest version to add Mali-G68 and Mali-G78 support.
     
  8. forestrf

    forestrf

    Joined:
    Aug 28, 2010
    Posts:
    122
    Thank you for this!
    It doesn't seem to work with my phone (probably it's the phone's fault, it's a Moto e4 and ARM Streamline reports that CONFIG_HW_PERF_EVENTS is not set and/or that the PMU isn't configured in the target device tree). I'm going to try rebuild the kernel (whenever I learn how to) (Edit: I enabled it and more stuff, but still didn't work) with CONFIG_HW_PERF_EVENTS set, but my feedback would be for the "Mail System Metrics" tab to report that the phone isn't compatible (specifying why is probably asking too much, and something that can be solved by the developer anyway) instead of showing all the stats at 0 (which could lead to confusion).
     
    Last edited: Feb 2, 2022
    alexeyzakharov likes this.
  9. antonk-unity

    antonk-unity

    Unity Technologies

    Joined:
    Oct 4, 2016
    Posts:
    27
    Good point, we'll look into how we can do that. Thanks!
     
    forestrf likes this.
  10. codestage

    codestage

    Joined:
    Jul 27, 2012
    Posts:
    1,734
    Hey @antonk-unity ,

    First of all, thank you for making this package - so much profiling details is something one could only dream before!

    But I couldn't get it working so far with my Mali-G76. More likely I'm missing something obvious, but this is what I see while profiling my Galaxy S10+ (Exynos 9820):

    upload_2022-2-9_17-0-50.png

    I have dev build, Frame Timing Stats option enabled and using package Version 1.0.0-pre.2 (pre.3 is not available in the latest public 2022.1.0b7).

    I also don't see the User counters group in the Profiler Module Editor:

    upload_2022-2-9_17-4-52.png

    What could I miss while setting things up?..

    P.S. tried to manually update up to the pre.3 and pre.5 versions - there were errors in console and no Mali System Metrics profiler module available at all in both cases (might be not compatible with 2022.1.0b7).
     
  11. richardpickeringunity3d

    richardpickeringunity3d

    Unity Technologies

    Joined:
    Jun 22, 2020
    Posts:
    2
    Hi @codestage

    I'm just investigating this for you now, I don't have the exact same device to hand but I've tried on a Galaxy Note 10 (Mali-G76) and it looks to be working for me. Did you happen to have any error messages shown to you when you tried

    Thanks
     
  12. codestage

    codestage

    Joined:
    Jul 27, 2012
    Posts:
    1,734
    Thanks for taking a look, but no, not a single error in the Unity Console.
    I've also checked adb logs and didn't find anything related. I've attached the adb logs here just in case.

    Furthermore, I also tried to profile the same device from an entirely another PC but without luck as well - still nothing for the Mali System Metrics module so far.

    Please let me know how can I debug it if possible to get more logs for you, I'll be glad to help.
     

    Attached Files:

    alexeyzakharov likes this.
  13. solidpixel

    solidpixel

    Joined:
    Jan 29, 2020
    Posts:
    3
    I'm from the Arm team who work on the backend counter interface the Unity profiler is using. Thanks for reporting the issue. A couple of questions:

    * Can you confirm the driver version in your device? (Open chrome://gpu and find GL_VERSION string in the table).
    * Can you try the Streamline profiler in Arm Mobile Studio and see if that manages to collect any data?

    Also, at the risk of sounding like an IT tech support, have you tried turning the device off and on again? The main causes of failure in the driver for this look related to the kernel driver being unable to allocate memory, so a reset may help (although I doubt it).

    Kind regards,
    Pete
     
    Last edited: Feb 11, 2022
    codestage likes this.
  14. codestage

    codestage

    Joined:
    Jul 27, 2012
    Posts:
    1,734
    Hey Pete,

    Thank you for chiming in,

    Sure here is my GL_VERSION from the device:
    OpenGL ES 3.2 v1.r32p1-01bet2-mbs2v39_0.9832bf74fac3612104002d946600017d


    I've tried, and it seems to work just fine:
    upload_2022-2-11_14-11-23.png

    While still nothing in the Unity Profiler's Mali System Metrics...

    =D yeah, this is a good one actually, and I know a lot of cases which were fixed with rebooting a device, but not this time unfortunately: rebooting didn't help - still nothing in the profiler for the Mali module =/
     

    Attached Files:

  15. codestage

    codestage

    Joined:
    Jul 27, 2012
    Posts:
    1,734
    I just tried to reproduce it with another device on the same SoC (SM-G970F with Exynos 9820 and Mali-G76) and it does reproduce for it as well.
     
  16. solidpixel

    solidpixel

    Joined:
    Jan 29, 2020
    Posts:
    3
    Thanks for the extra info. If Streamline is working then the underlying mechanism isn't broken, so I just need to work out why the driver connection is being rejected for the backend library the Unity profiler is using.

    Cheers,
    Pete
     
    codestage likes this.
  17. codestage

    codestage

    Joined:
    Jul 27, 2012
    Posts:
    1,734
    Just let me know if I can help you with any further debugging, I'll be glad to help!
     
    solidpixel likes this.
  18. antonk-unity

    antonk-unity

    Unity Technologies

    Joined:
    Oct 4, 2016
    Posts:
    27
    New 1.0.0 version is now available!

    Relative to 1.0.0-pre.3 changes are:

    Fixed
    • Fixed counters sampling in Profiler-only capture mode
    • Fixed example script.
    Added
    • Added usage guide documentation.

    Your issue should be fixed in 1.0 package version
     
    codestage likes this.
  19. codestage

    codestage

    Joined:
    Jul 27, 2012
    Posts:
    1,734
    @antonk-unity

    Thank you for releasing a new version!
    I've just updated the package to 1.0.0 version at Unity 2021.2.0f1 and this is what I see every time when entering Play Mode (I didn't make a build or connected to the Android device yet after fresh Unity instance start):

    Code (CSharp):
    1. SystemMetricsMali: Initialization failed
    2. UnityEngine.Debug:Log (object)
    3. Unity.Profiling.SystemMetrics.SystemMetricsMali:InitializeSystemMetricsMali () (at Library/PackageCache/com.unity.profiling.systemmetrics.mali@1.0.0/Runtime/SystemMetricsMali.cs:307)
    It does happen in Unity 2022.1.0b9 too.

    Is it expected behavior?

    Next thing I did:
    - Opened project with older SMM package in Unity 2022.1.0b9
    - Build and run on device
    - Attach Profiler and start capture (saw empty Mali profiler module)
    - Update SMM package
    - saw numerous warnings in Console like this:
    Code (CSharp):
    1. SystemMetrics: Failed to get handle for 'GpuCycles' id, check that native plugin id match
    2. UnityEngine.Debug:LogWarning (object)
    3. Unity.Profiling.SystemMetrics.SystemMetricsLibHWCP:GetProfilerRecorderHandle (string) (at Library/PackageCache/com.unity.profiling.systemmetrics.mali@1.0.0/Runtime/SystemMetricsLibHWCP.cs:40)
    4. Unity.Profiling.SystemMetrics.SystemMetricsMali:.ctor () (at Library/PackageCache/com.unity.profiling.systemmetrics.mali@1.0.0/Runtime/SystemMetricsMali.cs:325)
    5. Unity.Profiling.SystemMetrics.SystemMetricsMali:.cctor () (at Library/PackageCache/com.unity.profiling.systemmetrics.mali@1.0.0/Runtime/SystemMetricsMali.cs:312)
    6. Unity.Profiling.Editor.SystemMetrics.Mali.MaliProfilerModule:.cctor () (at Library/PackageCache/com.unity.profiling.systemmetrics.mali@1.0.0/Editor/ProfilerModule/MaliProfilerModule.cs:10)
    7.  
    closing with these errors:
    Code (CSharp):
    1. ArgumentException: ProfilerRecorderHandle is not initialized or is not available
    2. Parameter name: handle
    3. Unity.Profiling.LowLevel.Unsafe.ProfilerRecorderHandle.GetDescription (Unity.Profiling.LowLevel.Unsafe.ProfilerRecorderHandle handle) (at <b2c530e619ab42aeba489c5671c82671>:0)
    4. Unity.Profiling.Editor.SystemMetrics.Mali.MaliProfilerModule.GetDescriptorProfilerCounterHandle (Unity.Profiling.LowLevel.Unsafe.ProfilerRecorderHandle handle) (at Library/PackageCache/com.unity.profiling.systemmetrics.mali@1.0.0/Editor/ProfilerModule/MaliProfilerModule.cs:56)
    5. Unity.Profiling.Editor.SystemMetrics.Mali.MaliProfilerModule..cctor () (at Library/PackageCache/com.unity.profiling.systemmetrics.mali@1.0.0/Editor/ProfilerModule/MaliProfilerModule.cs:10)
    6. Rethrow as TypeInitializationException: The type initializer for 'Unity.Profiling.Editor.SystemMetrics.Mali.MaliProfilerModule' threw an exception.
    7.  
    8. Unable to create Profiler module of type Unity.Profiling.Editor.SystemMetrics.Mali.MaliProfilerModule. The Profiler module 'Mali System Metrics' cannot have no chart counters.
    9. UnityEditor.ProfilerWindow:OnEnable ()
    10.  
    And Mali module disappear from the Profiler entirely.
    Looks like an edge case with updating from the specific version to the 1.0.0 which might reproduce when updating from 1.0.0 to the newer version.

    Then I restarted Unity 2022.1.0b9, build&run with Auto-connect Profiler option enabled, and it worked fine, though it didn't show the graph until the profiler window resize.

    Hope this feedback helps!
     
  20. antonk-unity

    antonk-unity

    Unity Technologies

    Joined:
    Oct 4, 2016
    Posts:
    27
    Thanks for the feedback. Unfortunately, that's a Unity Editor limitation.
    You need to restart it, as Unity doesn't support plugin DLLs unload in Editor.
    And while you upgrade the package the DLL is being replaced, but the old one is still loaded.
    I've tested it locally and Unity Editor reload seems to help.
     
    Last edited: Mar 3, 2022
    codestage likes this.
  21. antonk-unity

    antonk-unity

    Unity Technologies

    Joined:
    Oct 4, 2016
    Posts:
    27
    New 1.0.1 version is now available!

    Relative to 1.0.0 changes are:

    Changed
    • Updated usage guide documentation.
     
unityunity