Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Introducing the Editor Iteration Profiler

Discussion in 'Profiler Previews' started by Rarepops, Jun 9, 2020.

  1. Rarepops

    Rarepops

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    24
    Editor Iteration Profiler (EIP)
    Before diving into the details I will start by defining what an iteration is. An iteration is a process that contains instructions that are repeated until a condition is met. Unity relies on different types of iterations.

    The Editor Iteration Profiler monitors iterations that are related to the scripting side of Unity, specifically entering and exiting playmode, assembly compilation, and assembly reload.

    The tool is an attempt to empower you, our users, to understand and help you solve the common question “Why does it take so long to compile my scripts/enter playmode?”. EIP accomplishes this by monitoring profiler frames and saving them in a window where you can more easily navigate through the data the profiler produces. Additionally, the data persists for the whole lifetime of the Editor (or until it’s cleared), as opposed to the Profiler, which has a limit to the number of previously stored frames.

    Disclaimer: This tool is still under development and anything could change on the API side.

    High-level features summary
    • Monitor and capture profiling data from assembly reload, assembly compilation, and enter playmode.

    • Export captured data or Profiler data to a number of formats such as HTML, JSON (for chrome://tracing, which is a fast flame-graph style data visualizer), CSV, and Plaintext.

    • Export to a special type of HTML (HTML Performance Report), which attempts to minimize the number of clicks needed to get to the important areas of the profiled data, as well as aggregate possible areas of interest which could be optimized (e.g. OnEnable calls)
    How to use
    Compatible with 2019.3 and later. It might work with earlier versions but it is not tested.
    1. Download the repository from GitHub - https://github.com/Unity-Technologies/com.unity.editoriterationprofiler

    2. Place the contents in a folder in your Project’s Packages folder

    3. Open up the project and open the window from Windows -> Analysis -> Editor Iteration Profiler -> Show Window

    4. You should see a window very similar to this

    The basics
    To enable the tool, you just need to click the Enable button in the window. The window doesn’t need to be open afterward: it will run in the background.


    After an iteration happens, it will show up in the window as follows:





    UI
    Please note that the UI might change/have changed slightly, however, the functionality should be the same.



    • Enable - enables the Profiler and sets it to run in Editor mode.

    • Deep Profile - useful to get all information about managed calls. (https://docs.unity3d.com/Manual/ProfilerWindow.html#deep-profiling)

    • Flattening - attempts to reduce the number of clicks needed to get to the place that is of interest. Works by ‘collapsing’ levels which contain multiple parented items with 1 child. Useful for GUI code and Deep Profiling.

    • User Code - is an attempt to filter out engine code and only show code the User might be interested in.

    • Clear - removes recorded events and events which the EIP is looking for.

    • Collapse All - recursively collapses every item.

    • Print to Console - logs the captured data into plaintext into the console/log file.

    • Export… - dropdown to export captured data in various formats.

    • Export Profiler Data… - dropdown to choose between exporting the selected frame in the Profiler Window or exporting multiple frames, frame by frame, between 2 ranges. It will automatically export the data to a selected folder.


    • Search Bar - works as you would expect. Selecting an item and either pressing the F key or clearing the search will automatically expand the tree view to that item.
    Exporters

    After you have captured some data, you might want to export it. You can do this by clicking the Export… dropdown and choosing a format.





    Here you can get the samples exported above for you to test out.

    The Formats

    Opening the HTML you will see something similar to this:



    It mostly contains the same data as the EIP Window, with the addition of the percentages. Items in square brackets “[ ]” are leaf items without any children.


    The JSON format is meant to be used within Google Chrome's chrome://tracing, which works on chromium-based browsers like Google Chrome, Microsoft Edge Chromium (edge://tracing - chrome://tracing will be replaced by it if typed), Opera, etc.. It is a fast, lightweight and Editor-agnostic way to view the captured data in the flame-graph style. In this gif, we show how to load and use the file into chrome://tracing.


    CSV can be used to load the data into other programs, for example. It is pretty difficult for humans to make sense of it. Also please be aware of the ‘header’ which contains environment information in case you plan to do further processing.


    Exporting Plaintext is the same as the one it is printed in the console with the “Print to Console” button. Using this you have the option to isolate the data and not have other information which is in the Editor.log file.


    The HTML Performance Report is based on the same structure as the regular HTML report, however, functionally it is different. The purpose of it is to distill the information into something humans would more easily understand. It basically does this by reducing the number of levels you have to click through to get to the areas of interest, where code diverges more.




    The items in curly brackets “{ }” represent items that have more children which were hidden due to the parents being under the minimum set threshold (currently a hard-coded value of 1%).

    The colored items represent ‘buckets’ of similar data, which is grouped in one place for convenience. The total time of these items is not added to the original time, it’s only there to give an estimate for that iteration.


    ⚠️Known Bugs/Limitations
    • When rebuilding the tree, Items under [AssemblyReload] get reparented to [AssetImport]

    • Code is not reported per-thread (useful for chrome://tracing)

    • Data might overlap on chrome://tracing because timestamp mismatches between the EIP's timestamp and the Profiler timestamp (to be fixed in the future)

    • Sorting data doesn't work in-window

    • In order to get the most accurate data (especially with DeepProfile enabled), close as many Editor Windows as possible, including the EIP Window itself. The reason is that rebuilding the Tree Hierarchy view and repainting the GUI takes time and it will "pollute" the data you might be looking for

    • When clicking the Profiler Window after clicking Enable in the EIP Window, the EIP's state will be overridden to whatever the original setting in the Profiler was

    • May cause unity to crash when closing it while the EIP is Enabled

    Feedback
    In terms of feedback, we're especially looking for:

    • Are there any useful use-cases not covered?

    • Are there any workflows that are unclear or missing?

    • Is there anything that is unclear or that you don’t understand?

    • Are there any issues or unclear parts in the documentation or this post?

    Please feel free to post any feedback in this sub-forum.

    How to report bugs
    Bugs can be reported through the regular means or in this sub-forum or thread, I will be keeping a close eye on them.


    Please describe the steps as well as include any relevant files necessary for reproduction.

    Troubleshooting
    If you encounter any problems, the simplest fix is usually to use the Clear button for the window. Another option you can try is through Window -> Analysis -> Editor Iteration Profiler -> Purge Caches.
     
    Last edited: Sep 11, 2020
    Vectrex, fherbst, phobos2077 and 11 others like this.
  2. sebas77

    sebas77

    Joined:
    Nov 4, 2011
    Posts:
    1,220
    "Place the contents in a folder in your Project’s Packages folder"

    doesn't this miss the part where we should change the manifest.json too?

    Code (CSharp):
    1. "dependencies": {
    2.     "com.unity.editoriterationprofiler": "com.unity.editoriterationprofiler-master",
     
  3. miniwolf_unity

    miniwolf_unity

    Unity Technologies

    Joined:
    Apr 10, 2018
    Posts:
    100
    That is not necessary. You can either embed a package in your Package folder or reference it in a local folder

    Code (CSharp):
    1.  "com.unity.editoriterationprofiler": "file:../path/to/com.unity.editoriterationprofiler-master"
    You can of course also do the local reference using the Package Manager UI.
     
    Rarepops likes this.
  4. sebas77

    sebas77

    Joined:
    Nov 4, 2011
    Posts:
    1,220
    I am confused, is this available in the package manager? If not (as I don't think it is), how else can be used outside the Asset Folder without changing the manifest? I am not aware of other ways.
     
  5. miniwolf_unity

    miniwolf_unity

    Unity Technologies

    Joined:
    Apr 10, 2018
    Posts:
    100
    phobos2077 and sebas77 like this.
  6. JesseSTG

    JesseSTG

    Joined:
    Jan 10, 2019
    Posts:
    83
    I added it with the GitHub repo link, like so:

    Code (csharp):
    1. "com.unity.editoriterationprofiler": "https://github.com/Unity-Technologies/com.unity.editoriterationprofiler.git"
    2.  
    Works like a charm; installs exactly as it should.
     
    phobos2077, Rarepops, JoNax97 and 3 others like this.
  7. Thygrrr

    Thygrrr

    Joined:
    Sep 23, 2013
    Posts:
    133
    I'm using this both privately and in a 13 person studio team.

    It's a godsent, especially when trying to discern what part of the time spent is in the domain reload etc, and what time is actually some loading of the game itself. (profiling profane wait states of async calls is always a bit of a pain)
     
    Rarepops and lukaszunity like this.
  8. JesseSTG

    JesseSTG

    Joined:
    Jan 10, 2019
    Posts:
    83
    On the latest commit (fdfcf97) on Unity 2019.4.0f1 on Ubuntu 19.10, the "AssemblyReload" seems to always report a duration of zero.
     
    phobos2077 likes this.
  9. Rarepops

    Rarepops

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    24
    I am not aware of this, sounds like a bug I don't know of. Mind sharing more info?

    There is a bug mentioned under Known Issues where the frame responsible for Assembly Reload gets reparented to AssetImport (see image below, red) when the tree is rebuilt.

    upload_2020-6-17_8-47-18.png
     
  10. JesseSTG

    JesseSTG

    Joined:
    Jan 10, 2019
    Posts:
    83
    Sure, what info would help? I don't know what to look for.
     
  11. Rarepops

    Rarepops

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    24
    How often it reproduces? Is it a Script Compilation event? If it's often what are the steps? Do you have Deep Profiling enabled?

    Can you try going to Window -> Analysis -> Editor Iteration Profiler -> Purge Caches and then Clear in the EIP Window? What about restarting the Editor?
     
  12. OwlchemyDawson

    OwlchemyDawson

    Joined:
    Aug 28, 2018
    Posts:
    14
    It seems that the package isn't actually compatible with 2019.2. After embedding the package into my project, I see the following errors:

    In checking the Unity C# reference source, functionality like EditorUtility.RequestScriptReload and EditorUtility.enterPlayModeOptionsEnabled are only available in 2019.3+.

    Was the 2019.2 reference in the description a typo or did I miss something?
     
  13. Rarepops

    Rarepops

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    24
    My bad, changed it to 2019.3+

    The issue is simply that those API's aren't available in 2019.2.
     
  14. Bryan-Legend

    Bryan-Legend

    Joined:
    Sep 8, 2012
    Posts:
    67
    I'm getting NUnit missing compile errors. Maybe put some #ifdefs around the using NUnit.Framework; and the asserts?
     
    Rarepops likes this.
  15. Rarepops

    Rarepops

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    24
    Thanks for the heads up! I replaced them with UnityEngine.Assertions :)
     
  16. NikoBay

    NikoBay

    Joined:
    Aug 15, 2018
    Posts:
    20
    Hi guys, I run into this bug. At first install, everything works fine. Then I duplicate my project (in afraid of messing s.t up) and open duplicated project's Iteration Editor Window show no texts at all, just like attachment. Then I remove duplicated project and open original project. Now Iteration Editor Window also show no texts. Removing and reinstalling, even reseting computer does not fix it. Any ideas what I should do?
    Also, other functions like Print to console still works and texts in console show fine.
     

    Attached Files:

  17. OmegaNemesis28

    OmegaNemesis28

    Joined:
    Dec 1, 2012
    Posts:
    84
    I tried to run it while doing a project export, and it worked - kind of. It captured all the script compilation iteration events which is what I wanted. But at the end of the build, it started to spam these errors temporarily before coming back to life. Then I could look inside the profiler details. Not sure if these are important.

    Code (CSharp):
    1. NullReferenceException: Object reference not set to an instance of an object
    2. UnityEditor.EditorIterationProfiler.DataCollector.EditorEvent (UnityEditor.EditorIterationProfiler.UnityEditorEvents+Event evt, System.String data) (at Packages/com.unity.editoriterationprofiler/Editor/DataCollector.cs:70)
    3. UnityEditor.EditorIterationProfiler.UnityEditorEvents.AssemblyCompilationStarted (System.String assembly) (at Packages/com.unity.editoriterationprofiler/Editor/UnityEditorEvents.cs:64)
    4. UnityEditor.Compilation.CompilationPipeline+<>c.<SubscribeToEvents>b__16_2 (System.String assemblyPath) (at <0a2a5ea3c8ab4e3394576dd407a984f6>:0)
    5. UnityEditor.BuildPipeline:BuildPlayer(String[], String, BuildTarget, BuildOptions)
    6. BuildTools:BuildPlayer(BuildSettings, String, String, String) (at Assets/Scripts/Build/Editor/BuildTools.cs:144)
    7. BuildWindow:DoBuild(StartInfo, Boolean, Boolean, Boolean, Boolean, Boolean) (at Assets/Scripts/Build/Editor/BuildWindow_BuildTools.cs:319)
    8. BuildWindow:DrawBuildTools() (at Assets/Scripts/Build/Editor/BuildWindow_BuildTools.cs:182)
    9. BuildWindow:OnGUI() (at Assets/Scripts/Build/Editor/BuildWindow.cs:169)
    10. UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr)
    11.  
    Looks like you're already aware that sorting the columns doesn't work which is unfortunate. But also, how about sorting them on export? The HTML report is thankfully nice enough to have a % breakdown on each level (god send), but would be cool if it auto descending or ascending.
     
  18. Rarepops

    Rarepops

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    24
    Looks like you are running on OSX? Can you tell me which version of Unity and OSX you are running? There are a few bugs in OSX apparently that I am becoming aware of because I developed this tool on Windows mostly. It might be a bug with IMGUI potentially
     
  19. Rarepops

    Rarepops

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    24
    Hi, thanks for the feedback!

    The errors you are seeing are probably because the Editor Iteration Profiler is looking for Domain Reloads (they happen while Building), but then it tries to add it to a root which is not there, as it's created in another code path. I will try to fix this if it's quick but I also have some time planned to also add the ability for the EIP to capture information about building the Players, so awareness EIP might have about Domain Reloads during Building Players might be the right thing to do instead of going around it :).

    I am aware the sorting is broken and was suggested I should sort before exporting as well. The idea is if I implement sorting for the UI I can also do sorting when exporting. I do have a few days set aside for this quarter for EIP improvements, I think I will focus on this as it is an important feature :)

    Stay tuned!
     
    Last edited: Jul 14, 2020
    OmegaNemesis28 likes this.
  20. NikoBay

    NikoBay

    Joined:
    Aug 15, 2018
    Posts:
    20
    hi @Rarepops , glad you're taking care of this. I even tried to create new project but except the first time, I've never managed to make the text appears again.
    I'm running macOS Catalina 10.15.5 and Unity 2019.4.3f1
     
  21. Rarepops

    Rarepops

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    24
    I will try it out on 2019.4, haven't seen this issue otherwise. It's also curious that there is no header for the columns either... and no errors are thrown anywhere? (picture is from Windows) upload_2020-7-14_12-5-33.png
     
  22. NikoBay

    NikoBay

    Joined:
    Aug 15, 2018
    Posts:
    20
    nope, just checked. Still no texts no headers nor errors anywhere. Though Print to Console returns normal it seems. So I guess problem is the text display only.
     
    Rarepops likes this.
  23. TJHeuvel-net

    TJHeuvel-net

    Joined:
    Jul 31, 2012
    Posts:
    517
    Can we report excessive startup costs as bugs?

    upload_2020-7-16_14-36-23.png

    upload_2020-7-16_14-36-40.png
    I'm sure the terrain team, and burst team would like to know about this. We dont even use Burst, there is nothing to burst compile.
     
  24. Rarepops

    Rarepops

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    24
    Thanks for this, I've let them know!
     
    phobos2077 and TJHeuvel-net like this.
  25. TJHeuvel-net

    TJHeuvel-net

    Joined:
    Jul 31, 2012
    Posts:
    517
    It probably has to do with the many, many scripts we have. That is one thing the editor struggles to scale with, a lot of assemblies and scripts.
     
    Rarepops likes this.
  26. xoofx

    xoofx

    Unity Technologies

    Joined:
    Nov 5, 2016
    Posts:
    296
    Indeed, the Burst initialization code you are seeing in the profiling is happening once per domain reload and it is basically gathering the list of assemblies used for Burst. As you have lots of assemblies it seems that it is a direct effect from that. Not sure there is a workaround for that unfortunately. Disabling Burst in the menu won't help, because it is only for disabling compilation, we don't support disabling dynamically Burst entirely from the editor as it would complicate too much the state of kick-starting Burst in the middle of a running editor. But you can disable Burst entirely from the Unity command line by passing
    --burst-disable-compilation
     
    phobos2077 and TJHeuvel-net like this.
  27. SugoiDev

    SugoiDev

    Joined:
    Mar 27, 2013
    Posts:
    325
    I hope there is a workaround, since Unity not scaling well in larger projects is a common performance issue. In particular linear scaling with the number of types in a solution.

    Extracting metadata (like attributes) and in general scanning the entire assembly should be done with the help of the unmanaged side to speed things up. Too many performance issues come from this.
     
  28. MartinTilo

    MartinTilo

    Unity Technologies

    Joined:
    Aug 16, 2017
    Posts:
    896
    Yep, we have a type cache for that.
     
  29. SugoiDev

    SugoiDev

    Joined:
    Mar 27, 2013
    Posts:
    325
    Yeah, the TypeCache is an awesome step in the right direction. I've been closely following the development of all things editor-iteration-performance for a while now, and after testing 2020.2, I think there's been some substancial improvements.

    I've been wary of actually using Burst because of increased iteration times, but I'm hopeful that this will be dealt with eventually.
     
  30. TJHeuvel-net

    TJHeuvel-net

    Joined:
    Jul 31, 2012
    Posts:
    517
    Thanks for the in depth response!
     
  31. phobos2077

    phobos2077

    Joined:
    Feb 10, 2018
    Posts:
    224
    Doesn't work in 2019.4.3f1. AssemblyReload is always shown as 0...

    upload_2020-8-15_0-52-10.png

    Trying to use "Deep profile" just results in Unity hanging forever and eating all available 10Gb of RAM. Seems like this tool is broken.
     
    Last edited: Aug 14, 2020
  32. Rarepops

    Rarepops

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    24
    Hi, does the AssemblyReload showing as 0 happen only with Deep Profile on? I've had 30GB of RAM easily used up when Deep Profile was on.

    As a quick workaround, you could try closing the EIP and Profiler window as the GUI won't have to rebuild the hierarchy trees and draw them (you don't need to have those windows open for EIP to work, you can check what it recorded after the fact).
     
  33. phobos2077

    phobos2077

    Joined:
    Feb 10, 2018
    Posts:
    224
    With Deep Profile on - Unity just eats up all ram and hangs, with no hope of recovering. So I've only seen the results w/o deep profile, which are always 0 for me. I've tried closing the window and reloading, reloading several times over - the same result. I use Rider and just change one line of code, then alt-tab to Unity to trigger recompile/reload.
     
  34. Rarepops

    Rarepops

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    24
    I will look into that, sounds like a bug :)
     
    firstuser and phobos2077 like this.
  35. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    669
    Thanks for creating this, this is tremendously useful. I recently put together a Compilation Visualizer (on OpenUPM) and will look into extending it to contain the type of data you're capturing here.

    That being said, 2020.2 seems crazy broken in terms of compilation times, I'm seeing 10x regressions compared to 2019.4, especially when adding/removing packages (in contrast to "just" triggering a full recompile in Unity). This is for projects with only Unity packages, NO USER CODE AT ALL.

    Full recompile on 2020.2 (triggered through CompilationPipeline.RequestScriptCompilation) is > 100s:

    20200823-021309_Unity.png

    Changing a single package in the manifest takes > 560s (that's more than 9 minutes) - some of that time seems missing from the Iteration Profiler as AssemblyReload is shown as 0s on that operation, in contrast to the correct reading for a recompile:

    upload_2020-8-23_2-29-27.png

    Both of these operations take ~35s on 2019.3 + 4.

    Reported as Case 1272396
     
    Last edited: Aug 23, 2020
    Rarepops and phobos2077 like this.
  36. Rarepops

    Rarepops

    Unity Technologies

    Joined:
    Sep 5, 2019
    Posts:
    24
    We are aware of these issues and there are efforts to focus on these issues in the near future. There are some bugs with EIP, it's used more as a concept to help us determine the usefulness of these types of tools for developers. I don't think I can say too many details about it but you should see some interesting features coming to the profiler in the future :)
     
  37. fherbst

    fherbst

    Joined:
    Jun 24, 2012
    Posts:
    669
    Just to clarify, when you say
    are you referring to
    ?

    If yes, could you please
    • post the Issue Tracker link here
    • elaborate a bit more on what the issue actually is, people are trying to do work with these versions, and it would be great to know more about what's going on
    • add this issue to the "Known Issues" for 2020.1 / 2020.2 release notes?

    Thank you!
     
    Last edited: Aug 27, 2020
    firstuser and phobos2077 like this.
unityunity