Official Code Coverage Package - Release Announcements

Hi and welcome! We will be using this thread for announcements related to the Code Coverage package.

Current release: 1.2.5
This release is intended for: Unity 2019.3 and above

Resources

Quickstart - Code Coverage Tutorial
Code Coverage package - Release Notes
Code Coverage package - Documentation

Note that this thread is locked for comments. If you have issues/comments/feedback related to the Code Coverage package, please visit the Testing & Automation section to browse current conversations or start a new thread. Please use the code coverage tag.

Introduction

Use the Code Coverage package with the Test Runner to gather and present test coverage information. When you run your tests with code coverage enabled you can see exactly which lines of your code are executed when the tests ran in addition to whether the tests passed or failed. See Using Code Coverage with Test Runner.


Example HTML coverage report generated by the package

Once a test run has completed, the Code Coverage package will generate an HTML coverage report showing which lines of your code are covered by tests. Code Coverage currently supports PlayMode and EditMode tests run in the Editor only; Standalone/Player support is planned for a future release. It also allows you to track the code coverage changes through time.

Additionally, the Code Coverage package offers a Coverage Recording feature which allows capturing coverage data on demand, in case you do not have tests in your project or doing manual testing.

The Quickstart guide will give you an insight into the package.


Code Coverage window


The discussion thread used while the package was in preview can be found here.

 

Code Coverage package - version 0.2.1-preview is out

• Added pathFilters batchmode coverageOption
• Improved globbing for pathFilters and assemblyFilters - see https://docs.unity3d.com/Packages/c...overage.html#using-code-coverage-in-batchmode
• Added new sections and examples in documentation
• Added warning and button to switch to debug mode, when using Code Optimization in release mode in 2020.1 and above
• Added confirmation dialogs when selecting Clear Data and Clear History buttons

Documentation: https://docs.unity3d.com/Packages/com.unity.testtools.codecoverage@0.2


[Unsupported] Package installation in 2019.2

To get the package in 2019.2:

• Install it in a new project in 2019.3
• Go to the project's folder into Library/PackageCache and copy the com.unity.testtools.codecoverage@0.2.x-preview folder
• Paste it into your 2019.2 project's Packages folder

Please note that the Code Coverage package is officially supported in 2019.3 and above.

It is highly recommended having the latest version of 2019.2 installed.

 

Code Coverage package - version 0.2.2-preview is out

• The default Included Assemblies are now only the assemblies found under the project's Assets folder, instead of all project assemblies
• After the report is generated, the file viewer window highlights the index.htm file, if Generate HTML Report is selected
• Fixed unassigned CodeCoverageWindow.m_IncludeWarnings warning in 2019.3

Documentation: https://docs.unity3d.com/Packages/com.unity.testtools.codecoverage@0.2

 

Code Coverage package - version 0.2.3-preview is out

• If more than one instance of the -coverageOptions command-line argument is specified, they will now be merged into a single instance
• If more than one instance of the -coverageResultsPath command-line argument is specified, only the first instance will be accepted
• Added Generate combined report from EditMode and PlayMode tests section in documentation, under Using Code Coverage in batchmode
• When closing (selecting outside of) the Included Assemblies dropdown, input is not accidentally propagated to the Code Coverage window
• Included Assemblies dropdown is now resizing to the longest assembly name (case 1215600)

Documentation: https://docs.unity3d.com/Packages/com.unity.testtools.codecoverage@0.2

 

Code Coverage package @ 0.3.0-preview is live

• Added coverageHistoryPath and generateHtmlReportHistory in -coverageOptions for batchmode
• Added History Location and Generate History settings in the Code Coverage window
• Added generateAdditionalMetrics in -coverageOptions for batchmode and removed enableCyclomaticComplexity (it is now included in Additional Metrics)
• Added Generate Additional Metrics setting in the Code Coverage window and removed Cyclomatic Complexity (it is now included in Additional Metrics)
• Added Crap Score in Additional Metrics
• Using the Settings Manager package to handle the serialization of project settings
• Added Code Coverage Workshop sample project
• Added a DisplayDialog warning when Code Optimization is set to Release mode
• Execute Stop Recording on the update loop, instead of the OnGUI (removes an EndLayoutGroup error)
• Make sure operator and anonymous function names are generated correctly
• Refactored code; in OpenCoverReporter class (to reduce Cyclomatic Complexity), in CodeCoverageWindow class and others
• Updated Report Generator to version v4.5.8
• Updated documentation to match version 0.3.0-preview

Documentation: https://docs.unity3d.com/Packages/com.unity.testtools.codecoverage@latest

 

Code Coverage package @ 0.3.1-preview is live

• Fixed issue where CRAP calculation was incorrect when generic methods were parsed (case 1261159)
• Corrected Six Labors License copyright in Third Party Notices (case 1257869)
• If assemblyFilters is not specified in -coverageOptions in batchmode, include only the assemblies found under the Assets folder
• Updated Report Generator to version v4.6.4

Documentation: https://docs.unity3d.com/Packages/com.unity.testtools.codecoverage@latest

 

Code Coverage package @ 0.4.0-preview is live

• Moved Code Coverage window under Window > Analysis
• Added Included Paths and Excluded Paths as ReorderableLists in the Code Coverage window
• Included Assemblies now use a single dropdown
• Added support for

  ExcludeFromCoverage

  and

  ExcludeFromCodeCoverage

  attributes
• Added CommandLineParser and removed dependency to internals in Test Framework
• Implemented

  {ProjectPath}

  alias in

  Settings.json

• Added Burst Compilation enabled console warning and info HelpBox with button to disable
• Added Analytics to help improve the user experience
• Removed the old EditorPref workflow from CoveragePreferences
• Updated Report Generator to version 4.7.1
• Refactored code; in Utils, Filtering, ResultWriter, Window and API classes
• Added CoverageWindow and Filtering folders
• Moved Generate History outside of Generate HTML Report. It is now disabled only if both Generate HTML Report and Generate Badges are not selected
• Disabled Generate from Last button when there are no assemblies selected
• Display info HelpBox when there are no assemblies selected
• Paths are now stored with forward slashes on Windows
• Added

  CodeCoverage.VerbosityLevel

  to set the verbosity level used in editor and console logs
• Added warning about Code Coverage not being supported currently when running PlayMode tests in standalone player
• Updated documentation and workshop to match version 0.4.0-preview

Documentation: https://docs.unity3d.com/Packages/com.unity.testtools.codecoverage@0.4

 

Code Coverage package @ 0.4.1-preview is live

Fixes

• Fixed multiple reports generated in batchmode when passing generateHtmlReport in -coverageOptions without passing -runTests

Changes

• All project assemblies are included when there are included paths specified in pathFilters but no included assemblies in assemblyFilters, when running in batchmode
• Updated Report Generator to version 4.8.4
• Updated documentation to match version 0.4.1-preview

Improvements

• Introduced new assemblyFilters aliases in batchmode, used for referencing a group of assemblies to include or exclude. These are <user>, <project> and <packages>

Documentation: https://docs.unity3d.com/Packages/com.unity.testtools.codecoverage@0.4

 

Code Coverage package @ 0.4.2-preview is live

Fixes

• Updated Include Platforms to Editor only in the ReportGeneratorMerged.dll settings. Fixes an Android build error introduced in 0.4.1-preview (case 1306557)

Documentation: https://docs.unity3d.com/Packages/com.unity.testtools.codecoverage@0.4

 

Code Coverage package versions 1.0.0-pre.4 & 0.4.3-preview are live

Fixes

• Fixed assembly version validation error due to internal libraries included in the ReportGeneratorMerged.dll (case 1312121)

Changes

• Added Enable Code Coverage checkbox under Settings in Code Coverage window

Note: In Unity 2019 and 2020 you can enable Code Coverage in General Preferences. This was removed in Unity 2021; the user interface for managing Code Coverage is now entirely inside the Code Coverage package.​

• The settings and options passed in the command line override/disable the settings in the Code Coverage window and relevant warnings display to indicate this
• Updated Report Generator to version 4.8.5
• Updated documentation and workshop to match version 1.0.0-pre.4/0.4.3-preview

Improvements

• Added verbosity in -coverageOptions for batchmode
• Added Generate combined report from separate projects section in documentation, under Using Code Coverage in batchmode

Documentation: https://docs.unity3d.com/Packages/com.unity.testtools.codecoverage@1.0

Note: 1.0.0-pre.4 will become the released version 1.0.0 (due in the next couple of weeks) which will be available in Unity 2019 and above, at which point we'll drop support for the preview version.

 

Code Coverage package version 1.0.0 is now available

Fixes

• Fixed issues with Path Filtering (cases 1318896, 1318897)

Improvements

• Selection/focus is cleared when mouse is clicked outside of the individual settings' areas
• Added Quickstart guide in documentation
• Renamed the Code Coverage Workshop sample to Code Coverage Tutorial
• Updated documentation and worksheet to match version 1.0.0

Note: In Unity 2019 and 2020 you can enable Code Coverage in General Preferences. This was removed in Unity 2021; the user interface for managing Code Coverage is now entirely inside the Code Coverage package.

Note: We have stopped supporting the preview version as the package is now released and version 1.0 is available in Unity 2019 and above.

Quickstart - Code Coverage Tutorial
Code Coverage package - Release Notes
Code Coverage package - Documentation

 

Code Coverage package version 1.1.0 is now available

Fixes

• Ensure Results and History folders are created if they do not exist (case 1334551)
• Added support for ExcludeFromCoverage/ExcludeFromCodeCoverage for lambda expressions and yield statements (case 1338636)
• Added support for ExcludeFromCodeCoverage for getter/setter properties (case 1338665)
• -coverageOptions are only parsed when running from the command line (feedback)

Changes

• Updated Report Generator to version 4.8.9

Improvements

• Implemented changes to support Test Framework package version 1.2
• Logs while the Report is generated are output per message rather than at the end of the generation
• Do not log burst warning when -burst-disable-compilation is passed in the command line
• Added Ignoring tests for Code Coverage section in documentation
• Updated the Generate combined report from separate projects section in documentation
• Updated documentation to match version 1.1.0

Features

• Added Code Coverage session Events API to subscribe to events invoked during a Code Coverage session
• Added useProjectSettings in -coverageOptions for batchmode which allows using the settings specified in ProjectSettings/Settings.json
• Added pathStrippingPatterns in -coverageOptions for batchmode which allows stripping specific sections from the paths that are stored in the coverage results xml files
• Added sourcePaths in -coverageOptions for batchmode which allows specifying the source directories which contain the corresponding source code

Quickstart - Code Coverage Tutorial
Code Coverage package - Release Notes
Code Coverage package - Documentation

 

Code Coverage package version 1.1.1 is now available

Fixes

• Ensure assemblies are removed from the Included Assemblies field if they no longer exist (case 1318668)

Changes

• Updated Report Generator to version 4.8.13

Improvements

• Added Help IconButton in the Code Coverage window for Unity versions 2021.2.2f1 and above
• Added What's new and Upgrade guide pages in the documentation
• Updated documentation to match version 1.1.1

Quickstart - Code Coverage Tutorial
Code Coverage package - Release Notes
Code Coverage package - Documentation

 

Code Coverage package version 1.2.0 is now available

Fixes

• Ensure hidden sequence points are ignored (case 1372305).

Changes

• Updated Report Generator to version 5.0.4.
• Updated the UI of the Code Coverage window moving the action buttons into a toolbar at the top.
• Renamed assemblyFilters aliases in batchmode; <user> was renamed to <assets> and <project> was renamed to <all>.
• Replaced pathStrippingPatterns with pathReplacePatterns in batchmode. The pathReplacePatterns option allows stripping and replacing specific sections from the paths that are stored in the coverage results xml files.

See the Upgrade guide if upgrading to Code Coverage package version 1.2.

Improvements

• The size of the coverage result files and the Code Coverage session duration have been optimized. At the start of the session a coverage xml result file is generated which includes all the lines but with zero coverage. The following coverage xml result files that are generated within a Code Coverage session include only the coverage data of the visited lines.
• Added Help IconButton in the toolbar in the Code Coverage window.
• Updated the mechanic for opening the containing folder, change the location or reset to the default location for Results Location and Report History Location.
• Refactored the Code Coverage window UI to include a new Report Options section and removing the word 'Generate' from the options.
• Introduced new selection buttons under the Included Assemblies dropdown in the Code Coverage window; use the All button to select all the assemblies in the project. Use the Assets button to select only the assemblies under the Assets folder. Use the Packages button to select only the Packages' assemblies. If searching, the buttons will apply only to the assemblies visible in the list.
• Updated What's new and Upgrade guide pages in the documentation.
• Added Using relative paths in path filters section in documentation.
• Updated the editor and console logs; added information about the assembly and path filters, improved coverage session logs.
• Improved the progress bars for Writing coverage results and Generating the report.
• Added an icon for the Code Coverage window.
• Updated documentation to match version 1.2.0.

Features

• Added Pause Recording and Resume Recording buttons in the toolbar in the Code Coverage window.
• Added Log Verbosity Level setting in the Code Coverage window which allows setting the verbosity level for the editor and console logs.
• Added Additional Reports option in the Code Coverage window which if checked SonarQube, Cobertura and LCOV reports will be generated. Added generateAdditionalReports in -coverageOptions for batchmode.
• Added Test Runner References report option in the Code Coverage window which if checked includes test references to the generated coverage results and enables the Coverage by test methods section in the HTML report, allowing you to see how each test contributes to the overall coverage. Added generateTestReferences in -coverageOptions for batchmode.
• Added Auto Open Report option in the Code Coverage window which if checked the coverage report will open automatically after it has been generated.
• Added pathFiltersFromFile in -coverageOptions for batchmode which allows specifying an external file which contains a list of path filters. When this file contains relative paths, the sourcePaths option can be used to specify the source directories.
• Added dontClear in -coverageOptions for batchmode which allows coverage results to be accumulated after every code coverage session. If not passed the results are cleared before a new session. For more information see Generate combined report from EditMode and PlayMode tests.
• When the pathFilters option or the pathFiltersFromFile option in -coverageOptions contains relative paths, the sourcePaths option can be used to specify the source directories.
  
  Quickstart - Code Coverage Tutorial
  What's new
  Release Notes
  Documentation
  Upgrade Guide

 

Code Coverage package version 1.2.1 is now available

Fixes

• Fixed compatibility with Test Framework package version 1.3.

Improvements

• A single file summary version of the report is now generated in Json format, in addition to the XML and Markdown formats.
• Added a warning when an invalid coverage option is passed in -coverageOptions in batchmode.
  

Quickstart - Code Coverage Tutorial
What's new
Release Notes
Documentation
Upgrade Guide

 

Code Coverage package version 1.2.2 is now available

Fixes

• Temporary fix for the Test Framework 1.3 issue where the RunFinished callback is not called when running from the command line and there is a domain reload (case DSTR-692).

Improvements

• Reduced the number of logs for the default Verbosity:Info.
• Added Uncoverable lines definition in How to interpret the results page in the documentation.
• Updated documentation to match version 1.2.2.
  

Quickstart - Code Coverage Tutorial
What's new
Release Notes
Documentation
Upgrade Guide

 

Code Coverage package version 1.2.3 is now available

Fixes

• Fixed failing to gather code coverage for generic methods (case COV-17).

Improvements

• Added filtersFromFile in -coverageOptions for batchmode. This allows specifying an external Json file which contains path and assembly filtering rules. When this file contains relative paths, the sourcePaths option can be used to specify the source directories.
  Note: The pathFiltersFromFile option will be deprecated in the next package major release. Please use the new filtersFromFile option instead.
  
  
• Make sure --burst-disable-compilation is expected to be passed with two dashes, unlike other editor command line options.
  
  Quickstart - Code Coverage Tutorial
  What's new
  Release Notes
  Documentation
  Upgrade Guide

 

Code Coverage package version 1.2.4 is now available

Fixes

• Fixed failing to gather code coverage for normal methods in generic classes (case COV-27).
• Documentation: Corrected Settings.json path in useProjectSettings section in -coverageOptions (case COV-26).
• Make sure Auto Generate Report defaults to true when running from the command line (case COV-25).

Quickstart - Code Coverage Tutorial
What's new
Release Notes
Documentation
Upgrade Guide

 

Code Coverage package version 1.2.5 is now available

Fixes

• Fixed failing to generate code coverage for assemblies whose name starts with 'system' (case COV-38).
• Fixed results xml file not been correctly stored under the EditMode folder if it is written after a domain reload (case COV-36).

Quickstart - Code Coverage Tutorial
What's new
Release Notes
Documentation
Upgrade Guide