Search Unity

Resolved Need help for displaying coverage history

Discussion in 'Testing & Automation' started by mwarvik, Mar 20, 2021.

  1. mwarvik

    mwarvik

    Joined:
    Nov 10, 2016
    Posts:
    4
    I am currently working on implementing the code coverage package on our projects on DevOps. The basic functionality works, but I cannot get the coverage history graph or risk hotspots to show in DevOps.

    The current steps are as follows:
    1. Run edit mode tests, publish file to folder.
    2. Convert the Unity OpenCover files to Cobertura with ReportGenerator tool. historydir input is set, and each run creates a new xml file. reports input is set to opencov/EditMode subfolder xml.
    3. Coverage results is published with PublishCodeCoverageResults task.

    I have run the pipeline around 20-30 times with different variations of parameters, and also with disable.coverage.autogenerate. This did not seem to work either, and also removed the styling for the coverage page.

    Any ideas or pointers on what to do?
     
    JimboJonesBL likes this.
  2. mwarvik

    mwarvik

    Joined:
    Nov 10, 2016
    Posts:
    4
    Currently it looks something like this. Have also tried to add
    generateHtmlReport;generateHtmlReportHistory and coverageHistoryPath to Unity arguments, but should not be needed as I understand it, as this is overriden by report generator anyway.


    Code (CSharp):
    1. - powershell: |
    2.  
    3.     $logFile = New-Item -Path .\editmode-test-run.log -ItemType File -Force
    4.  
    5.     $proc = Start-Process -FilePath $env:UNITY_EXE_PATH -PassThru -ArgumentList "
    6.        -projectPath $env:PROJECT_PATH
    7.        -runTests
    8.        -testPlatform editmode
    9.        -debugCodeOptimization
    10.        -enableCodeCoverage
    11.        -coverageResultsPath .\test-coverage-results
    12.        -coverageOptions generateAdditionalMetrics;pathFilters:$env:COVERAGE_PATH_FILTERS
    13.        -batchmode
    14.        -logFile $($logFile.Name)
    15.        -editorTestsResultFile .\test-editmode-default.xml"
    16.  
    17.     $ljob = Start-Job -ScriptBlock { param($log) Get-Content "$log" -Wait } -ArgumentList $logFile.FullName
    18.  
    19.     while (-not $proc.HasExited -and $ljob.HasMoreData)
    20.     {
    21.         Receive-Job $ljob
    22.         Start-Sleep -Milliseconds 500
    23.     }
    24.     Receive-Job $ljob
    25.  
    26.     Stop-Job $ljob
    27.  
    28.     Remove-Job $ljob
    29.     Stop-Process $proc
    30.     exit $proc.ExitCode
    31.   displayName: 'Run EditMode tests'
    32.   env:
    33.     UNITY_EXE_PATH: '${{parameters.unityInstallFolder}}$(unitygetprojectversion.projectVersion)${{parameters.unityEditorPath}}'
    34.     PROJECT_PATH: ${{ parameters.projectPath }}
    35.     COVERAGE_PATH_FILTERS: ${{ parameters.testCoveragePathFilters }}
    36.  
    37. - task: Palmmedia.reportgenerator.reportgenerator-build-release-task.reportgenerator@4
    38.   displayName: 'Convert coverage report'
    39.   inputs:
    40.     reports: $(Build.SourcesDirectory)\test-coverage-results\*-opencov\EditMode\*.xml;
    41.     targetdir: $(Build.SourcesDirectory)\test-coverage
    42.     reporttypes: 'Cobertura'
    43.     historydir: C:\coverage-history\${{parameters.packageName}}
    44.     #customSettings: 'settings:renderPngFallBackImagesForHistoryCharts=true'
    45.  
    46. # Publish test results:
    47. - task: PublishTestResults@2
    48.   displayName: 'Publish test results'
    49.   inputs:
    50.     testResultsFormat: NUnit
    51.     testResultsFiles: 'test*.xml'
    52.     failTaskOnFailedTests: true
    53.  
    54. # Publish code coverage results
    55. - task: PublishCodeCoverageResults@1
    56.   displayName: 'Publish code coverage results'
    57.   inputs:
    58.     codeCoverageTool: 'cobertura'
    59.     summaryFileLocation: '$(Build.SourcesDirectory)\test-coverage\Cobertura.xml'
     
  3. mwarvik

    mwarvik

    Joined:
    Nov 10, 2016
    Posts:
    4
    After 47 runs of the pipeline it worked! I set disable.coverage.autogenerate to true, generated HtmlInline_AzureDevops;Cobertura reports from ReportGenerator, and added the reportDirectory in PublishCodeCoverageResults task.
     
    Unity-Nikos likes this.
  4. Unity-Nikos

    Unity-Nikos

    Unity Technologies

    Joined:
    Sep 30, 2015
    Posts:
    87
    Hi @mwarvik - I am glad you resolved it! I wanted to add that currently for pathFilters only absolute paths are supported. However, you can use globbing to make them shorter e.g. **/Assets/Scripts/. We are exploring allowing relative paths in a future release.
     
    mwarvik likes this.
  5. mwarvik

    mwarvik

    Joined:
    Nov 10, 2016
    Posts:
    4
    As it is now, it works fine for our use case :) We just include everything in Packages folder by default (eg '+*/Packages/*'), with an optional parameter that can be sent in for special cases.