Search Unity

Resolved Use Settings.json for coverage from command line

Discussion in 'Testing & Automation' started by marco-nutonomy, May 5, 2021.

  1. marco-nutonomy

    marco-nutonomy

    Joined:
    Jul 20, 2018
    Posts:
    2
    Hi,
    Is there a way to have code coverage use the Settings.json config file instead of having to specify all the options every time from command line? Having to list the assemblies/exclusions/inclusions in two different places (Settings.json and command line for CI) is extremely prone to misalignment.
    If not, can we have this feature please? It would be even better to be able to specify the Settings.json to use, as well as overrides if needed (e.g., output the report in a different folder, or "do not generate html/badge").

    I Apologize if this has been asked somewhere else but i couldn't find anything related tagged as code coverage.

    Thank you,
    Marco
     
  2. Unity-Nikos

    Unity-Nikos

    Unity Technologies

    Joined:
    Sep 30, 2015
    Posts:
    62
    Hi Marco. Thank you for the suggestion. We can introduce a new command line -coverageOption (eg. useProjectSettings) to use the settings in Settings.json instead, and any settings still passed in the command line will override the settings from Settings.json. Let me know what you think.

    Thank you,
    Nikos
     
    bdovaz likes this.
  3. marco-nutonomy

    marco-nutonomy

    Joined:
    Jul 20, 2018
    Posts:
    2
    Hi Nikos,
    Yes, that would be ideal!

    On a separate note, I i've also found a bug with the normal coverage window. Let's say that you set up a folder (e.g.
    "value": "{\"m_Value\":\"{ProjectPath}/Coverage\"}"
    . When you reopen the window, if that folder doesn't exist (it happens to us because we .gitignore it), the path gets reset to
    "value": "{\"m_Value\":\"{ProjectPath}\"}"
    . This happens both for Path and HistoryPath.

    Repro steps:
    1. Create a folder (e.g. Coverage under the project root)
    2. Open the coverage window
    3. Select folder created at (1) for both Path and History
    4. Close coverage window
    5. Delete folder created at (1)
    6. Open the coverage window again
    7. Observe the Results Location and the History Location, as well as the values in the Settings.json being reset to project root
    Moreover, from this point, if you:
    1. Close the coverage window
    2. Create the folder again
    3. Revert the changes to Settings.json manually
    4. Reopen the coverage window
    It somehow goes back to root again, changing yet again the Settings.json file to root too.
    This is extremely problematic for us, and it would be nice if the behavior was to create the folder on demand when the report/history have to be saved instead.

    Thank you again,
    Marco
     
  4. JuliusJ

    JuliusJ

    QA Minion of the month Unity Technologies

    Joined:
    May 20, 2015
    Posts:
    13
    Hey Marco. Thank you for your report. I'll open a bug for this and will share a link to issue tracker once it's ready.

    Edit:
    Issue can be tracked here https://issuetracker.unity3d.com/product/unity/issues/guid/1334551/
     
    Last edited: May 27, 2021
  5. Unity-Nikos

    Unity-Nikos

    Unity Technologies

    Joined:
    Sep 30, 2015
    Posts:
    62
unityunity