Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Bug Code coverage not working when multiple test assemblies are run via Rider IDE

Discussion in 'Testing & Automation' started by Xarbrough, May 20, 2021.

  1. Xarbrough

    Xarbrough

    Joined:
    Dec 11, 2014
    Posts:
    1,188
    When I run edit mode tests and then play mode tests one after the other in the Unity test runner, code coverage seems to be mostly working fine (except a few missing lines here and there):

    UnityRunner.PNG

    However, when I run all of my tests (edit + play mode) via the Rider IDE test runner, there are issues:

    RiderRunner.PNG

    In my project I have two runtime assemblies A and B and three test assemblies A-EditMode, B-EditMode, B-PlayMode. From looking at the report I can see that when using Rider only the code that is exercise by B-PlayMode is covered. This also happens to be the test assembly that is run last (it seems Rider calls a test run for each assembly one after the other). It seems like the results of the most recent run overwrite the other assemblies each time. If I only run a single assembly, the results change, but then I only have a single assembly covered.

    What might be going wrong here? Is this something that the Code Coverage package has control over or an issue with the Rider TestRunner?
     
  2. Unity-Nikos

    Unity-Nikos

    Unity Technologies

    Joined:
    Sep 30, 2015
    Posts:
    87
    Hi @Xarbrough, this is because Rider runs separate Test Sessions per Test Assembly and the Code Coverage package clears the coverage results at the beginning of a Test Session for the same assembly. We are looking at what would be the best way to fix this together with the Rider team. In the meantime there is a not-so-elegant workaround:

    You can select Start Recording in the Code Coverage window in the editor, run your tests in Rider and when they finish select Stop Recording. That way all the code coverage capturing happens in the same coverage session.
     
  3. Xarbrough

    Xarbrough

    Joined:
    Dec 11, 2014
    Posts:
    1,188
    Thanks for letting me know! That’ll do for a workaround.
     
  4. Unity-Nikos

    Unity-Nikos

    Unity Technologies

    Joined:
    Sep 30, 2015
    Posts:
    87
    A ticket was opened for Rider for refactoring Unity UnitTesting support in a way to run all tests at once, instead of running them assembly by assembly.
     
    nowsprinting and Xarbrough like this.