Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

Question Using build configs with Test Runner for ECS project in built player, playmode tests

Discussion in 'Testing & Automation' started by lovewessman, Jan 4, 2023.

  1. lovewessman

    lovewessman

    Joined:
    Jun 15, 2019
    Posts:
    6
    I'm using Entities 0.17 and Test Framework 1.1.33. It's my understanding that builds with ECS need to be done through build configs, otherwise you run into some bugs such as the scene catalog not being loaded. This only occurs in built playmode tests, not playmode tests in the editor. Is there any way to make the test runner build using a build config? If not, how do you use Test Runner and ECS in conjunction with each other when using playmode tests on a built game?
     
  2. Warnecke

    Warnecke

    Unity Technologies

    Joined:
    Nov 28, 2017
    Posts:
    93
    Yes, the 2.0.1-exp.2 version of the test framework has support for using build configs for tests. In its new ui you will find them as options in the dropdown in the top left.
     
    lovewessman likes this.
  3. lovewessman

    lovewessman

    Joined:
    Jun 15, 2019
    Posts:
    6
    Fantastic! It seems to work very well. One question: Is there any way to do this via command line? I checked the docs but couldn't find anything there.
     
  4. lovewessman

    lovewessman

    Joined:
    Jun 15, 2019
    Posts:
    6
    Do you happen to know about how to do this via command line @Warnecke ?
     
  5. Warnecke

    Warnecke

    Unity Technologies

    Joined:
    Nov 28, 2017
    Posts:
    93
    Yes, you should be able to achieve that with e.g.
    -playerBuilderName BuildConfiguration
    for most platforms. Let me know if that works for you
     
    lovewessman likes this.
  6. lovewessman

    lovewessman

    Joined:
    Jun 15, 2019
    Posts:
    6
    Hm, when I include this, it seems to only run the tests in the editor, not actually make a build with build configs. For reference, my command line looks like this:
    Code (CSharp):
    1. "%UNITY_EDITOR_PATH%" -runTests -playerBuilderName BuildConfiguration -logFile %UNITY_LOG_FILE_PATH% -projectPath "%CURRENT_DIR%\Game" -testResults "%CURRENT_DIR%\buildOutput\%UNITY_UNIT_TESTS_RESULTS_PATH%" -forgetProjectPath -assemblyNames %ASSEMBLY_NAMES% %*
    It seems like the command
    -playerBuilderName BuildConfiguration
    isn't doing anything. Maybe I'm missing something?

    Edit: I forgot to include
    -testPlatform StandaloneWindows64
    as one of the commands. If I do
    Code (CSharp):
    1. "%UNITY_EDITOR_PATH%" -runTests -playerBuilderName BuildConfiguration -testPlatform StandaloneWindows64 -logFile %UNITY_LOG_FILE_PATH% -projectPath "%CURRENT_DIR%\Game" -testResults "%CURRENT_DIR%\buildOutput\%UNITY_UNIT_TESTS_RESULTS_PATH%" -forgetProjectPath -assemblyNames %ASSEMBLY_NAMES% %*
    it seems to work. Thanks!
     
    Last edited: Jan 26, 2023
  7. lovewessman

    lovewessman

    Joined:
    Jun 15, 2019
    Posts:
    6
    Although, I am getting some strange errors. It seems that Unity "fails" running the result. Apparently there is no build result found for StandaloneWindows64. upload_2023-1-30_19-28-31.png
    However, it does produce an executable that runs without error. But this behaviour is still not desirable since it makes jenkins quite sad when getting a bunch of errors. By the way, this error also occurs for Unity's own playmode tests, not just my own.
    Build <a instanceID="-12364">StandaloneWindows64</a> succeeded after 9.07m.
    Result can be found in <a directory="D:\jenkins_home\.jenkins\workspace\projectm\Game\Temp\UnityTempFile-401e17b422269ee4cb19f7ea641d9573\PlayerWithTests">D:\jenkins_home\.jenkins\workspace\projectm\Game\Temp\UnityTempFile-401e17b422269ee4cb19f7ea641d9573\PlayerWithTests</a>.
    Run <a instanceID="-12364">StandaloneWindows64</a> failed after 0ms.
    No build result found for <a instanceID="-12364">StandaloneWindows64</a>.
    UnityEngine.StackTraceUtility:ExtractStackTrace ()
    UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[])
    UnityEngine.Logger:Log (UnityEngine.LogType,object,UnityEngine.Object)
    UnityEngine.Debug:LogError (object,UnityEngine.Object)
    Unity.Build.BuildPipelineResult:LogResult () (at Library/PackageCache/com.unity.platforms@0.51.1-preview.21/Editor/Unity.Build/BuildPipelineResult.cs:44)
    Unity.Build.Playmode.TestRunner.BuildConfigurationPlayerLauncher:Run () (at Library/PackageCache/com.unity.platforms@0.51.1-preview.21/Tests/Editor/Unity.Build.Playmode.TestRunner/BuildConfigurationPlayerLauncher.cs:109)
    Unity.Build.Playmode.TestRunner.BuildConfigurationPlayerBuilder/<BuildAndRun>d__4:MoveNext () (at Library/PackageCache/com.unity.platforms@0.51.1-preview.21/Tests/Editor/Unity.Build.Playmode.TestRunner/BuildConfigurationPlayerBuilder.cs:28)
    UnityEditor.TestTools.TestRunner.TestRun.Tasks.BuildAndRunPlayerTask/<Execute>d__0:MoveNext () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/Tasks/Player/BuildAndRunPlayerTask.cs:14)
    UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteStep () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs:171)
    UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteCallback () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs:109)
    UnityEditor.EditorApplication:Internal_CallUpdateFunctions ()

    (Filename: Library/PackageCache/com.unity.platforms@0.51.1-preview.21/Editor/Unity.Build/BuildPipelineResult.cs Line: 44)

    TestLaunchFailedException: Player run failed
    at Unity.Build.Playmode.TestRunner.BuildConfigurationPlayerLauncher.Run () [0x0009e] in D:\jenkins_home\.jenkins\workspace\projectm\Game\Library\PackageCache\com.unity.platforms@0.51.1-preview.21\Tests\Editor\Unity.Build.Playmode.TestRunner\BuildConfigurationPlayerLauncher.cs:114
    at Unity.Build.Playmode.TestRunner.BuildConfigurationPlayerBuilder+<BuildAndRun>d__4.MoveNext () [0x00028] in D:\jenkins_home\.jenkins\workspace\projectm\Game\Library\PackageCache\com.unity.platforms@0.51.1-preview.21\Tests\Editor\Unity.Build.Playmode.TestRunner\BuildConfigurationPlayerBuilder.cs:28
    at UnityEditor.TestTools.TestRunner.TestRun.Tasks.BuildAndRunPlayerTask+<Execute>d__0.MoveNext () [0x00068] in D:\jenkins_home\.jenkins\workspace\projectm\Game\Library\PackageCache\com.unity.test-framework@2.0.1-pre.18\UnityEditor.TestRunner\TestRun\Tasks\Player\BuildAndRunPlayerTask.cs:14
    at UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner.ExecuteStep () [0x0015e] in D:\jenkins_home\.jenkins\workspace\projectm\Game\Library\PackageCache\com.unity.test-framework@2.0.1-pre.18\UnityEditor.TestRunner\TestRun\TestJobRunner.cs:171
    UnityEngine.DebugLogHandler:Internal_LogException(Exception, Object)
    UnityEngine.DebugLogHandler:LogException(Exception, Object)
    UnityEngine.Logger:LogException(Exception, Object)
    UnityEngine.Debug:LogException(Exception)
    UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteStep() (at Library\PackageCache\com.unity.test-framework@2.0.1-pre.18\UnityEditor.TestRunner\TestRun\TestJobRunner.cs:214)
    UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteCallback() (at Library\PackageCache\com.unity.test-framework@2.0.1-pre.18\UnityEditor.TestRunner\TestRun\TestJobRunner.cs:109)
    UnityEditor.EditorApplication:Internal_CallUpdateFunctions()

    (Filename: Library/PackageCache/com.unity.platforms@0.51.1-preview.21/Tests/Editor/Unity.Build.Playmode.TestRunner/BuildConfigurationPlayerLauncher.cs Line: 114)

    An unexpected error happened while running tests.
    UnityEngine.StackTraceUtility:ExtractStackTrace ()
    UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[])
    UnityEngine.Logger:Log (UnityEngine.LogType,object)
    UnityEngine.Debug:LogError (object)
    UnityEditor.TestTools.TestRunner.Api.CallbacksDelegator:RunFailed (string) (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/Api/CallbacksDelegator.cs:64)
    UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteStep () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs:215)
    UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteCallback () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs:109)
    UnityEditor.EditorApplication:Internal_CallUpdateFunctions ()

    (Filename: Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/Api/CallbacksDelegator.cs Line: 64)

    Executing IPostBuildCleanup for: Unity.PerformanceTesting.Editor.TestRunBuilder.
    Registering precompiled user dll's ...
    Registered in 0.018718 seconds.
    Refreshing native plugins compatible for Editor in 9.55 ms, found 8 plugins.
    Preloading 1 native plugins for Editor in 0.05 ms.
    Refresh completed in 4.703978 seconds.
    RefreshInfo: RefreshV2(NoUpdateAssetOptions) scanfilter:
    RefreshProfiler: Total: 4703.843ms
    InvokeBeforeRefreshCallbacks: 0.000ms
    ApplyChangesToAssetFolders: 0.068ms
    Scan: 234.223ms
    OnSourceAssetsModified: 6.706ms
    UnregisterDeletedAssets: 0.296ms
    InitializeImportedAssetsSnapshot: 19.383ms
    GetAllGuidsForCategorization: 3.975ms
    CategorizeAssets: 206.695ms
    ImportAndPostprocessOutOfDateAssets: 4103.072ms (33.949ms without children)
    PostProcessAllAssets: 4022.466ms
    EnsureUptoDateAssetsAreRegisteredWithGuidPM: 13.212ms
    InitializingProgressBar: 0.000ms
    PostProcessAllAssetNotificationsAddChangedAssets: 4.836ms
    OnDemandSchedulerStart: 4.939ms
    RestoreLoadedAssetsState: 7.882ms
    UpdateImportedAssetsSnapshot: 15.788ms
    ReloadSourceAssets: 2.772ms
    UnloadImportedAssets: 2.832ms
    Hotreload: 16.629ms
    FixTempGuids: 0.031ms
    GatherAllCurrentPrimaryArtifactRevisions: 2.682ms
    UnloadStreamsBegin: 0.381ms
    LoadedImportedAssetsSnapshotReleaseGCHandles: 5.923ms
    GetLoadedSourceAssetsSnapshot: 15.424ms
    PersistCurrentRevisions: 9.491ms
    UnloadStreamsEnd: 0.377ms
    Untracked: 72.879ms
    Too many instant steps in test execution mode: Error. Current task ConditionalTask.
    UnityEngine.StackTraceUtility:ExtractStackTrace ()
    UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[])
    UnityEngine.Logger:Log (UnityEngine.LogType,object)
    UnityEngine.Debug:LogError (object)
    UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteCallback () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs:120)
    UnityEditor.EditorApplication:Internal_CallUpdateFunctions ()

    (Filename: Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs Line: 120)
     

    Attached Files: