Question Using build configs with Test Runner for ECS project in built player, playmode tests

I'm using Entities 0.17 and Test Framework 1.1.33. It's my understanding that builds with ECS need to be done through build configs, otherwise you run into some bugs such as the scene catalog not being loaded. This only occurs in built playmode tests, not playmode tests in the editor. Is there any way to make the test runner build using a build config? If not, how do you use Test Runner and ECS in conjunction with each other when using playmode tests on a built game?

 

Fantastic! It seems to work very well. One question: Is there any way to do this via command line? I checked the docs but couldn't find anything there.

 

Do you happen to know about how to do this via command line @Warnecke ?

 

Hm, when I include this, it seems to only run the tests in the editor, not actually make a build with build configs. For reference, my command line looks like this:

Code (CSharp):

1. "%UNITY_EDITOR_PATH%" -runTests -playerBuilderName BuildConfiguration -logFile %UNITY_LOG_FILE_PATH% -projectPath "%CURRENT_DIR%\Game" -testResults "%CURRENT_DIR%\buildOutput\%UNITY_UNIT_TESTS_RESULTS_PATH%" -forgetProjectPath -assemblyNames %ASSEMBLY_NAMES% %*

It seems like the command

-playerBuilderName BuildConfiguration

isn't doing anything. Maybe I'm missing something?

Edit: I forgot to include

-testPlatform StandaloneWindows64

as one of the commands. If I do

Code (CSharp):

1. "%UNITY_EDITOR_PATH%" -runTests -playerBuilderName BuildConfiguration -testPlatform StandaloneWindows64 -logFile %UNITY_LOG_FILE_PATH% -projectPath "%CURRENT_DIR%\Game" -testResults "%CURRENT_DIR%\buildOutput\%UNITY_UNIT_TESTS_RESULTS_PATH%" -forgetProjectPath -assemblyNames %ASSEMBLY_NAMES% %*

it seems to work. Thanks!

 

Although, I am getting some strange errors. It seems that Unity "fails" running the result. Apparently there is no build result found for StandaloneWindows64.
However, it does produce an executable that runs without error. But this behaviour is still not desirable since it makes jenkins quite sad when getting a bunch of errors. By the way, this error also occurs for Unity's own playmode tests, not just my own.

Spoiler: log file

Build <a instanceID="-12364">StandaloneWindows64</a> succeeded after 9.07m.
Result can be found in <a directory="D:\jenkins_home\.jenkins\workspace\projectm\Game\Temp\UnityTempFile-401e17b422269ee4cb19f7ea641d9573\PlayerWithTests">D:\jenkins_home\.jenkins\workspace\projectm\Game\Temp\UnityTempFile-401e17b422269ee4cb19f7ea641d9573\PlayerWithTests</a>.
Run <a instanceID="-12364">StandaloneWindows64</a> failed after 0ms.
No build result found for <a instanceID="-12364">StandaloneWindows64</a>.
UnityEngine.StackTraceUtility:ExtractStackTrace ()
UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[])
UnityEngine.Logger:Log (UnityEngine.LogType,object,UnityEngine.Object)
UnityEngine.Debug:LogError (object,UnityEngine.Object)
Unity.Build.BuildPipelineResult:LogResult () (at Library/PackageCache/com.unity.platforms@0.51.1-preview.21/Editor/Unity.Build/BuildPipelineResult.cs:44)
Unity.Build.Playmode.TestRunner.BuildConfigurationPlayerLauncher:Run () (at Library/PackageCache/com.unity.platforms@0.51.1-preview.21/Tests/Editor/Unity.Build.Playmode.TestRunner/BuildConfigurationPlayerLauncher.cs:109)
Unity.Build.Playmode.TestRunner.BuildConfigurationPlayerBuilder/<BuildAndRun>d__4:MoveNext () (at Library/PackageCache/com.unity.platforms@0.51.1-preview.21/Tests/Editor/Unity.Build.Playmode.TestRunner/BuildConfigurationPlayerBuilder.cs:28)
UnityEditor.TestTools.TestRunner.TestRun.Tasks.BuildAndRunPlayerTask/<Execute>d__0:MoveNext () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/Tasks/Player/BuildAndRunPlayerTask.cs:14)
UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteStep () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs:171)
UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteCallback () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs:109)
UnityEditor.EditorApplication:Internal_CallUpdateFunctions ()

(Filename: Library/PackageCache/com.unity.platforms@0.51.1-preview.21/Editor/Unity.Build/BuildPipelineResult.cs Line: 44)

TestLaunchFailedException: Player run failed
at Unity.Build.Playmode.TestRunner.BuildConfigurationPlayerLauncher.Run () [0x0009e] in D:\jenkins_home\.jenkins\workspace\projectm\Game\Library\PackageCache\com.unity.platforms@0.51.1-preview.21\Tests\Editor\Unity.Build.Playmode.TestRunner\BuildConfigurationPlayerLauncher.cs:114
at Unity.Build.Playmode.TestRunner.BuildConfigurationPlayerBuilder+<BuildAndRun>d__4.MoveNext () [0x00028] in D:\jenkins_home\.jenkins\workspace\projectm\Game\Library\PackageCache\com.unity.platforms@0.51.1-preview.21\Tests\Editor\Unity.Build.Playmode.TestRunner\BuildConfigurationPlayerBuilder.cs:28
at UnityEditor.TestTools.TestRunner.TestRun.Tasks.BuildAndRunPlayerTask+<Execute>d__0.MoveNext () [0x00068] in D:\jenkins_home\.jenkins\workspace\projectm\Game\Library\PackageCache\com.unity.test-framework@2.0.1-pre.18\UnityEditor.TestRunner\TestRun\Tasks\Player\BuildAndRunPlayerTask.cs:14
at UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner.ExecuteStep () [0x0015e] in D:\jenkins_home\.jenkins\workspace\projectm\Game\Library\PackageCache\com.unity.test-framework@2.0.1-pre.18\UnityEditor.TestRunner\TestRun\TestJobRunner.cs:171
UnityEngine.DebugLogHandler:Internal_LogException(Exception, Object)
UnityEngine.DebugLogHandler:LogException(Exception, Object)
UnityEngine.Logger:LogException(Exception, Object)
UnityEngine.Debug:LogException(Exception)
UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteStep() (at Library\PackageCache\com.unity.test-framework@2.0.1-pre.18\UnityEditor.TestRunner\TestRun\TestJobRunner.cs:214)
UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteCallback() (at Library\PackageCache\com.unity.test-framework@2.0.1-pre.18\UnityEditor.TestRunner\TestRun\TestJobRunner.cs:109)
UnityEditor.EditorApplication:Internal_CallUpdateFunctions()

(Filename: Library/PackageCache/com.unity.platforms@0.51.1-preview.21/Tests/Editor/Unity.Build.Playmode.TestRunner/BuildConfigurationPlayerLauncher.cs Line: 114)

An unexpected error happened while running tests.
UnityEngine.StackTraceUtility:ExtractStackTrace ()
UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[])
UnityEngine.Logger:Log (UnityEngine.LogType,object)
UnityEngine.Debug:LogError (object)
UnityEditor.TestTools.TestRunner.Api.CallbacksDelegator:RunFailed (string) (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/Api/CallbacksDelegator.cs:64)
UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteStep () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs:215)
UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteCallback () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs:109)
UnityEditor.EditorApplication:Internal_CallUpdateFunctions ()

(Filename: Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/Api/CallbacksDelegator.cs Line: 64)

Executing IPostBuildCleanup for: Unity.PerformanceTesting.Editor.TestRunBuilder.
Registering precompiled user dll's ...
Registered in 0.018718 seconds.
Refreshing native plugins compatible for Editor in 9.55 ms, found 8 plugins.
Preloading 1 native plugins for Editor in 0.05 ms.
Refresh completed in 4.703978 seconds.
RefreshInfo: RefreshV2(NoUpdateAssetOptions) scanfilter:
RefreshProfiler: Total: 4703.843ms
InvokeBeforeRefreshCallbacks: 0.000ms
ApplyChangesToAssetFolders: 0.068ms
Scan: 234.223ms
OnSourceAssetsModified: 6.706ms
UnregisterDeletedAssets: 0.296ms
InitializeImportedAssetsSnapshot: 19.383ms
GetAllGuidsForCategorization: 3.975ms
CategorizeAssets: 206.695ms
ImportAndPostprocessOutOfDateAssets: 4103.072ms (33.949ms without children)
PostProcessAllAssets: 4022.466ms
EnsureUptoDateAssetsAreRegisteredWithGuidPM: 13.212ms
InitializingProgressBar: 0.000ms
PostProcessAllAssetNotificationsAddChangedAssets: 4.836ms
OnDemandSchedulerStart: 4.939ms
RestoreLoadedAssetsState: 7.882ms
UpdateImportedAssetsSnapshot: 15.788ms
ReloadSourceAssets: 2.772ms
UnloadImportedAssets: 2.832ms
Hotreload: 16.629ms
FixTempGuids: 0.031ms
GatherAllCurrentPrimaryArtifactRevisions: 2.682ms
UnloadStreamsBegin: 0.381ms
LoadedImportedAssetsSnapshotReleaseGCHandles: 5.923ms
GetLoadedSourceAssetsSnapshot: 15.424ms
PersistCurrentRevisions: 9.491ms
UnloadStreamsEnd: 0.377ms
Untracked: 72.879ms
Too many instant steps in test execution mode: Error. Current task ConditionalTask.
UnityEngine.StackTraceUtility:ExtractStackTrace ()
UnityEngine.DebugLogHandler:LogFormat (UnityEngine.LogType,UnityEngine.Object,string,object[])
UnityEngine.Logger:Log (UnityEngine.LogType,object)
UnityEngine.Debug:LogError (object)
UnityEditor.TestTools.TestRunner.TestRun.TestJobRunner:ExecuteCallback () (at Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs:120)
UnityEditor.EditorApplication:Internal_CallUpdateFunctions ()

(Filename: Library/PackageCache/com.unity.test-framework@2.0.1-pre.18/UnityEditor.TestRunner/TestRun/TestJobRunner.cs Line: 120)