Modifying BuildSettings in before running tests in PlayMode.

Hi,

I'm trying to create tests for the Loading System for my game. It does a lot of additional work before and after using calls to SceneManager.LoadScene().

For my use case, I've created three empty test scenes. In order to test the system, I need to have them added to BuildSettings. I would love not to have the need of having them kept there forever though, and to have them present in BuildSettings only when the tests are running.

I wrote this piece of code to add them there, before test runs.

Code (CSharp):

1. [assembly: TestRunCallback(typeof(SetBuildSettingsTestCallbacks))]
2. namespace Core.Tests.Editor
3. {
4.     public class SetBuildSettingsTestCallbacks : ITestRunCallback
5.     {
6.         private EditorBuildSettingsScene[] _originalScenes;
7.      
8.         public void RunStarted(ITest testsToRun)
9.         {
10.             _originalScenes = EditorBuildSettings.scenes;
11.  
12.             var additionalScenes = TestSettings.Instance.AdditionalScenes;
13.          
14.             var newScenes = new EditorBuildSettingsScene[_originalScenes.Length + additionalScenes.Length];
15.  
16.             for (int i = 0; i < _originalScenes.Length; i++)
17.                 newScenes[i] = _originalScenes[i];
18.  
19.             for (int i = 0; i < additionalScenes.Length; i++)
20.                 newScenes[_originalScenes.Length + i] = new EditorBuildSettingsScene(additionalScenes[i].FullPath, true);
21.  
22.             EditorBuildSettings.scenes = newScenes;
23.          
24.             EditorApplication.playModeStateChanged += RestoreOriginalScenesOnExitPlayMode;
25.         }
26.  
27.         private void RestoreOriginalScenesOnExitPlayMode(PlayModeStateChange state)
28.         {
29.             if(state == PlayModeStateChange.ExitingPlayMode)
30.                 RestoreOriginalScenes();
31.         }
32.  
33.         public void RunFinished(ITestResult testResults)
34.         {
35.             RestoreOriginalScenes();
36.         }
37.  
38.         private void RestoreOriginalScenes()
39.         {
40.             EditorBuildSettings.scenes = _originalScenes;
41.         }
42.  
43.         public void TestStarted(ITest test)
44.         {
45.             //Intentionally left empty.
46.         }
47.  
48.         public void TestFinished(ITestResult result)
49.         {
50.             //Intentionally left empty.
51.         }
52.     }
53. }
54.  

The issue is, that even though the settings are modified correctly - and I can verify it by opening BuildSettings when the tests are running - I still get

Scene 'SceneToLoad_1' couldn't be loaded because it has not been added to the build settings or the AssetBundle has not been loaded.

error. I think it is caused by the RunStarted being invoked in PlayMode instead of before running tests. Is there a way to detect the start of the test run before it enters to PlayMode?

I've also tried to use the [SetUp] attribute but it has the same result.

I know about the

EditorSceneManager.LoadSceneInPlayMode

, for those who would propose to use it instead. I don't want to use it because that would require modifying the runtime Loading System to use editor code and that is not something that I want to do because it would mean that there is different behaviour during tests.

 

You can solve this problem by using the ITestPlayerBuildModifier.

The sample code is available below.
https://github.com/nowsprinting/Uni...Scripts/Editor/BuildAllScenesForTestPlayer.cs

 

I've actually tried it once before and it didn't work.

@nowsprinting, do I need to trigger it somehow? Because It doesn't seem to get invoked when pressing Run All from Test Runner window. I can't catch a breakpoint or get the Debug.Log to show up.

I wrote it like this:

Code (CSharp):

1. [assembly: TestPlayerBuildModifier(typeof(TestScenesBuildSettingsModifier))]
2. namespace Core.Tests.Editor
3. {
4.     public class TestScenesBuildSettingsModifier : ITestPlayerBuildModifier
5.     {
6.         public BuildPlayerOptions ModifyOptions(BuildPlayerOptions playerOptions)
7.         {
8.             Debug.Log("Modifying BuildPlayerOptions"); //Not showing up when clicking Run All button in Test Runner
9.          
10.             var buildSceneList = new List<string>(playerOptions.scenes);
11.  
12.             //This is my scriptable object with scenes I want to add, I've checked it three times if the correct scenes are added.
13.             var scenesToAdd = TestSettings.Instance.AdditionalScenes;
14.  
15.             for (int i = 0; i < scenesToAdd.Length; i++)
16.             {
17.                 if (!buildSceneList.Contains(scenesToAdd[i].FullPath))
18.                     buildSceneList.Add(scenesToAdd[i].FullPath);
19.             }
20.  
21.             playerOptions.scenes = buildSceneList.ToArray();
22.  
23.             return playerOptions;
24.         }
25.     }
26. }

 

I've just read the docs of the ITestPlayerBuildModifier and I see what the issue is. This is run when building a player for Tests.

I want this to work as well in Editor when pressed Run All.

 

All right, I've figured out how to do this.

Apparently, the thing I needed to use was IPrebuildSetup and IPostBuildCleanup. They both run once before and after starting tests via Test Runner in Editor, and adding scenes to the Build Settings during the Prebuild seems to work fine.

The only issue is - that I think what happens, haven't confirmed it - they create their own instance of the class when running the code that is separate for IPrebuildSetup and IPostBuildCleanup. So using them both in one class and caching the original build scenes setup in PreBuild and then using it in PostBulid will result in a Null Reference Exception because the variable containing the cache won't be defined.

I've managed to work around it by caching it to a singleton scriptable object and using the value from there.

 

The only problem with using IPrebuildSetup is that you have to change the "Scenes in Build" back.

I use EditorSceneManager.LoadSceneAsyncInPlayMode to load it when I run it in the Unity Editor.

Here's an example.
https://github.com/nowsprinting/Uni...xample/Tests/Runtime/Utils/TestSceneLoader.cs