Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Resolved Pre-Build Integration Tests

Discussion in 'Testing & Automation' started by Nigey, May 23, 2018.

  1. Nigey

    Nigey

    Joined:
    Sep 29, 2013
    Posts:
    1,115
    Hi Guys,

    I've been playing with the Unity test runner, and unit testing. That's working okay so far, but it doesn't fit my use-case. I want this ability:

    To be able to on-demand and as a pre-build process run a list of tests on objects in each scene. So not actually testing the code itself, but testing the implementation of those script components, and making sure they all have their variables assigned and all work correctly in an expected way. It would be nice to run those tests prior to build, and for the build itself to fail without those tests on prefabs and GameObjects working.

    Can anyone think of a way to do this? I can see how to write post-build processes, but not pre-build processes. I want to avoid ANY mistakes with clients, and I'm finding unit tests doesn't cover this hole.

    Thoughts?

    Thanks!
     
  2. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    5,091
    You can use the IPreprocessBuild interface for that.

    Code (csharp):
    1. public class RunTestsBeforeBuild : IPreprocessBuild {
    2.  
    3.     public int callbackOrder => 0;
    4.     public void OnPreprocessBuild(BuildTarget target, string path) {
    5.         foreach (string scene in scenes) { //scenes defined somewhere
    6.             EditorSceneManager.OpenScene(scene);
    7.             var testResult = RunTests();
    8.             if (testResult == failed) {
    9.                 throw new BuildFailedException("Tests in scene " + scene + " failed!");
    10.             }
    11.         }
    12.     }
    13. }
     
    hjohnsen and Nigey like this.
  3. Nigey

    Nigey

    Joined:
    Sep 29, 2013
    Posts:
    1,115
    Thank you @Baste! Perfect.
     
  4. NGC6543

    NGC6543

    Joined:
    Jun 3, 2015
    Posts:
    213
    @Baste, I'm also playing with BuildPipeline and need some of your advice.
    Currently I'm writing a script extending a ScriptableObject that controls version, build count, build date etc. I implemented both
    Code (CSharp):
    1. IPreprocessBuildWithReport
    and
    Code (CSharp):
    1. IPostprocessBuildWithReport
    .

    Currently I'm capable of building a project with a single click and it sets the output file name with version, build count, wether the built project is debug or release and build date. Once the build is complete(OnPostprocessBuild) it also copies a release note next to the built project.

    What I want to do is that on pre-process build, I want to update the current build count and current build date prior to actual build process, so that I can use that info on runtime on a built project.
    Below is the OnPreprocessBuild I implemented :
    Code (CSharp):
    1. public void OnPreprocessBuild(BuildReport report)
    2.         {
    3.             Debug.Log("[Version and Build] OnPreprocessBuild");
    4.             switch (report.summary.result)
    5.             {
    6.                 case BuildResult.Cancelled:
    7.                     Debug.LogWarning("[Version and Build] Build cancelled!");
    8.                     return;
    9.  
    10.                 case BuildResult.Failed:
    11.                     Debug.LogError("[Version and Build] Build failed!");
    12.                     return;
    13.  
    14.                 case BuildResult.Succeeded:
    15.                     Debug.Log("[Version and Build] Build succeeded!");
    16.                     break;
    17.  
    18.                 case BuildResult.Unknown:
    19.                     Debug.Log("[Version and Build] Unknown build result!");
    20.                     break;
    21.             }
    22.            
    23.             //UNDONE enabling below will cause the editor to halt.
    24.             // AssetDatabase.StartAssetEditing();
    25.            
    26.             // Instance.lastBuildDate = CurrentDateYYMMDD;
    27.  
    28.             // EditorUtility.SetDirty(Instance);
    29.             // AssetDatabase.SaveAssets();
    30.             // AssetDatabase.Refresh();
    31.         }
    As you can see, the commented lines at the end of the method causes Unity editor to halt.
    Even the debug messages above is not logged to the console.
    I also think that messing with AssetDatabase right before a build process is not a good Idea. Could you recommend another way to do it?
     
    tigerleapgorge likes this.
  5. lordofduct

    lordofduct

    Joined:
    Oct 3, 2011
    Posts:
    7,352
    Personally I'm not a big fan of the build pipeline in Unity. Especially working on our current project where I have several different build targets, and build configurations (it's episodic, so the included scenes vary from build to build).

    So I wrote mine own ScriptableObject that allows me to create build settings as a configurable asset:
    https://github.com/lordofduct/space...nityFrameworkEditor/Settings/BuildSettings.cs

    That's just the base implementation. I didn't consider having events on it to extend it... that could be easily added though.

    Regardless, how I extend it on a per project basis is just by inheriting from BuildSettings. Here's the one for my episodic release:
    Code (csharp):
    1.  
    2. using System.Collections;
    3. using System.Collections.Generic;
    4. using UnityEngine;
    5. using UnityEditor;
    6.  
    7. using com.spacepuppy;
    8. using com.spacepuppy.Collections;
    9. using com.spacepuppy.Project;
    10. using com.spacepuppyeditor;
    11.  
    12. using com.mansion;
    13.  
    14. namespace com.mansioneditor
    15. {
    16.  
    17.     [CreateAssetMenu(fileName = "EpisodeBuildSettings", menuName = "Spacepuppy/Episode Build Settings", order = int.MaxValue)]
    18.     public class EpisodeBuildSettings : BuildSettings
    19.     {
    20.         #region Fields
    21.  
    22.         [SerializeField]
    23.         private EpisodeSettings _episodeSettings;
    24.         [SerializeField]
    25.         private string _resourcesFolderName;
    26.      
    27.         #endregion
    28.  
    29.         #region Properties
    30.      
    31.         public EpisodeSettings EpisodeSettings
    32.         {
    33.             get { return _episodeSettings; }
    34.             set { _episodeSettings = value; }
    35.         }
    36.  
    37.         public string ResourcesFolderName
    38.         {
    39.             get { return _resourcesFolderName; }
    40.             set { _resourcesFolderName = value; }
    41.         }
    42.      
    43.         #endregion
    44.  
    45.     }
    46.  
    47.     [CustomEditor(typeof(EpisodeBuildSettings))]
    48.     public class EpisodeBuildSettingsEditor : BuildSettingsEditor
    49.     {
    50.  
    51.         public const string PROP_EPISODESETTINGS = "_episodeSettings";
    52.         public const string PROP_RESOURCESFOLDERNAME = "_resourcesFolderName";
    53.  
    54.         public override void DrawScenes()
    55.         {
    56.             this.DrawPropertyField(PROP_BOOTSCENE);
    57.             this.DrawPropertyField(PROP_SCENES);
    58.         }
    59.  
    60.         public override void DrawBuildOptions()
    61.         {
    62.             base.DrawBuildOptions();
    63.  
    64.             this.DrawPropertyField(PROP_EPISODESETTINGS);
    65.             this.DrawPropertyField(PROP_RESOURCESFOLDERNAME);
    66.         }
    67.  
    68.         public override string[] GetScenePaths()
    69.         {
    70.             using (var lst = TempCollection.GetList<string>())
    71.             {
    72.                 var settings = this.target as EpisodeBuildSettings;
    73.                 if (settings.BootScene != null) lst.Add(AssetDatabase.GetAssetPath(settings.BootScene));
    74.  
    75.                 foreach (var scene in settings.Scenes)
    76.                 {
    77.                     lst.Add(AssetDatabase.GetAssetPath(scene));
    78.                 }
    79.  
    80.                 if (settings.EpisodeSettings != null && settings.EpisodeSettings.TitleScreen.SceneAsset != null)
    81.                 {
    82.                     var path = AssetDatabase.GetAssetPath(settings.EpisodeSettings.TitleScreen.SceneAsset);
    83.                     if (lst.Contains(path)) lst.Remove(path);
    84.                     lst.Insert(1, path);
    85.                 }
    86.  
    87.                 return lst.ToArray();
    88.             }
    89.         }
    90.  
    91.         public override void SyncToGlobalBuild()
    92.         {
    93.             var settings = this.target as EpisodeBuildSettings;
    94.             var gameSettings = AssetDatabase.LoadAssetAtPath<Game>("Assets/Resources/GameSettings.asset");
    95.             if (gameSettings != null)
    96.             {
    97.                 Undo.RecordObject(gameSettings, "Sync Build Settings to GameSettings");
    98.                 gameSettings.SetEpisodeSettings(settings.EpisodeSettings);
    99.             }
    100.  
    101.             base.SyncToGlobalBuild();
    102.         }
    103.  
    104.  
    105.         protected override IEnumerator DoBuild(PostBuildOption postBuildOption)
    106.         {
    107.             var settings = this.target as EpisodeBuildSettings;
    108.          
    109.             if(!string.IsNullOrEmpty(settings.ResourcesFolderName))
    110.             {
    111.                 var spath = @"Assets/Build/" + settings.ResourcesFolderName;
    112.                 AssetHelper.MoveFolder(spath, @"Assets/Build/Resources");
    113.                 AssetDatabase.Refresh();
    114.             }
    115.          
    116.             var gameSettings = AssetDatabase.LoadAssetAtPath<Game>(@"Assets/Resources/GameSettings.asset");
    117.             EpisodeSettings backupEpSettings = null;
    118.             if (gameSettings != null)
    119.             {
    120.                 backupEpSettings = gameSettings.GetEpisodeSettings();
    121.                 gameSettings.SetEpisodeSettings(settings.EpisodeSettings);
    122.             }
    123.  
    124.             try
    125.             {
    126.                 this.Build(postBuildOption);
    127.             }
    128.             finally
    129.             {
    130.                 if (gameSettings != null)
    131.                 {
    132.                     gameSettings.SetEpisodeSettings(backupEpSettings);
    133.                 }
    134.  
    135.                 if (!string.IsNullOrEmpty(settings.ResourcesFolderName))
    136.                 {
    137.                     AssetHelper.MoveFolder("Assets/Build/Resources", "Assets/Build/" + settings.ResourcesFolderName);
    138.                     AssetDatabase.Refresh();
    139.                 }
    140.             }
    141.          
    142.             yield break;
    143.         }
    144.  
    145.  
    146.     }
    147.  
    148. }
    149.  
    And how it looks in the editor:
    BuildSettingsExample.png

    And if you're curious about the InputSettings, this is them:
    https://github.com/lordofduct/space...nityFrameworkEditor/Settings/InputSettings.cs

    It allows you to define different settings for the input manager on a per build basis. This way if you are targeting different platforms and need the inputs to be different on each, you set them up here, and the build settings automatically applies them before building.
     
    Last edited: Jun 5, 2018
    NGC6543 likes this.
  6. NGC6543

    NGC6543

    Joined:
    Jun 3, 2015
    Posts:
    213
    Wow, you have a grand set of works! Thanks for the info! I'll look into it!
     
  7. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    5,091
    I assume it's because you're messing with the asset database in the middle of a build. You don't really need any of that, it should be fine to just:

    Code (csharp):
    1. Instance.lastBuildDate = CurrentDateYYMMDD;
    2. EditorUtility.SetDirty(Instance);
    In addition, I think hat the reason you're getting the BuildReport as an argument to OnPreprocessBuild is so that you can write data to it - like setting the result to "Failed" due to some precondition not being met. It doesn't make any sense for the value to be set to eg. success during OnPreprocessBuild. So your switch statement can probably go away too.
     
    NGC6543 likes this.
  8. NGC6543

    NGC6543

    Joined:
    Jun 3, 2015
    Posts:
    213
    I did that because the data is stored as an asset(ScriptableObject) and I want it to be saved and included into build.

    And for those switch statement, the IPreprocessBuild and IPostprocessBuild are marked as 'obsolete' and there were new interfaces with 'Report'. I did think that receiving Report on 'Pre'processBuild was weird, but on PostprocessBuild I thought it would tell wether the build was successful or not.
    I was wondering why the Report parameter always returns the result as "Unknown" on both pre and post build. After what you've said, I have to set it to 'Succeeded'.

    Thanks for your Tip, again!
     
unityunity