Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Stripping unused Universal render pipeline shaders and textures on iOS build

Discussion in 'Universal Render Pipeline' started by lostminds, Dec 3, 2019.

  1. lostminds

    lostminds

    Joined:
    Jan 17, 2019
    Posts:
    51
    I'm building an iOS app using the Universal Render Pipeline and I'm trying to get the file size as small as possible. When taking a look at the build report I noticed that Universal Render Pipeline seems to include a bunch of resources I don't use in my project, but that are still included in the build package.

    Primarily it's a big UberPost shader 1.7 mb, Packages/com.unity.render-pipelines.universal/Shaders/PostProcessing/UberPost.shader
    Along with a bunch of 256k textures in Packages/com.unity.render-pipelines.universal/Textures/

    These all seem to be related to post processing, that I don't use. Is there any way make sure these unused (?) resources aren't included in the build?

    While it's not such a problem for the file size I also notice that a lot of resources are included from Packages/com.unity.render-pipelines.core/Runtime/Debugging/ (various UI widgets, prefabs and images) even though this isn't supposed to be a debugging build. Shouldn't these be stripped as well when you're building for release on iOS?
     
  2. lostminds

    lostminds

    Joined:
    Jan 17, 2019
    Posts:
    51
    I should add that this is in 2019.3.0f1 with the latest render pipeline package
     
  3. FardinHaque

    FardinHaque

    Joined:
    Apr 13, 2015
    Posts:
    3
    Facing the same issue. My UberPost.shader size is 4.6mb even though i am not using any post processing. Is there any way to exclude this shader from the build?
     
    charl3ss likes this.
  4. lostminds

    lostminds

    Joined:
    Jan 17, 2019
    Posts:
    51
    Still the same in 2019.3.0f3 for me, though I just get a 1.7MB UberPost.shader. For some reason they closed the bug I reported on this back in October 2019, but it doesn't seem to have been fixed so I've contacted them and asked for the bug to be re-opened.
     
  5. Brady

    Brady

    Joined:
    Sep 25, 2008
    Posts:
    2,386
    I'm still on 7.1.7 due to some incompatibility with some other assets I'm using, but my Android builds take FOREVER because it spends a ton of time compiling the Standard shader and other shaders meant for the built-in pipeline which I'm not even using anymore. Do I take it that 7.1.8 is supposed to solve this?
     
  6. lostminds

    lostminds

    Joined:
    Jan 17, 2019
    Posts:
    51
    I haven't seen anything about URP v7.1.8 solving this anywhere (I'm not sure Unity even considers this a bug any more), and as far as I can tell the problem is still the same on 2019.3.0f6 and URP v 7.1.8
     
  7. Brady

    Brady

    Joined:
    Sep 25, 2008
    Posts:
    2,386
    According to the docs here:
    https://docs.unity3d.com/Packages/c...es.universal@7.1/manual/shader-stripping.html

    It should at least perform some kind of stripping according to the settings in the URP asset. I would think, at minimum, built-in shaders for the built-in pipeline should fall under that stripping, given that they can't even work with the URP. Yet my builds spend TONS of time compiling all the built-in pipeline shaders.
     
    chrismarch likes this.
  8. kaarloew

    kaarloew

    Joined:
    Nov 1, 2018
    Posts:
    216
    Same here.

    Shader is:
    4.6 mb 1.1% Packages/com.unity.render-pipelines.universal/Shaders/PostProcessing/UberPost.shader

    Textures are:
    256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Thin02.png
    256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Thin01.png
    256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Medium06.png
    256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Medium05.png
    256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Medium04.png
    256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Medium03.png
    256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Medium02.png
    256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Medium01.png
    256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Large02.png
    256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Large01.png
     
  9. kaarloew

    kaarloew

    Joined:
    Nov 1, 2018
    Posts:
    216
    Small workaround for UberPost.shader, create new file called StripShaders.cs to Assets/Editor folder and copy paste following code to it
    Code (CSharp):
    1. using System.Collections.Generic;
    2. using UnityEditor;
    3. using UnityEditor.Build;
    4. using UnityEditor.Rendering;
    5. using UnityEngine;
    6. using UnityEngine.Rendering;
    7.  
    8. // Simple example of stripping of a debug build configuration
    9. class ShaderDebugBuildProcessor : IPreprocessShaders
    10. {
    11.     public ShaderDebugBuildProcessor()
    12.     {
    13.  
    14.     }
    15.  
    16.     // Multiple callback may be implemented.
    17.     // The first one executed is the one where callbackOrder is returning the smallest number.
    18.     public int callbackOrder { get { return 0; } }
    19.  
    20.     public void OnProcessShader(
    21.         Shader shader, ShaderSnippetData snippet, IList<ShaderCompilerData> shaderCompilerData)
    22.     {
    23.         //UnityEngine.Debug.Log($"Shader {shader.name}");
    24.         if (shader.name.Contains("UberPost"))
    25.         {
    26.             //UnityEngine.Debug.Log($"- Shader cleared");
    27.             shaderCompilerData.Clear();
    28.         }
    29.     }
    30. }
    UberPost.shader will still be included in the build, but its size will be around 0.3 kb
     
  10. Moritz5thPlanet

    Moritz5thPlanet

    Joined:
    Feb 5, 2019
    Posts:
    41
    I use a solution based on this hacky github repo:
    https://github.com/SixWays/UnityShaderStripper/wiki

    I use the Whitelist feature of the ShaderStripperVariantCollection. Works like a charm (after realizing tha there are TWO activation checkboxes, one in the stripper and one in each settings asset). You need to add some excludes manually, such as "Sprites/Diffuse" if you want a splash screen that's not pink.

    It has cut down build times by over 20 minutes.
     
  11. kaarloew

    kaarloew

    Joined:
    Nov 1, 2018
    Posts:
    216
    Another strip script, this time for textures in FilmGrain folder. So create script file called RemoveURPTextures.cs to Assets/Editor folder and copy paste following code to it
    Code (CSharp):
    1. using System.IO;
    2. using UnityEditor;
    3. using UnityEditor.Build;
    4. using UnityEditor.Build.Reporting;
    5. using UnityEngine;
    6.  
    7. public class RemoveURPTextures : IPreprocessBuildWithReport
    8. {
    9.     public int callbackOrder { get { return 0; } }
    10.     public void OnPreprocessBuild(BuildReport report)
    11.     {
    12.         Debug.Log("MyCustomBuildProcessor.OnPreprocessBuild for target " + report.summary.platform + " at path " + report.summary.outputPath);
    13.  
    14.         string packagesPath = Application.dataPath.Replace("/Assets", "") + "/Library/PackageCache";
    15.         Debug.Log($"packagesPath: {packagesPath}");
    16.         string[] directories = Directory.GetDirectories(packagesPath);
    17.  
    18.         foreach (string directory in directories)
    19.         {
    20.             Debug.Log($"directory: {directory}");
    21.             if (directory.Contains("com.unity.render-pipelines.universal"))
    22.             {
    23.                 Debug.Log($"Modifying entries in: {directory}");
    24.                 string texturesPath = Path.Combine(directory, "Textures/FilmGrain");
    25.                 string[] files = Directory.GetFiles(texturesPath, "*.meta");
    26.                 foreach (string filename in files)
    27.                 {
    28.                     Debug.Log($"Modifying entry: {filename}");
    29.                     string[] lines = File.ReadAllLines(filename);
    30.  
    31.                     for (int i = 0; i < lines.Length; i++)
    32.                     {
    33.                         lines[i] = lines[i].Replace("maxTextureSize: 2048", "maxTextureSize: 32");
    34.                     }
    35.  
    36.                     File.WriteAllLines(filename, lines);
    37.                 }
    38.             }
    39.         }
    40.         //throw new BuildFailedException("aa");
    41.     }
    42. }
    Filmgrain textures will still be in package, but their sizes will be 1.1 kb. This script reduces APK size with 1.2 MB in our project.
     
  12. grapefrukt

    grapefrukt

    Joined:
    Sep 26, 2016
    Posts:
    5
    I had this same issue and decided to take the time to combine both these techniques into one script:
    https://gist.github.com/grapefrukt/c6b90a9d427d6176f58f8b981ebc4779

    This strips any shader that has "Universal Render Pipeline" in the name, which surprisingly does not seem to break anything, at least for me.
    It also sets any textures originating from that folder to 32x32 (which also catches the SMAA/AreaTex.tga, that isn't caught by the earlier code posted here). It should be possible to set the files as hidden to keep them out altogether, but I'm not sure if that would make it more annoying to undo if you change your mind. Either way, this is good enough for my purposes.

    For me this saves 3.5mb, which isn't a ton, but seeing as it shouldn't even have been in there to begin with I think it's pretty good.
     
  13. lostminds

    lostminds

    Joined:
    Jan 17, 2019
    Posts:
    51
    Great, seems to work for me too, removes about 3MB of resources and really speeds up build time, thanks!
     
  14. Alexaroth

    Alexaroth

    Joined:
    Feb 4, 2019
    Posts:
    28
    Any dev answers to this? Still happening in latest
     
  15. phil_lira

    phil_lira

    Unity Technologies

    Joined:
    Dec 17, 2014
    Posts:
    547
    We are working on the ability to strip post-processing shaders/assets.
     
  16. Vaultboy_444

    Vaultboy_444

    Joined:
    Sep 3, 2019
    Posts:
    4
    Thank you!
    It works well.
     
  17. bigbrainz

    bigbrainz

    Joined:
    Jul 21, 2015
    Posts:
    121
    Any chance we could get an enhancement that would let us strip MOST of the shaders, but not all? We're using a couple of them, but would love to strip out the others.
     
  18. HeshamAkmal

    HeshamAkmal

    Joined:
    Oct 1, 2016
    Posts:
    5
    Use something like this in the "OnProcessShader" method

    upload_2020-6-20_0-14-41.png

    where "Blabla" is something unique in the shader name you are using, you could just use shader.name.equals("shader exact name")
     
  19. bigbrainz

    bigbrainz

    Joined:
    Jul 21, 2015
    Posts:
    121
    We'll try it!
     
    HeshamAkmal likes this.
  20. Alexaroth

    Alexaroth

    Joined:
    Feb 4, 2019
    Posts:
    28
    OnPreprocessBuild fix no longer works, seems the meta files for those textures get overwritten before they can be read and used in generating the resources

    PS: On Unity 2020 beta version
     
  21. Alexaroth

    Alexaroth

    Joined:
    Feb 4, 2019
    Posts:
    28
    Found out why, now unity doesn't allow you do modify those at all, and immediately repairs the changed files:

    "The package cache was invalidated and rebuilt because the following immutable asset(s) were unexpectedly altered"
     
  22. Brady

    Brady

    Joined:
    Sep 25, 2008
    Posts:
    2,386
    Maybe I'm missing something, but why are you modifying assets?
     
  23. Alexaroth

    Alexaroth

    Joined:
    Feb 4, 2019
    Posts:
    28
    Because for android, the APK I build contains about 4-5 mb of stuff it doesn't need, like post process shader or those assets that I never even reference. Might not be much for a desktop title but 5mb for an app is a lot
     
  24. Brady

    Brady

    Joined:
    Sep 25, 2008
    Posts:
    2,386
    That doesn't require modifying assets in any way. You just need to use the shader stripping API to tell it not to include them. Use something like the tool mentioned in this post above to do that.
     
  25. bigbrainz

    bigbrainz

    Joined:
    Jul 21, 2015
    Posts:
    121
    Hmmm . . . a nice shader stripping tool would be nice. In the tool recommended above I note the developer's warning:

    I haven't heard of anything better though and would love to hear of a tool that folks have battle-tested.
     
  26. Brady

    Brady

    Joined:
    Sep 25, 2008
    Posts:
    2,386
    I actually made a custom stripper extension for that tool that works much better at not stripping stuff you don't want. Instead of white-listing, it just lets you blacklist things you definitely know you want to strip. I'll see if I can easily post it here.
     
  27. bigbrainz

    bigbrainz

    Joined:
    Jul 21, 2015
    Posts:
    121
    That would be stellar!!
     
  28. Brady

    Brady

    Joined:
    Sep 25, 2008
    Posts:
    2,386
    @bigbrainz add this class to the UnityShaderStripper folder and then you should be able to go to Assets -> Create -> Sigtrap -> Shader Stripper Blacklist Except. That'll create the asset that lets you specify exceptions to your blacklist. You'll also want to create a Shader Stripper Simple asset, which lets you specify things to blacklist. Then go to Tools -> Sigtrap -> Shader Stripper. You may then need to click "Refresh Settings" in the window that comes up. And you should then see the newly created blacklist listed there. Then you just fill in criteria for the blacklist, then add shader variant collections to the "Blacklisted Collections" list (which I apologize is misnamed, it should be something like "Blacklist exception collections"). This makes sure that the stuff you don't want doesn't also wind up catching stuff you DO want. But be conservative with what you decide to blacklist. I hope that helps. Let me know if it doesn't compile for you. It may need a handful of modifications I made to the core tool as well, but I'm not sure.

    Code (csharp):
    1.  
    2. #if UNITY_2018_2_OR_NEWER
    3. using System.Collections;
    4. using System.Collections.Generic;
    5. using System.Collections.ObjectModel;
    6. using UnityEngine;
    7. using UnityEngine.Rendering;
    8. using System.Text;
    9. using System.Linq;
    10. #if UNITY_EDITOR
    11. using UnityEditor;
    12. using UnityEditor.Rendering;
    13. #endif
    14.  
    15. namespace Sigtrap.Editors.ShaderStripper
    16. {
    17.     /// <summary>
    18.     /// Strips ALL shaders and variants except those in the supplied ShaderVariantCollection assets.
    19.     /// Does not strip built-in shaders.
    20.     /// </summary>
    21.     [CreateAssetMenu(menuName = "Sigtrap/Shader Stripper Blacklist Except")]
    22.     public class ShaderStripper_BlacklistExcept : ShaderStripperBase, ISerializationCallbackReceiver
    23.     {
    24.         [SerializeField]
    25.         [Tooltip("Set a path like Assets/.../<name> (no extension) to merge whitelisted collections into a new collection asset.\nPath to a whitelisted collection (to overwrite) IS allowed.")]
    26.         string _mergeToFile = null;
    27.         [SerializeField]
    28.         [Tooltip("The shaders in these collections will be stripped except for the specific variants listed in the collections.")]
    29.         List<ShaderVariantCollection> _blacklistedCollections;
    30.         [SerializeField]
    31.         [HideInInspector]
    32.         List<string> _collectionPaths;
    33.         [SerializeField]
    34.         [Tooltip("Strip Hidden shaders. Be careful - shaders in Resources might get stripped.\nHidden shaders in collections will always have their variants stripped.")]
    35.         bool _stripHidden = false;
    36.         [SerializeField]
    37.         [Tooltip("Allow VR versions of variants in collection even when VR keywords not in collection.")]
    38.         bool _allowVrVariants;
    39.         [SerializeField]
    40.         [Tooltip("Allow GPU instanced versions of variants in collection even when instancing keywords not in collection.")]
    41.         bool _allowInstancedVariants;
    42.  
    43.         bool _valid = false;
    44.         bool _dirty = false;
    45.  
    46.         #region Serialization
    47.         // Automagically pick up new collections which have overwritten existing ones
    48.         public void OnAfterDeserialize()
    49.         {
    50.             _dirty = true;
    51.         }
    52.         public void OnBeforeSerialize()
    53.         {
    54.             _dirty = true;
    55.         }
    56.         #endregion
    57.  
    58. #if UNITY_EDITOR
    59.         static readonly string[] VR_KEYWORDS = new string[]{
    60.             "UNITY_SINGLE_PASS_STEREO", "STEREO_INSTANCING_ON", "STEREO_MULTIVIEW_ON"
    61.         };
    62.         static readonly string[] INSTANCING_KEYWORDS = new string[]{
    63.             "INSTANCING_ON"
    64.         };
    65.         static List<string> _tempExcludes = new List<string>();
    66.         Dictionary<Shader, Dictionary<PassType, List<ShaderVariantCollection.ShaderVariant>>> _variantsByShader = new Dictionary<Shader, Dictionary<PassType, List<ShaderVariantCollection.ShaderVariant>>>();
    67.  
    68.         void ReplaceOverwrittenCollections()
    69.         {
    70.             if (_collectionPaths != null)
    71.             {
    72.                 for (int i = 0; i < _blacklistedCollections.Count; ++i)
    73.                 {
    74.                     if (_blacklistedCollections[i] == null)
    75.                     {
    76.                         _blacklistedCollections[i] = AssetDatabase.LoadAssetAtPath<ShaderVariantCollection>(_collectionPaths[i]);
    77.                     }
    78.                 }
    79.             }
    80.             else
    81.             {
    82.                 _collectionPaths = new List<string>();
    83.             }
    84.             _collectionPaths.Clear();
    85.             foreach (var c in _blacklistedCollections)
    86.             {
    87.                 if (c != null)
    88.                 {
    89.                     _collectionPaths.Add(AssetDatabase.GetAssetPath(c));
    90.                 }
    91.             }
    92.         }
    93.  
    94.         #region Parse YAML - thanks Unity for not having a simple ShaderVariantCollection.GetVariants or something
    95.         static List<string> _tempCompareShaderVariants = new List<string>();
    96.         bool ShaderVariantsEqual(ShaderVariantCollection.ShaderVariant a, ShaderVariantCollection.ShaderVariant b)
    97.         {
    98.             if (a.shader != b.shader || a.passType != b.passType) return false;
    99.             if ((a.keywords == null) != (b.keywords == null)) return false;
    100.             if (a.keywords == null && b.keywords == null) return true;
    101.             if (a.keywords.Length != b.keywords.Length) return false;
    102.             _tempCompareShaderVariants.Clear();
    103.             _tempCompareShaderVariants.AddRange(a.keywords);
    104.             for (int i = 0; i < b.keywords.Length; ++i)
    105.             {
    106.                 if (!_tempCompareShaderVariants.Contains(b.keywords[i]))
    107.                 {
    108.                     return false;
    109.                 }
    110.             }
    111.             return true;
    112.         }
    113.         public override void Initialize()
    114.         {
    115.             ReplaceOverwrittenCollections();
    116.  
    117.             _tempExcludes.Clear();
    118.             if (_allowVrVariants)
    119.             {
    120.                 _tempExcludes.AddRange(VR_KEYWORDS);
    121.             }
    122.             if (_allowInstancedVariants)
    123.             {
    124.                 _tempExcludes.AddRange(INSTANCING_KEYWORDS);
    125.             }
    126.  
    127.             foreach (var c in _blacklistedCollections)
    128.             {
    129.                 // Load asset YAML
    130.                 var file = new List<string>(System.IO.File.ReadAllLines(
    131.                     (Application.dataPath + AssetDatabase.GetAssetPath(c)).Replace("AssetsAssets", "Assets")
    132.                 ));
    133.  
    134.                 #region Pre-process to get rid of mid-list line breaks
    135.                 var yaml = new List<string>();
    136.  
    137.                 // Find shaders list
    138.                 int i = 0;
    139.                 for (; i < file.Count; ++i)
    140.                 {
    141.                     if (YamlLineHasKey(file[i], "m_Shaders")) break;
    142.                 }
    143.  
    144.                 // Process and fill
    145.                 int indent = 0;
    146.                 for (; i < file.Count; ++i)
    147.                 {
    148.                     string f = file[i];
    149.                     int myIndent = GetYamlIndent(f);
    150.                     if (myIndent > indent)
    151.                     {
    152.                         // If no "<key>: ", continuation of previous line
    153.                         if (!f.EndsWith(":") && !f.Contains(": "))
    154.                         {
    155.                             yaml[yaml.Count - 1] += " " + f.Trim();
    156.                             continue;
    157.                         }
    158.                     }
    159.  
    160.                     yaml.Add(f);
    161.                     indent = myIndent;
    162.                 }
    163.                 #endregion
    164.  
    165.                 #region Iterate over shaders
    166.                 for (i = 0; i < yaml.Count; ++i)
    167.                 {
    168.                     string y = yaml[i];
    169.                     if (yaml[i].Contains("first:"))
    170.                     {
    171.                         string guid = GetValueFromYaml(y, "guid");
    172.                         Shader s = AssetDatabase.LoadAssetAtPath<Shader>(AssetDatabase.GUIDToAssetPath(guid));
    173.  
    174.                         // Move to variants contents (skip current line, "second:" and "variants:")
    175.                         i += 3;
    176.  
    177.                         if (i >= yaml.Count)
    178.                             break;
    179.  
    180.                         indent = GetYamlIndent(yaml[i]);
    181.                         var sv = new ShaderVariantCollection.ShaderVariant();
    182.                         for (; i < yaml.Count; ++i)
    183.                         {
    184.                             y = yaml[i];
    185.  
    186.                             // If indent changes, variants have ended
    187.                             if (GetYamlIndent(y) != indent)
    188.                             {
    189.                                 // Outer loop will increment, so counteract
    190.                                 i -= 1;
    191.                                 break;
    192.                             }
    193.  
    194.                             if (IsYamlLineNewEntry(y))
    195.                             {
    196.                                 // First entry will be a new entry but no variant info present yet, so skip
    197.                                 // Builtin shaders will also be null
    198.                                 if (sv.shader != null)
    199.                                 {
    200.                                     Dictionary<PassType, List<ShaderVariantCollection.ShaderVariant>> variantsByPass = null;
    201.                                     if (!_variantsByShader.TryGetValue(sv.shader, out variantsByPass))
    202.                                     {
    203.                                         variantsByPass = new Dictionary<PassType, List<ShaderVariantCollection.ShaderVariant>>();
    204.                                         _variantsByShader.Add(sv.shader, variantsByPass);
    205.                                     }
    206.                                     List<ShaderVariantCollection.ShaderVariant> variants = null;
    207.                                     if (!variantsByPass.TryGetValue(sv.passType, out variants))
    208.                                     {
    209.                                         variants = new List<ShaderVariantCollection.ShaderVariant>();
    210.                                         variantsByPass.Add(sv.passType, variants);
    211.                                     }
    212.                                     bool dupe = false;
    213.                                     foreach (var existing in variants)
    214.                                     {
    215.                                         if (ShaderVariantsEqual(existing, sv))
    216.                                         {
    217.                                             dupe = true;
    218.                                             break;
    219.                                         }
    220.                                     }
    221.                                     if (!dupe)
    222.                                     {
    223.                                         variants.Add(sv);
    224.                                     }
    225.                                 }
    226.                                 sv = new ShaderVariantCollection.ShaderVariant();
    227.                                 sv.shader = s;
    228.                             }
    229.  
    230.                             // Get contents
    231.                             if (YamlLineHasKey(y, "passType"))
    232.                             {
    233.                                 sv.passType = (PassType)int.Parse(GetValueFromYaml(y, "passType"));
    234.                             }
    235.                             if (YamlLineHasKey(y, "keywords"))
    236.                             {
    237.                                 sv.keywords = GetValuesFromYaml(y, "keywords", _tempExcludes);
    238.                             }
    239.                         }
    240.                         // Get final variant
    241.                         if (sv.shader != null)
    242.                         {
    243.                             Dictionary<PassType, List<ShaderVariantCollection.ShaderVariant>> variantsByPass = null;
    244.                             if (!_variantsByShader.TryGetValue(sv.shader, out variantsByPass))
    245.                             {
    246.                                 variantsByPass = new Dictionary<PassType, List<ShaderVariantCollection.ShaderVariant>>();
    247.                                 _variantsByShader.Add(sv.shader, variantsByPass);
    248.                             }
    249.                             List<ShaderVariantCollection.ShaderVariant> variants = null;
    250.                             if (!variantsByPass.TryGetValue(sv.passType, out variants))
    251.                             {
    252.                                 variants = new List<ShaderVariantCollection.ShaderVariant>();
    253.                                 variantsByPass.Add(sv.passType, variants);
    254.                             }
    255.                             bool dupe = false;
    256.                             foreach (var existing in variants)
    257.                             {
    258.                                 if (ShaderVariantsEqual(existing, sv))
    259.                                 {
    260.                                     dupe = true;
    261.                                     break;
    262.                                 }
    263.                             }
    264.                             if (!dupe)
    265.                             {
    266.                                 variants.Add(sv);
    267.                             }
    268.                         }
    269.                     }
    270.                 }
    271.                 #endregion
    272.  
    273.                 LogMessage(this, "Parsing ShaderVariantCollection " + c.name);
    274.                 // Loop over shaders
    275.                 foreach (var s in _variantsByShader)
    276.                 {
    277.                     string log = "Shader: " + s.Key.name;
    278.                     // Loop over passes
    279.                     foreach (var p in s.Value)
    280.                     {
    281.                         log += string.Format("\n   Pass: ({1:00}){0}", p.Key, (int)p.Key);
    282.                         // Loop over variants
    283.                         for (int v = 0; v < p.Value.Count; ++v)
    284.                         {
    285.                             log += string.Format("\n      Variant [{0}]:\t", v);
    286.                             // Loop over keywords
    287.                             var ks = p.Value[v].keywords;
    288.                             if (ks != null && ks.Length != 0)
    289.                             {
    290.                                 bool first = true;
    291.                                 foreach (var k in ks)
    292.                                 {
    293.                                     if (!first) log += ", ";
    294.                                     log += k;
    295.                                     first = false;
    296.                                 }
    297.                             }
    298.                             else
    299.                             {
    300.                                 log += "<no keywords>";
    301.                             }
    302.                         }
    303.                     }
    304.                     LogMessage(this, log);
    305.                 }
    306.             }
    307.  
    308.             // Merge collections
    309.             if (!string.IsNullOrEmpty(_mergeToFile) && _blacklistedCollections.Count > 1)
    310.             {
    311.                 var svc = new ShaderVariantCollection();
    312.                 foreach (var a in _variantsByShader)
    313.                 {
    314.                     foreach (var b in a.Value)
    315.                     {
    316.                         foreach (var s in b.Value)
    317.                         {
    318.                             svc.Add(s);
    319.                         }
    320.                     }
    321.                 }
    322.                 try
    323.                 {
    324.                     string file = _mergeToFile + ".shadervariants";
    325.                     string log = string.Format("Merged following ShaderVariantCollections into {0}:\n", file);
    326.                     foreach (var s in _blacklistedCollections)
    327.                     {
    328.                         log += "    " + s.name + "\n";
    329.                     }
    330.  
    331.                     if (AssetDatabase.LoadAssetAtPath<ShaderVariantCollection>(file) != null)
    332.                     {
    333.                         AssetDatabase.DeleteAsset(file);
    334.                     }
    335.                     AssetDatabase.SaveAssets();
    336.                     AssetDatabase.Refresh();
    337.                     AssetDatabase.CreateAsset(svc, file);
    338.                     AssetDatabase.SaveAssets();
    339.                     AssetDatabase.Refresh();
    340.  
    341.                     Debug.Log(log, AssetDatabase.LoadAssetAtPath<ShaderVariantCollection>(file));
    342.                 }
    343.                 catch (System.Exception ex)
    344.                 {
    345.                     Debug.LogError("Error merging ShaderVariantCollections. Exception follows.");
    346.                     throw;
    347.                 }
    348.             }
    349.  
    350.             _valid = (_variantsByShader != null && _variantsByShader.Count > 0);
    351.         }
    352.         int GetYamlIndent(string line)
    353.         {
    354.             for (int i = 0; i < line.Length; ++i)
    355.             {
    356.                 if (line[i] != ' ' && line[i] != '-') return i;
    357.             }
    358.             return 0;
    359.         }
    360.         bool IsYamlLineNewEntry(string line)
    361.         {
    362.             foreach (var c in line)
    363.             {
    364.                 // If a dash (before a not-space appears) this is a new entry
    365.                 if (c == '-') return true;
    366.                 // If not a dash, must be a space or indent has ended
    367.                 if (c != ' ') return false;
    368.             }
    369.             return false;
    370.         }
    371.         int GetIndexOfYamlValue(string line, string key)
    372.         {
    373.             int i = line.IndexOf(key + ":", System.StringComparison.Ordinal);
    374.             if (i >= 0)
    375.             {
    376.                 // Skip to value
    377.                 i += key.Length + 2;
    378.             }
    379.             return i;
    380.         }
    381.         bool YamlLineHasKey(string line, string key)
    382.         {
    383.             return GetIndexOfYamlValue(line, key) >= 0;
    384.         }
    385.         string GetValueFromYaml(string line, string key)
    386.         {
    387.             int i = GetIndexOfYamlValue(line, key);
    388.             if (i < 0)
    389.             {
    390.                 return "";
    391.                 //throw new System.Exception((string.Format("Value not found for key {0} in YAML line {1}", key, line)));
    392.             }
    393.             StringBuilder sb = new StringBuilder();
    394.             for (; i < line.Length; ++i)
    395.             {
    396.                 char c = line[i];
    397.                 if (c == ',' || c == ' ') break;
    398.                 sb.Append(c);
    399.             }
    400.             return sb.ToString();
    401.         }
    402.         string[] GetValuesFromYaml(string line, string key, List<string> exclude = null)
    403.         {
    404.             int i = GetIndexOfYamlValue(line, key);
    405.             if (i < 0)
    406.             {
    407.                 throw new System.Exception((string.Format("Value not found for key {0} in YAML line {1}", key, line)));
    408.             }
    409.             List<string> result = new List<string>();
    410.             StringBuilder sb = new StringBuilder();
    411.             for (; i < line.Length; ++i)
    412.             {
    413.                 char c = line[i];
    414.                 bool end = false;
    415.                 bool brk = false;
    416.                 if (c == ',')
    417.                 {
    418.                     // Comma delimits keys
    419.                     // Add the current entry and stop parsing
    420.                     end = brk = true;
    421.                 }
    422.                 if (c == ' ')
    423.                 {
    424.                     // Space delimits entries
    425.                     // Add current entry, move to next
    426.                     end = true;
    427.                 }
    428.                 if (end)
    429.                 {
    430.                     result.Add(sb.ToString());
    431.                     sb.Length = 0;
    432.                     if (brk) break;
    433.                 }
    434.                 else
    435.                 {
    436.                     sb.Append(c);
    437.                 }
    438.             }
    439.             // Catch last entry if line ends
    440.             if (sb.Length > 0)
    441.             {
    442.                 var s = sb.ToString();
    443.                 if (exclude == null || exclude.Count == 0 || !exclude.Contains(s))
    444.                 {
    445.                     result.Add(sb.ToString());
    446.                 }
    447.             }
    448.             return result.ToArray();
    449.         }
    450.         #endregion
    451.  
    452.         static List<string> _tempRequestedKeywordsToMatch = new List<string>();
    453.         static List<string> _tempRequestedKeywordsToMatchCached = new List<string>();
    454.         static List<string> _tempCollectedKeywordsSorted = new List<string>();
    455.         protected override bool StripCustom(Shader shader, ShaderSnippetData passData, IList<ShaderCompilerData> variantData)
    456.         {
    457.             // Don't strip anything if no collections present
    458.             if (!_valid) return true;
    459.             // Always ignore built-in shaders
    460.             if (string.IsNullOrEmpty(AssetDatabase.GetAssetPath(shader))) return true;
    461.  
    462.             // Try to match shader
    463.             Dictionary<PassType, List<ShaderVariantCollection.ShaderVariant>> collectedVariantsByPass = null;
    464.             if (_variantsByShader.TryGetValue(shader, out collectedVariantsByPass))
    465.             {
    466.                 // Try to match pass
    467.                 List<ShaderVariantCollection.ShaderVariant> collectedPassVariants = null;
    468.                 if (collectedVariantsByPass.TryGetValue(passData.passType, out collectedPassVariants))
    469.                 {
    470.                     // Loop over supplied variants
    471.                     // Iterate backwards over supplied variants to allow index-based removal
    472.                     int count = variantData.Count;
    473.                     for (int i = count - 1; i >= 0; --i)
    474.                     {
    475.  
    476.                         // Fill temp buffer to fill OTHER temp buffer each time SIGH
    477.                         _tempRequestedKeywordsToMatchCached.Clear();
    478.                         var sks = variantData[i].shaderKeywordSet.GetShaderKeywords();
    479.                         foreach (var sk in sks)
    480.                         {
    481.                             string n = GetKeywordName(sk);
    482.                             bool add = true;
    483.                             // Don't look for VR or instanced variants
    484.                             if (_tempExcludes.Count > 0)
    485.                             {
    486.                                 if (_tempExcludes.Contains(n))
    487.                                 {
    488.                                     add = false;
    489.                                 }
    490.                             }
    491.                             if (add)
    492.                             {
    493.                                 _tempRequestedKeywordsToMatchCached.Add(n);
    494.                             }
    495.                         }
    496.                         bool variantMatched = false;
    497.  
    498.                         // Loop over cached variants
    499.                         foreach (var collectedVariant in collectedPassVariants)
    500.                         {
    501.  
    502.                             // Must match ALL keywords
    503.                             _tempRequestedKeywordsToMatch.Clear();
    504.                             _tempRequestedKeywordsToMatch.AddRange(_tempRequestedKeywordsToMatchCached);
    505.  
    506.                             // Early out (match) if both have no keywords
    507.                             if (_tempRequestedKeywordsToMatch.Count == 0 && (collectedVariant.keywords == null || collectedVariant.keywords.Length == 0))
    508.                             {
    509.                                 variantMatched = true;
    510.                                 break;
    511.                             }
    512.  
    513.                             // No match
    514.                             if (collectedVariant.keywords == null)
    515.                                 continue;
    516.  
    517.                             // Early out (no match) if keyword counts don't match
    518.                             if (_tempRequestedKeywordsToMatch.Count != collectedVariant.keywords.Length)
    519.                                 continue;
    520.  
    521.                             // Check all keywords
    522.                             _tempCollectedKeywordsSorted.Clear();
    523.                             if (collectedVariant.keywords != null)
    524.                                 _tempCollectedKeywordsSorted.AddRange(collectedVariant.keywords);
    525.                             _tempCollectedKeywordsSorted.Sort((a, b) => { return string.CompareOrdinal(a, b); });
    526.                             foreach (var k in _tempCollectedKeywordsSorted)
    527.                             {
    528.                                 bool isVROrInstancingKeyword = false;
    529.  
    530.                                 if (
    531.                                         (
    532.                                             _allowVrVariants
    533.                                             &&
    534.                                             (
    535.                                                 string.Compare(k, VR_KEYWORDS[0]) == 0 ||
    536.                                                 string.Compare(k, VR_KEYWORDS[1]) == 0 ||
    537.                                                 string.Compare(k, VR_KEYWORDS[2]) == 0
    538.                                             )
    539.                                         )
    540.                                         ||
    541.                                         (
    542.                                             _allowInstancedVariants
    543.                                             &&
    544.                                             (
    545.                                                 string.Compare(k, INSTANCING_KEYWORDS[0]) == 0
    546.                                             )
    547.                                         )
    548.                                     )
    549.                                     isVROrInstancingKeyword = true;
    550.  
    551.                                 bool keywordMatched = _tempRequestedKeywordsToMatch.Remove(k);
    552.  
    553.                                 if (!keywordMatched && !isVROrInstancingKeyword) break;
    554.                             }
    555.  
    556.                             // Make sure any desired VR or instanced keywords are removed:
    557.                             if (_allowVrVariants)
    558.                             {
    559.                                 for (int j = 0; j < VR_KEYWORDS.Length; ++j)
    560.                                     _tempRequestedKeywordsToMatch.Remove(VR_KEYWORDS[j]);
    561.                             }
    562.                             if (_allowInstancedVariants)
    563.                             {
    564.                                 for (int j = 0; j < INSTANCING_KEYWORDS.Length; ++j)
    565.                                     _tempRequestedKeywordsToMatch.Remove(INSTANCING_KEYWORDS[j]);
    566.                             }
    567.  
    568.                             // If all keywords removed, all keywords matched
    569.                             if (_tempRequestedKeywordsToMatch.Count == 0)
    570.                             {
    571.                                 variantMatched = true;
    572.                                 break;
    573.                             }
    574.                         }
    575.  
    576.                         // Strip this variant
    577.                         if (!variantMatched)
    578.                         {
    579.                             LogRemoval(this, shader, passData, i, count, variantData[i]);
    580.                             variantData.RemoveAt(i);
    581.                         }
    582.                     }
    583.                 }
    584.                 else
    585.                 {
    586.                     // If not matched pass, this is not blacklisted:
    587.                     return true;
    588.                 }
    589.             }
    590.             else
    591.             {
    592.                 // If not matched shader, this is not blacklisted:
    593.                 return true;
    594.             }
    595.  
    596.             return true;
    597.         }
    598.  
    599.         public override string description { get { return "Strips ALL (non-built-in) shaders not in selected ShaderVariantCollection assets."; } }
    600.         public override string help
    601.         {
    602.             get
    603.             {
    604.                 string result = _stripHidden ? "WILL strip Hidden shaders." : "Will NOT strip Hidden shaders.";
    605.                 result += " Will NOT strip built-in shaders. Use other strippers to remove these.";
    606.                 return result;
    607.             }
    608.         }
    609.  
    610.         protected override bool _checkShader { get { return false; } }
    611.         protected override bool _checkPass { get { return false; } }
    612.         protected override bool _checkVariants { get { return false; } }
    613.  
    614.         public override void OnGUI()
    615.         {
    616.             if (_dirty)
    617.             {
    618.                 ReplaceOverwrittenCollections();
    619.             }
    620.             _dirty = false;
    621.         }
    622. #endif
    623.     }
    624. }
    625. #endif
    626.  
     
    Last edited: Jul 4, 2020
    andreiagmu likes this.
  29. bigbrainz

    bigbrainz

    Joined:
    Jul 21, 2015
    Posts:
    121
    Sweet--we'll check it out. Thank you! (Not sure why I didn't get a notification email about your post, so sorry for the delay).
     
  30. bigbrainz

    bigbrainz

    Joined:
    Jul 21, 2015
    Posts:
    121
    OK, setup seems to go pretty well and basic functionality is there. However, it's kicking out a bit of an error. And if I add or drag a Variant Collection it doesn't actually add it.

    And just to confirm, the "Blacklisted Collections" should be a Shader Variant Collection that I want to KEEP, right?

    ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
    Parameter name: index
    System.ThrowHelper.ThrowArgumentOutOfRangeException (System.ExceptionArgument argument, System.ExceptionResource resource) (at <fb001e01371b4adca20013e0ac763896>:0)
    System.ThrowHelper.ThrowArgumentOutOfRangeException () (at <fb001e01371b4adca20013e0ac763896>:0)
    System.Collections.Generic.List`1[T].get_Item (System.Int32 index) (at <fb001e01371b4adca20013e0ac763896>:0)
    Sigtrap.Editors.ShaderStripper.ShaderStripper_BlacklistExcept.ReplaceOverwrittenCollections () (at Assets/3rd Party/ShaderStripper/Sigtrap.cs:76)
    Sigtrap.Editors.ShaderStripper.ShaderStripper_BlacklistExcept.OnGUI () (at Assets/3rd Party/ShaderStripper/Sigtrap.cs:618)
    Sigtrap.Editors.ShaderStripper.ShaderStripperEditor.OnGUI () (at Assets/3rd Party/ShaderStripper/Editor/ShaderStripperEditor.cs:173)
    System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at <fb001e01371b4adca20013e0ac763896>:0)
    Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation.
    System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at <fb001e01371b4adca20013e0ac763896>:0)
    System.Reflection.MethodBase.Invoke (System.Object obj, System.Object[] parameters) (at <fb001e01371b4adca20013e0ac763896>:0)
    UnityEditor.HostView.Invoke (System.String methodName, System.Object obj) (at <bccb16a88ec4456dbf07978c418f407b>:0)
    UnityEditor.HostView.Invoke (System.String methodName) (at <bccb16a88ec4456dbf07978c418f407b>:0)
    UnityEditor.HostView.InvokeOnGUI (UnityEngine.Rect onGUIPosition, UnityEngine.Rect viewRect) (at <bccb16a88ec4456dbf07978c418f407b>:0)
    UnityEditor.DockArea.DrawView (UnityEngine.Rect viewRect, UnityEngine.Rect dockAreaRect) (at <bccb16a88ec4456dbf07978c418f407b>:0)
    UnityEditor.DockArea.OldOnGUI () (at <bccb16a88ec4456dbf07978c418f407b>:0)
    UnityEngine.UIElements.IMGUIContainer.DoOnGUI (UnityEngine.Event evt, UnityEngine.Matrix4x4 parentTransform, UnityEngine.Rect clippingRect, System.Boolean isComputingLayout, UnityEngine.Rect layoutSize, System.Action onGUIHandler, System.Boolean canAffectFocus) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
    UnityEngine.UIElements.IMGUIContainer.HandleIMGUIEvent (UnityEngine.Event e, UnityEngine.Matrix4x4 worldTransform, UnityEngine.Rect clippingRect, System.Action onGUIHandler, System.Boolean canAffectFocus) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
    UnityEngine.UIElements.IMGUIContainer.HandleIMGUIEvent (UnityEngine.Event e, System.Action onGUIHandler, System.Boolean canAffectFocus) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
    UnityEngine.UIElements.IMGUIContainer.HandleIMGUIEvent (UnityEngine.Event e, System.Boolean canAffectFocus) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
    UnityEngine.UIElements.IMGUIContainer.SendEventToIMGUI (UnityEngine.UIElements.EventBase evt, System.Boolean canAffectFocus) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
    UnityEngine.UIElements.IMGUIContainer.HandleEvent (UnityEngine.UIElements.EventBase evt) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
    UnityEngine.UIElements.EventDispatchUtilities.PropagateEvent (UnityEngine.UIElements.EventBase evt) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
    UnityEngine.UIElements.MouseEventDispatchingStrategy.DispatchEvent (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel panel) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
    UnityEngine.UIElements.EventDispatcher.ApplyDispatchingStrategies (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel panel, System.Boolean imguiEventIsInitiallyUsed) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
    UnityEngine.UIElements.EventDispatcher.ProcessEvent (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel panel) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
    UnityEngine.UIElements.EventDispatcher.Dispatch (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel panel, UnityEngine.UIElements.DispatchMode dispatchMode) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
    UnityEngine.UIElements.BaseVisualElementPanel.SendEvent (UnityEngine.UIElements.EventBase e, UnityEngine.UIElements.DispatchMode dispatchMode) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
    UnityEngine.UIElements.UIElementsUtility.DoDispatch (UnityEngine.UIElements.BaseVisualElementPanel panel) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
    UnityEngine.UIElements.UIElementsUtility.ProcessEvent (System.Int32 instanceID, System.IntPtr nativeEventPtr) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
    UnityEngine.GUIUtility.ProcessEvent (System.Int32 instanceID, System.IntPtr
     
  31. bigbrainz

    bigbrainz

    Joined:
    Jul 21, 2015
    Posts:
    121
    As for the textures, since the previous fix no longer works, I found them and manually resized them to tiny.
     
  32. Brady

    Brady

    Joined:
    Sep 25, 2008
    Posts:
    2,386
    Yes, it is unfortunately labeled "Blacklist collections" in this particular case, as it is actually the ones you want to exempt from blacklisting. The desired combined effect of the two lists (both the "Simple" and "Except" objects) is to define a set of shaders to blacklist by name, and then have a list of specific shader variants to exempt from this.

    There is a good chance that the out-of-bounds error you're experiencing is due to one of the various bugs I discovered in the original Shader Stripper package. I've come across perhaps three different ones over the course of the last six months, and fixed each one as I went, and haven't kept detailed logs of those modifications. But if you DM me, I could send you my entire modified version of Shader Stripper that you can use as-is that should work.
     
    andreiagmu likes this.
  33. chrismarch

    chrismarch

    Joined:
    Jul 24, 2013
    Posts:
    409
    andreiagmu likes this.
  34. drallcom3

    drallcom3

    Joined:
    Feb 12, 2017
    Posts:
    79
    Is there any way to remove/reduce those Filmgrain textures?
    Modifying them causes Unity to rebuild them. Even setting them to read-only doesn't work.
     
    KamilDA likes this.
  35. KamilDA

    KamilDA

    Joined:
    May 21, 2020
    Posts:
    39
    Submitted a bug report to get a quick-fix until the post-processing stripping feature is added to URP:
    https://issuetracker.unity3d.com/is...e-unused-asset-files-that-can-not-be-stripped

    You can vote for it to get traction.
     
    andreiagmu and chrismarch like this.
  36. andreiagmu

    andreiagmu

    Joined:
    Feb 20, 2014
    Posts:
    36
  37. KamilDA

    KamilDA

    Joined:
    May 21, 2020
    Posts:
    39
  38. KamilDA

    KamilDA

    Joined:
    May 21, 2020
    Posts:
    39
    chrismarch and andreiagmu like this.
  39. chrismarch

    chrismarch

    Joined:
    Jul 24, 2013
    Posts:
    409
unityunity