Stripping unused Universal render pipeline shaders and textures on iOS build

I'm building an iOS app using the Universal Render Pipeline and I'm trying to get the file size as small as possible. When taking a look at the build report I noticed that Universal Render Pipeline seems to include a bunch of resources I don't use in my project, but that are still included in the build package.

Primarily it's a big UberPost shader 1.7 mb, Packages/com.unity.render-pipelines.universal/Shaders/PostProcessing/UberPost.shader
Along with a bunch of 256k textures in Packages/com.unity.render-pipelines.universal/Textures/

These all seem to be related to post processing, that I don't use. Is there any way make sure these unused (?) resources aren't included in the build?

While it's not such a problem for the file size I also notice that a lot of resources are included from Packages/com.unity.render-pipelines.core/Runtime/Debugging/ (various UI widgets, prefabs and images) even though this isn't supposed to be a debugging build. Shouldn't these be stripped as well when you're building for release on iOS?

 

I should add that this is in 2019.3.0f1 with the latest render pipeline package

 

Facing the same issue. My UberPost.shader size is 4.6mb even though i am not using any post processing. Is there any way to exclude this shader from the build?

 

Still the same in 2019.3.0f3 for me, though I just get a 1.7MB UberPost.shader. For some reason they closed the bug I reported on this back in October 2019, but it doesn't seem to have been fixed so I've contacted them and asked for the bug to be re-opened.

 

I'm still on 7.1.7 due to some incompatibility with some other assets I'm using, but my Android builds take FOREVER because it spends a ton of time compiling the Standard shader and other shaders meant for the built-in pipeline which I'm not even using anymore. Do I take it that 7.1.8 is supposed to solve this?

 

I haven't seen anything about URP v7.1.8 solving this anywhere (I'm not sure Unity even considers this a bug any more), and as far as I can tell the problem is still the same on 2019.3.0f6 and URP v 7.1.8

 

According to the docs here:
https://docs.unity3d.com/Packages/c...es.universal@7.1/manual/shader-stripping.html

It should at least perform some kind of stripping according to the settings in the URP asset. I would think, at minimum, built-in shaders for the built-in pipeline should fall under that stripping, given that they can't even work with the URP. Yet my builds spend TONS of time compiling all the built-in pipeline shaders.

 

Same here.

Shader is:
4.6 mb 1.1% Packages/com.unity.render-pipelines.universal/Shaders/PostProcessing/UberPost.shader

Textures are:
256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Thin02.png
256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Thin01.png
256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Medium06.png
256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Medium05.png
256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Medium04.png
256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Medium03.png
256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Medium02.png
256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Medium01.png
256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Large02.png
256.1 kb 0.1% Packages/com.unity.render-pipelines.universal/Textures/FilmGrain/Large01.png

 

Small workaround for UberPost.shader, create new file called StripShaders.cs to Assets/Editor folder and copy paste following code to it

Code (CSharp):

1. using System.Collections.Generic;
2. using UnityEditor;
3. using UnityEditor.Build;
4. using UnityEditor.Rendering;
5. using UnityEngine;
6. using UnityEngine.Rendering;
7.  
8. // Simple example of stripping of a debug build configuration
9. class ShaderDebugBuildProcessor : IPreprocessShaders
10. {
11.     public ShaderDebugBuildProcessor()
12.     {
13.  
14.     }
15.  
16.     // Multiple callback may be implemented.
17.     // The first one executed is the one where callbackOrder is returning the smallest number.
18.     public int callbackOrder { get { return 0; } }
19.  
20.     public void OnProcessShader(
21.         Shader shader, ShaderSnippetData snippet, IList<ShaderCompilerData> shaderCompilerData)
22.     {
23.         //UnityEngine.Debug.Log($"Shader {shader.name}");
24.         if (shader.name.Contains("UberPost"))
25.         {
26.             //UnityEngine.Debug.Log($"- Shader cleared");
27.             shaderCompilerData.Clear();
28.         }
29.     }
30. }

UberPost.shader will still be included in the build, but its size will be around 0.3 kb

 

I use a solution based on this hacky github repo:
https://github.com/SixWays/UnityShaderStripper/wiki

I use the Whitelist feature of the ShaderStripperVariantCollection. Works like a charm (after realizing tha there are TWO activation checkboxes, one in the stripper and one in each settings asset). You need to add some excludes manually, such as "Sprites/Diffuse" if you want a splash screen that's not pink.

It has cut down build times by over 20 minutes.

 

Another strip script, this time for textures in FilmGrain folder. So create script file called RemoveURPTextures.cs to Assets/Editor folder and copy paste following code to it

Code (CSharp):

1. using System.IO;
2. using UnityEditor;
3. using UnityEditor.Build;
4. using UnityEditor.Build.Reporting;
5. using UnityEngine;
6.  
7. public class RemoveURPTextures : IPreprocessBuildWithReport
8. {
9.     public int callbackOrder { get { return 0; } }
10.     public void OnPreprocessBuild(BuildReport report)
11.     {
12.         Debug.Log("MyCustomBuildProcessor.OnPreprocessBuild for target " + report.summary.platform + " at path " + report.summary.outputPath);
13.  
14.         string packagesPath = Application.dataPath.Replace("/Assets", "") + "/Library/PackageCache";
15.         Debug.Log($"packagesPath: {packagesPath}");
16.         string[] directories = Directory.GetDirectories(packagesPath);
17.  
18.         foreach (string directory in directories)
19.         {
20.             Debug.Log($"directory: {directory}");
21.             if (directory.Contains("com.unity.render-pipelines.universal"))
22.             {
23.                 Debug.Log($"Modifying entries in: {directory}");
24.                 string texturesPath = Path.Combine(directory, "Textures/FilmGrain");
25.                 string[] files = Directory.GetFiles(texturesPath, "*.meta");
26.                 foreach (string filename in files)
27.                 {
28.                     Debug.Log($"Modifying entry: {filename}");
29.                     string[] lines = File.ReadAllLines(filename);
30.  
31.                     for (int i = 0; i < lines.Length; i++)
32.                     {
33.                         lines[i] = lines[i].Replace("maxTextureSize: 2048", "maxTextureSize: 32");
34.                     }
35.  
36.                     File.WriteAllLines(filename, lines);
37.                 }
38.             }
39.         }
40.         //throw new BuildFailedException("aa");
41.     }
42. }

Filmgrain textures will still be in package, but their sizes will be 1.1 kb. This script reduces APK size with 1.2 MB in our project.

EDIT:
This script does NOT work with Unity 2020 or newer.

 

I had this same issue and decided to take the time to combine both these techniques into one script:
https://gist.github.com/grapefrukt/c6b90a9d427d6176f58f8b981ebc4779

This strips any shader that has "Universal Render Pipeline" in the name, which surprisingly does not seem to break anything, at least for me.
It also sets any textures originating from that folder to 32x32 (which also catches the SMAA/AreaTex.tga, that isn't caught by the earlier code posted here). It should be possible to set the files as hidden to keep them out altogether, but I'm not sure if that would make it more annoying to undo if you change your mind. Either way, this is good enough for my purposes.

For me this saves 3.5mb, which isn't a ton, but seeing as it shouldn't even have been in there to begin with I think it's pretty good.

 

Great, seems to work for me too, removes about 3MB of resources and really speeds up build time, thanks!

 

Any dev answers to this? Still happening in latest

 

Thank you!
It works well.

 

Any chance we could get an enhancement that would let us strip MOST of the shaders, but not all? We're using a couple of them, but would love to strip out the others.

 

Use something like this in the "OnProcessShader" method



where "Blabla" is something unique in the shader name you are using, you could just use shader.name.equals("shader exact name")

 

We'll try it!

 

OnPreprocessBuild fix no longer works, seems the meta files for those textures get overwritten before they can be read and used in generating the resources

PS: On Unity 2020 beta version

 

Found out why, now unity doesn't allow you do modify those at all, and immediately repairs the changed files:

"The package cache was invalidated and rebuilt because the following immutable asset(s) were unexpectedly altered"

 

Maybe I'm missing something, but why are you modifying assets?

 

Because for android, the APK I build contains about 4-5 mb of stuff it doesn't need, like post process shader or those assets that I never even reference. Might not be much for a desktop title but 5mb for an app is a lot

 

That doesn't require modifying assets in any way. You just need to use the shader stripping API to tell it not to include them. Use something like the tool mentioned in this post above to do that.

 

Hmmm . . . a nice shader stripping tool would be nice. In the tool recommended above I note the developer's warning:

I haven't heard of anything better though and would love to hear of a tool that folks have battle-tested.

 

I actually made a custom stripper extension for that tool that works much better at not stripping stuff you don't want. Instead of white-listing, it just lets you blacklist things you definitely know you want to strip. I'll see if I can easily post it here.

 

That would be stellar!!

 

@bigbrainz add this class to the UnityShaderStripper folder and then you should be able to go to Assets -> Create -> Sigtrap -> Shader Stripper Blacklist Except. That'll create the asset that lets you specify exceptions to your blacklist. You'll also want to create a Shader Stripper Simple asset, which lets you specify things to blacklist. Then go to Tools -> Sigtrap -> Shader Stripper. You may then need to click "Refresh Settings" in the window that comes up. And you should then see the newly created blacklist listed there. Then you just fill in criteria for the blacklist, then add shader variant collections to the "Blacklisted Collections" list (which I apologize is misnamed, it should be something like "Blacklist exception collections"). This makes sure that the stuff you don't want doesn't also wind up catching stuff you DO want. But be conservative with what you decide to blacklist. I hope that helps. Let me know if it doesn't compile for you. It may need a handful of modifications I made to the core tool as well, but I'm not sure.

Code (csharp):

1.  
2. #if UNITY_2018_2_OR_NEWER
3. using System.Collections;
4. using System.Collections.Generic;
5. using System.Collections.ObjectModel;
6. using UnityEngine;
7. using UnityEngine.Rendering;
8. using System.Text;
9. using System.Linq;
10. #if UNITY_EDITOR
11. using UnityEditor;
12. using UnityEditor.Rendering;
13. #endif
14.  
15. namespace Sigtrap.Editors.ShaderStripper
16. {
17.     /// <summary>
18.     /// Strips ALL shaders and variants except those in the supplied ShaderVariantCollection assets.
19.     /// Does not strip built-in shaders.
20.     /// </summary>
21.     [CreateAssetMenu(menuName = "Sigtrap/Shader Stripper Blacklist Except")]
22.     public class ShaderStripper_BlacklistExcept : ShaderStripperBase, ISerializationCallbackReceiver
23.     {
24.         [SerializeField]
25.         [Tooltip("Set a path like Assets/.../<name> (no extension) to merge whitelisted collections into a new collection asset.\nPath to a whitelisted collection (to overwrite) IS allowed.")]
26.         string _mergeToFile = null;
27.         [SerializeField]
28.         [Tooltip("The shaders in these collections will be stripped except for the specific variants listed in the collections.")]
29.         List<ShaderVariantCollection> _blacklistedCollections;
30.         [SerializeField]
31.         [HideInInspector]
32.         List<string> _collectionPaths;
33.         [SerializeField]
34.         [Tooltip("Strip Hidden shaders. Be careful - shaders in Resources might get stripped.\nHidden shaders in collections will always have their variants stripped.")]
35.         bool _stripHidden = false;
36.         [SerializeField]
37.         [Tooltip("Allow VR versions of variants in collection even when VR keywords not in collection.")]
38.         bool _allowVrVariants;
39.         [SerializeField]
40.         [Tooltip("Allow GPU instanced versions of variants in collection even when instancing keywords not in collection.")]
41.         bool _allowInstancedVariants;
42.  
43.         bool _valid = false;
44.         bool _dirty = false;
45.  
46.         #region Serialization
47.         // Automagically pick up new collections which have overwritten existing ones
48.         public void OnAfterDeserialize()
49.         {
50.             _dirty = true;
51.         }
52.         public void OnBeforeSerialize()
53.         {
54.             _dirty = true;
55.         }
56.         #endregion
57.  
58. #if UNITY_EDITOR
59.         static readonly string[] VR_KEYWORDS = new string[]{
60.             "UNITY_SINGLE_PASS_STEREO", "STEREO_INSTANCING_ON", "STEREO_MULTIVIEW_ON"
61.         };
62.         static readonly string[] INSTANCING_KEYWORDS = new string[]{
63.             "INSTANCING_ON"
64.         };
65.         static List<string> _tempExcludes = new List<string>();
66.         Dictionary<Shader, Dictionary<PassType, List<ShaderVariantCollection.ShaderVariant>>> _variantsByShader = new Dictionary<Shader, Dictionary<PassType, List<ShaderVariantCollection.ShaderVariant>>>();
67.  
68.         void ReplaceOverwrittenCollections()
69.         {
70.             if (_collectionPaths != null)
71.             {
72.                 for (int i = 0; i < _blacklistedCollections.Count; ++i)
73.                 {
74.                     if (_blacklistedCollections[i] == null)
75.                     {
76.                         _blacklistedCollections[i] = AssetDatabase.LoadAssetAtPath<ShaderVariantCollection>(_collectionPaths[i]);
77.                     }
78.                 }
79.             }
80.             else
81.             {
82.                 _collectionPaths = new List<string>();
83.             }
84.             _collectionPaths.Clear();
85.             foreach (var c in _blacklistedCollections)
86.             {
87.                 if (c != null)
88.                 {
89.                     _collectionPaths.Add(AssetDatabase.GetAssetPath(c));
90.                 }
91.             }
92.         }
93.  
94.         #region Parse YAML - thanks Unity for not having a simple ShaderVariantCollection.GetVariants or something
95.         static List<string> _tempCompareShaderVariants = new List<string>();
96.         bool ShaderVariantsEqual(ShaderVariantCollection.ShaderVariant a, ShaderVariantCollection.ShaderVariant b)
97.         {
98.             if (a.shader != b.shader || a.passType != b.passType) return false;
99.             if ((a.keywords == null) != (b.keywords == null)) return false;
100.             if (a.keywords == null && b.keywords == null) return true;
101.             if (a.keywords.Length != b.keywords.Length) return false;
102.             _tempCompareShaderVariants.Clear();
103.             _tempCompareShaderVariants.AddRange(a.keywords);
104.             for (int i = 0; i < b.keywords.Length; ++i)
105.             {
106.                 if (!_tempCompareShaderVariants.Contains(b.keywords[i]))
107.                 {
108.                     return false;
109.                 }
110.             }
111.             return true;
112.         }
113.         public override void Initialize()
114.         {
115.             ReplaceOverwrittenCollections();
116.  
117.             _tempExcludes.Clear();
118.             if (_allowVrVariants)
119.             {
120.                 _tempExcludes.AddRange(VR_KEYWORDS);
121.             }
122.             if (_allowInstancedVariants)
123.             {
124.                 _tempExcludes.AddRange(INSTANCING_KEYWORDS);
125.             }
126.  
127.             foreach (var c in _blacklistedCollections)
128.             {
129.                 // Load asset YAML
130.                 var file = new List<string>(System.IO.File.ReadAllLines(
131.                     (Application.dataPath + AssetDatabase.GetAssetPath(c)).Replace("AssetsAssets", "Assets")
132.                 ));
133.  
134.                 #region Pre-process to get rid of mid-list line breaks
135.                 var yaml = new List<string>();
136.  
137.                 // Find shaders list
138.                 int i = 0;
139.                 for (; i < file.Count; ++i)
140.                 {
141.                     if (YamlLineHasKey(file[i], "m_Shaders")) break;
142.                 }
143.  
144.                 // Process and fill
145.                 int indent = 0;
146.                 for (; i < file.Count; ++i)
147.                 {
148.                     string f = file[i];
149.                     int myIndent = GetYamlIndent(f);
150.                     if (myIndent > indent)
151.                     {
152.                         // If no "<key>: ", continuation of previous line
153.                         if (!f.EndsWith(":") && !f.Contains(": "))
154.                         {
155.                             yaml[yaml.Count - 1] += " " + f.Trim();
156.                             continue;
157.                         }
158.                     }
159.  
160.                     yaml.Add(f);
161.                     indent = myIndent;
162.                 }
163.                 #endregion
164.  
165.                 #region Iterate over shaders
166.                 for (i = 0; i < yaml.Count; ++i)
167.                 {
168.                     string y = yaml[i];
169.                     if (yaml[i].Contains("first:"))
170.                     {
171.                         string guid = GetValueFromYaml(y, "guid");
172.                         Shader s = AssetDatabase.LoadAssetAtPath<Shader>(AssetDatabase.GUIDToAssetPath(guid));
173.  
174.                         // Move to variants contents (skip current line, "second:" and "variants:")
175.                         i += 3;
176.  
177.                         if (i >= yaml.Count)
178.                             break;
179.  
180.                         indent = GetYamlIndent(yaml[i]);
181.                         var sv = new ShaderVariantCollection.ShaderVariant();
182.                         for (; i < yaml.Count; ++i)
183.                         {
184.                             y = yaml[i];
185.  
186.                             // If indent changes, variants have ended
187.                             if (GetYamlIndent(y) != indent)
188.                             {
189.                                 // Outer loop will increment, so counteract
190.                                 i -= 1;
191.                                 break;
192.                             }
193.  
194.                             if (IsYamlLineNewEntry(y))
195.                             {
196.                                 // First entry will be a new entry but no variant info present yet, so skip
197.                                 // Builtin shaders will also be null
198.                                 if (sv.shader != null)
199.                                 {
200.                                     Dictionary<PassType, List<ShaderVariantCollection.ShaderVariant>> variantsByPass = null;
201.                                     if (!_variantsByShader.TryGetValue(sv.shader, out variantsByPass))
202.                                     {
203.                                         variantsByPass = new Dictionary<PassType, List<ShaderVariantCollection.ShaderVariant>>();
204.                                         _variantsByShader.Add(sv.shader, variantsByPass);
205.                                     }
206.                                     List<ShaderVariantCollection.ShaderVariant> variants = null;
207.                                     if (!variantsByPass.TryGetValue(sv.passType, out variants))
208.                                     {
209.                                         variants = new List<ShaderVariantCollection.ShaderVariant>();
210.                                         variantsByPass.Add(sv.passType, variants);
211.                                     }
212.                                     bool dupe = false;
213.                                     foreach (var existing in variants)
214.                                     {
215.                                         if (ShaderVariantsEqual(existing, sv))
216.                                         {
217.                                             dupe = true;
218.                                             break;
219.                                         }
220.                                     }
221.                                     if (!dupe)
222.                                     {
223.                                         variants.Add(sv);
224.                                     }
225.                                 }
226.                                 sv = new ShaderVariantCollection.ShaderVariant();
227.                                 sv.shader = s;
228.                             }
229.  
230.                             // Get contents
231.                             if (YamlLineHasKey(y, "passType"))
232.                             {
233.                                 sv.passType = (PassType)int.Parse(GetValueFromYaml(y, "passType"));
234.                             }
235.                             if (YamlLineHasKey(y, "keywords"))
236.                             {
237.                                 sv.keywords = GetValuesFromYaml(y, "keywords", _tempExcludes);
238.                             }
239.                         }
240.                         // Get final variant
241.                         if (sv.shader != null)
242.                         {
243.                             Dictionary<PassType, List<ShaderVariantCollection.ShaderVariant>> variantsByPass = null;
244.                             if (!_variantsByShader.TryGetValue(sv.shader, out variantsByPass))
245.                             {
246.                                 variantsByPass = new Dictionary<PassType, List<ShaderVariantCollection.ShaderVariant>>();
247.                                 _variantsByShader.Add(sv.shader, variantsByPass);
248.                             }
249.                             List<ShaderVariantCollection.ShaderVariant> variants = null;
250.                             if (!variantsByPass.TryGetValue(sv.passType, out variants))
251.                             {
252.                                 variants = new List<ShaderVariantCollection.ShaderVariant>();
253.                                 variantsByPass.Add(sv.passType, variants);
254.                             }
255.                             bool dupe = false;
256.                             foreach (var existing in variants)
257.                             {
258.                                 if (ShaderVariantsEqual(existing, sv))
259.                                 {
260.                                     dupe = true;
261.                                     break;
262.                                 }
263.                             }
264.                             if (!dupe)
265.                             {
266.                                 variants.Add(sv);
267.                             }
268.                         }
269.                     }
270.                 }
271.                 #endregion
272.  
273.                 LogMessage(this, "Parsing ShaderVariantCollection " + c.name);
274.                 // Loop over shaders
275.                 foreach (var s in _variantsByShader)
276.                 {
277.                     string log = "Shader: " + s.Key.name;
278.                     // Loop over passes
279.                     foreach (var p in s.Value)
280.                     {
281.                         log += string.Format("\n   Pass: ({1:00}){0}", p.Key, (int)p.Key);
282.                         // Loop over variants
283.                         for (int v = 0; v < p.Value.Count; ++v)
284.                         {
285.                             log += string.Format("\n      Variant [{0}]:\t", v);
286.                             // Loop over keywords
287.                             var ks = p.Value[v].keywords;
288.                             if (ks != null && ks.Length != 0)
289.                             {
290.                                 bool first = true;
291.                                 foreach (var k in ks)
292.                                 {
293.                                     if (!first) log += ", ";
294.                                     log += k;
295.                                     first = false;
296.                                 }
297.                             }
298.                             else
299.                             {
300.                                 log += "<no keywords>";
301.                             }
302.                         }
303.                     }
304.                     LogMessage(this, log);
305.                 }
306.             }
307.  
308.             // Merge collections
309.             if (!string.IsNullOrEmpty(_mergeToFile) && _blacklistedCollections.Count > 1)
310.             {
311.                 var svc = new ShaderVariantCollection();
312.                 foreach (var a in _variantsByShader)
313.                 {
314.                     foreach (var b in a.Value)
315.                     {
316.                         foreach (var s in b.Value)
317.                         {
318.                             svc.Add(s);
319.                         }
320.                     }
321.                 }
322.                 try
323.                 {
324.                     string file = _mergeToFile + ".shadervariants";
325.                     string log = string.Format("Merged following ShaderVariantCollections into {0}:\n", file);
326.                     foreach (var s in _blacklistedCollections)
327.                     {
328.                         log += "    " + s.name + "\n";
329.                     }
330.  
331.                     if (AssetDatabase.LoadAssetAtPath<ShaderVariantCollection>(file) != null)
332.                     {
333.                         AssetDatabase.DeleteAsset(file);
334.                     }
335.                     AssetDatabase.SaveAssets();
336.                     AssetDatabase.Refresh();
337.                     AssetDatabase.CreateAsset(svc, file);
338.                     AssetDatabase.SaveAssets();
339.                     AssetDatabase.Refresh();
340.  
341.                     Debug.Log(log, AssetDatabase.LoadAssetAtPath<ShaderVariantCollection>(file));
342.                 }
343.                 catch (System.Exception ex)
344.                 {
345.                     Debug.LogError("Error merging ShaderVariantCollections. Exception follows.");
346.                     throw;
347.                 }
348.             }
349.  
350.             _valid = (_variantsByShader != null && _variantsByShader.Count > 0);
351.         }
352.         int GetYamlIndent(string line)
353.         {
354.             for (int i = 0; i < line.Length; ++i)
355.             {
356.                 if (line[i] != ' ' && line[i] != '-') return i;
357.             }
358.             return 0;
359.         }
360.         bool IsYamlLineNewEntry(string line)
361.         {
362.             foreach (var c in line)
363.             {
364.                 // If a dash (before a not-space appears) this is a new entry
365.                 if (c == '-') return true;
366.                 // If not a dash, must be a space or indent has ended
367.                 if (c != ' ') return false;
368.             }
369.             return false;
370.         }
371.         int GetIndexOfYamlValue(string line, string key)
372.         {
373.             int i = line.IndexOf(key + ":", System.StringComparison.Ordinal);
374.             if (i >= 0)
375.             {
376.                 // Skip to value
377.                 i += key.Length + 2;
378.             }
379.             return i;
380.         }
381.         bool YamlLineHasKey(string line, string key)
382.         {
383.             return GetIndexOfYamlValue(line, key) >= 0;
384.         }
385.         string GetValueFromYaml(string line, string key)
386.         {
387.             int i = GetIndexOfYamlValue(line, key);
388.             if (i < 0)
389.             {
390.                 return "";
391.                 //throw new System.Exception((string.Format("Value not found for key {0} in YAML line {1}", key, line)));
392.             }
393.             StringBuilder sb = new StringBuilder();
394.             for (; i < line.Length; ++i)
395.             {
396.                 char c = line[i];
397.                 if (c == ',' || c == ' ') break;
398.                 sb.Append(c);
399.             }
400.             return sb.ToString();
401.         }
402.         string[] GetValuesFromYaml(string line, string key, List<string> exclude = null)
403.         {
404.             int i = GetIndexOfYamlValue(line, key);
405.             if (i < 0)
406.             {
407.                 throw new System.Exception((string.Format("Value not found for key {0} in YAML line {1}", key, line)));
408.             }
409.             List<string> result = new List<string>();
410.             StringBuilder sb = new StringBuilder();
411.             for (; i < line.Length; ++i)
412.             {
413.                 char c = line[i];
414.                 bool end = false;
415.                 bool brk = false;
416.                 if (c == ',')
417.                 {
418.                     // Comma delimits keys
419.                     // Add the current entry and stop parsing
420.                     end = brk = true;
421.                 }
422.                 if (c == ' ')
423.                 {
424.                     // Space delimits entries
425.                     // Add current entry, move to next
426.                     end = true;
427.                 }
428.                 if (end)
429.                 {
430.                     result.Add(sb.ToString());
431.                     sb.Length = 0;
432.                     if (brk) break;
433.                 }
434.                 else
435.                 {
436.                     sb.Append(c);
437.                 }
438.             }
439.             // Catch last entry if line ends
440.             if (sb.Length > 0)
441.             {
442.                 var s = sb.ToString();
443.                 if (exclude == null || exclude.Count == 0 || !exclude.Contains(s))
444.                 {
445.                     result.Add(sb.ToString());
446.                 }
447.             }
448.             return result.ToArray();
449.         }
450.         #endregion
451.  
452.         static List<string> _tempRequestedKeywordsToMatch = new List<string>();
453.         static List<string> _tempRequestedKeywordsToMatchCached = new List<string>();
454.         static List<string> _tempCollectedKeywordsSorted = new List<string>();
455.         protected override bool StripCustom(Shader shader, ShaderSnippetData passData, IList<ShaderCompilerData> variantData)
456.         {
457.             // Don't strip anything if no collections present
458.             if (!_valid) return true;
459.             // Always ignore built-in shaders
460.             if (string.IsNullOrEmpty(AssetDatabase.GetAssetPath(shader))) return true;
461.  
462.             // Try to match shader
463.             Dictionary<PassType, List<ShaderVariantCollection.ShaderVariant>> collectedVariantsByPass = null;
464.             if (_variantsByShader.TryGetValue(shader, out collectedVariantsByPass))
465.             {
466.                 // Try to match pass
467.                 List<ShaderVariantCollection.ShaderVariant> collectedPassVariants = null;
468.                 if (collectedVariantsByPass.TryGetValue(passData.passType, out collectedPassVariants))
469.                 {
470.                     // Loop over supplied variants
471.                     // Iterate backwards over supplied variants to allow index-based removal
472.                     int count = variantData.Count;
473.                     for (int i = count - 1; i >= 0; --i)
474.                     {
475.  
476.                         // Fill temp buffer to fill OTHER temp buffer each time SIGH
477.                         _tempRequestedKeywordsToMatchCached.Clear();
478.                         var sks = variantData[i].shaderKeywordSet.GetShaderKeywords();
479.                         foreach (var sk in sks)
480.                         {
481.                             string n = GetKeywordName(sk);
482.                             bool add = true;
483.                             // Don't look for VR or instanced variants
484.                             if (_tempExcludes.Count > 0)
485.                             {
486.                                 if (_tempExcludes.Contains(n))
487.                                 {
488.                                     add = false;
489.                                 }
490.                             }
491.                             if (add)
492.                             {
493.                                 _tempRequestedKeywordsToMatchCached.Add(n);
494.                             }
495.                         }
496.                         bool variantMatched = false;
497.  
498.                         // Loop over cached variants
499.                         foreach (var collectedVariant in collectedPassVariants)
500.                         {
501.  
502.                             // Must match ALL keywords
503.                             _tempRequestedKeywordsToMatch.Clear();
504.                             _tempRequestedKeywordsToMatch.AddRange(_tempRequestedKeywordsToMatchCached);
505.  
506.                             // Early out (match) if both have no keywords
507.                             if (_tempRequestedKeywordsToMatch.Count == 0 && (collectedVariant.keywords == null || collectedVariant.keywords.Length == 0))
508.                             {
509.                                 variantMatched = true;
510.                                 break;
511.                             }
512.  
513.                             // No match
514.                             if (collectedVariant.keywords == null)
515.                                 continue;
516.  
517.                             // Early out (no match) if keyword counts don't match
518.                             if (_tempRequestedKeywordsToMatch.Count != collectedVariant.keywords.Length)
519.                                 continue;
520.  
521.                             // Check all keywords
522.                             _tempCollectedKeywordsSorted.Clear();
523.                             if (collectedVariant.keywords != null)
524.                                 _tempCollectedKeywordsSorted.AddRange(collectedVariant.keywords);
525.                             _tempCollectedKeywordsSorted.Sort((a, b) => { return string.CompareOrdinal(a, b); });
526.                             foreach (var k in _tempCollectedKeywordsSorted)
527.                             {
528.                                 bool isVROrInstancingKeyword = false;
529.  
530.                                 if (
531.                                         (
532.                                             _allowVrVariants
533.                                             &&
534.                                             (
535.                                                 string.Compare(k, VR_KEYWORDS[0]) == 0 ||
536.                                                 string.Compare(k, VR_KEYWORDS[1]) == 0 ||
537.                                                 string.Compare(k, VR_KEYWORDS[2]) == 0
538.                                             )
539.                                         )
540.                                         ||
541.                                         (
542.                                             _allowInstancedVariants
543.                                             &&
544.                                             (
545.                                                 string.Compare(k, INSTANCING_KEYWORDS[0]) == 0
546.                                             )
547.                                         )
548.                                     )
549.                                     isVROrInstancingKeyword = true;
550.  
551.                                 bool keywordMatched = _tempRequestedKeywordsToMatch.Remove(k);
552.  
553.                                 if (!keywordMatched && !isVROrInstancingKeyword) break;
554.                             }
555.  
556.                             // Make sure any desired VR or instanced keywords are removed:
557.                             if (_allowVrVariants)
558.                             {
559.                                 for (int j = 0; j < VR_KEYWORDS.Length; ++j)
560.                                     _tempRequestedKeywordsToMatch.Remove(VR_KEYWORDS[j]);
561.                             }
562.                             if (_allowInstancedVariants)
563.                             {
564.                                 for (int j = 0; j < INSTANCING_KEYWORDS.Length; ++j)
565.                                     _tempRequestedKeywordsToMatch.Remove(INSTANCING_KEYWORDS[j]);
566.                             }
567.  
568.                             // If all keywords removed, all keywords matched
569.                             if (_tempRequestedKeywordsToMatch.Count == 0)
570.                             {
571.                                 variantMatched = true;
572.                                 break;
573.                             }
574.                         }
575.  
576.                         // Strip this variant
577.                         if (!variantMatched)
578.                         {
579.                             LogRemoval(this, shader, passData, i, count, variantData[i]);
580.                             variantData.RemoveAt(i);
581.                         }
582.                     }
583.                 }
584.                 else
585.                 {
586.                     // If not matched pass, this is not blacklisted:
587.                     return true;
588.                 }
589.             }
590.             else
591.             {
592.                 // If not matched shader, this is not blacklisted:
593.                 return true;
594.             }
595.  
596.             return true;
597.         }
598.  
599.         public override string description { get { return "Strips ALL (non-built-in) shaders not in selected ShaderVariantCollection assets."; } }
600.         public override string help
601.         {
602.             get
603.             {
604.                 string result = _stripHidden ? "WILL strip Hidden shaders." : "Will NOT strip Hidden shaders.";
605.                 result += " Will NOT strip built-in shaders. Use other strippers to remove these.";
606.                 return result;
607.             }
608.         }
609.  
610.         protected override bool _checkShader { get { return false; } }
611.         protected override bool _checkPass { get { return false; } }
612.         protected override bool _checkVariants { get { return false; } }
613.  
614.         public override void OnGUI()
615.         {
616.             if (_dirty)
617.             {
618.                 ReplaceOverwrittenCollections();
619.             }
620.             _dirty = false;
621.         }
622. #endif
623.     }
624. }
625. #endif
626.  

 

Sweet--we'll check it out. Thank you! (Not sure why I didn't get a notification email about your post, so sorry for the delay).

 

OK, setup seems to go pretty well and basic functionality is there. However, it's kicking out a bit of an error. And if I add or drag a Variant Collection it doesn't actually add it.

And just to confirm, the "Blacklisted Collections" should be a Shader Variant Collection that I want to KEEP, right?

Spoiler: Error Message

ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
System.ThrowHelper.ThrowArgumentOutOfRangeException (System.ExceptionArgument argument, System.ExceptionResource resource) (at <fb001e01371b4adca20013e0ac763896>:0)
System.ThrowHelper.ThrowArgumentOutOfRangeException () (at <fb001e01371b4adca20013e0ac763896>:0)
System.Collections.Generic.List`1[T].get_Item (System.Int32 index) (at <fb001e01371b4adca20013e0ac763896>:0)
Sigtrap.Editors.ShaderStripper.ShaderStripper_BlacklistExcept.ReplaceOverwrittenCollections () (at Assets/3rd Party/ShaderStripper/Sigtrap.cs:76)
Sigtrap.Editors.ShaderStripper.ShaderStripper_BlacklistExcept.OnGUI () (at Assets/3rd Party/ShaderStripper/Sigtrap.cs:618)
Sigtrap.Editors.ShaderStripper.ShaderStripperEditor.OnGUI () (at Assets/3rd Party/ShaderStripper/Editor/ShaderStripperEditor.cs:173)
System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at <fb001e01371b4adca20013e0ac763896>:0)
Rethrow as TargetInvocationException: Exception has been thrown by the target of an invocation.
System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) (at <fb001e01371b4adca20013e0ac763896>:0)
System.Reflection.MethodBase.Invoke (System.Object obj, System.Object[] parameters) (at <fb001e01371b4adca20013e0ac763896>:0)
UnityEditor.HostView.Invoke (System.String methodName, System.Object obj) (at <bccb16a88ec4456dbf07978c418f407b>:0)
UnityEditor.HostView.Invoke (System.String methodName) (at <bccb16a88ec4456dbf07978c418f407b>:0)
UnityEditor.HostView.InvokeOnGUI (UnityEngine.Rect onGUIPosition, UnityEngine.Rect viewRect) (at <bccb16a88ec4456dbf07978c418f407b>:0)
UnityEditor.DockArea.DrawView (UnityEngine.Rect viewRect, UnityEngine.Rect dockAreaRect) (at <bccb16a88ec4456dbf07978c418f407b>:0)
UnityEditor.DockArea.OldOnGUI () (at <bccb16a88ec4456dbf07978c418f407b>:0)
UnityEngine.UIElements.IMGUIContainer.DoOnGUI (UnityEngine.Event evt, UnityEngine.Matrix4x4 parentTransform, UnityEngine.Rect clippingRect, System.Boolean isComputingLayout, UnityEngine.Rect layoutSize, System.Action onGUIHandler, System.Boolean canAffectFocus) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
UnityEngine.UIElements.IMGUIContainer.HandleIMGUIEvent (UnityEngine.Event e, UnityEngine.Matrix4x4 worldTransform, UnityEngine.Rect clippingRect, System.Action onGUIHandler, System.Boolean canAffectFocus) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
UnityEngine.UIElements.IMGUIContainer.HandleIMGUIEvent (UnityEngine.Event e, System.Action onGUIHandler, System.Boolean canAffectFocus) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
UnityEngine.UIElements.IMGUIContainer.HandleIMGUIEvent (UnityEngine.Event e, System.Boolean canAffectFocus) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
UnityEngine.UIElements.IMGUIContainer.SendEventToIMGUI (UnityEngine.UIElements.EventBase evt, System.Boolean canAffectFocus) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
UnityEngine.UIElements.IMGUIContainer.HandleEvent (UnityEngine.UIElements.EventBase evt) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
UnityEngine.UIElements.EventDispatchUtilities.PropagateEvent (UnityEngine.UIElements.EventBase evt) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
UnityEngine.UIElements.MouseEventDispatchingStrategy.DispatchEvent (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel panel) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
UnityEngine.UIElements.EventDispatcher.ApplyDispatchingStrategies (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel panel, System.Boolean imguiEventIsInitiallyUsed) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
UnityEngine.UIElements.EventDispatcher.ProcessEvent (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel panel) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
UnityEngine.UIElements.EventDispatcher.Dispatch (UnityEngine.UIElements.EventBase evt, UnityEngine.UIElements.IPanel panel, UnityEngine.UIElements.DispatchMode dispatchMode) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
UnityEngine.UIElements.BaseVisualElementPanel.SendEvent (UnityEngine.UIElements.EventBase e, UnityEngine.UIElements.DispatchMode dispatchMode) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
UnityEngine.UIElements.UIElementsUtility.DoDispatch (UnityEngine.UIElements.BaseVisualElementPanel panel) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
UnityEngine.UIElements.UIElementsUtility.ProcessEvent (System.Int32 instanceID, System.IntPtr nativeEventPtr) (at <78b213560771414e9fa6f1b95f5ad8bb>:0)
UnityEngine.GUIUtility.ProcessEvent (System.Int32 instanceID, System.IntPtr

 

As for the textures, since the previous fix no longer works, I found them and manually resized them to tiny.

 

Yes, it is unfortunately labeled "Blacklist collections" in this particular case, as it is actually the ones you want to exempt from blacklisting. The desired combined effect of the two lists (both the "Simple" and "Except" objects) is to define a set of shaders to blacklist by name, and then have a list of specific shader variants to exempt from this.

There is a good chance that the out-of-bounds error you're experiencing is due to one of the various bugs I discovered in the original Shader Stripper package. I've come across perhaps three different ones over the course of the last six months, and fixed each one as I went, and haven't kept detailed logs of those modifications. But if you DM me, I could send you my entire modified version of Shader Stripper that you can use as-is that should work.

 

You can vote to prioritize shader stripping work in URP, at the link below. This might be the work @phil_lira was referring to.
https://portal.productboard.com/unity/1-unity-graphics/c/12-improved-shader-stripping

 

Is there any way to remove/reduce those Filmgrain textures?
Modifying them causes Unity to rebuild them. Even setting them to read-only doesn't work.

 

Submitted a bug report to get a quick-fix until the post-processing stripping feature is added to URP:
https://issuetracker.unity3d.com/is...e-unused-asset-files-that-can-not-be-stripped

You can vote for it to get traction.

 

There's a new comment in that Unity Graphics link:
https://portal.productboard.com/uni.../12?utm_medium=social&utm_source=portal_share

"Improvements for shader stripping has landed and is available in beta through 2020.2b and URP v10"

Did anyone tested it yet?

 

Using URP v10 Preview 26 with 2020.2b, no dice. Still includes all PP assets and shaders in build even if PP is disabled.

 

Hey all, seems like a fix is in the works:
https://github.com/Unity-Technologies/Graphics/pull/1873

 

while waiting for the fix in progress, I have posted a temporary fix for getting the post processing textures out of memory, if the asset post processor script no longer works: https://forum.unity.com/threads/how...p-textures-out-of-memory.984945/#post-6402901

 

Fix got merged in the master branch for URP:
https://github.com/Unity-Technologies/Graphics/pull/1873#commits-pushed-05a4ff9

It's coming soon.

 

Has anyone tested the fix yet?

Edit: I found a fix that works for me, by setting to strip all shaders, then i manually added the ones in my project to the always include list. Build size went down 80mb (50% of my build. Not sure how to reduce unitys dll? I've only got a few of the inbuilt modules enabled)

 

Tested in 2021.1.0b3. Appears to work, stripped 10-15% of my build size.

 

Honestly. reading this just hurts. we have 2021! this engine shouldn't cause such issues. I feel like working with a kindergarden-tool. As a developer I need full control. It has nothing to do with personal demands or so. It is simply essential to have full control over your build! Especially if you make a living from it. Workarounds don't count for me. I am not here to find out how the engine has to be handled if it has flaws. It should be vice versa! Strong tickets with a lot of petitors should be solution and not by showing the company that we are willing to fight out their mistakes! Unity brought so many flaws. The concept itself also doesn't go through either. Because at the moment you only need to click in a list, somewhere in project for example, and a bar will open with calculations and you have to wait minimum 30 seconds. for what? And it happens all the time. And the project isn't even big on data or else. For only two projects I have twenty different Unity Versions installed, because one version makes more problem than the other. Mobile/VR btw. For example, these are all reasons why still few Oculus Quest titles are available in the store, after minimum two years of its existence!!! There are something about two pages with BIG BIG thumbnails!

I really appreciate, also admire the doggedness and the effort that this community brings in, but it's not without reason that Helgason left!

There is still that rat in the company and this pressures many good people who are working there. If you don't know what happend, try to find some articles. Everything because of one horny motherf.... :/, who doesn't care what happens to Unity and the company!

Sorry for being that 'destructive' :/ However, I can't else anymore. I've been doing this for too long. Lost so much money, projects because of so many issues since unity 2019.x upwards. (From URP flaws, to delta-time problems, and so much more) Unity 2021.x seems to have fixed a lot of these issues, that made the engine on certain platforms not suitable for production! Still have to wait for the 1st 2021.x LTS version, otherwise I can't guarantee flawless production for my clients. Meanwhile, I can go sell flowers or shoes. And not even that because of that great time we were pushed in.

peace...

 

Feel the pain @SOIL.

I will say that I did notice a push from Unity to improve things and listen more but you have to use the proper channels. Notably, the customer satisfaction team... they do seem to push in the same direction as devs and can skip many hoops.

Cheers.

 

Thank you for your sympathy, however I don't believe in knocking on the door to ask for a better software. If you got my drift ^^

 

I understand that not everyone has that inclination. That said, it can be hard to walk that line and not be 'destructive' like you said. In some case, we have to use Unity because, even with its quirks, it does the job most of the time if you know what you're doing. I personally just decided to bite the bullet and push where I can... and one loud voice well timed can sometimes do wonders.

 

Hi @phil_lira. Any plan to implement proper tool to inspect all the available shaders and able to choose the shader to strip to massively improve build speed and build size.

 

Not exactly the same but you can always install Unity's "Build Report" package to look at what is being built, what is using it and how big it is. A lot more user friendly than going through the build-report log file.

 

TBH that tool is a joke. The asset list renders 1 fps on my Ryzen 5950. If I want to select an asset, I need to click ten times for it to register. Feels like they didn't test it on a real project (surprised are: nobody).

 

In case someone (like me) is stuck to 2020 for some reason (assets), and stumbles on this thread, here a simple workaround that strips filmgrain textures etc.:

- delete the pngs in the file explorer, not the metas
- repeat above step every time you reimport / before you build ah, gotta love unity

pngs will get restored in the explorer, but the package manager will not include them until you reimport. this worked for me even with PP on. modifying/downscaling did not work for me.

Cheers