Search Unity

Surface-shader compilation error/timeout with many multi_compile

Discussion in 'Shaders' started by MaxProude, Jan 15, 2019.

  1. MaxProude

    MaxProude

    Joined:
    Feb 24, 2013
    Posts:
    175
    I have run into a problem creating a shader that reconstructs billboards (and other shapes) to be used in a GPU Particle System. I'm using many multi_compile's and found that they seem to be working independently. I couldn't get it to work with all features enabled because it stops compiling after 10 min.
    Maybe you guys have an idea why surface-shaders behave like this?!

    Attached are the shaders that are used to calculate and display. You can find the shaders with the problem here: Shaders/InternalParticleAppearance/ParticlePositionBillboardSurface...
    I duplicated it 4 times to show you what's wrong.

    Not compiling with error:
    Usually:
    Shader compiler: internal error processing shader: Protocol error - unexpected end of data (did the Editor crash?))
    Sometimes:
    Shader compiler: internal error processing shader: Read timeout from shader compiler process


    #pragma multi_compile __ MAINTEX
    #pragma multi_compile __ METALLIC_SMOOTHNESS
    #pragma multi_compile __ NORMAL_MAP
    #pragma multi_compile __ EMISSION_MAP
    #pragma multi_compile __ POINT TRIANGLE BILLBOARD H_BILLBORD V_BILLBOARD S_BILLBOARD TS_BILLBOARD MESH
    #pragma multi_compile __ ROTATION
    #pragma multi_compile __ LINEAR_SIZE SMOOTH_SIZE CURVE_SIZE RANDOM2CURVES_SIZE
    #pragma multi_compile __ LINEAR_ROTATION SMOOTH_ROTATION CURVE_ROTATION RANDOM2CURVES_ROTATION


    Compiling v1:

    #pragma multi_compile __ MAINTEX
    #pragma multi_compile __ METALLIC_SMOOTHNESS
    #pragma multi_compile __ NORMAL_MAP
    #pragma multi_compile __ EMISSION_MAP
    #pragma multi_compile __ POINT TRIANGLE BILLBOARD H_BILLBORD V_BILLBOARD S_BILLBOARD TS_BILLBOARD MESH

    #pragma multi_compile __ ROTATION
    //#pragma multi_compile __ LINEAR_SIZE SMOOTH_SIZE CURVE_SIZE RANDOM2CURVES_SIZE

    //#pragma multi_compile __ LINEAR_ROTATION SMOOTH_ROTATION CURVE_ROTATION RANDOM2CURVES_ROTATION


    Compiling v2

    #pragma multi_compile __ MAINTEX
    #pragma multi_compile __ METALLIC_SMOOTHNESS
    #pragma multi_compile __ NORMAL_MAP
    #pragma multi_compile __ EMISSION_MAP
    #pragma multi_compile __ POINT
    //TRIANGLE BILLBOARD H_BILLBORD V_BILLBOARD S_BILLBOARD TS_BILLBOARD MESH

    //#pragma multi_compile __ ROTATION
    #pragma multi_compile __ LINEAR_SIZE SMOOTH_SIZE CURVE_SIZE RANDOM2CURVES_SIZE

    #pragma multi_compile __ LINEAR_ROTATION SMOOTH_ROTATION CURVE_ROTATION RANDOM2CURVES_ROTATION


    Compiling v3

    #pragma multi_compile __ MAINTEX
    //#pragma multi_compile __ METALLIC_SMOOTHNESS
    //#pragma multi_compile __ NORMAL_MAP
    //#pragma multi_compile __ EMISSION_MAP
    #pragma multi_compile __ POINT TRIANGLE BILLBOARD H_BILLBORD V_BILLBOARD S_BILLBOARD TS_BILLBOARD MESH

    //#pragma multi_compile __ ROTATION[/B]
    #pragma multi_compile __ LINEAR_SIZE SMOOTH_SIZE CURVE_SIZE RANDOM2CURVES_SIZE
    #pragma multi_compile __ LINEAR_ROTATION SMOOTH_ROTATION CURVE_ROTATION RANDOM2CURVES_ROTATION


    The shaders are part of the Ultimate GPU Particle System asset, using v1 of the deactivated multi_compiles that disables particle and rotation over lifetime when using surface shaders. I couldn't find a solution yet.
    The shader is still work in progress. Changing multi_compile to shader_feature also didn't work and is not an option, since the particle system needs to have all variants ready on the device. I hope that I'm not missing anything obvious here :D

    @LukasCh any idea?
     

    Attached Files:

  2. LukasCh

    LukasCh

    Unity Technologies

    Joined:
    Mar 9, 2015
    Posts:
    102
    Okey, so I have not checked the shaders deeply yet, but I found few potential issues.

    I noticed that you use a lots of multi_compile in one shader:

            #pragma multi_compile __ MAINTEX
    #pragma multi_compile __ METALLIC_SMOOTHNESS
    #pragma multi_compile __ NORMAL_MAP
    #pragma multi_compile __ EMISSION_MAP
    #pragma multi_compile __ POINT TRIANGLE BILLBOARD H_BILLBORD V_BILLBOARD S_BILLBOARD TS_BILLBOARD MESH
    #pragma multi_compile __ ROTATION
    #pragma multi_compile __ LINEAR_SIZE SMOOTH_SIZE CURVE_SIZE RANDOM2CURVES_SIZE
    #pragma multi_compile __ LINEAR_ROTATION SMOOTH_ROTATION CURVE_ROTATION RANDOM2CURVES_ROTATION


    These few lines, will result in 2*2*2*2*8*2*5*5=6400 variants. Which is quite a number, especially if your shaders are big. So I'm not sure you actually need all these variants.

    By changing it to shader_feature your shader compilation must likely will cut down from 10 minutes to few seconds. You mentioned that you need all this variants on device. However if you can figure out all variants that you will need at built time - you can easily get away with shader_feature (In you scenario I'm guessing you have particle script that allow toggling options for particles, if it is true - when you can generate material at editor with specific keywords enabled).

    So the first error indicates that some corruption happened, as editor is communicating through socket with shader compiler and it got unexpected symbol.
    The second error mostly happens when compiler crashes.
     
  3. MaxProude

    MaxProude

    Joined:
    Feb 24, 2013
    Posts:
    175
    I know, that's a lot of shader variants. I just wanted to support the possibility of creating new particle systems on the device programmatically. Using shader_features is something I'll consider when everything else fails, but Im not giving up yet. For some reason shader_feature still fails, if all features are enabled.

    My gut feeling tells me, that the compilation takes too long and the editor is frozen during compilation. The compiler tries to communicate with the editor but it does not react. It then stops after exactly 10 min. and unfreezes the editor.
    If I add more shader features, compilation time increases. if compilation takes longer than 10 min the problem occurs.

    I wonder if the new VFX pipeline will be able to create effects during runtime as well.