Search Unity

  1. Unity 2019.1 is now released.
    Dismiss Notice

GL_APPLE_shader_framebuffer_fetch

Discussion in 'Shaders' started by Martin-Kraus, Jun 20, 2012.

  1. Martin-Kraus

    Martin-Kraus

    Joined:
    Feb 18, 2011
    Posts:
    617
    Hi,

    I just saw that Apple will support an extension GL_APPLE_shader_framebuffer_fetch in OpenGL ES 2.0 on all iOS 6 devices which allows programmable blending by using a built-in variable gl_LastFragData[0] in a fragment shader.

    Is there any chance that this will be accessible any time soon in Unity?

    Thanks
     
    Last edited: Jun 20, 2012
  2. Dreamora

    Dreamora

    Joined:
    Apr 5, 2008
    Posts:
    26,601
    who knows what Unity 4 will bring
    I would take it for granted that Unity 3 will not get an iOS 6 upgrade
     
  3. Martin-Kraus

    Martin-Kraus

    Joined:
    Feb 18, 2011
    Posts:
    617
    Aras?

    Thinking about it, I'm not sure how much support from Unity is necessary: maybe one doesn't even have to call any OpenGL function but the variable gl_LastFragData[0] is just available in GLSL fragment shaders on iOS 6.
     
  4. Aras

    Aras

    Graphics Plumber Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,528
    Well, we learned about this feature at exactly the same time you did, which is to say, "a couple of days ago" ;)

    Features take time to expose, but yeah, if you're writing raw GLSL yourself, then it should (haven't checked) "just work", just write GLSL shader that uses the variable.

    If you want shaders written in Cg/HLSL to be able to do this, then it's more complex for us to implement. We'd need to extend our HLSL2GLSL translator and GLSL Optimizer to "understand" this, and produce valid iOS6 GLSL in the end.

    But you know what? Since both of those pieces are open source... hint hint... pull request... https://github.com/aras-p/hlsl2glslfork and https://github.com/aras-p/glsl-optimizer ;)
     
  5. Martin-Kraus

    Martin-Kraus

    Joined:
    Feb 18, 2011
    Posts:
    617
    Thanks for the reply, Aras! And, yes, I do write raw, untamed GLSL code. :)
     
  6. Aras

    Aras

    Graphics Plumber Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,528
    Then I think it should work (well I haven't checked). All the extension requires is adding an extension statement to your shader, and then using gl_LastFragData.
     
  7. brn

    brn

    Joined:
    Feb 8, 2011
    Posts:
    316
    I haven't read up about this functionality properly yet, so i might be jumping to conclusions.

    But it should open up a whole world of possibilities. Looking forward to it being implemented some time in the future. Aras will this be mapped to unitys the grabpass functionality, or will you keep it as a separate feature?

    Right now the hoops I'm jumping through to avoid reading from the frame buffer on IOS are quite inconvenient, worth the performance gain though.
     
  8. Aras

    Aras

    Graphics Plumber Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,528
    It has nothing to do with GrabPass, I think.

    This new feature lets you read the color of the curent pixel in your pixel shader. Common use cases are pretty much what is mentioned in the extension specification: implementing custom blending modes (think all of the Photoshop ones) and doing really simple postprocessing effects that only need to read the current pixel (color correction etc.).
     
  9. Martin-Kraus

    Martin-Kraus

    Joined:
    Feb 18, 2011
    Posts:
    617
    Where can I read the extension specification? I didn't see it at http://www.khronos.org/registry/gles/ and I didn't find it in Apple's developer resources (apart from the WWDC presentation).
     
  10. Dreamora

    Dreamora

    Joined:
    Apr 5, 2008
    Posts:
    26,601

    I would assume that the only reason apple even added it is the new image processing effects in Core Image, but I might be wrong there ;)
     
  11. Martin-Kraus

    Martin-Kraus

    Joined:
    Feb 18, 2011
    Posts:
    617
  12. Martin-Kraus

    Martin-Kraus

    Joined:
    Feb 18, 2011
    Posts:
    617
    I agree. :)
     
  13. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    25,108
    Correct me if I'm wrong but it seems like a nice speed boost. Shame it does not translate to droid well (unless I'm mistaken and it's supported there).
     
  14. Aras

    Aras

    Graphics Plumber Unity Technologies

    Joined:
    Nov 7, 2005
    Posts:
    4,528
    In theory all PowerVR GPUs can do it; whether the drivers expose the functionality or not is a different story.

    A lot of other GPUs can't do it and won't be able to anytime soon, so yeah, not very much "cross platform appeal".
     
  15. Martin-Kraus

    Martin-Kraus

    Joined:
    Feb 18, 2011
    Posts:
    617
    I just learned that Nvidia's Tegra chip has been supporting this functionality for a long time (GL_NV_shader_framebuffer_fetch: http://developer.download.nvidia.com/assets/mobile/files/tegra_gles2_development.pdf ); but I assume that not all Tegra-based devices expose it. Khronos now lists the (multi-vendor) extension GL_EXT_shader_framebuffer_fetch: http://www.khronos.org/registry/gles/extensions/EXT/EXT_shader_framebuffer_fetch.txt . Since Qualcomm's and ARM's mobile GPUs are tile-based, they are probably also able to implement this without too much effort (at least in future hardware).

    UPDATE: I just tried it on an iPod touch 4 with iOS 6 and it works fine except that you have to ask for GL_EXT_shader_framebuffer_fetch instead of GL_APPLE_shader_framebuffer_fetch (I guess Apple used the GL_APPLE_... name in earlier pre-release builds of iOS 6). I also noted that you have to use gl_LastFragData[0] while using gl_LastFragColor results in an error. (GL_NV_shader_framebuffer_fetch allows to use gl_LastFragColor instead of gl_LastFragData[0]).
     
    Last edited: Sep 22, 2012
  16. Elringus

    Elringus

    Joined:
    Oct 3, 2012
    Posts:
    332
    Hello,

    With Unity 5 already out, I would like to know if there are any news/plans implementing the framebuffer fetch function to the Cg language?
     
  17. VIC20

    VIC20

    Joined:
    Jan 19, 2008
    Posts:
    2,395
    I haven't tested it but I think I saw it mentioned in the release notes, I guess it should already work.
     
    Elringus likes this.
  18. Elringus

    Elringus

    Joined:
    Oct 3, 2012
    Posts:
    332
    Wow, I skimmed through the notes, but couldn’t find it. Will look again than, thank you! :)
     
  19. VIC20

    VIC20

    Joined:
    Jan 19, 2008
    Posts:
    2,395
    At least Aras mentioned it in another thread some months ago.
     
  20. VIC20

    VIC20

    Joined:
    Jan 19, 2008
    Posts:
    2,395
    from 4.6.3 release notes:

    • Shaders: Support for GL_EXT_shader_framebuffer_fetch. If you have a pixel shader with an "inout" color, it gets translated to use shader framebuffer fetch on GLES2/GLES3/Metal. Use UNITY_FRAMEBUFFER_FETCH_AVAILABLE macro in shaders to conditionally enclose that.
     
    Elringus likes this.
  21. Elringus

    Elringus

    Joined:
    Oct 3, 2012
    Posts:
    332
    Yep, I found the note. But now I'm really confused about how to use it: "pixel shader with an "inout" color" is a somewhat vague hint... I'm currently can only think about something like:

    Code (CSharp):
    1. fixed4 frag (v2f i, inout fixed4 fetchColor : COLOR1) : SV_Target
    2. {
    3.     ....
    4. }
    But it's not working...
     
    Last edited: Apr 7, 2015
  22. Elringus

    Elringus

    Joined:
    Oct 3, 2012
    Posts:
    332
  23. VIC20

    VIC20

    Joined:
    Jan 19, 2008
    Posts:
    2,395
    There is a bug in the HLSL2GLSL "gl_FragData[0] = " is always added TWICE to the compiled shader

    But this shader compiles fine without errors and basically is what you will need.

    Code (CSharp):
    1. Shader "Custom/BasicFrameBufferFetch"
    2. {
    3.     SubShader
    4.     {
    5.         Tags { "Queue" = "Overlay" }
    6.         ZWrite Off
    7.  
    8.         Pass
    9.         {
    10.  
    11.             CGPROGRAM
    12.             #pragma vertex vert
    13.             #pragma fragment frag
    14.        
    15.             struct vertexInput
    16.             {
    17.                 float4 vertex : POSITION;
    18.             };
    19.  
    20.             struct vertexOutput
    21.             {          
    22.                  float4 pos : SV_POSITION;
    23.             };
    24.        
    25.               vertexOutput vert(vertexInput i)
    26.             {
    27.                 vertexOutput o;
    28.                 o.pos =  mul(UNITY_MATRIX_MVP, i.vertex);
    29.                 return o;
    30.             }
    31.        
    32.             fixed4 frag(vertexOutput i      
    33.             #ifdef UNITY_FRAMEBUFFER_FETCH_AVAILABLE
    34.                 , inout fixed4 FetchedFrameBufferColor : COLOR0
    35.             #endif  
    36.             ) : SV_Target
    37.             {
    38.          
    39.                 fixed4 fragcolor;
    40.  
    41.                 #ifdef UNITY_FRAMEBUFFER_FETCH_AVAILABLE
    42.                     // on supported platforms do something with last fragment color          
    43.                     fragcolor = FetchedFrameBufferColor * 0.5;
    44.                 #else
    45.                     // other platforms do something else
    46.                     fragcolor = fixed4(1.0, 1.0, 1.0, 1.0);
    47.                 #endif
    48.          
    49.                 return fragcolor;
    50.             }
    51.  
    52.             ENDCG
    53.         }
    54.     }
    55. }
    Bug in compiled shader code commented out:

    Code (CSharp):
    1. SubProgram "gles " {
    2. // Stats: 1 math
    3. "!!GLES
    4.  
    5.  
    6. #ifdef VERTEX
    7.  
    8. attribute vec4 _glesVertex;
    9. uniform highp mat4 glstate_matrix_mvp;
    10. void main ()
    11. {
    12.  gl_Position = (glstate_matrix_mvp * _glesVertex);
    13. }
    14.  
    15.  
    16.  
    17. #endif
    18. #ifdef FRAGMENT
    19.  
    20. #extension GL_EXT_shader_framebuffer_fetch : enable
    21. void main ()
    22. {
    23.  lowp vec4 xlt_FetchedFrameBufferColor_1;
    24.  mediump vec4 tmpvar_2;
    25.  tmpvar_2 = gl_LastFragData[0];
    26.  xlt_FetchedFrameBufferColor_1 = tmpvar_2;
    27.  lowp vec4 tmpvar_3;
    28.  tmpvar_3 = (xlt_FetchedFrameBufferColor_1 * 0.5);
    29. // gl_FragData[0] = xlt_FetchedFrameBufferColor_1;
    30.  gl_FragData[0] = tmpvar_3;
    31. }
    32.  
    33.  
    34.  
    35. #endif"
    36. }
    37.  
     
    Last edited: Apr 8, 2015
    colin299, Jessy and Elringus like this.
  24. Elringus

    Elringus

    Joined:
    Oct 3, 2012
    Posts:
    332
    Ah, so I should have used COLOR0 semantic. Thank you very much, it's working!