Search Unity

  1. Unity 2019.1 is now released.
    Dismiss Notice

Has any one experimented with Box Projection Correction Environment Mapping?

Discussion in 'Shaders' started by Dev.D1, Nov 30, 2011.

  1. Dev.D1

    Dev.D1

    Joined:
    Dec 29, 2010
    Posts:
    99
    I am wondering if anyone has experimented with "Box Projected Cubemap Environment Mapping in Unity"?

    I saw a couple of examples, one with an implementation in the Blender Game Engine . Looks much nicer than the out of the box cube mapping - when done right :). This solution looks particularly good when rendering box-shaped interior scenes. Bascially this approach changes the way we sample the cube map.

    The math for this is described here.


    Implementing this in Unity3D shouldn't be too difficult, however I managed to 'bump' into some problems :) . Hence a call out to any one with better experiance writing shaders. What I did was take the code of the "Reflect-Bumped.shader" from the builtin shaders, added the bpcem method. I'm using worldPos in the surface shaders input struct . I'd assume it would be similar to the world position in the original algorithm.


    The shader looks like this :
    Code (csharp):
    1. // Box Projection Cubemap Environ Mapping Test in CG
    2.  
    3. // Adapted from the work of Bartosz Czuba and Martins Upitis
    4.  
    5. //
    6.  
    7. // http://devlog-martinsh.blogspot.com/2011/09/box-projected-cube-environment-mapping.html
    8.  
    9.  
    10.  
    11.  
    12.  
    13. Shader "BPCEM_Test_CG" {
    14.  
    15. Properties {
    16.  
    17.     _Color ("Main Color", Color) = (1,1,1,1)
    18.  
    19.     _ReflectColor ("Reflection Color", Color) = (1,1,1,0.5)
    20.  
    21.     _MainTex ("Base (RGB) RefStrength (A)", 2D) = "white" {}
    22.  
    23.     _Cube ("Reflection Cubemap", Cube) = "_Skybox" { TexGen CubeReflect }
    24.  
    25.     _BumpMap ("Normalmap", 2D) = "bump" {}
    26.  
    27. }
    28.  
    29.  
    30.  
    31. SubShader {
    32.  
    33.     Tags { "RenderType"="Opaque" }
    34.  
    35.     LOD 300
    36.  
    37.    
    38.  
    39. CGPROGRAM
    40.  
    41. #pragma surface surf Lambert
    42.  
    43. #pragma target 3.0
    44.  
    45.  
    46.  
    47. // Debug pragmas
    48.  
    49. #pragma only_renderers d3d9
    50.  
    51. #pragma debug
    52.  
    53.  
    54.  
    55. sampler2D _MainTex;
    56.  
    57. sampler2D _BumpMap;
    58.  
    59. samplerCUBE _Cube;
    60.  
    61.  
    62.  
    63. fixed4 _Color;
    64.  
    65. fixed4 _ReflectColor;
    66.  
    67.  
    68.  
    69. struct Input
    70.  
    71. {
    72.  
    73.     float2 uv_MainTex;
    74.  
    75.     float2 uv_BumpMap;
    76.  
    77.     float3 worldRefl;
    78.  
    79.     float3 worldPos;
    80.  
    81.     float3 viewDir;
    82.  
    83.     INTERNAL_DATA
    84.  
    85. };
    86.  
    87.  
    88.  
    89.  
    90.  
    91. //env map parameters
    92.  
    93. //in this case the box dimensions are 2x2x2 and it is positioned in the middle of the world
    94.  
    95. const float3 EnvBoxMax = float3(1.0,1.0,1.0); //AABB max value
    96.  
    97. const float3 EnvBoxMin = float3(-1.0,-1.0,-1.0); //AABB min value
    98.  
    99. const float3 EnvBoxPos = float3(0.0,0.0,0.0); // the world location of AABB box
    100.  
    101.  
    102.  
    103. //Box Projected Cube Environment Mapping by Bartosz Czuba
    104.  
    105. float3 bpcem (in float3 v, float3 Emax, float3 Emin, float3 Epos, float3 Pos)
    106.  
    107. {  
    108.  
    109.     // All calculations in worldspace
    110.  
    111.     float3 nrdir = normalize(v);
    112.  
    113.     float3 rbmax = (Emax - Pos)/nrdir;
    114.  
    115.     float3 rbmin = (Emin - Pos)/nrdir;
    116.  
    117.    
    118.  
    119.     float3 rbminmax;
    120.  
    121.     rbminmax.x = (nrdir.x>0.0)?rbmax.x:rbmin.x;
    122.  
    123.     rbminmax.y = (nrdir.y>0.0)?rbmax.y:rbmin.y;
    124.  
    125.     rbminmax.z = (nrdir.z>0.0)?rbmax.z:rbmin.z;    
    126.  
    127.     float fa = min(min(rbminmax.x, rbminmax.y), rbminmax.z);
    128.  
    129.    
    130.  
    131.     float3 posonbox = Pos + nrdir* fa;
    132.  
    133.     return posonbox - Epos;
    134.  
    135. }
    136.  
    137.                
    138.  
    139. void surf (Input IN, inout SurfaceOutput o)
    140.  
    141. {
    142.  
    143.     fixed4 tex = tex2D(_MainTex, IN.uv_MainTex);
    144.  
    145.     fixed4 c = tex * _Color;
    146.  
    147.     o.Albedo = c.rgb;
    148.  
    149.    
    150.  
    151.     o.Normal = UnpackNormal(tex2D(_BumpMap, IN.uv_BumpMap));
    152.  
    153.    
    154.  
    155.     // Reflection  
    156.  
    157.     float3 worldRefl = WorldReflectionVector (IN, o.Normal);   
    158.  
    159.  
    160.  
    161.     //bpcem-izing reflection coordinates
    162.  
    163.     // IN.worldPos comes bound to a texture coordinate. This should be an interpolated value.  
    164.  
    165.     float3 rVec = bpcem(worldRefl,EnvBoxMax,EnvBoxMin,EnvBoxPos, IN.worldPos);
    166.  
    167.    
    168.  
    169.     // Instead of the usual cubemapping , we figure out the updated sampling point from above bpcem function.
    170.  
    171.     //fixed4 reflcol = texCUBE (_Cube, worldRefl);
    172.  
    173.     fixed4 reflcol = texCUBE (_Cube, rVec);
    174.  
    175.  
    176.  
    177.     reflcol *= tex.a;
    178.  
    179.     o.Emission = reflcol.rgb * _ReflectColor.rgb;
    180.  
    181.     o.Alpha = reflcol.a * _ReflectColor.a;
    182.  
    183. }
    184.  
    185. ENDCG
    186.  
    187. }
    188.  
    189.  
    190.  
    191. FallBack "Reflective/VertexLit"
    192.  
    193. }
    194.  
    195.  
    Think I might have buggered up with a matrix transform somewhere.Attaching an image of what I'm observing as compared to regular cubemapping. I'm also enclosing my whole test scene in a unity package for quick testing.

    Hope someone out there can give me some pointers. Cheers!
     

    Attached Files:

    Last edited: Nov 30, 2011
  2. Steva

    Steva

    Joined:
    Sep 9, 2010
    Posts:
    60
    Sorry for digging up an old thread, but i wanted to say that BPCEM is indeed possible in unity and not too difficult :)

    The only problem i encountered was that for some reason two faces of the cubemap get inverted



    here link to webplayer:
    www.megaupload.com/?d=Z5YGVCSP

    here's the shader:
    Code (csharp):
    1.  
    2. Shader "Custom/Reflective BPCEM Diffuse" {
    3. Properties {
    4.     //Box Projected Cubemap Environment Mapping
    5.     _Color ("Main Color", Color) = (1,1,1,1)
    6.     _ReflectColor ("Reflection Color", Color) = (1,1,1,0.5)
    7.     _MainTex ("Base (RGB) RefStrength (A)", 2D) = "white" {}
    8.     _Cube ("Reflection Cubemap", Cube) = "_Skybox" { TexGen CubeReflect }
    9.     _BumpMap ("Normalmap", 2D) = "bump" {}
    10.    
    11.     _EnvBoxStart ("Env Box Start", Vector) = (0, 0, 0)
    12.     _EnvBoxSize ("Env Box Size", Vector) = (10, 10, 10)
    13. }
    14.  
    15. SubShader {
    16.     Tags { "RenderType"="Opaque" }
    17.     LOD 300
    18.    
    19. CGPROGRAM
    20. #pragma surface surf Lambert
    21.  
    22. sampler2D _MainTex;
    23. sampler2D _BumpMap;
    24. samplerCUBE _Cube;
    25.  
    26. fixed4 _Color;
    27. fixed4 _ReflectColor;
    28. fixed4 _EnvBoxStart;
    29. fixed4 _EnvBoxSize;
    30.  
    31. struct Input {
    32.     float2 uv_MainTex;
    33.     float2 uv_BumpMap;
    34.     float3 worldRefl;
    35.     fixed3 worldPos;
    36.     float3 worldNormal;
    37.     INTERNAL_DATA
    38. };
    39.  
    40. void surf (Input IN, inout SurfaceOutput o) {
    41.    
    42.    
    43.    
    44.     fixed4 tex = tex2D(_MainTex, IN.uv_MainTex);
    45.     fixed4 c = tex * _Color;
    46.     o.Albedo = c.rgb;
    47.    
    48.     fixed3 n = UnpackNormal(tex2D(_BumpMap, IN.uv_BumpMap));
    49.    
    50.     float3 dir = IN.worldPos - _WorldSpaceCameraPos;                            // pos fragment relativo a pos camera
    51.     float3 worldNorm = IN.worldNormal;
    52.     worldNorm.xy -= n;
    53.     float3 rdir = reflect (dir, worldNorm);                             // vettore riflesso da normale
    54.    
    55.     //BPCEM
    56.     float3 nrdir = normalize (rdir);                                            // vettore riflesso normalizzato
    57.     float3 rbmax = (_EnvBoxStart + _EnvBoxSize - IN.worldPos)/nrdir;            // AABB max value +...
    58.     float3 rbmin = (_EnvBoxStart - IN.worldPos)/nrdir;                          // AABB min value +...
    59.     float3 rbminmax = (nrdir>0.0f)?rbmax:rbmin;                                 // ...?
    60.     float fa = min(min(rbminmax.x, rbminmax.y), rbminmax.z);                    // ...?
    61.     float3 posonbox = IN.worldPos + nrdir*fa;                                   // ...?
    62.     rdir = posonbox - (_EnvBoxStart +_EnvBoxSize/2);                            // ...? - posizione (centro) del box
    63.     //PBCEM end
    64.    
    65.     float3 env = texCUBE (_Cube, rdir);
    66.    
    67.     float3 luminosity = float3 (0.30, 0.59, 0.11);
    68.     float reflectivity = dot(luminosity, env.rgb);
    69.     o.Emission = env.rgb *reflectivity *_ReflectColor;
    70.    
    71. }
    72. ENDCG
    73. }
    74.  
    75. FallBack "Reflective/VertexLit"
    76. }
     
  3. brn

    brn

    Joined:
    Feb 8, 2011
    Posts:
    316
    Ive been wanting to look at this technique, nice work
     
  4. Dev.D1

    Dev.D1

    Joined:
    Dec 29, 2010
    Posts:
    99

    [Edit: See my next post]

    Hey Steva,

    I worked the BPCEM stuff out the same day , must have forgotten to post the results.

    In my original post if you move the parameters EnvBoxMax ,EnvBoxMin , EnvBoxPos into the BCPEM function or pass them via the function arguments, the shader works correctly. The problem in my version was that the 'const' globals were not being mapped correctly by unity. That was the only problem. I haven't noticed any inverted cubemap faces. That may happen if you are not setting the above parameters correctly according to your scene. The device I'm on doesn't let me view webplayers so i can't check your stuff right now.

    In my project I ended up supplying the BPCEM parameters to the shader via Unity shader's parameter block ( as you are doing in yours)
     
    Last edited: Jan 12, 2012
  5. Dev.D1

    Dev.D1

    Joined:
    Dec 29, 2010
    Posts:
    99
    Ok I had another look at this.

    Steva, are you sure your generated cubemap is accurate ? Please try capturing the cubemap again in your scene.

    I just plugged your shader into my setup and after adjusting the params a bit by hand I got what i think is the correct reflection behaviour. I'm enclosing a couple of screen caps. Your shader has been applied to the wooden floor. Notice that the windows and the door reflection of 3 sides of room are correctly rendered. Windows have my BPCEM shader as a control case.




    For comparison I'm enclosing a screencap with my shader on the wooden floor. Both kind of match.
    $BPCEM.png

    Please omit the distortions due to slightly inaccurate BPCEM parameters and the odd shape of the room.
     
  6. brn

    brn

    Joined:
    Feb 8, 2011
    Posts:
    316
    Looks great to me, nice one!
     
  7. Dev.D1

    Dev.D1

    Joined:
    Dec 29, 2010
    Posts:
    99
    Cheers. All credit to Bartosz Czuba and Martins Upitis for coming up with this technique. There are, however, some limitations:

    As the original article points out this technique in its current avatar is best suited for cubic rooms which are axis aligned. The system can be trivially extended to support non axis aligned rooms using a matrix transformation. Support for odd shaped rooms in a bit more involved and requires breaking up the room into a set of bounding boxes breaking up the reflecting surfaces as well. This is the approach that was used in the original blender engine game demo. This has the unfortunate side effect of introducing discontinuities in the mapped reflections ( see images in the above article). For subtle reflections as with wooden floors the discontinuities are not very glaring. But I think inspite of these issues it's great for what it is.

    I wish we could get this to work on iOS targets but since no cubemap support ...
     
    Last edited: Jan 12, 2012
  8. Steva

    Steva

    Joined:
    Sep 9, 2010
    Posts:
    60
    No, i'm not. In fact i believe they are wrong, but i didn't look into it (the scene was pretty anyway :)).

    Also, my shader probably should be cleaned and optimized a bit.

    Nice bed btw, did you model it yourself?
     
  9. Dev.D1

    Dev.D1

    Joined:
    Dec 29, 2010
    Posts:
    99
    Likewise, I would have liked to release the optimised final shader backend setup so that others could benefit. I have a few editor tools that help setup BPCEM params automatically, generate cubemaps ( similar to the one on unifycommunity). This helps with larger projects and constant art iterations . However since this was commissioned work and the client didn't agree, the prototype shader in the first post is the most I can make public. :(. With your shader, atleast other folk can get this working off the bat.

    Thanks, this WIP bed was made by our artist for the commercial project. The final asset is actually much nicer :)
     
  10. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    Really cool stuff - the improved results are very nice.

    How badly does this sort of thing mess up if you're not using AABB or cube-shaped rooms?
     
  11. Dev.D1

    Dev.D1

    Joined:
    Dec 29, 2010
    Posts:
    99
    Non AABB room :
    Results look wrong if you use the stock BPCEM shader. Can be solved by defining a matrix transform. Best way to solve this is to take a cube, orient it to your non AABB room, calculate the matrix transform and take care of this in the shader which sampling the cubemap. If you look at the shader itself, it depends on the world position and the reflection vector, bounding box extremas etc. Transform them and your good to go.

    Non cubic room : This is a hard one to solve. The result do not look correct, esp if there are lots of curved walls or corridors. Best approach I found is to subdivide the irregular shaped room into smaller AABBs . You also need to partition the rooms reflecting surfaces so their materials can reference the correct cubemaps. See the original blender demo. In our cases the reflection effects required were subtle so the doscontinuties were not that noticeable.

    Practically the above demo and the one posted by Steva are simplistic. The real issues arise when you have beveled geometry ( eg: a extruded window ledge) on the walls. No matter what you do you will not get a 100% accurate result, as you would get by realtime planer reflection. But this is stupid cheap!
     
  12. Steva

    Steva

    Joined:
    Sep 9, 2010
    Posts:
    60
    I just checked, the cubemaps do seem to work on ARMV7 devices (iPhone4, iPad)... the shader doesn't (for now), maybe i'll convert it in GLSL and see...
     
  13. Farfarer

    Farfarer

    Joined:
    Aug 17, 2010
    Posts:
    2,249
    Cool, that doesn't seem too much of an imposition to get such nice results. Can all be pre-baked and fed into the shader as required.

    Thanks for the explanation :)
     
  14. Dev.D1

    Dev.D1

    Joined:
    Dec 29, 2010
    Posts:
    99
    I think this is probably a limitation of Unity iOS. Or it could be a limitation of my understanding of Unity iOS :).

    I was having a look at another (unrelated) post and bumped across this comment from user : aNTeNNa trEE about cubemaps in AngryBots on mobile targets. Might be a good place to start if you're bothered.
     
  15. Steva

    Steva

    Joined:
    Sep 9, 2010
    Posts:
    60
    Thanks for the tip, i'll have a look at the AngryBots demo :)
     
  16. tatoforever

    tatoforever

    Joined:
    Apr 16, 2009
    Posts:
    3,744
    I got this technique working fine for our project but not the original work. That one seems to be wrong on the projection. I've implemented a new one which is 100% accurate, here is a example from our current project Forggoten Memories iOS:

    $21-10-2012 7-26-28 AM.jpg $19-10-2012 1-16-28 PM.jpg $19-10-2012 1-14-46 PM.jpg $21-10-2012 7-29-11 AM.jpg
    $18-10-2012 3-03-54 AM.jpg

    Notice the reflections on metallic surfaces and the blood in the floor. I've also made different versions, transparent, bump, diffuse, etc.
     
    Last edited: Oct 25, 2012
  17. Dev.D1

    Dev.D1

    Joined:
    Dec 29, 2010
    Posts:
    99
    Care to share what you did differently for the basic BPCEM technique? The one listed at the start of the thread works well for us, haven't noticed any projection anomalies - apart from the constraints of cubemap capturing. I can see that making various versions , blending using a texture, perturbing with a normal map etc would indeed make it useful esp for a grungy env like yours :) . Reflecting pool of blood ... shivers :S

    Since starting out with this technique - we have cut down on its use in our own projects. Essentially the lack of support for reflections for objects placed within the room ( as opposed to windows , paintings which are flush with the BPCEM boundaries ) is the major limiting factor. We use it sparingly now and usually blur the reflections a bit now for a faux matt / ceramic finish.
     
  18. Ippokratis

    Ippokratis

    Joined:
    Oct 13, 2008
    Posts:
    1,504
    tatorforever : Very impressing work !
     
  19. I am da bawss

    I am da bawss

    Joined:
    Jun 2, 2011
    Posts:
    2,574
    I am curious has anyone convert this shader to sphere mapping?



    Tatoforever, did you use cubemap or sphere map for those?
     
    Last edited: Jan 25, 2013
  20. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,349
    +1 care to share on what change did you made?
     
  21. Blenderik

    Blenderik

    Joined:
    May 14, 2013
    Posts:
    58
    Great Method!
    Is there anything else to know about this shader? It doesn't seem to be working for me. I made a seamless cubemap, applied it to the floor etc. I can adjust the width and start sliders until it looks "OK", but not very accurate. The biggest problem though is that it does not work from all angles, is that just so (it would probably work for me, but I'd like to know if I'm doing something wrong.
    Thanks.
     
  22. Blenderik

    Blenderik

    Joined:
    May 14, 2013
    Posts:
    58
    Nevermind, I had a look at the Python version. For everyone else experiencing weird behavior:
    Box start: pos of your object - size of your object / 2
    box size: Dimensions of your object.
    Please tell me I'm wrong, but I didn't find a tool that displays the size of your object in the Viewer, so here is a script that prints the size for you:

    function Start() {
    var mesh : Mesh = GetComponent(MeshFilter).mesh;
    print(mesh.bounds.size);
    }
    attach this to your object and start the game.
     
  23. mrmadprofessor

    mrmadprofessor

    Joined:
    May 22, 2012
    Posts:
    53
    Did anyone get this shader working for iOS? I am working on a scene right now where I have reflective water on the floor and want the cubbemapped reflection to line up with the environment.

    If anyone has the shader for this and want to share I would be forever grateful! :)
     
  24. OH NO

    OH NO

    Joined:
    Oct 16, 2013
    Posts:
    14
    @ Dev.D1 Steva

    That is really nice !!! it works prettty well for me. Thanks :)

    is it possible to integrate this code into another shader especially the marmoset shader ?

    If anyone can help me, this would be really awesome.
     
  25. Harry64

    Harry64

    Joined:
    Jan 21, 2013
    Posts:
    18
    It looks soo good ingame. I tryed it out now and I need to say this is what I want in my game.
    but....

    is it really necessary to render the Cubemap always from the center of the room?
    What if I have a loong hallway and I want to place 2 or more cubemaps along this hallway and change the cubemaps when I am close to it so I can still have the correct reflections?

    so what I want to say is, is there a way to code this into this shader that it takes also the position where the cubemap was made ito account?
    then I would not be limited to have only 1 cubemap per room and it would solve also the problem with the room height because you would place the cubemaps normally in that hight where your head is.
     
  26. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,349
    it's possible but not in shader, you still need a script to sample multiple cubemaps from predefined position and pass those information to shader.
    You might want to read on these, http://seblagarde.wordpress.com/201...ng-approaches-and-parallax-corrected-cubemap/
     
  27. Harry64

    Harry64

    Joined:
    Jan 21, 2013
    Posts:
    18
    omg thank you I think I overlooked it somehow.

    now I am able to set the cubemap position.

    this line in the article was it what I needed.

    ReflDirectionWS= EnvMapOffset * (PositionWS - CubemapPositionWS) + ReflDirectionWS;

    this is in the code of this thread this line:

    rdir = posonbox - (_EnvBoxStart +_EnvBoxSize/2);

    I just added to the shader this:

    and changed the line from above to this:
    now I can build in a room more cubemaps and define the position of them in the shader.


    1 problem solved.

    now to the blending between the cubemaps.... this is for me a very hard task because when you think the standard technic for this would be to find the nearest cubemap in the scene and do a blending.... but this has 2 big problems...

    for this problem I still got no solution...
     
  28. frogsbo

    frogsbo

    Joined:
    Jan 16, 2014
    Posts:
    79
    i wrote a box projection texture mapping, that rewrites mesh UV's:

    $find+procedural+normals.jpg
     
  29. Harry64

    Harry64

    Joined:
    Jan 21, 2013
    Posts:
    18
    I run now into a problem that I am not able to solve... please help cause I need this shader.

    I wanted to add Normal Mapping to this shader and suddenly the shader run out of Interpolators...

    I just changed this lines from this:
    to this:
    If I would not use o.Normal the shader would work correct and all reflections are ok. but If I would write the o.Normal inside the original shader then it would have wrong reflections... it actually looks like it mapps the cubemap so that it appears to be transparent...

    I only get my version to run if I would use #pragma glsl or #pragma target 4.0

    I stopp now experimenting with my shader because I have really no idea what to do.
     
  30. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,349
    What do you mean by wrong reflection?? can you show any screenshot?? because the shader from Steva in second post is works fine
     
  31. Harry64

    Harry64

    Joined:
    Jan 21, 2013
    Posts:
    18
    in this picture above you see my cubemap Testmap with my moded shader.
    as you can see if I use #pragma target 4.0 or #pragma glsl the shader works and I have the correct reflections + normalmapping.
    this is the shader that has these lines changed.
    $ECM_0.jpg

    in the picture below you see the original shader with only the lines added:
    it looks like its transparent but it actually uses my cubemap. if I would lower the cubemap resolution you would see that.
     
    Last edited: Jan 30, 2014
  32. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,349
    have you tried to normalized the n?
     
  33. Harry64

    Harry64

    Joined:
    Jan 21, 2013
    Posts:
    18
    I did not change anything important from the original exept these lines what I have written above + some other things like Rimlight or that the reflections are now in the o.Albedo
     
    Last edited: Feb 1, 2014
  34. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,349
    Sooo.....your code work just fine in my place, even the reflection is correct. I guess it's on your side then :/
     
  35. Harry64

    Harry64

    Joined:
    Jan 21, 2013
    Posts:
    18
    ? really? even without #pragma target 4.0 or glsl?

    EDIT: I updated the code above so you have the full shader I am using now!
    remove #pragma glsl and try the shader again please!

    I have a AMD Radeon HD 7900 graphic card.
     
    Last edited: Jan 30, 2014
  36. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,349
    This line,
    Code (csharp):
    1. rdir = posonbox - _CubemapPos;    
    It's the one that broke your reflection
    oh and it works without target 4.0
     
  37. Harry64

    Harry64

    Joined:
    Jan 21, 2013
    Posts:
    18
    I changed that line back to its original
    Code (csharp):
    1. rdir = posonbox- (_EnvBoxStart +_EnvBoxSize/2);
    but this makes no sense... this line of code only tells the shader where the position of the cubemap was... now it only tells the shader that the cubemap is exactly in the middle of the box. in my code I was able to tell the shader from wich position the cubemap was made.
    and I also changed these lines back: Bold text is now changed:

    CGPROGRAM
    #pragma surface surf BlinnPhong
    #pragma target 3.0
    #pragma glsl // I deleted this line
    .....

    float3 dir = IN.worldPos - _WorldSpaceCameraPos;
    float3 worldNorm = IN.worldNormal;
    worldNorm.xy -= n;
    float3 rdir = reflect (dir, worldNorm);
    ...
    ...
    rdir = posonbox - (_EnvBoxStart +_EnvBoxSize/2);

    NOW the shader dont work!!! I get this error!
    Code (csharp):
    1. Too many interpolators used (maybe you want #pragma glsl?) at line 108
    if I add the #pragma glsl again then the shader works but the reflections are like in my pictures before wrong and the objects seems to be transparent! but because I changed the rdir = posonbox - (_EnvBoxStart +_EnvBoxSize/2); line now the reflections comes from the center of the Box and this looks now like this:
    $ECM_3.jpg

    I need to be able to tell the shader from wich position the cubemap was made. first because the cubemap should be made from the average camera hight. and second that I can make 2 or more cubemaps in one room and switch between them.

    but since I think it cant be displayed on my pc without #pragma 4.0 or glsl I stick with them. my game should then not be run on a pc with dx 9.0c or lower.... need to stick then to dx 10.1....


    EDIT:
    Found out that the normal map distortion of the cubemap was not correct in my shader. if looked from world -z direction the distortion was like 0%.
    in my shader above I removed now the line worldNorm -= n;.
     
    Last edited: Feb 1, 2014
  38. Reanimate_L

    Reanimate_L

    Joined:
    Oct 10, 2009
    Posts:
    2,349
    Oh hey sorry i forgot about this thread,
    your code works fine in my place, and strangely even with this line added it works fine.

    Code (csharp):
    1. rdir = posonbox - _CubemapPos;
    and it's also runs fine under pragma target 3.
     
  39. Harry64

    Harry64

    Joined:
    Jan 21, 2013
    Posts:
    18
    ok... I dont know why my pc cannot display the shader with only pragma target 3... I will stick with pragma glsl.
    I have now created a nice Cubemap system for my game that works with Sectors. the only thing that would make it even better would be if I would be able to make the projection to local coordinates.... so I would be able to create a box projection like 45°...

    I will make this weekend a little test scene in unity and post a video how my system now works and what the pros and cons are.
    maybe others can then build up on this system. ^_^
     
  40. bcoyle

    bcoyle

    Joined:
    May 22, 2013
    Posts:
    56
    is this right? so I make a cube and scale it to fit the space - and then use the numbers from that to pipe into the shader?

    I'm using Harry64's version of the shader - I like that additional function of being able to call out where the cubemap is in the space!
     
  41. Harry64

    Harry64

    Joined:
    Jan 21, 2013
    Posts:
    18
    yes thats how I made it. I use 3 emptys with 2 scripts where I use OnDrawGizmosSelected to display the cubes when I select the first empty.
    The first empty has the OnDrawGizmosSelected and it has a script that will cache the other 2 emptys positions for the shader. the other empty is parented to the first one and the last empty is parented to the second empty and has also the OnDrawGizmosSelected to display the cube.
    I made it like this so if I select the second empty I can still see the cube and I can move that empty around and set its start position. when I select the third empty I can see the box again because he has the OnDrawGizmoSelected and so I can position the end position.

    then I have made what I call "Sectors" its another empty with also a GizmoBox and that one is looking at the start of the scene for static stuff that needs a cubemap and for rigidbodys that needs a Cubemap.
    whit the sectors I tryed to prevent that cubemaps from other levelparts or from rooms on the other side of walls get applyed to the wrong objects or levelparts. it works great.

    my system works very good. but after I heared that Unity 5 is on the way and it has dynamic Cubemaps I was like I worked that all out for nothing...
    I also coded a Cubemap Generator so that I only need to push a button and it creates all cubemaps in the scene.

    I have an old Youtube video that shows a first impression on how it looks ingame.

    http://youtu.be/PhDHSlBja5s

    now the testmap is much bigger and would also show 400 rigidbodys that get all dynamicly set the right cubemap.
    maybe I should record this tomorrow and upload it.
     
    Last edited: May 4, 2014
  42. HonoraryBob

    HonoraryBob

    Joined:
    May 26, 2011
    Posts:
    1,056
    Has anyone tried to get this to work reliably with a landscape (e.g. a river?)
     
  43. Dev.D1

    Dev.D1

    Joined:
    Dec 29, 2010
    Posts:
    99
    What do you mean by "dynamic" Cubemaps? You can capture cubemaps dynamically right now in Unity3D. Just get the camera to render to a cubemap using Camera.RenderToCubemap. I trust that is what you are doing with your cubemap generator.
     
  44. Dev.D1

    Dev.D1

    Joined:
    Dec 29, 2010
    Posts:
    99
    This technique is something you'd typically want to use in an interior scenario, like in a rectangular room. In larger areas , such as a large landscape , especially for rivers you will get distortions for sure.

    Try and work out the basic idea of the concept - first thing you do is to "capture " a cubemap. For a room this is usually the center of the volume of the room using a camera. In variations of this technique you can subdivide the room in sectors (each usually rectangular themselves). This technique is awesome for features that exist on the peripheries, such as walls , illuminated windows, celing lights and doorways etc.

    This technique is not good when you have too much geometry within the room that you wan to reflect. If you have, say a chair on the floor in the room, left un-culled the cubemap camera's captures this too. Because of the view angle, you will get a perspective incorrect image of a chair in the cubemap. This will not match with the physical chair when you apply such a cubemap using BPCEM . If someone has ideas how to get past this , do let me know. I ended up just setting geometry I was interested in getting in the Cubemap (eg: windows and walls) in separate layers. I would then only render these layers (using the cubemap camera). The chair would not be set to the layers visible to the cubemap camera. Yes the chair would be absent from the reflection, but thats better than a distorted chair drawn disconnected.

    I imagine iou'd have the same issue in your river where reflections of a boat in the water (say) will not end up aligning with the boat properly. With the amount of effort required for setting up "capture sectors" as people have suggested above and texture memory required, I gather a simple planer reflection would serve the river case much better. What would perhaps give you better mileage would be to capture lower resolutions reflections and just blur them with something like bilateral filtering , a Gaussian blur or some other computationally cheap convolution. Adding some ripples on to of that is exactly the Smoke on the water you may require (pun )

    Over the years I have ended up using this technique sparingly and only for low frequency cubemaps ( read blurred cubemaps) for faux matte reflection and for Fake GI volumetrics.
     
  45. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    1,505
    As far as i'm aware the general approach is to use image proxies for the reflection of these objects, though i'm not sure they would be good enough for a lone chair in the middle of a room. They do work well for sliding doors or for characters.

    Sébastien Lagarde describes them in relation to the graphics in Remember me.

    Here is a great write up on FxGuide, it covers PBS, but towards the end is a short section on reflections where you can see image proxies being used. Though I could never exactly work out how the planar image proxies were rendered into the scene.

    Various links
    pro-4-practical-planar-reflections
    Video
     
    Arkade and Dev.D1 like this.
  46. Dev.D1

    Dev.D1

    Joined:
    Dec 29, 2010
    Posts:
    99
    That's an awesome link there @noisecrime , I hadn't seen this before, remarkable work indeed.I've experimented with imposters (aka image proxies) but only for highlights in other reflection schemes (eg: in automotive shaders )- not couldn't with BPCEM per se cause of other constraints . These guys made significant investment in their workflow and pipeline . And it works very well for their specific case.

    Although I'm still reading the article, I would imagine the imposters are just blended in over the cubemap reflection ( as they are more "local" for the reflection) in the shader. The imposters for static geometry themselves maybe captured in scene or "proxies" produced by artists etc. Even approximate reflection grounding the prop in the game is enough to fool the player in a usual game level.

    I had a few difficulties with imposters in my case cause I was using BPCEM for architectural visualisation where it is a bit more difficult to cheat ( read not too may noisy textures) and artifacts were pretty bad for my projects. One important thing to note- these guys are using imposters not just for boosting reflections, they are also using them to block / occlude cubemap reflections! This is very clever indeed ! :) ( See 3:08 to 3:20 in your Video link)

    Imposter placement follows the usual reflection paradigm, for dynamic actors they are probably using smaller textures for each bone and are animating them (which would in effect be almost free) . Each 2D imposter representation of a character is being rendered in no more than 256x256 and then "blended in " over the cubemap reflections.

    I want to try this !
     
  47. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    1,505
    Yeah his work is very impressive, he has a blog with much of this stuff on and more, well worth reading. I'd forgotten they also used them as occulders, that was a nice idea too.

    Yeah thats the bit I don't get, do you literally mean have the imposters rendered from the same shader? If so how do you know where to render them and apply any transformations needed? If not in the same shader, then I can see the blending working, though do you then have a pass per image proxy? Its probably real easy I just can't wrap my head around it for some reason.

    Please post any progress you make on this, would be interesting to see what can be done in Unity. BPCEM on its own is pretty amazing, had so much fun using that, but getting an idea how to do image proxies would be icing on the cake.
     
  48. HonoraryBob

    HonoraryBob

    Joined:
    May 26, 2011
    Posts:
    1,056
    [QUOTE="Dev.D1, post: 1691836, member: 31069" I want to try this ![/QUOTE]

    I was also thinking of attempting something like this, since it would work even for outdoor scenes so long as you could find a way to store the location of all reflectable objects along rivers, lakes, etc. But I'm not good at anything related to cubemapping - I mostly write procedural shaders that generate patterns - so I don't know if I could figure it out. If you get something to work, I'm hoping you'll post it here. :)
     
  49. Dev.D1

    Dev.D1

    Joined:
    Dec 29, 2010
    Posts:
    99
    @HonoraryBob Sure! I have limited time at the moment but I intend to do a full implementation based on Sébastien Lagarde's work. Would be awesome for my portfolio. I am not sure if I'd do it in Unity3D though - I need a similar intensive experiment to work on and justify my UE4 subscription :) . But if I do it all in Unity3D, I will definitely want to use Lux , another thing I really want to check out.

    I still think using a standard planar reflection using unity's mirror script will give you the best results in a production environment. Anything else has a lot more steps involved, consequently more stuff to get wrong, Especially for your river example. I am not convinced using cubemaps workflow is suitable for you. If you care, send me a rough idea what you want to achieve and the look and feel of your project.

    However, you can and should experiment with image proxies/imposters ! You can use image proxies with planar reflections like the mirror script straight away, imagine getting lights and other highlights to pop in the water. I'll see if I can get a quick framework for that . Any art assets you can share would also be welcome.

    @noisecrime In Remember Me it seems they used forward rendering, so i'm just going to defer thinking of deferred for the moment (pun). I think having a separate render pass for each image proxy/ imposter is counter productive. It'd imagine I'd aggregate all active proxies and render them together in one pass (or get the engine to do it for me) . You know what? I'll just have a look at the mirror script and see how that's set up and see if we can get something running. This should be fairly easy. Once that's done I'll see if we can get that to work with BPCEM. How's that ?
     
  50. Noisecrime

    Noisecrime

    Joined:
    Apr 7, 2010
    Posts:
    1,505
    Sounds cool, but don't feel pressured, I just wanted you to know that if you did do any work on it i'd be interested to see what results you got. I would take a look myself but i'm deep into a different pet project shader at the moment.