Rendering inside a "fishtank"

Hi, I'm working on a game where I want to only render geometry that sits inside the confines of 6 sided box, somewhat like a fishtank. Any geometry outside the tank should be invisible. I will be using Unity Pro with Windows. The effect I want is like the example on the right in this image :-

http://imgur.com/bWaDXJp
The objects that might intersect the tank it are animated, so I need to do this at runtime.

Maybe I could do this by checking all geometry for an intersection every frame. But presumably it could be very slow. Is it possible to do something like this using a shader? I'm not asking for a full solution, just suggestions that I could research.

Thanks for any suggestions.

 

Create a box.
Add script to it.
In the script:
Send "world to object" matrix to shader.
In the vertex shader:
Transform vertex position from object space to world, then from world to the space from sended in previous step matrix.
"Send" resulting position to fragment shader.
In fragment shader:
Discard fragment if any component of position(absolut value from any axis) greater then 0.5.
---------
Here it's almost what you want.

Code (csharp):

1.  
2. usingUnityEngine;
3. usingSystem.Collections;
4.  
5. [ExecuteInEditMode]
6. publicclass SetDecalMatr : MonoBehaviour
7. {
8.    publicTransform proj;
9.  
10.    void Update()
11.    {
12.         SetMatr();
13.    }
14.  
15.    [ContextMenu("set matrix")]
16.        void SetMatr()
17.    {
18.         Matrix4x4 mat = proj.worldToLocalMatrix*transform.localToWorldMatrix;
19.        renderer.sharedMaterial.SetMatrix("_DecalMatr", mat);
20.        }
21. }
22.  

Code (csharp):

1.  
2. Shader "ProjectedDecal"
3. {
4.    Properties
5.    {
6.       _MainTex ("Texture Image", 2D)="white"{}
7.       _DecalTex ("DecalTex Image", 2D)="white"{}
8.    }
9.    SubShader
10.    {
11.       Pass
12.          {
13.            
14.          CGPROGRAM
15.                  #pragma only_renderers d3d9 gles
16.          #pragma glsl
17.          #pragma vertex vert  
18.          #pragma fragment frag
19.  
20.          uniformsampler2D _MainTex;  
21.          uniformsampler2D _DecalTex;  
22.          uniform float4x4 _DecalMatr;  
23.  
24.          struct vertexInput
25.                  {
26.             float4 vertex : POSITION;
27.             float2 texcoord : TEXCOORD0;
28.          };
29.          struct vertexOutput
30.                  {
31.             float4 pos : SV_POSITION;
32.             float3 pTex : TEXCOORD1;
33.             float2 tex : TEXCOORD0;
34.          };
35.  
36.          vertexOutput vert(vertexInput input)
37.          {
38.             vertexOutput output;
39.  
40.                         float4 trasformed = mul(_DecalMatr,input.vertex);
41.            output.pTex= trasformed.xyz+float3(0.5, 0.5, 0);
42.  
43.                        output.tex=input.texcoord;
44.  
45.            output.pos= mul(UNITY_MATRIX_MVP,input.vertex);
46.            returnoutput;
47.          }
48.  
49.          float4 frag(vertexOutput input): COLOR
50.          {
51.                         float4 pc = tex2D(_DecalTex, float2(input.pTex));
52.                        if(input.pTex.z<-0.5 ||input.pTex.z>0.5)
53.                        {
54.                                 pc.a=0;
55.                        }
56.  
57.                         float4 c = tex2D(_MainTex,input.tex);
58.  
59.            return lerp(c, pc, pc.a);
60.          }
61.  
62.          ENDCG
63.      }
64.    }
65. }
66.  
67.  

 

Stencils might be worth a look as well.

 

You could do something like this:
- Draw the geometry that goes into the box normally.
- Draw the box with no color or z writes, no backface culling and a xor stencil. (So the stencil buffer is 1 if the geometry is inside the box and 0 if outside the box.)
- Put a background color in where the stencil buffer is 0. You could also draw a square at "infinite" distance without z check where the stencil buffer is 0. That way you'd reset the z buffer and you can continue drawing other objects into the scene.

Just rejecting pixels in the pixel shader is easier if you only need a few shaders for the objects inside the box. Using the stencil buffer gives you the freedom to pretty much use any existing shader on the object inside the box. (As long as they do a z write.)

 

if you can use the same shader on all objects visible in the tank, you can just do... if pixel worldspace min max is in box, alpha = alpha, else alpha = 0. will make everything outside box transparent.

 

Transparent, but still writing to the z buffer if not rejected. I'd prefer to use the clip instruction (in the pixel shader.) Which can work on a vector.

Code (csharp):

1.  
2. clip(pos-min);
3. clip(max-pos);
4.  

Where pos, min and max are float3's.

 

Could you explain this a bit more, the XOR stencil?

Is the background color where the stencil buffer is 0 what would create the effect of only seeing the geometry in the box?

If I draw a quad without z check where the stencil buffer is 0, would that be drawn in something like OnPostRender for example?

 

The exclusive or method is described very shortly here:
http://en.wikipedia.org/wiki/Shadow_volume#Exclusive-or

What it comes down to is that you flip the stencil buffer when you go into the volume and flip it again when you go out. If you "hit" any geometry while inside the volume, this will leave the stencil buffer valued at 1. In code:

Code (csharp):

1.  
2. Stencil {
3.   ReadMask 1
4.   WriteMask 1
5.   Comp always
6.   Pass invert
7. }
8.  

Then you can draw a plane very far away where the geometry is outside the volume to reset the color and depth:

Code (csharp):

1.  
2. ZTest Always
3.  
4. Stencil {
5.   Ref 1
6.   Comp notEqual
7. }
8.  

Yes, the rest will be cleared.

No, you do that after the fishtank and before the rest of the rendering:
Opaque-3: Fishtank objects
Opaque-2: Stencil volume
Opaque-1: Clearing plane
Opaque: Main rendering