UI and masking 3d meshes in a scrollrect

You guys may want to try it out

. Though, I'm not entirely sure how we can add it onto the UI.

hi phil, can you guys update us on what we can do in the URP and HDRP? the solution posted here doesn't seem to work on the new render pipelines.

I'm getting an error from the shader:

Code (CSharp):

1. Shader error in 'Universal Render Pipeline/Simple Lit': Couldn't open include file 'Packages/com.unity.render-pipelines.universal/Shaders/SimpleLitInput.hlsl'. at line 134

 

Proper support of URP is coming. The basics have worked since day 1 but anything requiring lighting needs to be redone and they've just not gotten around to supporting it yet. I'm working with on of their devs to get URP full support into a release soon.

As for you issue i'm not 100% sure as i'm not fully informed on how URP shaders are working.

 

I would love an update on this for URP, if there's a solution?
I want to make a scrollrect with 3D objects.
Thanks!

 

Hey All!
I approached this question in a rather non-standard way. I hide the 3D meshes if they are not visible in the viewport. Maybe someone will come in handy:

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using UnityEngine;
4.  
5. public class ScrollItemCtl : MonoBehaviour
6. {
7.     public RectTransform scrollViewViewport;
8.     public GameObject go3dMesh;
9.  
10.     Vector4 offset = new Vector4( 0, 17, 0, 20);
11.  
12.     private void Update()
13.     {
14.         if (go3dMesh != null)
15.             go3dMesh.SetActive(RectTransformUtility.RectangleContainsScreenPoint(scrollViewViewport, transform.position, null , offset));
16.     }
17. }

Oh yeah .. you can add offsets to fine tune the hiding of objects.

 

Couldn't find any news on this so here's something I made for my needs. It renders a mesh using the Canvas Renderer component and allows said mesh to be masked just like any other UI element, as long as its material shaders are set up correctly. (Tested on URP 2021.3)

I believe this is everything you need to add to your shader(s) in order for masking to work:

Code (CSharp):

1. Shader "UI/Maskable"
2. {
3.     Properties
4.     {
5.  
6.         _StencilComp("Stencil Comparison", Float) = 8
7.         _Stencil("Stencil ID", Float) = 0
8.         _StencilOp("Stencil Operation", Float) = 0
9.         _StencilWriteMask("Stencil Write Mask", Float) = 255
10.         _StencilReadMask("Stencil Read Mask", Float) = 255
11.  
12.         _ColorMask("Color Mask", Float) = 15
13.  
14.     }
15.  
16.     SubShader
17.         {
18.  
19.                 Stencil
20.                 {
21.                     Ref[_Stencil]
22.                     Comp[_StencilComp]
23.                     Pass[_StencilOp]
24.                     ReadMask[_StencilReadMask]
25.                     WriteMask[_StencilWriteMask]
26.                 }
27.  
28.                 ZWrite Off
29.                 ZTest[unity_GUIZTestMode]
30.                 ColorMask[_ColorMask]
31.  
32.                 Pass
33.                 {
34.  
35.                     // ...
36.  
37.                 }
38.  
39.         }
40. }

Or you can use an existing shader that's already compatible with masking.

Now simply add the UIConvertMeshRenderer component to a game object that is within a UI canvas hierarchy. Make sure it has a MeshFilter and MeshRenderer component attached, and that they include the desired mesh and material(s).

A few things to note:

- You'll probably need to set the transform scale of your mesh pretty high when it's in a canvas.

- I only tested this on a canvas using Screen Space - Camera.

- Meshes used for this must have the read/write flag enabled.

- I didn't need my materials to be lit, so no idea if lighting is possible for this.

- If you allow the script to create a control parent transform, you should be able to use its anchors and such to manipulate the mesh just like any other UI element.

 

This looks awesome, thanks!

 

Use such settings and archive with all files from Allen0012. Don't forget to add a Mask component for the parent object

 

For those using URP, I solved this by adding two features to the Forward Renderer: the first one would render the masking object (e.g. a panel with alpha = 0) and would replace the stencil buffer with value 1. The second feature would render the 3D object to be masked, but only if the stencil buffer was greater than 1. Of course for this to work you need to create two layers (one for the mask, another one for the masked) and the new features must render these layers (and the main renderer must not render these layers). I'm not at work now, but if you need more details just text me.

 

After almost 7 years since my original post with the solution I found myself needing to mask 3D meshes again and had completely forgotten about this thread, searched Google, lo and behold.. it is still not natively supported and my post to the rescue!

I've come back to post a slightly improved version that is more of a drag & drop for the non-technical folks.

Improvements:

• Supports multiple renderers under the same GameObject hierarchy
• Supports SkinnedMeshRenderers as well as MeshRenderers
• No manual addition of components or adjusting of shaders

The requirements are still the same:

• You need a UI Mask on a parent element setup properly that works for 2D UI elements
  
• You need a Screen Space - Camera canvas

I've also added these two little additions to the shader to support double-sided mesh as well as more complex mesh that might clip into itself

Remember that if you enable ZWrite Off you need to adjust the World Space z of your UI and 3D elements to get the desired outcome.

Just add UIMaskableMesh component to the root of the mesh you are trying to mask.

 

I can't get this to work, I followed every instruction but nothing. Can you show it step by step? do I need to add a new material to my meshes? can the script be in a parent object that has no mesh? what do I do with the shader?

 

@francismoy I would be very interested in a more detailed explanation of your approach. It seems like the "correct" way using URP. I have never worked with Renderer Features before so I am bit lost here

 

Hi! Ok, so I'm going to try to explain the different things I did with more detail:

- In the Forward Renderer asset I created two Renderer Features, let's call them RF1 and RF2:
RF1 has the layer mask (e.g. with name LayerMask1) of the UI element that is going to mask the 3D object. That is, in the Canvas, the UI element that is going to mask the object (e.g. an Image) is assigned to layer mask LayerMask1.
RF2 has the layer mask where the masked object is, e.g. with name LayerMask2. That is, say you want to mask a 3D model of a dog: you'd place the prefab of that dog in LayerMask2.

- RF1 has stencil buffer enabled (stencil checkbox enabled) with value 1 and Compare Function = Always, with Pass Replace and the rest of values to Keep. This will make sure that the UI element is going to write to the stencil buffer in the area of the screen where it's rendered.
- RF2 has stencil buffer enabled (stencil checkbox enabled) with value 1 and Compare Function = Greater (the rest with Keep). This will make sure that the 3D object will only be rendered in areas where the stencil buffer is different than 1. That is, when a portion of the 3D object covers the area where the UI element wrote to the stencil buffer, that portion won't be rendered.

Finally, don't forget to add the new created layer masks (LayerMask1 and LayerMask2) to the Culling Mask of the camera that is rendering the game at that point.

Depending on your configuration of cameras and complexity of the UI/Canvas, you might need to experiment a little with assigning the different objects to different layers, but this is the general idea with the most important points. Don't hesitate to ask if something is not clear and you don't get the results you want.

 

For anyone who are looking solution to this. I have used replied comment's method but struggled to find suitable 3D maskable mesh shader. So I have modified unity's default UI shader to render 3D mesh with masking support (even RectMask2D). This shader supports softness of the RectMask too. Feel free to use. Tested on Unity 2022.3.16, URP.

Spoiler: If somehow the file is broken, here is the code

Code (CSharp):

1. Shader "UI/Default-Front"
2. {
3.     Properties
4.     {
5.         [MainTexture] _MainTex ("Sprite Texture", 2D) = "white" {}
6.         _Color ("Tint", Color) = (1,1,1,1)
7.  
8.         _StencilComp ("Stencil Comparison", Float) = 8
9.         _Stencil ("Stencil ID", Float) = 0
10.         _StencilOp ("Stencil Operation", Float) = 0
11.         _StencilWriteMask ("Stencil Write Mask", Float) = 255
12.         _StencilReadMask ("Stencil Read Mask", Float) = 255
13.  
14.         _ColorMask ("Color Mask", Float) = 15
15.  
16.         [Toggle(UNITY_UI_ALPHACLIP)] _UseUIAlphaClip ("Use Alpha Clip", Float) = 0
17.     }
18.  
19.     SubShader
20.     {
21.         Tags
22.         {
23.             "Queue"="Transparent"
24.             "IgnoreProjector"="True"
25.             "RenderType"="Transparent"
26.             "PreviewType"="Plane"
27.             "CanUseSpriteAtlas"="True"
28.         }
29.  
30.         Stencil
31.         {
32.             Ref [_Stencil]
33.             Comp [_StencilComp]
34.             Pass [_StencilOp]
35.             ReadMask [_StencilReadMask]
36.             WriteMask [_StencilWriteMask]
37.         }
38.  
39.         Cull Back
40.         Lighting Off
41.         ZWrite Off
42.         ZTest [unity_GUIZTestMode]
43.         Blend SrcAlpha OneMinusSrcAlpha
44.         ColorMask [_ColorMask]
45.  
46.         Pass
47.         {
48.             Name "Default"
49.         CGPROGRAM
50.             #pragma vertex vert
51.             #pragma fragment frag
52.             #pragma target 2.0
53.  
54.             #include "UnityCG.cginc"
55.             #include "UnityUI.cginc"
56.  
57.             #pragma multi_compile_local _ UNITY_UI_CLIP_RECT
58.             #pragma multi_compile_local _ UNITY_UI_ALPHACLIP
59.  
60.             struct appdata_t
61.             {
62.                 float4 vertex   : POSITION;
63.                 float4 color    : COLOR;
64.                 float2 texcoord : TEXCOORD0;
65.                 UNITY_VERTEX_INPUT_INSTANCE_ID
66.             };
67.  
68.             struct v2f
69.             {
70.                 float4 vertex   : SV_POSITION;
71.                 fixed4 color    : COLOR;
72.                 float2 texcoord  : TEXCOORD0;
73.                 float4 worldPosition : TEXCOORD1;
74.                 half4  mask : TEXCOORD2; // Required for rectmask softness
75.                 UNITY_VERTEX_OUTPUT_STEREO
76.             };
77.  
78.             sampler2D _MainTex;
79.             fixed4 _Color;
80.             fixed4 _TextureSampleAdd;
81.             float4 _ClipRect;
82.             float4 _MainTex_ST;
83.  
84.             // Required parameters to use rectmask softness
85.             float _UIMaskSoftnessX;
86.             float _UIMaskSoftnessY;
87.  
88.             v2f vert(appdata_t v)
89.             {
90.                 v2f OUT;
91.                 UNITY_SETUP_INSTANCE_ID(v);
92.                 UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(OUT);
93.                 OUT.worldPosition = v.vertex;
94.                 OUT.vertex = UnityObjectToClipPos(OUT.worldPosition);
95.  
96.                 OUT.texcoord = TRANSFORM_TEX(v.texcoord, _MainTex);
97.  
98.                 // Rect mask softness calculations
99.                 float2 pixelSize = OUT.vertex.w;
100.                 pixelSize /= float2(1, 1) * abs(mul((float2x2)UNITY_MATRIX_P, _ScreenParams.xy));
101.  
102.                 float4 clampedRect = clamp(_ClipRect, -2e10, 2e10);
103.                 float2 maskUV = (v.vertex.xy - clampedRect.xy) / (clampedRect.zw - clampedRect.xy);
104.                 OUT.texcoord = TRANSFORM_TEX(v.texcoord.xy, _MainTex);
105.                 OUT.mask = half4(v.vertex.xy * 2 - clampedRect.xy - clampedRect.zw, 0.25 / (0.25 * half2(_UIMaskSoftnessX, _UIMaskSoftnessY) + abs(pixelSize.xy)));
106.                 // End of Rect mask softness calculations
107.  
108.                 OUT.color = v.color * _Color;
109.                 return OUT;
110.             }
111.  
112.             fixed4 frag(v2f IN) : SV_Target
113.             {
114.                 half4 color = (tex2D(_MainTex, IN.texcoord) + _TextureSampleAdd) * IN.color;
115.  
116.                 #ifdef UNITY_UI_CLIP_RECT
117.                 // color.a *= UnityGet2DClipping(IN.worldPosition.xy, _ClipRect); // Default code, doesn't work with RectMask softness
118.                 half2 m = saturate((_ClipRect.zw - _ClipRect.xy - abs(IN.mask.xy)) * IN.mask.zw);
119.                 color.a *= m.x * m.y;
120.                 #endif
121.  
122.                 #ifdef UNITY_UI_ALPHACLIP
123.                 clip (color.a - 0.001);
124.                 #endif
125.  
126.                 return color;
127.             }
128.         ENDCG
129.         }
130.     }
131. }
132.