Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Applying decal texture at runtime without planar projection

Discussion in 'General Graphics' started by MaeL0000, Jun 17, 2019.

  1. MaeL0000

    MaeL0000

    Joined:
    Aug 8, 2015
    Posts:
    35
    I'm trying to achieve a system similar to what is shown in this video:


    Basically a way to apply textures on meshes at runtime. I've tried using tools from the Asset Store like EasyDecal and UVPaint but they both don't allow for the decal texture to wrap around the mesh, instead of just doing a planar projection. (the shortcomings of decal projections are succinctly explained in this clip:
    )

    What would the correct technique be to recreate something like this?
     
  2. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    9,268
    APB is doing regular projection mapping, using a combination of cylindrical, and planar projections. The difference is it's doing that projection against that static pose you see during placement and baking it to a texture that gets used on the live animated mesh.

    You need a mesh with unique per surface UVs. Basically no part of the UVs can overlap. Then you need to render your decals in UV space, either by having a textures with the world positions and normals baked into it that you project onto in a shader, or by rendering the character mesh in UV space every time.

    Then there will be issues of seams, and handling masking, etc. Understand that the devs for APB stated that half their staff and man hours spent on the game were for making customization work, and the projection is not a small part of that task.
     
    xVergilx likes this.
  3. MaeL0000

    MaeL0000

    Joined:
    Aug 8, 2015
    Posts:
    35
    I've tried searching for cylindrical projection mapping with no results. Doing the projection on a static mesh and baking it is fine for my use case, but at this point I really can't figure out how to make a cylindrical projector and am kind of freaked out that the internet doesn't know what I'm talking about, always a bad omen that signals that I'm out of my depth. Is that what you're trying to tell me too?
     
  4. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    9,268
    If you search for "cylindrical projection" you'll find hundreds if not thousands of sites talking about it... just maybe not in the context of shaders. You'll find a bunch of sites talking about the math of cartography, or map making. Here's a hint: those sites are talking about the same thing as what I mentioned.

    Basic planar / orthographic projection uses a transform matrix to transform a position from the local or world position to the projection relative position. You then use the projection space x and y coordinates to determine the texture's UV, and the Z can be used for depth fading, or at least clipping the depth range the projection applies to. Basically you have a box that you're finding the original mesh's relative position within.

    Cylindrical projection uses the same kind of transform matrix, but instead of using the x and y as the UVs, you're using the x and y (or any two components) together to calculate the angle around a cylinder with the third component as the axis height.

    Code (csharp):
    1. // a world to local transform matrix with a 1 unit box with a center pivot as the preview
    2. float4 decalPos = mul(worldToDecalMatrix, float4(worldPos.xyz, 1.0));
    3.  
    4. // cylindrical projection
    5. float2 cylindricalDecalUV = saturate(float2(atan2(decalPos.y, decalPos.x), decalPos.z + 0.5));
    6. // if radius position is > 0.5 units away from z axis, clip
    7. clip(0.25 - dot(decalPos.xy, decalPos.xy));
    8.  
    9. // planar projection
    10. float2 planarDecalUV = saturate(float2(decalPos.xy + 0.5));
    11. // if z position is > 0.5 untis away from pivot, clip
    12. clip(0.5 - abs(decalPos.z));
     
  5. MaeL0000

    MaeL0000

    Joined:
    Aug 8, 2015
    Posts:
    35
    I tried writing a shader:
    Code (CSharp):
    1.  
    2. Shader "AddTattoo" {
    3. Properties {
    4.      _MainTex ("Base (RGB) Trans (A)", 2D) = "white" {}
    5. }
    6. SubShader {
    7.      Tags {"Queue"="Transparent" "IgnoreProjector"="True" "RenderType"="Transparent"}
    8.      LOD 100
    9.    
    10.      ZWrite Off
    11.      Blend SrcAlpha OneMinusSrcAlpha
    12.    
    13.      Pass {
    14.          CGPROGRAM
    15.              #pragma vertex vert
    16.              #pragma fragment frag
    17.  
    18.              #include "UnityCG.cginc"
    19.  
    20.              float4x4 worldToDecalMatrix;
    21.              struct appdata_t {
    22.                  float4 vertex : POSITION;
    23.                  float2 texcoord : TEXCOORD0;
    24.              };
    25.              struct v2f {
    26.                  float4 vertex : SV_POSITION;
    27.                  half2 texcoord : TEXCOORD0;
    28.              };
    29.  
    30.              sampler2D _MainTex;
    31.              float4 _MainTex_ST;
    32.            
    33.              v2f vert (appdata_t v)
    34.              {
    35.                  float3 worldPos;
    36.                  v2f o;
    37.                  worldPos = mul (unity_ObjectToWorld, v.vertex);
    38.                  float4 decalPos = mul(worldToDecalMatrix, float4(worldPos.xyz, 1.0));
    39.                  float2 cylindricalDecalUV = saturate(float2(atan2(decalPos.y, decalPos.x), decalPos.z + 0.5));
    40.                  // can't use clip because generates following error: Shader error in 'AddTattoo': cannot map expression to vs_4_0 instruction set at line 50 (on d3d11)
    41.                  //clip(0.25 - dot(decalPos.xy, decalPos.xy));
    42.                  o.vertex = UnityObjectToClipPos(v.vertex);
    43.                  o.texcoord = cylindricalDecalUV;
    44.  
    45.                  return o;
    46.              }
    47.            
    48.              fixed4 frag (v2f i) : SV_Target
    49.              {
    50.                  fixed4 col = tex2D(_MainTex, i.texcoord);
    51.                  return col;
    52.              }
    53.          ENDCG
    54.      }
    55. }
    56. }
    worldToDecalMatrix is calculated and passed as input through the following script:
    Code (CSharp):
    1. using UnityEngine;
    2.  
    3. public class TattooSetDecalMatrix : MonoBehaviour
    4. {
    5.     public Material tattooMaterial;
    6.     private Renderer renderer;
    7.  
    8.     public void Start()
    9.     {
    10.         renderer = GetComponent<Renderer>();
    11.     }
    12.  
    13.     public void Update()
    14.     {
    15.         Matrix4x4 m = renderer.worldToLocalMatrix;
    16.         tattooMaterial.SetMatrix("worldToDecalMatrix", m);
    17.     }
    18. }
    The shader is in a material applied to the Mesh onto which I want to project the texture.
    The script is attached to a cylinder GO that envelopes a limb of the mesh.
    Rotating and moving the cylinder changes the projection onto the limb. It's working quite nicely for a PoC, although the texture is getting projected onto both sides of the mesh and the seems are causing problems as you said. The texture's tiling and offset parameters aren't working as expected anymore though, changing them to anything other than (1, 1) & (0, 0) makes the texture warp and disappear, so I'm not sure how to scale the texture dimensions.
     
  6. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    9,268
    The clip() function only works in a fragment shader. Really most of that code should be run only in the fragment shader as cylindrical projection won't interpolate properly. If you want to do some of the work in the vertex shader, calculate the decalPos there and pass it onto the fragment shader as a float3 value. Alternatively you could try passing the the value used inside of clip() as a Z component of the "texcoord" (making that a float3), but you're guaranteed to get some weird distortions as, again, cylindrical values will not interpolate properly.

    Well, in the shader code above it shouldn't be doing anything since you're not calling TRANSFORM_TEX or otherwise using _MainTex_ST anywhere. However the way to apply the scale / offset would be to apply it to the calculated UVs. Again, probably best done in the fragment shader instead of the vertex shader before supplying the UVs to the tex2D() function.

    The TRANSFORM_TEX macro is super cheap, and the only reason why Unity does it in the vertex shader at all is mostly due to legacy hardware reasons, like >6 year old phones and 15 year old desktop GPUs.

    One other note. The shader code only clips along the axis that the texture UVs don't cover. The example code I provided assumes the texture itself wasn't going to extend outside of the 0.0 to 1.0 range of the UVs. If you want to fade out past that you'll want to add more code to fade out the edges or otherwise clip outside of the UV range.
     
  7. MaeL0000

    MaeL0000

    Joined:
    Aug 8, 2015
    Posts:
    35
    Ok so I moved almost everything into the fragment shader:

    Code (CSharp):
    1.  Shader "AddTattoo" {
    2.     Properties {
    3.         _MainTex ("Base (RGB) Trans (A)", 2D) = "white" {}
    4.     }
    5.  
    6.     SubShader {
    7.         Tags {"Queue"="Transparent" "IgnoreProjector"="True" "RenderType"="Transparent"}
    8.         LOD 100
    9.      
    10.         ZWrite Off
    11.         Blend SrcAlpha OneMinusSrcAlpha
    12.      
    13.         Pass {
    14.             CGPROGRAM
    15.                 #pragma vertex vert
    16.                 #pragma fragment frag
    17.  
    18.                 #include "UnityCG.cginc"
    19.  
    20.                 float4x4 worldToDecalMatrix;
    21.  
    22.                 struct appdata_t {
    23.                     float4 vertex : POSITION;
    24.                     float2 texcoord : TEXCOORD0;
    25.                 };
    26.  
    27.                 struct v2f {
    28.                     float4 vertex : SV_POSITION;
    29.                     half2 texcoord : TEXCOORD0;
    30.                     float3 worldPos : TEXCOORD8;
    31.                 };
    32.  
    33.                 sampler2D _MainTex;
    34.                 float4 _MainTex_ST;
    35.              
    36.                 v2f vert (appdata_t v)
    37.                 {
    38.                     v2f o;
    39.                     o.vertex = UnityObjectToClipPos(v.vertex);
    40.                     o.worldPos = mul (unity_ObjectToWorld, v.vertex);
    41.                     o.texcoord = v.texcoord;
    42.                     return o;
    43.                 }
    44.              
    45.                 fixed4 frag (v2f i) : SV_Target
    46.                 {
    47.                     float4 decalPos = mul(worldToDecalMatrix, float4(i.worldPos.xyz, 1.0));
    48.                     clip(0.25 - dot(decalPos.xy, decalPos.xy));
    49.                     float2 cylindricalDecalUV = saturate(float2(atan2(decalPos.y, decalPos.x), decalPos.z + 0.5));
    50.                     i.texcoord = cylindricalDecalUV;
    51.                     i.texcoord = TRANSFORM_TEX(i.texcoord, _MainTex);
    52.                     fixed4 col = tex2D(_MainTex, i.texcoord);
    53.                     return col;
    54.                 }
    55.             ENDCG
    56.         }
    57.     }
    58. }
    and it seems to be working. The projected texture comes out a bit warped. What are the factors in play here? Is it about the receiving model's uv mapping? I'll make a test scene and come back to post a video and zipped asset for anyone interested in this discussion.
     
  8. MaeL0000

    MaeL0000

    Joined:
    Aug 8, 2015
    Posts:
    35
    Here's the Unity package built with Unity3D 2019.3.0a4. Just run in editor, switch to Scene view and rotate/move the cylinder around.

    There's still a lot of things I'm not understanding, like why changing the cylinder scale really messes up the projection and that whole thing about cliping outside the UV range. Also, what's the difference between this whole method and simply using a GO on the cylinder to do a perpendicular raycast onto the mesh, using RaycastHit.textureCoord to get the uv coordinates and then using those in a decal material applied to the mesh as the offset for the decal texture?
     

    Attached Files:

  9. Chaiker

    Chaiker

    Joined:
    Apr 14, 2014
    Posts:
    50
    If you will use uv just as center of decal, you will get pain on uv seams.
     
  10. MaeL0000

    MaeL0000

    Joined:
    Aug 8, 2015
    Posts:
    35
    @bgolus the decal still doesn't seem to be projected cylindrically, as seen here: https://pasteboard.co/In0WifU.png
    Am I messing up the implementation or does the formula for finding the decal's UVs need some working on?
     
  11. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    9,268
    Looks correct to me. However the code I provided is doing the UVs for a cylinder with its axis around z, but the default Unity cylinders use y, so your expectations are probably being subverted a bit.

    try:
    saturate(float2(atan2(decalPos.z, decalPos.x), decalPos.y + 0.5));

    Also atan2 returns a value with a range between -pi and pi, so you probably actually want:
    saturate(float2(atan2(decalPos.z, decalPos.x) / (0.5 * UNITY_PI) + 0.5, decalPos.y + 0.5));
     
  12. MaeL0000

    MaeL0000

    Joined:
    Aug 8, 2015
    Posts:
    35
    Ok so searching for finding uv on a cylinder and cylindrical coordinates on google gave a lot of good reading material.

    Based off of what I read, the right formula for my case seems to be:

    float2 cylindricalDecalUV = (float2( (atan2(decalPos.z, decalPos.x) + pi) / (2*pi), decalPos.y + 0.5));


    adding pi translates the result from [-pi, pi] to [0, 2pi], dividing it by 2pi then normalizes the value to the [0,1] range so there you have your u. Also, as you said the z and y axis coordinates need to be swapped in my case because of how it's set up.

    This seems to be working great! The texture is now wrapping all the way around:
    https://pasteboard.co/In9yvzDT.png

    I think this is pretty good for now, next step is figuring out how to bake the decal into the texture on the mesh!
     
    bgolus likes this.
  13. grejtoth

    grejtoth

    Joined:
    Aug 7, 2015
    Posts:
    10
    Hi @MaeL0000 , want to know if you had success in baking the decal into a Texture? Thanks!
     
    Last edited: Oct 15, 2019
  14. MaeL0000

    MaeL0000

    Joined:
    Aug 8, 2015
    Posts:
    35
    No, I haven't had the need to tackle the problem yet, but will have to get to it in the near future. I'll try to remember to update this thread with insights and probably questions.
     
  15. grejtoth

    grejtoth

    Joined:
    Aug 7, 2015
    Posts:
    10
    Okay! Thanks! I'll try it myself too and if I came up with something I'll post it here!
     
unityunity