Official Ground shader

I've been working on a prototype ground shader based on Joyce [Minions Art] patreon tutorials.
Unity Polybrush and Shaders
ShaderGraph Conversion - Vertex Colors(Specular/Texture Painting)

So far I think I have been able to replicate the base functionality but I'm not sure if I've added vertex color support properly, if at all.

Here is my main graph:


And the ground subgraph:


And here is the current functionality:

This is a result of the ground mesh's geometry in the TestingGround scene.


Last night I was looking into how to add support for vertex colors for shader graph (or, if I already have vertex color support, to eliminate the warning message in the polybrush window) but I haven't yet found an answer.

I use the ToonShading subgraph because I receive this error when I use the internal lighting subgraphs:


To avoid this error I made the ground shader a shader add-on to use with the ToonShading subgraph. During my process I decided to move the outline subgraph out of the main ToonShading subgraph as the ground shader is not outlined. I made this change in all the shader variants using the outline.

The ground shader would use the combined diffuse lighting from the ToonLightingModel and AdditionalLightsToon subgraphs but ignore specular light. Ideally I can use the internal subgraphs and avoid unused properties such as Specular Map or Specular Color.

 

@cirocontinisio
Here's a small update:

I made a duplicate test scene and subdivided the ground mesh a few times.

Edit: Oops! Looks like I had the noise scale set to 0. Here's an image with noise at 0.1.

 

I'll look into that chromatic aberration. Right now I'm merging the current project into my fork and then I can make a pull request. I can also work on the ground debug shader.

Any idea why using the following graph setup returns this error?

Parse error: syntax error, unexpected $end, expecting TOK_SHADER

I've also seen this error pop up:

Error in Graph at Assets/Ground Shader Test/Toon_Ground.shadergraph at node Unlit Master: Graph is generating too many variants. Either delete Keywords, reduce Keyword variants or increase the Shader Variant Limit in Preferences > Shader Graph.

UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr)

 

Here's the debug shader:

Just added a boolean keyword:


Edit: Since one of the requirements of the debug shader was to enable the visualization of individual channels, I could replace this boolean keyword with the W channel of a Vector 4 property I would use as a channel mixer.

I'd prefer to display this in the inspector as checkboxes (as the codecks card recommends) and I could look into some custom shader GUI to achieve that. Even though the debug feature will be editor-only I'd like to avoid unnecessary booleans. Of course I could also display this as one Boolean with three vector 1 sliders.

 

@cirocontinisio I've removed the ghosting effect that was happening along the contours. I was able to ask Joyce if she had any ideas what might be the cause and she noticed that I was using thevector 4 output from the triplanar's noise texture. Adding a split node to use only one channel fixed this.

I can put together a pull request but I had thought about making a custom shader GUI to better communicate how to use the vertex color debug function.

 

Pull request is up!
Demo video:

 

I didn't found any relevant threads for this, so I'm opening this thread to work on it
This is the card in Codecks
https://open.codecks.io/unity-open-project-1/decks/15-code/card/19s-ground-material-detector
And a reminder that there is another thread for the ground shader which is related to this task
https://forum.unity.com/threads/ground-shader.1041673/

So,
As the card suggests, I was thinking of adding a script for the player to make a raycast to the ground, get the mesh color from the collider and return a value according to a dictionary matching the colors with the material,
also, we have the case of rocks and other materials that don't depend on the color of the mesh, so for these ones we would be returning the name, the tag, or maybe the material itself could be mapped into the dictionary too
and for the "run every x frames" requirement I was wondering if an if in the update function with a serialize variable would be enough or if we need to implement something using InvokeRepeating or something like that

well that my initial thoughts for this task, let me know what you all think of this.

 

Hey! I found this post, I didn't test it or anything and I'm not sure if we can use it because it's using the Terrain component and we aren't, but perhaps you'll find it useful. The idea seems pretty clever, by getting the alpha maps at a specific location, it's then using those values to blend the footstep sounds together according to the textures. I really don't know much about this topic so I have no idea if this is something we can do or if it's specific to the Terrain component, just throwing it out there.

I think we should use the same animation event we already have to play the footstep sounds and put the ground detection logic there, that way every footstep will have the correct sound, what do you think?

 

Terrain is entirely different. The alphamaps are basically a 3 dimensional array, one dimension for the different terrain textures and two dimensions for the position (without height). It means you can calculate the index in the array from the player position.

In our case the terrain is a mesh and the texture information is in the vertex colors. Conveniently if we use Raycast on it's mesh collider, the raycast hit also returns the hit triangle index, which is an array of integers in which the indices of the vertices come in groups of 3.

Essentially you would get the 3 vertex colors this way:
Color vertexColor1 = mesh.colors[mesh.triangles[hit.triangleIndex * 3 + 0]];
Color vertexColor2 = mesh.colors[mesh.triangles[hit.triangleIndex * 3 + 1]];
Color vertexColor3 = mesh.colors[mesh.triangles[hit.triangleIndex * 3 + 2]];

Typically you would define local variables to store mesh.colors and mesh.triangles, yet I don't know whether that is efficient concerning memory usage.

 

Thanks for the explanation, @Smurjo!

I didn't know about that property in the raycast hit, that is indeed convenient. I don't think memory would be an issue there, we only need to store the info of the mesh we're currently standing on.

 

ok as an update on this task
@Smurjo i tried your idea

Code (CSharp):

1. Color vertexColor1 = mesh.colors[mesh.triangles[hit.triangleIndex * 3 + 0]];
2. Color vertexColor2 = mesh.colors[mesh.triangles[hit.triangleIndex * 3 + 1]];
3. Color vertexColor3 = mesh.colors[mesh.triangles[hit.triangleIndex * 3 + 2]];

but I'm getting an error

Not allowed to access colors on mesh 'Combined Mesh (root: scene) 6 Instance' (isReadable is false; Read/Write must be enabled in import settings)
UnityEngine.Mesh:get_colors()

I think this is because the mesh is static and we can not get the values that way,
I'm trying to found a workaround for this, if anyone knows how I could do this I would appreciate the help

 

That's indeed a show-stopper. I can't see how we can use vertex colors at all if we can't read them.

We would have to keep a copy of the mesh which we can read. But having closer look at the card it also asks for consideration of rocks as well. I think we might be better off with a 2D texture especially made for the purpose. Here we simply calculate the index from the player position (without height - meaning the same footstep sounds on top of the arch as under the arch of the glade). The texture wouldn't have to have a very high resolution - I guess one pixel per 0.5 m should do. Thinking about it, a "sound texture" is a very interesting idea indeed - we could also code ambient sounds like birds singing, fat sizzling or soup bubbling in it.

 

The issue is coming from all of them, I tried to remove the static and it works for the ground, but palms and rock are still unreadable,
though I think those don't matter since they don't have a color mesh so for those I was thinking in use their tag to identify if it is a palm, rock, bush, etc, and another one for the ground, so if it's ground we run the color logic to get the right material else we return the tag

 

It's an exception though that the player can be on two very different heights less than 0.5 m apart in the same scene with different sounds. Mostly we use high rocks to separate scenes or the height doesn't matter if we e.g. hear birds twitter in the forest. Neither would it matter if the footstep sound in the arch and on top of the arch (not many would go there) were the same. I doubt it is much of restriction if you can't put sizzling fat (or whatever is making sound on a specific spot) within 0.5 m of a rock, being aware of the systems limitations.

 

Hey,
could someone please link this forum topic to the roadmap card? (https://open.codecks.io/unity-open-project-1/decks/15-code/card/19s-ground-material-detector)

I've done a little bit of experimenting before I found this topic. I came to the same conclusions @Eddrico reached.

My script shoots a raycast down from its game object. If the raycast hits something, the script checks whether the found object is ground or not (this check is not implemented yet).
If the raycast hits the ground, the script searches for the 3 vertices belonging to the found triangle and reads their colors.
You can see the result here (check the 3 colors in the inspector):

There are a few problems with this method:

1. To be able to read the vertex colors, vertices and triangles of the mesh of the ground, I've found that the ground objects need to be dynamic objects. I've tried to expose a generated texture from the ground shader in order to read it from my C# script but I couldn't find any solution. (To be honest I don't have much experience with shaders or game development, but I know that usually it's not an easy task to transfer GPU data to the CPU.)

2. I'm not really sure how to implement an adequate method that could convert the 3 vertex colors into one "material" information. This could be done by measuring the distance between the hit point and the vertices and choose the color of the closest vertex. Or it could be used another approach that somehow combines the colors together using the distances as weights. A more complex approach could use the neighboring triangles as well.

3. The algorithm calculates the result based on the vertex colors. If I understand correctly the Toon_Ground shader uses a noise texture to randomize the borders between the other four textures. This means that in order to get a more precise result the algorithm should take this noise texture into account.

 

@murkesz in order of points
1. I had the same issue, and I'm being working with a dynamic (non-Static) mesh, that way is easy to get the vertex
2. yes, it's not easy to calculate it, I was working on an idea that may work, I'll expose it later in this post
3. the noise is used to blend the textures, I think we want to handle each triangle by its own color, so if sand and dirt are mixed we will play 50% Sand sound and 50% Dirt sound (of course each triangle has 3 vertexes making the calculation more complicated than that)

Note.- I'm no expert in this, nor have any authority over the direction of the project, so if I'm assuming anything wrong please let me know.

So, as I was saying I have an idea of what we can do to calculate the material
We have 4 materials, in RGB form those would be:

• 0,0,0 = Grass
• 1,0,0 = Dirt
• 0,1,0 = Cobblestone
• 0,0,1 = Sand

note.- I'm not taking into account the alpha channel, because all vertex has their alpha as 1

And we want to set a sound according to the material in the triangle we are stood on (just taking into account the "terrain" case, since other objects will be handled by tag), to do this, I was adding the 3 vertex colors to get the mixed color
(colorVertex1 + colorVertex2 + colorVertex3) / 3
note.- this is done for each channel on its own.

I'm taking 3 cases that I was able to look in the current terrain
case 1.- primary colors based materials are mixed
suppose we have the next vertex colors

• 1,0,0
• 0,1,0
• 0,0,1

this is a mix of three of the materials we have, the mix would be
0.333, 0.333, 0.333 that means 1/3 dirt, 1/3 cobblestone, 1/3 sand
if we pass this mixed color, we could set the sound using them as percentages

case 2.- Grass(black) is involved in the mix
suppose we have the next vertex colors

• 0,0,0
• 0,0,0
• 1,0,0

the mix would be 0.333, 0, 0, knowing that the max value is 1, we can check if any of the colors are black and add a 0.333 for each one of them, which gives us a 0.666 or a 66% of volume in grass
we can say that this means 1/3 dirt, 2/3 grass

case 3.- the dark side is strong in this one (or primary colors are not set to 1)
3.1.- suppose we have the next vertex color

• 0,0,0
• 0.5, 0, 0
• 0, 0.5, 0

the mix would be 0.166, 0.166, 0 again we add 0.333 for black values
let the result side for a moment

3.2.- suppose we have the next vertex colors

• 0,0,0
• 1,0,0
• 0,0.5,0

the mix would be 0.333, 0.166, 0 again we add 0.333 for black values

now summarizing, case 3 is a real case though simplify,
this is the way I was planning in getting the volume for each material in all previous cases
Results
we can consider the sum of the values in 3.1 and 3.2 each as 100% meaning:

3.1.- 0.166 + 0.166 + 0 + 0.333 = 0,665 = 100%
this would mean

• Grass% = 0.166 * 100% / 0.665 = 24.962 %
• Dirt% = 0.166 * 100% / 0.665 = 24.962 %
• Sand% = 0 * 100% / 0.665 = 0 %
• Coblestone% = 0.333 * 100% / 0.665 = 50.075 %

3.2.- 0.333 + 0.166 + 0 + 0.333 = 0,832 = 100%
the same way

• Grass% = 0.333 * 100% / 0,832 = 40.024 %
• Dirt% = 0.166 * 100% / 0,832 = 19.951 %
• Sand% = 0 * 100% / 0,832 = 0 %
• Coblestone% = 0.333 * 100% / 0,832 = 40.024 %

@cirocontinisio
I've been testing the values and the math, it's a simple spell but quite unbreakable, let me know what you think

 

In my whiteboxing I have use an additional "rock"-texture instead of the sand (sand you would only find on the beach). I think it would be appropriate if we can associate the ground material with the vertex colors per scene - else I fear the 4 textures are quite limiting. Actually the same information is already in the textures given to the ground shader in the ground material, so maybe you could just return the name of the texture in there.

 

Oh, you're right. We can play more than one sound in the same time, so we don't have to narrow down the output to one "material". It definitely worth to check how does that sound.

I was thinking about giving a sound material component to every object which can interact with a character. So the material detector system would use two different components (scripts): one (sound material detector) for the characters and another (sound material) for every object which can be interacted by the characters.

Sound Material Component:
It could have a public method with the following declaration:

Code (CSharp):

1. public MySoundEffect GetMySoundEffect(RaycastHit hit);

This method could be implemented differently in child classes (I'm not sure how does unity handle inheritance in case of game object components) or simply just give back different results based on the actual property values (for example: the name of the used texture) of the Sound Material component.

Sound Material Detector component:
It could simply shoot a raycast down. If the ray hit something then the Sound Material Detector asks for the Sound Material component:

Code (CSharp):

1. var sm = hit.transform.gameObject.GetComponent<SoundMaterial>();
2. if (sm != null)
3. {
4. var mySoundEffect = sm.GetMySoundEffect();
5. ...
6. }

I don't have a computer on me at the moment, but I can test the solution tonight (GMT +1) and give you further explanation if needed.

 

I think I'm done with the above mentioned 2 scripts and an interface.

The interface is simple:

Code (CSharp):

1. /// <summary>
2. /// Every game object having a component which class implements this interface can be interacted by
3. /// the <class>SoundMaterialDetector</class> component.
4. /// </summary>
5. interface ISoundMaterial
6. {
7.     void EmitSound(RaycastHit hit);
8. }
9.  

Then I created a MonoBehaviour script called GroundSoundMaterial. The script implements the ISoundMaterial and can be used as a component on a non-static "ground" game object:


As you can see my logic was to give a name to the four different vertex color used by the ground. This way we can reuse the script on ground objects with different textures.

At the moment the EmitSound method of the component simply prints out four texts every time it is asked to emit a sound by someone.

Code (CSharp):

1.         print("Emitting " + _redSoundMaterialName + " sound material. Ratio: " + redRatio);
2.         print("Emitting " + _greenSoundMaterialName + " sound material. Ratio: " + greenRatio);
3.         print("Emitting " + _blueSoundMaterialName + " sound material. Ratio: " + blueRatio);
4.         print("Emitting " + _blackSoundMaterialName + " sound material. Ratio: " + blackRatio);
5.  
6.         //TODO: instead of printing the material names and ratios the method should somehow call the sound system

The ratio is calculated by the algorithm thought out by @Eddrico.


The second script I created is another MonoBehaviour. This can be used as a component on a game object (mostly on a character). It checks periodically whether a sound material object (a game object with a component which implements the ISoundMaterial interface) is under the game object. If it finds a sound material object then it calls the EmitSound method of it.


The Detect Rate property controls the time between two detection cycle. The raycast can hit only one object at a time. (I'm not sure if this is enough.)

You can see the result here (see console messages). Let me know what you think about it:

Enu

 

I have read all the posts and thanks to everyone, I have come up with a PR.
this is the link to the PR: https://github.com/UnityTechnologies/open-project-1/pull/476

In my opinion, for detecting the ground type, the mesh triangle is too big, and mixing all the 3 vertex colors is not a very good idea, only the nearest vertex color to the character should be enough for detecting the ground type.

If this PR was good, for the next step, we can use the GroundTypeDetector to call the right footstep sound.

 

I haven't been able to contribute much to the project since submitting the ground shader PR. I am very out of the loop but I got notification for this thread and thought I would compare the current PR with a ground detection method a game jam team member once made for use with this ground shader.

@hosseinpanahloo's PR looks like a very robust non-game jam approach using scriptable objects and a couple fallback methods for GetGroundType. In terms of underlying logic one difference I noticed in the method I had seen was the use of the barycentric coordinate of the raycast hit point to determine what vertex color was beneath the player.

I sadly don't know all the specifics but I thought it would be better to share than not. I've put some pseudo-code and/or comments at the start of this first script.

Code (CSharp):

1. // CharacterController.cs
2. ...
3. private TerrainColor _lastTerrainColor = TerrainColor.Black;
4. ...
5. ...
6. // Check ground type when player jumps
7. private void OnJump(InputAction.CallbackContext obj)
8. {
9.         animator.SetTrigger(Jump);
10.         var terrainColor = GetUnderlyingTerrainColor();
11.         jumpingSource.clip = jumpingClips.GetMatchingClip(terrainColor);
12.         StartCoroutine(timeJumpSound());
13. }
14.  
15. private void Update()
16. {
17. ...
18. ...
19. ...
20.     if (walkSpeed > 0.01f)
21.     {
22.         var terrainColor = GetUnderlyingTerrainColor();
23.         if (!footstepSource.isPlaying || terrainColor != _lastTerrainColor)
24.                 {
25.                     if (isCrouching)
26.                     {
27.                         footstepSource.clip = slowFootstepClips.GetMatchingClip(terrainColor);
28.                     }
29.                     else
30.                     {
31.                         footstepSource.clip = footstepClips.GetMatchingClip(terrainColor);
32.                     }
33.                     footstepSource.Play();
34.                     _lastTerrainColor = terrainColor;
35.                 }
36.             }
37.             else
38.             {
39.                 if (footstepSource.isPlaying)
40.                 {
41.                     footstepSource.Stop();
42.                 }
43.             }
44.     }
45.  
46.     private TerrainColor GetUnderlyingTerrainColor()
47.     {
48.         Ray ray = new Ray(transform.position, Vector3.down);
49.         if (!Physics.Raycast(ray, out RaycastHit hit))
50.         {
51.             return TerrainColor.Black;
52.         }
53.  
54.         MeshCollider meshCollider = hit.collider as MeshCollider;
55.         if (!meshCollider || !meshCollider.sharedMesh)
56.         {
57.             return TerrainColor.Black;
58.         }
59.  
60.         Mesh mesh = meshCollider.sharedMesh;
61.         Color[] colors = mesh.colors;
62.         if (colors == null || colors.Length < 1)
63.         {
64.             return TerrainColor.Black;
65.         }
66.         int[] triangles = mesh.triangles;
67.  
68.         // Extract local space normals of the triangle we hit
69.         Color c0 = colors[triangles[hit.triangleIndex * 3 + 0]];
70.         Color c1 = colors[triangles[hit.triangleIndex * 3 + 1]];
71.         Color c2 = colors[triangles[hit.triangleIndex * 3 + 2]];
72.  
73.         // interpolate using the barycentric coordinate of the hitpoint
74.         Vector3 baryCenter = hit.barycentricCoordinate;
75.  
76.         // Use barycentric coordinate to interpolate normal*
77.         Color interpolatedColor = c0 * baryCenter.x + c1 * baryCenter.y + c2 * baryCenter.z;
78.  
79.         return AudioTerrainMap.GetTerrainColor(interpolatedColor);
80.     }
81. }
82.  

*note: not so sure if this comment is correct about interpolating normal

Some of AudioTerrainMap's functionality is replaced by Scriptable Object Architecture in the open project but I've included it for reference. It does include a function for settting the correct footstep audio clips.

Code (CSharp):

1. // AudioTerrainMap.cs
2. public enum TerrainColor
3. {
4.     Black,
5.     Red,
6.     Green,
7.     Blue
8. }
9.  
10. [System.Serializable]
11. public class AudioTerrainMap
12. {
13.     public AudioList blackTerrainList;
14.     public AudioList redTerrainList;
15.     public AudioList greenTerrainList;
16.     public AudioList blueTerrainList;
17.  
18.     private static (Vector3, TerrainColor)[] _map =
19.     {
20.         (new Vector3(0f, 0f, 0f), TerrainColor.Black),
21.         (new Vector3(1f, 0f, 0f), TerrainColor.Red),
22.         (new Vector3(0f, 1f, 0f), TerrainColor.Green),
23.         (new Vector3(0f, 0f, 1f), TerrainColor.Blue),
24.     };
25.  
26.     public static TerrainColor GetTerrainColor(Color color)
27.     {
28.         Vector3 rgb = (Vector4) color;
29.         float best = Mathf.Infinity;
30.         TerrainColor bestColor = TerrainColor.Black;
31.         for (int i = 0; i < _map.Length; i++)
32.         {
33.             float d2 = (rgb -
34.                         _map[i]
35.                             .Item1).sqrMagnitude;
36.             if (d2 < best)
37.             {
38.                 best = d2;
39.                 bestColor = _map[i]
40.                     .Item2;
41.             }
42.         }
43.  
44.         return bestColor;
45.     }
46.  
47.     public AudioClip GetMatchingClip(TerrainColor color)
48.     {
49.         AudioList list = blackTerrainList;
50.         switch (color)
51.         {
52.             case TerrainColor.Black:
53.                 list = blackTerrainList;
54.                 break;
55.             case TerrainColor.Red:
56.                 list = redTerrainList;
57.                 break;
58.             case TerrainColor.Green:
59.                 list = greenTerrainList;
60.                 break;
61.             case TerrainColor.Blue:
62.                 list = blueTerrainList;
63.                 break;
64.         }
65.  
66.         return list.GetRandomClip();
67.     }
68. }

And our extremely game-jam list of audio clips to randomly cycle through

Code (CSharp):

1. //AudioList.cs
2. [System.Serializable]
3. public class AudioList
4. {
5.     public AudioClip[] clips;
6.  
7.     public AudioClip GetRandomClip()
8.     {
9.         if (clips.Length < 1)
10.         {
11.             return null;
12.         }
13.  
14.         int index = Random.Range(0, clips.Length);
15.         return clips[index];
16.     }
17. }

Anyways, it's great to see things have been going well. I've learned so much more about shaders since I made this ground shader in January but I'm glad to see it has been enough for the time being.

 

@daneobyrd It's an interesting solution.

here is my idea about it but I'm not so sure.
I think when the mesh triangles size is way too big in comparison to the character size, finding the nearest vertex color might be a better solution.
In case the mesh triangles size is not too big in comparison to the character size, using barycentric to find interpolated color works better.

Here is a picture to see the triangle size in comparison to the character using the Beach_Landmass prefab as the ground.


In some scenarios, one of these solutions might work better than the other. What do you guys think we should do?

 

I had no idea the mesh triangles were that big in some areas.

As an aside I can restructure the ground shader some if you all would prefer each texture be affected by the light differently. I know most assets in the game only have Albedo but if there are any additional detail textures for the ground I could implement them.

 

Here’s a small note about the ground shader that may have been forgotten. Currently the shader takes the maximum value as each new vertex color is introduced. This was to prevent some odd transparency where some textures were unintentionally visible where vertex colors overlapped. This essentially determines the sorting order for how the textures stack on top of one another where vertex colors overlap.

If I remember correctly it functioned like this.

max(max((max(Black, Red)), Green),Blue)

max(max((max(Grass, Dirt)), Cobblestone), Sand) 

The unintentional transparency can be seen in this post