Resolved HDRP Shader graph hybrid instanced instancing now working

Hi,

TLDR;
- Hybrid Instanced properties not working with MaterialPropertyBlock.
- A mixed bag of ideas, future features, abandonment and outdated documentation.
- Lot of research: bad, outdated documentation.
- Missing cookbook a-z documentation how to implement is for ShaderGraph.
- Some complaints to unity.
- A lot of wondering and amazement from a client perspective.

Background
I'm trying to create my own terrain shader to render a infinite terrain with a view of 16x16km.
I try to perform this feat by making fixed size chunks of 500 points with various unit sizes. Each point being larger the further the chunk is away from the character/camera.

This still creates a lot of verts, that aren't culled properly by unity (looking straight at sloped ground = 30mil). If it would have been culled properly, it would still have more verts then I would like, because chunks further away would be so large that there is a big change something of it would be rendered.

Requirement
So I need to use (lots of) performing small terrain chunks.

Batching
For performance I tried manually batching the terrain parts together by code. Lol. Takes ages to batch it. No solution. Dynamic batching also doesn't really help. Enabling dots does nothing.

Instancing
(Should not have to tell this instancing is something entirely different then batching)
So I need instancing so I can use (lots of) smaller terrain chunks without batch overhead.
Problem is that instancing can only be done on the same mesh + material. Solution: make a flat texture with a grid of vertexes. Displace the vertexes up and down as needed.
For that we need to use an atlased (/mega) texture or (what I use) Texture2dArray.

After a lot of tinkering and testing I found out that Texture2dArray does work with vertex displacement (is not the same as tessellation!) and instancing (more on that at the end of the post). So luckily the most difficult part works out of the box with HDRP & Shader Graph: yeay.

Instanced values
Only thing that doesn't work is the 'Hybrid Instanced (experimental)' values in the shader (graph).
It's supposed to work with usage of MaterialPropertyBlock. But the value (_terrainId) just stays 0.

I used the following code for instanced gameobjects:
MeshRenderer renderer = _terrainGround.GetComponent<MeshRenderer>();
renderer.GetPropertyBlock(_uniqueMaterialProperties);
_uniqueMaterialProperties.SetFloat("_pointsWidth", _pointSize);
_uniqueMaterialProperties.SetFloat("_terrainId", (float)_id);
renderer.SetPropertyBlock(_uniqueMaterialProperties);

It works great without instancing, but I need instancing.

The shader (de)code
By opening the shader graph and right clicking on the Lit Master (end-)node there is an option to 'Copy Shader'. The shader code will be shown by pasting it into a texteditor (I used notepad).

To my astonishment I find the following code that does enable instancing of the variable (and another one I have):
#define DOTS_CUSTOM_ADDITIONAL_MATERIAL_VARS UNITY_DEFINE_INSTANCED_PROP(float, _terrainId_Array)\
UNITY_DEFINE_INSTANCED_PROP(float, _pointsWidth_Array)

Though the property used is suddenly _terrainId_Array. Which isn't defined in the properties (where does it come from? Some one might know). How do I fill it if it's not a property? No tutorial found that shows it.

@Unity: To me it looks like it would be a nice Monday morning of easy coding to connect things in code for shader graph.

Help
Is there any one that knows how to get the (apparently) instanced property in my Shader Graph? For example some code in a Custom function node?

I'm using:
Unity 2019.3.12f1
HDRP 7.3.1
Shader Graph 7.3.1

Some other ramblings that might have worked
-It's not nice to shanghaiing people into using shader graph. It's even worse that the most required feature doesn't work or isn't documented properly. Especially as it doesn't look like a difficult, time consuming or impactful, but yet necessary thing to do.
-Hybrid Renderer V1 package is nowhere to be found in the package manager. Another disbanded project/documentation gone wrong?
-SRP Batcher sounds great. No were to be found in Unity either. Except for some obscure magical setting for code: GraphicsSettings.useScriptableRenderPipelineBatching. It's magic (disruption) on instancing and batching 'because in most cases it should be better anyways' is astounding.
-Dots looks awesome. Can't wait for it to come out (2020.2?)!

Tip
On another note: I would like to highlight (hard to find) that it is possible to read a Texture2dArray and use it for vertex displacement by creating a 'Custom function' node and using the following function in it (got if from the forums THNX!):
#ifndef SAMPLE_TEXTURE_ARRAY_SHADER_INCLUDED
#define SAMPLE_TEXTURE_ARRAY_SHADER_INCLUDED

void SampleTexture2dArrayLod_float(Texture2DArray TextureArray, float2 UV, uint Index, SamplerState Sampler, out float4 RGBA)
{
RGBA = SAMPLE_TEXTURE2D_ARRAY_LOD(TextureArray, Sampler, UV, Index, 0);
}
#endif

@Unity: If it's this simple, why isn't there a 'Sample Texture 2d Array Lod' node?

P.S. Sorry for the long post and the rambling.

 

Been a while but I found my own answer.
There are 2 ways of instancing:
-1). Automatic. Only makes object a bit faster.
-2). Real instancing. Complecated but uses 1 object to instance it in many places.

At the time I was using nr 1.
Works with MaterialPropertyBlock.
The solution is to not register you instancing variable as a property. This makes it that unity won't define it in the code.
This frees up the possibility to define it your self with a custom node.
This can be done with direct code included in the custom node or a reference (in the custom node) to a hlsl (include) file.

In this custom node you execute the normal code to read instanced variables:
#ifndef SOMERANDOMUNIQUENAME
#define SOMERANDOMUNIQUENAME
UNITY_INSTANCING_BUFFER_START(SomeRandomGroupDescription)
UNITY_DEFINE_INSTANCED_PROP(float, VARIABLE_NAME_INSTANCED)
UNITY_INSTANCING_BUFFER_END(SomeRandomGroupDescription)
#endif

With VARIABLE_NAME_INSTANCED being the variable you want instanced.

Option 2 is more cumbersome. With real instancing you have to use Graphics.DrawMeshInstancedIndirect.
Here you give a MaterialPropertyBlock with commandbuffers containing your data.
These commandbuffers can be read as StructuredBuffers:
uniform StructuredBuffer<int>_terrainPointWidth;
unity_InstanceID contains the instanceId so you can read you instanced variables with
_terrainPointWidth[unity_InstanceID]

Though you also have to handle setting unity_ObjectToWorld you self with the use of
#pragma instancing_options procedural:FunctionNameThatFixesUnity_InstanceID


On a side note. Pragma's won't be executed from included files. They have to be define in the 'string' type custom node. Also they have to be executed at the beginning of the code. So the have to be placed and make use of starting points.And be used by the following nodes!

Example:
#pragma instancing_options procedural:FunctionNameThatFixesUnity_InstanceID
Output = Input;

 

I tried to follow this approach exactly with the custom node, exactly like you described and it doesnt work (on Unity 2023.1) unfortuantely. It's a nice example though and it makes sense, but doesnt work now

 

I'm on 2021.3.36f1, HDRP 12.1.14
I tried different URP examples, but it didn't work because in HDRP you get this error:

1. "undeclared identifier 'Use_Macro_UNITY_MATRIX_M_instead_of_unity_ObjectToWorld' error"
2. Use UNITY_MATRIX_M and UNITY_MATRIX_I_M instead.
3. Then, you get "out parameters require l-value arguments"

I tried these 2 examples:
https://twitter.com/cyanilux/status/1396848736022802435
https://gist.github.com/ArieLeo/d7e6bc5485caa9ba99cd3a59d0f53404

What worked for me (yes, hacky #define abuse, but it works around the l-value thing, or suppressing the Use_Macro_UNITY_MATRIX_M stuff):

Code (CSharp):

1. #define unity_ObjectToWorld unity_ObjectToWorld
2. #define unity_WorldToObject unity_WorldToObject

I tried that myself but failed (put the define in the wrong place) but then found this example:

https://github.com/parcoder/hdrp_sh.../blob/master/Shaders/HDRP_indi_instance.cginc

which contained these lines. I took them and now it works in HDRP for me!

More Details: the setup function uses "inout" to modify the passed matrices:

Code (CSharp):

1. void vertInstancingMatrices(inout float4x4 objectToWorld, inout float4x4 worldToObject) {

and I'm calling it like this with the #define hack:

Code (CSharp):

1. void vertInstancingSetup() {
2.         #define unity_ObjectToWorld unity_ObjectToWorld
3.         #define unity_WorldToObject unity_WorldToObject
4.         vertInstancingMatrices(unity_ObjectToWorld, unity_WorldToObject);
5.     }