Question Which data is uploaded to the GPU every frame? What is "render state"?

Hello everyone,
I have dabbled a bit with shaders and creating meshes procedually, but I now wanted to understand how Unity does graphics and am trying to read the docs.

The main things I can think of that a GPU needs when executing:

1. Shader itself
2. Shader inputs (e.g. Texture)
3. ObjectToWorld Matrix (actually part of shader inputs)
   
4. Mesh
5. UVs
6. Other global state like lights etc.

Now the docs talk a lot a bout "not changing render state". I do not really understand what render state itself is. Which of the above items are part of render state, and which can be cheaply changed in each call?

It seems as if the only thing you are allowed to change is item 3 (and using MaterialBlockProperties you can maybe also have some other shader inputs like color that you can change).

What I do not understand is: For the other things, like the Shader, the Meshes etc., are they reuploaded to the GPU every single frame? To me it seems as if one could just upload all the shaders and meshes when the game starts to the GPU, and then just tell the GPU: Now execute shader `n` with mesh `j`. Here are the inputs. But then it would not be very costly to change the shader and mesh, since the GPU already has all the data - so what is this all about?

I think maybe my main question is, what data is uploaded every frame to the GPU and what data just stays there throughout. And why can one only modify some shader inputs (all shader inputs?) cheaply.

I know this is a really noob question, but I just can't understand it from the Docs. If anyone has any pointers for me to understand this better (and maybe GPU programming), that are not to long (this is to much of a sidetrack anyways already haha), that would also be great!

 

That's exactly how it works. Usually everything is loaded when the game starts (if you ignore streaming).

Well, you have to see what time frame we are talking about. There are only 16 ms in a 60 Hz frame and if you have thousands of draw calls, you are only left with microseconds to change render state. Render state changes are expensive because the GPU has to validate the state and reconfigure itself.

Having said that, DX12 and Vulkan have introduced so-called Pipeline State Objects (PSO) which are precompiled state configurations that can be changed much quicker at run-time. So this problem is not as severe on newer APIs.

In a perfect world, you only upload constant buffer changes and command lists every frame. Sometimes you have vertex buffers that are changed every frame, for example when you do CPU skinning. Unity also updates vertex buffers for UI elements when there is a layout or graphics change.

Render state is
- Which shader (pass) is being used
- Blend State
- Vertex Format Input Layout
- Primitive Topology
- Rasterizer State
- Depth Stencil State
- Number of render targets and render target formats
- Depth-stencil format
- Stream output buffer description

Not part of the render state are
- Resource bindings (vertex buffers, index buffers, constant buffers, textures, render targets, stream output buffers, ...)
- Viewports
- Scissor Rectangles
- Constant Blend Factor
- Stencil Reference Value

Unity distinguishes between SetPass calls and DrawCalls. SetPass call is basically a render state/shader change.

PS: You may wonder, does this mean that you can change a mesh cheaply as long as you don't change the shader? If the vertex format or topology changes, then it's an expensive render state change. If it doesn't then it's just a change of bindings. But this is where it gets a bit mushy. Eventually, you are gonna have to profile it because there are a lot of factors and differences between platforms, APIs and GPU vendors.

Also take a look at Unity's SRP batcher recommendations.

 

Man, thanks so much for the elaborate answer! Helps a lot. I also read about unities old and new rendering pipeline all day now.

What I do not understand is (I know that SRP changed this now, SRP seems much closer in general to how I thought it should work), is why did every material change before SRP result in a render state change?

A different material (if it is the same shader!) to me just means that it is a different input to the same shader? Also, do you have any resources what one should learn to understand these things? Is doing a vulkan or directx intro something that would teach how these things work?

 

Seriously, I'm asking this myself too!
Sounds like Unity didn't even have some basic render state cache before SRP.

Except for the keywords, yes.
Also, ShaderLab shaders have a way to feed some material inputs into render state changes.
But in general, you are right.

Yes, that would be helpful. Just grab one of those API books and follow along with a small toy engine. That's what I would recommend. I wouldn't start with Vulkan and DX12, though, if you don't have any prior experience. OpenGL is very easy to learn (but very outdated) and DirectX 11 isn't that hard either (a bit more modern).

By the way, you are asking excellent questions!