Why is AsyncGPUReadback limited to the main thread?

I know, odd question, but the purpose of AsyncGPUReadback is to offset something on the GPU.

But WaitForCompletion can only be called on the main thread, and it may hang up the thread for a while.

Would be nice to have the main thread free while we wait for the GPU.

WaitForCompletion is a very odd beast, as when you call it can be be quick, or take a long while. It depends of the GPU queue and other things I'm not too sure about.

 

I need it later in the same frame as when it was called. Any clue how to do that without blocking the main thread?

Since it returns NativeArray<>, would have been nice to offset it on a different thread.

 

Sadly, no can't do. We use it for depth culling.

We tried to dispatch as early as we can, and WaitForCompletion as late as possible... but that turned out to be rather bad, because how long it's going to wait is highly dependent on what's happening with the CPU/GPU at that moment.

Using PIX, we found out that our process only takes 0.22ms to complete on Xbox One. Calling Wait right away, makes it stall the CPU for 1-1.2 ms. If we call it later - say after processing physic - we sometime stall for MUCH longer, and the GPU is busy doing something else, such as updating Skinned objects.

Again... Why is this main thread only? It returns a NativeArray, which since it's used in Jobs, I would assume is thread safe. Or maybe have a AsyncGPUReadback that forces readback as soon as possible.

 

Because it wouldn't lockup the main thread. I could run physic, AI, etc while waiting to get back the culling result. I have literally 4ms of stuff to do between dispatching and before I start rendering stuff.

In some way, WaitForCompletion has an issue. PIX shows me my dispatch is handled right away, and takes 0.22ms to complete. But for some reason, if I put WaitForCompletion 4 ms later, it still lock up the main thread for 0.5-1.8ms.

I see here that WaitForCompletion does more than just waiting for "done" to be true. It calls Gfx.UpdateAsyncReadbackData. Would it be possible to have that Gfx.UpdateAsyncReadbackData call done right away without locking up the main thread?

 

I'm sorry... But what? That interface (PCIe) is as fast in both direction. That's why we have 3Gb/s SSD in a M.2 slot. The data involved here is 3000 floats. It's 12kb of data. It's nothing. Even at 60 frame per second, it's only 720kb.

That's exactly what I do. It still hang the CPU for a while. On XboxOne, I get up to 9-10ms of the main thread waiting... and that's with the GPU ended up the task 10-12 ms earlier. Currently on Xbox, the GPU is busy 15-20% of the time.

I verified with a friend if I wasn't crazy - 3D Prog on Assassin's Creed - and he concurs; that read back should take 0.1ms top on XboxOne, not 10 ms. Something's fishy.

 

I know all this, and it would make perfect sense if the GPU had tasks to perform already queued. But it's not the case. I have a starving GPU that the CPU can't keep up feeding. Even more, when we Dispatch, there's nothing else in the GPU queue at that moment. It's processed right away in 0.22ms. And then the next task happens 3-4ms second later with updating skinned meshes - post physic update.

But let's get back to the main point of this thread; why is there no "get result right away" method that is NOT main thread blocking.

Am I being unclear here? Something similar to WaitForCompletion, but would be WaitForCompletionAsync.

I want the results as soon as possible - similar to WaitForCompletion - but I don't want to block the main thread while waiting. Is it dumb? If so, why?

 

Hmm.. I think I missed that bit, sorry.

 

I feel like there was a fundamental misunderstanding here, and the topic was never really resolved.

It sounds like @LightStriker wants to do the following: [Dispatch some work] [game code runs] [Get the result], like @richardkettlewell suggests. However, the issue here is that the [Get the result] step is taking way longer than it should. Richard mentioned latency, and I think that is the main factor thats wrong here. No matter how long you wait to [Get the result], the GPU is going to be busy doing something at the time you actually call it. You need to stall, wait for it to finish, and then actually get the result. Because the GPU is always going to be busy doing something, you're going to have that latency.

I think what LightStriker is suggesting is an off-thread alternative so that we can get the data (introduce that sync point) from the GPU. Yeah, this is still going to stall the gpu because it needs to stop what its doing and send data to the cpu. But, the point here is that the main thread will not be blocked during this latency.

There is AsyncGPUReadbackRequest callback, but it seems to still have the same issue. At some point on the main thread, it asks for data back from the GPU, and in doing so it seems the GPU has to finish up whatever its currently schedule to do first, so the request's GetData can be randomly pretty slow. Depending on the game, you may get lucky and request the data when the GPU isnt actually busy, and in those scenarios its as fast as it should be.

 

AFAIK, even on consoles, getting GPU data to the CPU at the same frame is going to have a cost. Even if you could wait in another thread, there's no guarantee the data will be available before the point in the frame where the main thread needs the data (if you're doing culling, this is before preparing the next frame for rendering) unless you force a GPU stall.

If you look at games that use GPU occlusion queries, most of them use data from the previous frame, which is why sometimes a fast camera turn causes objects to pop in. The only way to use the data on the same frame reliably is when your rendering is also managed by the GPU, using indirect rendering or DX12/Vulkan indirect execution to pipeline the data.