Search Unity

Resolved Entity Rendering and Compute Buffer Begin Write GPU Synchronization

Discussion in 'DOTS Dev Blitz Day 2022 - Q&A' started by Kmsxkuse, Dec 8, 2022.

  1. Kmsxkuse

    Kmsxkuse

    Joined:
    Feb 15, 2019
    Posts:
    306
    As mentioned on this thread https://forum.unity.com/threads/computebuffer-beginwrite.1252560/

    > The only thing we need to ensure here is that the upload buffers are not reused too soon. Initially we did a fixed N frame multibuffering but it seems that is not to be trusted. So currently we keep track of where the GPU is using a tiny AsyncReadback per frame. It's certainly not nice but currently there is no exposed API to keep track of what frame is done on the GPU.

    How exactly is this done? Ive attempted to do my own computebuffer.begin write but as soon as the GPU process extends longer than CPU execution (gpu bottleneck), the CPU begins writing before the GPU finishes reading from the buffer, causing graphical artifacts.

    How does the Entity Renderering team solve this problem?
     
    Kmsxkuse, Dec 8, 2022
    #1

  2. JussiKnuuttila

    JussiKnuuttila

    Unity Technologies

    Joined:
    Jun 7, 2019
    Posts:
    351
    The current solution used by Entities Graphics is to wait for a given number of frames before reusing a GraphicsBuffer. We have a fixed wait period of 3 frames for most graphics APIs, and 4 frames for Apple Metal and Nintendo Switch.

    We put all buffers used during a given frame in an array, and when we need a new array we will reuse buffers from any arrays that are older than the wait period. If there are no such buffers left, then we create a new buffer.

    This ensures that by the time GraphicsBuffer.LockBufferForWrite is called again, the GPU is no longer using that buffer, and no artifacts or stalls will occur.

    If you are interested in the exact code we use, I would recommend taking a look at the SparseUploader.cs file in the Entities Graphics package source, and in particular the RecoverBuffers() method.

    It's also worth pointing out that earlier version of the Entities Graphics package used a different approach, where we issued one AsyncGPUReadbackRequest each frame, and used its "done" property to check whether the request (and the frame it was performed in) had completed. However, this approach was discovered to cause a problem where the computer is unable to enter idle state, which is very bad for e.g. laptop or mobile device power use, so we don't recommend using that approach.
     
    JussiKnuuttila, Dec 8, 2022
    #2
    Kmsxkuse likes this.

  3. Kmsxkuse

    Kmsxkuse

    Joined:
    Feb 15, 2019
    Posts:
    306
    Ah, that's annoying... I wanted to use the buffer begin write to reduce memory utilization by skipping the intermediate holding array but having a rotating 3 buffer system would require even more memory than just using SetData and enforcing driver level write synchronization.
     
    Kmsxkuse, Dec 8, 2022
    #3

  4. JussiKnuuttila

    JussiKnuuttila

    Unity Technologies

    Joined:
    Jun 7, 2019
    Posts:
    351
    The Entities Graphics package tries to reduce the memory overhead this causes by using fixed size rotating buffers (currently 4 MiB).

    The destination GPU buffer that we copy to can be much larger than this (tens or hundreds of MiB), but the rotating buffers are relatively small and are only consumed on demand, and often only one or two rotating buffers is enough for a single frame unless there was a lot of data updated during that frame.
     
    JussiKnuuttila, Dec 8, 2022
    #4
    Kmsxkuse likes this.

  5. Kmsxkuse

    Kmsxkuse

    Joined:
    Feb 15, 2019
    Posts:
    306
    Thank you. I'll check that file you mentioned then benchmark the differences. You've brought me a lot to think about moving forward with my own rendering systems.
     
    Kmsxkuse, Dec 8, 2022
    #5
    JussiKnuuttila likes this.

  6. JussiKnuuttila

    JussiKnuuttila

    Unity Technologies

    Joined:
    Jun 7, 2019
    Posts:
    351
    The SparseUploader.cs file should also be pretty self-contained (i.e. it shouldn't have dependencies to the rest of the Entities Graphics package), so in case it's useful to you, you might be able to copy it inside your project and use it directly.
     
    JussiKnuuttila, Dec 8, 2022
    #6
    Krajca and Kmsxkuse like this.