Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Unite 2023 Registrations are now LIVE! We are thrilled to announce that Unite 2023 Registration is now live and open for all!
    Dismiss Notice
New Forum User Notice Unite 2023 Registrations are now LIVE!

Question Interaction between IssuePluginEventAndData and Graphics Jobs

Discussion in 'General Graphics' started by jdesantos, Dec 21, 2022.

  1. jdesantos

    jdesantos

    Joined:
    May 24, 2013
    Posts:
    304
    I don't see any kind of documentation regarding how CommandBuffer.IssuePluginEventAndData works when Graphics Jobs are active (for example in D3D12 standalone)

    I would expect that the callbacks passed to IssuePluginEventAndData are always invoked non-concurrently. But I am observing that, with D3D12 (Unity 2022.2, 2022.1), the given callbacks are executed in parallel sometimes.

    This obviously break our code, because without Graphic Jobs we don't expect this reentrancy into our native code.

    To avoid this behavior, we need to use the flag "kUnityD3D12EventConfigFlag_SyncWorkerThreads", but I assume this is quite expensive and shouldn't be necessary for this?

    Could someone please clarify how Graphics Jobs affects IssuePluginEventAndData?

    Thank you!
     
    jdesantos, Dec 21, 2022
    #1

  2. jdesantos

    jdesantos

    Joined:
    May 24, 2013
    Posts:
    304
    jdesantos, Dec 22, 2022
    #2

  3. aleksandrk

    aleksandrk

    Unity Technologies

    Joined:
    Jul 3, 2017
    Posts:
    2,845
    I'll ask someone to answer this. Please keep in mind that it's vacation season already, so it may take some time :)
     
    aleksandrk, Dec 22, 2022
    #3
    jdesantos likes this.

  4. tvirolai

    tvirolai

    Unity Technologies

    Joined:
    Jan 13, 2020
    Posts:
    77
    Hi, The plugin events are not called from graphics jobs themselves. However there is a place where you can potentially have concurrent execution: Your plugin callback might be called either from the renderthread which records commandbuffers. You use this if you want to add your own commands into the existing commandlist. Another way is if you ask for queue access. This callback happens on the submission thread that pretty much only submits commandbuffers to the GPU.

    So if you issue 2 callbacks, one that doesn't want command queue access and another that does they will be submitted from different threads and you might get them called simultaneously.

    If you don't use graphics jobs there is no separate submission thread so everything is serialized.
     
    tvirolai, Dec 22, 2022
    #4

  5. jdesantos

    jdesantos

    Joined:
    May 24, 2013
    Posts:
    304
    Thanks for your answer. Our callback is configured with:

    kUnityD3D12GraphicsQueueAccess_DontCare

    First frame is invoked from the following call-stack:

    Code (CSharp):
    1. Noesis.dll!Noesis_GetRenderOffscreenCallback::__l2::<lambda>(int __formal, void * data) Line 570    C++
    2. Noesis.dll!void <lambda>(int, void *)::<lambda_invoker_cdecl>(int __p1, void * data) Line 601    C++
    3. UnityPlayer.dll!GfxTaskExecutorD3D12::RunTask(class GfxTaskExecutorD3D12 &)    Unknown
    4. UnityPlayer.dll!GfxTaskExecutorD3D12::Task(void *)    Unknown
    5. UnityPlayer.dll!Thread::RunThreadWrapper(void *)    Unknown
    And in the second frame we are invoked from a different thread (while the execution of the callback in the first frame hasn't finished yet)

    Code (CSharp):
    1. Noesis.dll!Noesis_GetRenderOffscreenCallback::__l2::<lambda>(int __formal, void * data) Line 570    C++
    2. Noesis.dll!void <lambda>(int, void *)::<lambda_invoker_cdecl>(int __p1, void * data) Line 601    C++
    3. UnityPlayer.dll!GfxDeviceD3D12::InsertCustomMarkerCallbackAndDataWithFlags(void (*)(int,void *),int,enum CustomMarkerCallbackFlags,void *,unsigned __int64)    Unknown
    4. UnityPlayer.dll!GfxDeviceWorker::RunCommand(class ThreadedStreamBuffer &)    Unknown
    5. UnityPlayer.dll!GfxDeviceWorker::RunExt(class ThreadedStreamBuffer &)    Unknown
    6. UnityPlayer.dll!GfxDeviceWorker::RunGfxDeviceWorker(void *)    Unknown
    7. UnityPlayer.dll!Thread::RunThreadWrapper(void *)    Unknown
    By your comments the first invocation (directly from a job?) is unexpected?

    And yes, we know this is only happening with "Graphics Jobs" enabled, but we cannot disable them because we are a plugin and can't control the settings used by our clients.
     
    jdesantos, Dec 22, 2022
    #5

  6. tvirolai

    tvirolai

    Unity Technologies

    Joined:
    Jan 13, 2020
    Posts:
    77
    The first invocation is actually from the submission thread (GfxTaskExecutorD3D12 handles the commandqueue submission). So those callbacks match the issue I described.

    Are you changing any of the flags you're passing to the plugin interface? Because it should be deterministic. If you don't change any of the flags it should always go in the same way.
     
    tvirolai, Dec 22, 2022
    #6
    jdesantos likes this.

  7. jdesantos

    jdesantos

    Joined:
    May 24, 2013
    Posts:
    304
    Hmm, it seems in the first frame we are invoking the callback without configuring the event. And if the default is "QueueAccess_Allow" this explains everything.

    Thanks for the help!
     
    jdesantos, Dec 22, 2022
    #7
    tvirolai likes this.