Search Unity

NativeArray.CopyFrom() strange behavior

Discussion in 'Entity Component System' started by shirocro, May 10, 2018.

  1. shirocro

    shirocro

    Joined:
    Apr 30, 2018
    Posts:
    16
    Hi, I just found that if I copy one Native array to another with .CopyFrom() it takes low overhead (for 4*1920*1080 it takes 2ms), but if I copy managed array to Native one it takes x10 longer.

    But after some debugging I am able to conclude that when coping one native array to another it is not blocking function and after 2ms when my code is continued only small slice of array was been copied. The problem is, how I can know when whole array is copied and ready?
     
  2. shirocro

    shirocro

    Joined:
    Apr 30, 2018
    Posts:
    16
  3. GabrieleUnity

    GabrieleUnity

    Unity Technologies

    Joined:
    Sep 4, 2012
    Posts:
    116
    @shirocro The copy is definitely blocking, I'm not sure why you are having that issue, maybe it has to do with wrong parameters being provided to the function?

    That said, the Copy functions have performance issues are being solved. An update that fixes them and improves the Copy operation will land soon and be available in a future Unity release.
     
  4. shirocro

    shirocro

    Joined:
    Apr 30, 2018
    Posts:
    16
    It is maybe problem because I run code in editor. But cannot build and run, it crashes every time, but in editor it works, have to see what is problem, but for now I am focused to editor.

    I am using AsyncGPUReadback to take screenshot in OnRenderImage callback.
    To receive pixels I have:

    //global variables
    NativeArray<Color32> mbuffer;
    NativeArray<Color32> buffer;

    //when data is ready
    mbuffer = req.GetData<Color32>();
    buffer.CopyFrom(mbuffer);

    and after that, sending pointer to buffer to native plugin NativeArrayUnsafeUtility.GetUnsafePtr<Color32>(buffer);
    but native plugin get uncompleted frame, if it waits some miliseconds, frame is completed.

    Second strange thing, if I don't do copy and instead that I send pointer to mbuffer to native plugin, I got all zeros instead rendered pixels!
     
  5. GabrieleUnity

    GabrieleUnity

    Unity Technologies

    Joined:
    Sep 4, 2012
    Posts:
    116
    Looks like a synchronization issue with the AsyncGPUReadback request, or maybe invalid parameters to the AsyncGPUReadback?

    CopyFrom is slow but definitely synchronous.
     
  6. shirocro

    shirocro

    Joined:
    Apr 30, 2018
    Posts:
    16
    I don't know about what parameters you are talking... This is my async request. I don't have any parameters, just source.

    Code (CSharp):
    1.  
    2. void OnRenderImage(RenderTexture source, RenderTexture destination)
    3.     {
    4.         if (_requests.Count < 8)
    5.             _requests.Enqueue(AsyncGPUReadback.Request(source));
    6.         else
    7.             UnityEngine.Debug.Log("Too many requests.");
    8.  
    9.         Graphics.Blit(source, destination);
    10.     }
    11.  
     
  7. shirocro

    shirocro

    Joined:
    Apr 30, 2018
    Posts:
    16
    Yes, there is a problem with synchronization. I got done flag on request even if request is not completed.
    How many requests could I have in queue? I am trying to take texture only from peek of queue when I have two requests done, but then I got a lot requests with an error.?
     
  8. JulienF_Unity

    JulienF_Unity

    Unity Technologies

    Joined:
    Dec 17, 2015
    Posts:
    326
    What about the error flag? A request with an error will have it's done flag set as well as its error flag.

    Also what's the format of your source texture ? Does the number of elements in the native array you get with GetData match the number of pixels of your render target ?

    Also maybe you can post the code that polls the requests state. Error in request may also mean it has been disposed because you didnt fetch the data at the frame it arrives.

    In any case if you try to get the data of a request with an error (or that is not done yet), an exception should be thrown.
     
    Last edited: May 11, 2018
  9. shirocro

    shirocro

    Joined:
    Apr 30, 2018
    Posts:
    16
    First, I didn't know I have to fetch the data at the frame it arrives. It is then clear why I had errors. Here is code snippet which doesn't have errors but it has frames which are not complete rendered:


    Code (CSharp):
    1. while (_requests.Count > 0)
    2.             {
    3.                 var req = _requests.Peek();
    4.  
    5.                
    6.                 if (req.hasError)
    7.                 {
    8.                     UnityEngine.Debug.Log("GPU readback error detected.");
    9.                     _requests.Dequeue();
    10.                 }
    11.                 else if (req.done)
    12.                 {
    13.                                      
    14.                     mbuffer = req.GetData<Color32>();
    15.  
    16.                     buffer[(frameCnt[0]) % 10].CopyFrom(mbuffer);
    17.                    
    18.  
    19.                     frameCnt[0]++;
    20.  
    21.                     _requests.Dequeue();
    22.                 }
    23.                 else
    24.                 {
    25.                     break;
    26.                 }
    27.             }
    Format of my source texture is 1920*1080 but I was not set that from script but from editor game window. Yes, there is equal number of elements.

    Will try to make simple example without native plugin to demonstrate the problem.
     
    Last edited: May 11, 2018
  10. JulienF_Unity

    JulienF_Unity

    Unity Technologies

    Joined:
    Dec 17, 2015
    Posts:
    326
    Your code looks ok. Which rendering API are you using ? Is it DX11 ?

    As a side note, in 18.2 there's a new interface with a callback. You might want to try it to not have to poll / handle a queue: It might not solve your issue though.

    Code (CSharp):
    1. void OnRequestDone(AsyncGPUReadbackRequest request)
    2. {
    3.     if (!request.hasError)
    4.     {
    5.         var data = request.GetData<Color32>();
    6.         // consume data
    7.     }
    8.     else
    9.         Debug.Log("An error occured with the request");
    10. }
    11.  
    12. ...
    13.  
    14. // Request a readback with the callback
    15. AsyncGPUReadback.Request(source,OnRequestDone);
     
  11. shirocro

    shirocro

    Joined:
    Apr 30, 2018
    Posts:
    16
    Yes, I am using DX11.
    I have installed 18.2 it is nice callback feature but problem is still here. I have new problem, when I press stop in editor, got InvalidOperationException: The NativeArray has been deallocated.... and Unity stopped working. It is probably something with this new callback.. async buffers are deallocated before callback is disabled?

    I am still not sure is something wrong with native copy or async request. I have run one test where I have three nativeArray.copy actions one after another, sth like:

    mbuffer = req.GetData<Color32>();
    buffer[(frameCnt[0]) % 10].CopyFrom(mbuffer);
    buffer[(frameCnt[0]) % 10].CopyFrom(nColorRed);
    buffer[(frameCnt[0]) % 10].CopyFrom(nColorGreen);

    and immediately after that I got this (it was blue color in the buffer before first frame):

    Screenshot_20180512-190155.png
     
  12. JulienF_Unity

    JulienF_Unity

    Unity Technologies

    Joined:
    Dec 17, 2015
    Posts:
    326
    Ok, there is probably a bug here. Can you file a bug report with a minimal repro project?
     
  13. shirocro

    shirocro

    Joined:
    Apr 30, 2018
    Posts:
    16
    Tnx for answer, I will.
    Is there soulution for second problem:

     
    Last edited: May 14, 2018