Search Unity

  1. Tune in to get all the latest Unity news live from Berlin on June 19, 6pm CEST. Set up a YouTube reminder!
    Dismiss Notice
  2. Unity 2018.1 has arrived! Read about it here
    Dismiss Notice
  3. Scriptable Render Pipeline improvements, Texture Mipmap Streaming, and more! Check out what we have in store for you in the 2018.2 Beta.
    Dismiss Notice
  4. Improve your Unity skills with a certified instructor in a private, interactive classroom. Learn more.
    Dismiss Notice
  5. ARCore is out of developer preview! Read about it here.
    Dismiss Notice
  6. Magic Leap’s Lumin SDK Technical Preview for Unity lets you get started creating content for Magic Leap One™. Find more information on our blog!
    Dismiss Notice
  7. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice

DownloadHandlerBuffer.data GC Allocation problems

Discussion in 'Scripting' started by Icarus-yika, Jun 12, 2018.

  1. Icarus-yika

    Icarus-yika

    Joined:
    May 24, 2017
    Posts:
    10
    Hi.I recently wrote my own downloadUtil, using 'UnityWebRequest'. 'UnityWebRequest.downloadHandler' is 'DownloadHandlerBuffer'.Then each frame visits DownloadHandlerBuffer.data. After a while, Unity started to change card and finally causes the computer to freeze. Later I checked the Profiler and found that each time we visit 'downloadHandler.data'. GC it is as large as the currently downloaded file. This should beNot normal? 2018-06-13_05-42-46.png Unity_2018-06-13_05-42-17.png
     

    Attached Files:

  2. Icarus-yika

    Icarus-yika

    Joined:
    May 24, 2017
    Posts:
    10
    Unity version: 2018.1.1f1
     
  3. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    10,579
    This is only a guess, but my hypothesis is that the download is handled by native-side code which gives you a new managed-side copy every time you access the .data property.

    That's inconvenient, but is consistent with other properties which access raw data collections from native engine code, such as (from memory) Mesh.vertices. It also somewhat makes sense, though I don't personally like that it looks like a variable when it doesn't behave like one.

    So, when the download finishes, access .data once to store your own reference to that copy of the data (ie: "localData = downloadHandler.data"). Use that to work with the data, and null it out when you're finished.

    That said, I'm not sure why this would freeze your computer. If you look at Unity's total memory usage, does that keep increasing?
     
    AndersMalmgren likes this.
  4. AndersMalmgren

    AndersMalmgren

    Joined:
    Aug 31, 2014
    Posts:
    2,069
    Thats a school example on something that should have been a metod instead to indicate that something more than just variable access is going on
     
    angrypenguin likes this.
  5. Icarus-yika

    Icarus-yika

    Joined:
    May 24, 2017
    Posts:
    10
    Thanks, but I need to write files every time I download, so what you said is not for me
     
  6. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    10,579
    I don't understand how what you do with the downloaded data effects how you want to access it? Why will this not work for you.

    I completely and utterly agree.
     
    Last edited: Jun 14, 2018 at 10:08 AM
    AndersMalmgren likes this.
  7. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    10,579
    Reading the docs, the .data property just wraps a call to GetData(), and GetData() "Returns a copy of the contents of the native-memory data buffer as a byte array." So, hypothesis confirmed.

    Wait, my guess is that you're writing the file as it's downloading, so you need to access the most recently received bytes, and I see no other way to get access to them in UnityWebRequest. The files are largeish, so you don't just want to wait for isDone to be true.

    That's... upsetting. Looking at the documentation that use case isn't well supported at all, because there is no way toaccess the received data otherwise. Edit: Yes it is, see below.

    @AndersMalmgren is likely to know more on this that me, is there a .NET class that is a better fit for this than Unity's provided one that's also available in Unity's Mono?
     
    Last edited: Jun 15, 2018 at 2:21 AM
  8. Icarus-yika

    Icarus-yika

    Joined:
    May 24, 2017
    Posts:
    10
    Code (CSharp):
    1. //This is the code I handle
    2. IEnumerator _download()
    3. {
    4.     _webRequest.SendWebRequest();
    5.     int offset = 0;
    6.     ulong lastLenght = 0;
    7.     while (!_webRequest.downloadHandler.isDone)
    8.     {
    9.         if (!string.IsNullOrEmpty(_writingAsyncError))
    10.         {
    11.             _errorHandle?.Invoke(_writingAsyncError);
    12.             yield break;
    13.         }
    14.         _progressHandle?.Invoke(
    15.             new DownloadProgressInfo(_webRequest.downloadedBytes - lastLenght,
    16.             _webRequest.downloadProgress),
    17.             DownloadMessageCode.GetMessage((int)DownloadMessageCodeTable.Progress));
    18.  
    19.         if (_webRequest.downloadedBytes > lastLenght)
    20.         {
    21.             FileUtil.WritingFileAsync(_webRequest.downloadHandler.data, _saveFilePath, true, errorHandle: x =>
    22.             {
    23.                 _writingAsyncError = x;
    24.             }, offSet: offset);
    25.             offset = (int)(_webRequest.downloadedBytes - 1);
    26.             lastLenght = _webRequest.downloadedBytes;              
    27.         }
    28.  
    29.         yield return null;
    30.     }
    31.  
    32.     if (!string.IsNullOrEmpty(_webRequest.error))
    33.     {
    34.         _errorHandle?.Invoke($"Download Error.Error Message:{_webRequest.error},ResponseCode:{_webRequest.responseCode}");
    35.         yield break;
    36.     }
    37.     _progressHandle?.Invoke(
    38.            new DownloadProgressInfo(1),
    39.            DownloadMessageCode.GetMessage((int)DownloadMessageCodeTable.DownloadComplete));
    40.  
    41.     if (_webRequest.downloadedBytes > lastLenght)
    42.     {
    43.         FileUtil.WritingFileAsync(_webRequest.downloadHandler.data, _saveFilePath, true,
    44.             errorHandle: x =>
    45.             {
    46.                 _errorHandle?.Invoke(x);
    47.             },completeHandle: x =>
    48.             {
    49.                 FileUtil.DeleteFile(_serverFileInfoFilePath);
    50.                 _completeHandle?.Invoke(_saveFilePath);
    51.             }, offSet: offset);
    52.     }
    53. }
     
  9. Icarus-yika

    Icarus-yika

    Joined:
    May 24, 2017
    Posts:
    10
    Sorry, forgot to reply=-=
    >>Oh
     
  10. JJJohan

    JJJohan

    Joined:
    Mar 18, 2016
    Posts:
    80
    angrypenguin likes this.
  11. angrypenguin

    angrypenguin

    Joined:
    Dec 29, 2011
    Posts:
    10,579
    Icarus-yika likes this.
  12. Icarus-yika

    Icarus-yika

    Joined:
    May 24, 2017
    Posts:
    10
  13. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    1,748
    You may also want to have a look at DownloadHandlerFile. Just note that is fairly recent addition and may not be yet available in the version you are on.
     
  14. Icarus-yika

    Icarus-yika

    Joined:
    May 24, 2017
    Posts:
    10
    Okay thank you
     
  15. Icarus-yika

    Icarus-yika

    Joined:
    May 24, 2017
    Posts:
    10
    DownloadHandlerFile That's what I want! Thank you very much!
     
  16. Icarus-yika

    Icarus-yika

    Joined:
    May 24, 2017
    Posts:
    10
    How to make DownloadHandlerFile append write? There is no description of this block in the document. covered Download is now . This is not the result I want. I want to implement my own DownloadHandler, but I cannot achieve it. The error "is inaccessible due to itsProtection level"
     
  17. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    1,748
    If you want to append, then you need to write your own download handler by extending DownloadHandlerScript.
     
  18. Icarus-yika

    Icarus-yika

    Joined:
    May 24, 2017
    Posts:
    10
    Okay thank you