DownloadHandlerBuffer.data GC Allocation problems

Hi.I recently wrote my own downloadUtil, using 'UnityWebRequest'. 'UnityWebRequest.downloadHandler' is 'DownloadHandlerBuffer'.Then each frame visits DownloadHandlerBuffer.data. After a while, Unity started to change card and finally causes the computer to freeze. Later I checked the Profiler and found that each time we visit 'downloadHandler.data'. GC it is as large as the currently downloaded file. This should beNot normal?

 

Unity version: 2018.1.1f1

 

This is only a guess, but my hypothesis is that the download is handled by native-side code which gives you a new managed-side copy every time you access the .data property.

That's inconvenient, but is consistent with other properties which access raw data collections from native engine code, such as (from memory) Mesh.vertices. It also somewhat makes sense, though I don't personally like that it looks like a variable when it doesn't behave like one.

So, when the download finishes, access .data once to store your own reference to that copy of the data (ie: "localData = downloadHandler.data"). Use that to work with the data, and null it out when you're finished.

That said, I'm not sure why this would freeze your computer. If you look at Unity's total memory usage, does that keep increasing?

 

Thats a school example on something that should have been a metod instead to indicate that something more than just variable access is going on

 

Thanks, but I need to write files every time I download, so what you said is not for me

 

I don't understand how what you do with the downloaded data effects how you want to access it? Why will this not work for you.

I completely and utterly agree.

 

Reading the docs, the .data property just wraps a call to GetData(), and GetData() "Returns a copy of the contents of the native-memory data buffer as a byte array." So, hypothesis confirmed.

Wait, my guess is that you're writing the file as it's downloading, so you need to access the most recently received bytes, and I see no other way to get access to them in UnityWebRequest. The files are largeish, so you don't just want to wait for isDone to be true.

That's... upsetting. Looking at the documentation that use case isn't well supported at all, because there is no way toaccess the received data otherwise. Edit: Yes it is, see below.

@AndersMalmgren is likely to know more on this that me, is there a .NET class that is a better fit for this than Unity's provided one that's also available in Unity's Mono?

 

Code (CSharp):

1. //This is the code I handle
2. IEnumerator _download()
3. {
4.     _webRequest.SendWebRequest();
5.     int offset = 0;
6.     ulong lastLenght = 0;
7.     while (!_webRequest.downloadHandler.isDone)
8.     {
9.         if (!string.IsNullOrEmpty(_writingAsyncError))
10.         {
11.             _errorHandle?.Invoke(_writingAsyncError);
12.             yield break;
13.         }
14.         _progressHandle?.Invoke(
15.             new DownloadProgressInfo(_webRequest.downloadedBytes - lastLenght,
16.             _webRequest.downloadProgress),
17.             DownloadMessageCode.GetMessage((int)DownloadMessageCodeTable.Progress));
18.  
19.         if (_webRequest.downloadedBytes > lastLenght)
20.         {
21.             FileUtil.WritingFileAsync(_webRequest.downloadHandler.data, _saveFilePath, true, errorHandle: x =>
22.             {
23.                 _writingAsyncError = x;
24.             }, offSet: offset);
25.             offset = (int)(_webRequest.downloadedBytes - 1);
26.             lastLenght = _webRequest.downloadedBytes;              
27.         }
28.  
29.         yield return null;
30.     }
31.  
32.     if (!string.IsNullOrEmpty(_webRequest.error))
33.     {
34.         _errorHandle?.Invoke($"Download Error.Error Message:{_webRequest.error},ResponseCode:{_webRequest.responseCode}");
35.         yield break;
36.     }
37.     _progressHandle?.Invoke(
38.            new DownloadProgressInfo(1),
39.            DownloadMessageCode.GetMessage((int)DownloadMessageCodeTable.DownloadComplete));
40.  
41.     if (_webRequest.downloadedBytes > lastLenght)
42.     {
43.         FileUtil.WritingFileAsync(_webRequest.downloadHandler.data, _saveFilePath, true,
44.             errorHandle: x =>
45.             {
46.                 _errorHandle?.Invoke(x);
47.             },completeHandle: x =>
48.             {
49.                 FileUtil.DeleteFile(_serverFileInfoFilePath);
50.                 _completeHandle?.Invoke(_saveFilePath);
51.             }, offSet: offset);
52.     }
53. }

 

Sorry, forgot to reply=-=
>>Oh

 

If you absolutely have to stream your data rather than processing it when the download is complete, you could look at the ReceiveData method. It sounds like it's intended for streaming data as it arrives.

https://docs.unity3d.com/ScriptReference/Networking.DownloadHandler.ReceiveData.html

 

Oooh, you're right, I missed that!

 

Thank you, thank you very much!

 

Okay thank you

 

DownloadHandlerFile That's what I want! Thank you very much!

 

How to make DownloadHandlerFile append write? There is no description of this block in the document. covered Download is now . This is not the result I want. I want to implement my own DownloadHandler, but I cannot achieve it. The error "is inaccessible due to itsProtection level"

 

Okay thank you