Search Unity

  1. Unity 2020.1 has been released.
    Dismiss Notice
  2. We are looking for feedback on the experimental Unity Safe Mode which is aiming to help you resolve compilation errors faster during project startup.
    Dismiss Notice
  3. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

UnityWebRequest an multipart file download

Discussion in 'Editor & General Support' started by jariwake, Nov 13, 2019.

  1. jariwake

    jariwake

    Joined:
    Jun 2, 2017
    Posts:
    98
    I need to download big files (hundreds of megabytes). It is not feasible to download them "normally" with UnityWebRequest using a DownloadHandlerFile because if the download fails for some reason, you have to re-download the whole file.

    I found out in this thread that it should be possible to download files in chunks using the Range header. So when you add the header with myUnityWebRequest.SetRequestHeader("Range", "bytes=0-1023") you only received the first 1024 bytes and the responseCode is 206 (partial content), as expected. But what then? How does one set up the download handler so it would append the downloaded bytes to the file? Or do I have to ditch the DownloadHandler and make my own solution using FileStream .Net class and FileMode.Append?

    I would be glad to see some example code as I was not able find one.
     
    Last edited: Nov 14, 2019
  2. Zenix

    Zenix

    Joined:
    Nov 9, 2009
    Posts:
    207
    Hey,

    So it's been ages since I did this, so there may be a better way, but I'll try post the relevant bits of my code and hopefully that helps.

    At the start of my download, I send a HEAD request to see if the server supports resume.

    Code (CSharp):
    1.  
    2.    var doWeSupportResume = new UnityWebRequest(_url, UnityWebRequest.kHttpVerbHEAD);
    3.             doWeSupportResume.SetRequestHeader("Range", "bytes=10-20"); //we send a HEAD request with an arbitary value for the Range header, if it returns '206' then resume is supported
    4.             doWeSupportResume.Send();
    5.  
    6.    var resumeSupported = doWeSupportResume.responseCode == 206;
    7.  
    If resume is supported, I check if there's an existing file on disk, and if so get the length of it.

    Basically...

    Code (CSharp):
    1.  
    2.                 _fileStream = new FileStream(_path + ".temp", FileMode.Append);//make sure we're appending to the existing file
    3.                 resumeOffset = (int)_fileStream.Position;
    4.  
    Then we send the request to get the rest of the file, using the filestream position as the range.

    Code (CSharp):
    1.  
    2.  var rangeHeader = "bytes=" + resumeOffset + "-";//so range is the current filestream position, to infinite (eg. get the rest of the file)
    3. download.SetRequestHeader("Range", rangeHeader);
    4.  
    5.  
    That should be the important stuff... I do most of this in a custom download handler, and of course have lots of error handling around it. If the download response code is '416', it means we specified an invalid range, in which case I just mark the file on disk as corrupt and then re-download the whole thing.

    Obviously this isn't the full code, you'll need to wait for requests to finish and stuff before checking response codes.

    Hope that's enough to get you on the right track.
     
  3. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    2,599
  4. jariwake

    jariwake

    Joined:
    Jun 2, 2017
    Posts:
    98
    Thanks for the answers. I ended up writing my own DownloadHandlerScript that is attached to UnityWebRequests (which have the Contet-Range) header. So each UWR downloads a 10 megabyte range, and only when the range is received, it is appended to a file on the disk.

    The solution works, but is is a bit counter intuitive. Mainly because the DownloadHandlerScript and UnityWebRequest have a 1 to 1 relation - when multiple UWR:s are sent, also multiple DownloadHandlerScripts instances are required.
     
  5. Jackrohit

    Jackrohit

    Joined:
    Mar 29, 2019
    Posts:
    2
    Hi Jariwake, Can you share your code?
     
unityunity