Search Unity

UnityWebRequest an multipart file download

Discussion in 'Editor & General Support' started by jariwake, Nov 13, 2019.

  1. jariwake

    jariwake

    Joined:
    Jun 2, 2017
    Posts:
    100
    I need to download big files (hundreds of megabytes). It is not feasible to download them "normally" with UnityWebRequest using a DownloadHandlerFile because if the download fails for some reason, you have to re-download the whole file.

    I found out in this thread that it should be possible to download files in chunks using the Range header. So when you add the header with myUnityWebRequest.SetRequestHeader("Range", "bytes=0-1023") you only received the first 1024 bytes and the responseCode is 206 (partial content), as expected. But what then? How does one set up the download handler so it would append the downloaded bytes to the file? Or do I have to ditch the DownloadHandler and make my own solution using FileStream .Net class and FileMode.Append?

    I would be glad to see some example code as I was not able find one.
     
    Last edited: Nov 14, 2019
  2. Zenix

    Zenix

    Joined:
    Nov 9, 2009
    Posts:
    213
    Hey,

    So it's been ages since I did this, so there may be a better way, but I'll try post the relevant bits of my code and hopefully that helps.

    At the start of my download, I send a HEAD request to see if the server supports resume.

    Code (CSharp):
    1.  
    2.    var doWeSupportResume = new UnityWebRequest(_url, UnityWebRequest.kHttpVerbHEAD);
    3.             doWeSupportResume.SetRequestHeader("Range", "bytes=10-20"); //we send a HEAD request with an arbitary value for the Range header, if it returns '206' then resume is supported
    4.             doWeSupportResume.Send();
    5.  
    6.    var resumeSupported = doWeSupportResume.responseCode == 206;
    7.  
    If resume is supported, I check if there's an existing file on disk, and if so get the length of it.

    Basically...

    Code (CSharp):
    1.  
    2.                 _fileStream = new FileStream(_path + ".temp", FileMode.Append);//make sure we're appending to the existing file
    3.                 resumeOffset = (int)_fileStream.Position;
    4.  
    Then we send the request to get the rest of the file, using the filestream position as the range.

    Code (CSharp):
    1.  
    2.  var rangeHeader = "bytes=" + resumeOffset + "-";//so range is the current filestream position, to infinite (eg. get the rest of the file)
    3. download.SetRequestHeader("Range", rangeHeader);
    4.  
    5.  
    That should be the important stuff... I do most of this in a custom download handler, and of course have lots of error handling around it. If the download response code is '416', it means we specified an invalid range, in which case I just mark the file on disk as corrupt and then re-download the whole thing.

    Obviously this isn't the full code, you'll need to wait for requests to finish and stuff before checking response codes.

    Hope that's enough to get you on the right track.
     
    ep1s0de likes this.
  3. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,735
  4. jariwake

    jariwake

    Joined:
    Jun 2, 2017
    Posts:
    100
    Thanks for the answers. I ended up writing my own DownloadHandlerScript that is attached to UnityWebRequests (which have the Contet-Range) header. So each UWR downloads a 10 megabyte range, and only when the range is received, it is appended to a file on the disk.

    The solution works, but is is a bit counter intuitive. Mainly because the DownloadHandlerScript and UnityWebRequest have a 1 to 1 relation - when multiple UWR:s are sent, also multiple DownloadHandlerScripts instances are required.
     
    Symgy likes this.
  5. Jackrohit

    Jackrohit

    Joined:
    Mar 29, 2019
    Posts:
    10
    Hi Jariwake, Can you share your code?
     
    Symgy likes this.
  6. Symgy

    Symgy

    Joined:
    Apr 7, 2017
    Posts:
    1
    can you please share sample code for resume and pause?
     
  7. AgnosiaGames

    AgnosiaGames

    Joined:
    May 26, 2020
    Posts:
    57
    Code (CSharp):
    1. var download = UnityWebRequest.Post(url, form);
    2. download.SetRequestHeader("Range", "bytes=0-100");
    3. download.SendWebRequest();
    This gives full of data. but range is "bytes=0-100". What is problem in these codes?
     
  8. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,735
    Range download has to be supported by the server. If server doesn't support it, you'll get full data.
     
    AgnosiaGames likes this.