Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Unity 2017.3 HTTP POST request always empty (Django web app)

Discussion in 'Scripting' started by jameskanestl, Feb 11, 2018.

  1. jameskanestl

    jameskanestl

    Joined:
    Nov 3, 2017
    Posts:
    12
    I'm trying to use Unity 2017.3 to send a basic HTTP POST request from Unity scripting. I want to send an image data, which I can access in my script either as `file.png` or in a `byte[]`. I am only posting to a local server running Python/Django, and the server does register the POST request coming in - but no matter what I do the request arrives empty of any content, body, files or raw data at my web app.

    ``` IEnumerator WriteAndSendPng() {
    // extra code that gets bytes from a render texture goes here
    //can verify that drawing.png is a valid image for my purposes

    System.IO.File.WriteAllBytes("drawing.png", bytes);
    List<IMultipartFormSection> formData = new List<IMultipartFormSection>();
    formData.Add( new MultipartFormFileSection ("drawing", bytes, "drawing.png", "image/png") );
    UnityWebRequest www = UnityWebRequest.Post("http://127.0.0.1:8000/predict/", formData);
    yield return www.SendWebRequest();
    if(www.isNetworkError || www.isHttpError) Debug.Log(www.error);
    else Debug.Log("Form upload complete!");
    }```

    I'm following [the docs](https://docs.unity3d.com/Manual/UnityWebRequest-SendingForm.html) and there are [a bunch of constructors for MultipartFormFileSection](https://docs.unity3d.com/ScriptReference/Networking.MultipartFormFileSection-ctor.html), most of which I feel like I've tried. Also tried UploadHandlers, or the old AddBinaryField WWW API - still the request is always empty when it hits my app... I've read the thorough response to this SO ticket, https://stackoverflow.com/questions/46003824/sending-http-requests-in-c-sharp-with-unity. I have tried many of the implementations here but again, Django receives empty requests. Even submitting the simplest possible request from Unity sends empty requests. So weird.

    ``` List<IMultipartFormSection> formData = new List<IMultipartFormSection>();
    formData.Add (new MultipartFormDataSection ("someVar=something"));```

    The Python server just sees:

    ``` [11/Feb/2018 14:14:12] "POST /predict/ HTTP/1.1" 200 1
    >>> print(request.method) # POST
    >>> print(request.encoding) # None
    >>> print(request.content_type) #multipart/form-data
    >>> print(request.POST) # <QueryDict: {}>
    >>> print(request.body) # b''```

    I thought it might be a Django issue, but posting to the same app w/ Postman or from other sources, it sees the incoming data just fine. Anyone done this recently? I thought this would be a piece of cake and many hours into into it I remain stymied. All help appreciated! Thanks, all.
     
    karelium likes this.
  2. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,722
    The issue most likely is that your Django server does not support HTTP 1.1 chunked transfer. You can either try updating your server, or set chunkedTransfer property on UnityWebRequest to false.
     
  3. jameskanestl

    jameskanestl

    Joined:
    Nov 3, 2017
    Posts:
    12
    Holy sh*t, it appears you're right! Disabled that option in Unity, now I can see the simple test cases coming over the line in Django! :D Now that I can get data at all, time to figure out the image stuff. Great call, thanks much!
     
    karelium likes this.
  4. jameskanestl

    jameskanestl

    Joined:
    Nov 3, 2017
    Posts:
    12
    More interesting side effects. I can now get data across the wire, but Django's request.POST and request.FILES don't populate. If I write request.body to a .txt file, I can see it does look more or less like a reasonable web request, but the app doesn't know what to make of it for some reason. There's an empty line to start, and some --base64 stuff (I think) -- not sure if that is interfering... Will keep at it.

    --m3Q3WjvRKLySJZblTxrdeFiKhtuIQ7Crecom1yrm
    Content-Disposition: file; name="drawing"; filename="drawing.png"
    Content-Type: image/png

    ‰PNG
    IHDR ð ð :R¼ IDATxìØÁ 0±¶ûïœÐ1” Œœ—ïÌG€ @€ @€ ‚À+„”‘ @€ @€
    // and so on...
     
  5. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,722
    It could be because of Content-Disposition being 'file'. Newer Unity versions have this fixed. I thing it should be "form-data".
     
  6. jameskanestl

    jameskanestl

    Joined:
    Nov 3, 2017
    Posts:
    12
    Not sure, I see similar results even excluding the file part. After sending the simplest multipart form w/ formData.Add (new MultipartFormDataSection ("a=b"));, request.POST doesn't populate. Writing request.body to a .txt, I can still see what looks like a standard web request. May well be a Django issue/option/flag, going to check their forums.
     
    Last edited: Feb 12, 2018
  7. jameskanestl

    jameskanestl

    Joined:
    Nov 3, 2017
    Posts:
    12
    Yeah, no luck no matter what Content-Type I try to set.

    I can use Postman or a simple web form to send data and image files to the Django server and it serves it up into a QueryDict in request.POST and request.FILES as expected, but even the simplest Unity request stays tied up as raw byte[] data in their request.body object.

    It might have something to do with encoding? Here is a screenshot of the full request from requestbin. Nothing looks amiss, the boundary is generating correctly (afaik)... Anybody spot any trouble?



    As suggested by Aurimas, in the successful Postman requests, the Content-Disposition for each multipart/form-data field is set to form-data, not file... but I can't edit the Content-Disposition of each Unity multipart section. Aurimas suggests in newer versions of Unity this is fixed... I'm on 2017.3, but am downloading the 2018.1 beta to test it out. Here's hoping!
     
    Last edited: Feb 12, 2018
  8. jameskanestl

    jameskanestl

    Joined:
    Nov 3, 2017
    Posts:
    12
    Okay, this is solved. I was using Python 3.5 and the Django test server, erroneously thinking TensorFlow only supported <= 3.5.x. When moving the app to a Heroku production server, I had to bake a requirements.txt file and in doing so I figured out I could indeed use the latest Python version, 3.6.4. Switching to that version made the Django test server start receiving requests properly. I still need a whole mess of boundary generation string building code, but it actually works!
     
  9. bevangelisti

    bevangelisti

    Joined:
    Feb 16, 2018
    Posts:
    3
    Anyone knows if there is any security risk switching chunkedTransfer to false for POST requests?
    In my case there wasn't any update or changes in Unity or the server setup (I'm not using Django, just plain PHP) and POST requests began to arrive NULL to the server now. The only thing in between was a few xCode updates, i'm thinking maybe that was the cause.
    Thanks!
     
    Last edited: Jun 18, 2018
  10. Aurimas-Cernius

    Aurimas-Cernius

    Unity Technologies

    Joined:
    Jul 31, 2013
    Posts:
    3,722
    There is no security risk, either option is equally good in terms of security.
     
  11. bevangelisti

    bevangelisti

    Joined:
    Feb 16, 2018
    Posts:
    3
    Thank you @Aurimas-Cernius !