Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  3. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Stream overloads on the JSON API

Discussion in '5.3 Beta' started by Biro456, Oct 4, 2015.

  1. Biro456

    Biro456

    Joined:
    Dec 22, 2012
    Posts:
    8
  2. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,649
    Do you need full streams, or would 'FromJsonFile,' where you pass the file path as a string, be enough?
     
  3. mdrotar

    mdrotar

    Joined:
    Aug 26, 2013
    Posts:
    377
    Is there a technical reason that would make it difficult to accept a Stream input? I don't have a specific use case for it right now but it seems like supporting the more general case of a Stream would allow for a lot flexibility and future-proofing. Then you've got a file stream, memory stream and network stream all supported out of the box.
     
    spryx likes this.
  4. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,644
    The serializer/deserializer is implemented in C++, so it cannot really pull data from managed stream objects. I believe the best it could do is to read the stream to end before starting deserialization process.
     
  5. mdrotar

    mdrotar

    Joined:
    Aug 26, 2013
    Posts:
    377
    Ah, so difficult to do efficiently. You might not be able to pull data directly from the Stream objects but could you have an adapter class marshal the data in chunks? Maybe that's tough to do efficiently as well.

    Reading the stream to end before starting deserialization would probably be good enough for my typical uses (as long as that is documented) but it would be good to hear what others think about it. Then again, unless your JSON parser can handle parsing in chunks, then you'd be reading it to the end anyway.
     
  6. superpig

    superpig

    Drink more water! Unity Technologies

    Joined:
    Jan 16, 2011
    Posts:
    4,649
    The problem is that each read from the managed stream would involve a full trip from native code (where the JSON serializer is setting up its input) to managed code (to call Stream.Read()) and frequently back to native code again (implementation of e.g. FileStream)... and _then_ marshalling the returned data back along the whole pipe the other direction. I think we could probably implement it, but making it fast would be a whole other problem...

    What does make sense is adding something like ToJsonFile/FromJsonFile, where you pass the file path to native code once, and then all of the file handling and de/serialization can happen in native code as efficiently as possible. This is pretty high on the roadmap for the feature right now, after support for TextAsset and for deserialisation from background threads.

    If you want to read the stream to the end before deserialisation, you can do that right now: you can read the stream into a string, and then pass the string into JsonUtility.
     
    ImpossibleRobert likes this.
  7. mdrotar

    mdrotar

    Joined:
    Aug 26, 2013
    Posts:
    377
    Thanks for the explanation.
     
  8. Biro456

    Biro456

    Joined:
    Dec 22, 2012
    Posts:
    8
    The general idea is flexibility as mdrotar already talked about. I believe that, for most use cases, strings and files should be enough... but there will always be someone that needs this or that extra feature.
    Knowing that it's implemented in native land instead of managed already gives a good idea of what might (or not) be possible with it in the future.
    Thanks for the response.