Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Preferences file not big enough

Discussion in 'Editor & General Support' started by monark, Jan 21, 2010.

  1. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,559
    How do you handle writing data to a users machine that needs to be bigger than 1Mb?

    I looked into this before and the suggestion was to write to a db on the server, the problem with that is for large files it's too slow. Has anything moved forward in this regard, is there any other way to write more than 1Mb of data to a users computer, with their permission of course.
     
  2. defmech

    defmech

    Joined:
    Feb 24, 2007
    Posts:
    507
    Would you mind giving more detail about what you need to do? Hard to give suggestions if we don't know what kind of data you need to store and for what purposes
     
  3. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,559
    It's an xml file, if I could compress it and store that I could fit a bunch more stuff in it but 1Mb would still be pretty limiting, and as far as I can tell you can't put binary data in a preference file.

    The xml file contains a custom format for a bookmarking system that stores the current state of the 3d scene. My problem is that there can be 2,000 or more models visible and the xml file is then about 120kb in size meaning I can only store about 10 bookmarks before I run out of space.
     
  4. CoherentInk

    CoherentInk

    Joined:
    Jul 16, 2006
    Posts:
    216
    Sounds like you should save a file to the local filesystem. Have a look at this thread. You can write arbitrary amounts of data to the local filesystem that way.
     
  5. Eric5h5

    Eric5h5

    Volunteer Moderator Moderator

    Joined:
    Jul 19, 2006
    Posts:
    32,254
    You can't do that using a webplayer, and if you're not using a webplayer, you don't have the 1MB PlayerPrefs limit anyway.

    --Eric
     
  6. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,559
    Yeah this is for the webplayer.
     
  7. CoherentInk

    CoherentInk

    Joined:
    Jul 16, 2006
    Posts:
    216
    Sorry, I hadn't realised this was for the webplayer.

    1MB is a lot of text and you could have your own format for data and save it in PlayerPrefs as a string. You could compress and decompress the text when you save and load it which could reduce the filesize as well.
     
  8. Dreamora

    Dreamora

    Joined:
    Apr 5, 2008
    Posts:
    26,602
    You can't store more than 1MB in the playerprefs and your users will already hate you for storing that 1MB into their registry

    When doing a webplayer game I think you really should store the stuff serverside and use a login so they can use their stuff everywhere, not just on a single system.
    Otherwise you kind of defeat the use of the webplayer if it is just a "embedded install"
     
  9. zumwalt

    zumwalt

    Joined:
    Apr 18, 2007
    Posts:
    2,207
    cookies, chomp chomp chomp.. cookie monster loves cookies
     
  10. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,559
    @dreamora
    ok, so going back to my original thread. If I save this data server side how do I stop it grinding the app to a halt everytime I need to send 5Mb of data to the server?

    @CoherentInk
    Yes it's a lot of text but not enough, check my second post for details, how do you compress a string and still save it as a string? prefs can't store binary data.

    @zumwalt
    I've never used cookies before, any example code anywhere?
     
  11. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,559
    I've found a couple of interesting methods in .NET that might help here, assuming they are available in mono, haven't tried yet.

    System.IO.Compression

    and

    Convert.ToBase64String / Convert.FromBase64String

    So in theory I could compress the data, convert to a string and store that in prefs.
    In a crude test a 1.5Mb file compressed to a zip file became 124Kb, so this would be pretty good for me if it works.
     
  12. Dreamora

    Dreamora

    Joined:
    Apr 5, 2008
    Posts:
    26,602
    i would store them in related blocks and then load them block by block.
    I doubt you have 5MB of pure savegame bytedata that you need all at once, because thats a pretty massive amount given webplayers are normally in the 5 - 30 mb sizerange

    if you do I would drop the support a mail to find out if the optional caching license also allows you to store webplayer generated data


    as for System.IO: Unsure if that namespace can be used at all on webplayers
     
  13. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,559
    I thought about that but this is for a bookmarking system so i need to populate the bookmark menu all in one go before the user can access it, it's a lot of data because although I call them bookmarks its basically an entire scene file in the bookmark. I guess the way round that is just to store the bookmark names and then stream the actual data only when it's selected by the user, and then give them the traditional loading screen whilst the data is down loaded.

    it's an option but will need a fair amount of re-writing to impliment so I'm going to see if this compression thing is possible first. If not that may be my only option.
     
  14. Dreamora

    Dreamora

    Joined:
    Apr 5, 2008
    Posts:
    26,602
    I think what you need to check is what you store actually

    just transform wise that would be 139k objects which you are definitely not storing.

    what you should be storing is transform (what can be altered dynamically at all only) and the id of the object (a unique identifier, integer or less, by which the client can retrieve the asset)

    if thats what you use, you should be reconsidering if you want to let the users place 120k objects in a scene ;)
     
  15. Tempest

    Tempest

    Joined:
    Dec 10, 2008
    Posts:
    1,286
    Something I remember way back from MUD programming, was that we didn't save the entire character object at one time.

    We saved character stat information only when the character leveled up.
    We saved character quest information only when the character started/completed a quest.
    We saved character inventory once every 15 minutes.


    So you break that 1 MB file into several bite size chunks, accessing and storing only the pieces you need at a specific time.
     
  16. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,559
    I'm not storing transforms or anything like that, I'm storing a list of object names and shader names.
    If I have around 4,000 in a single bookmark that file comes out at 120kb.
     
  17. Dreamora

    Dreamora

    Joined:
    Apr 5, 2008
    Posts:
    26,602
    and those names are dynamic?
    if not you can create UUIDs from them too (-> stringtable), reduces it to 4 - 16kb :)
     
  18. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,559
    Now we're getting somewhere! that's a good idea. I just used the object names for readibilty, but yes that's a very sensible suggestion.
     
  19. Dreamora

    Dreamora

    Joined:
    Apr 5, 2008
    Posts:
    26,602
    cool :)

    generally anything the player can't change can be stored like that for retrieval :)
     
  20. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,559
    how would you store these in prefs without losing the benefit of creating them? It looks like they would still have to get stored as a string, so wouldn't that nullify the point?
     
  21. monark

    monark

    Joined:
    May 2, 2008
    Posts:
    1,559
    I think I've managed to get string compression working using SharpZipLib, see this thread for more details.

    http://forum.unity3d.com/viewtopic.php?p=259485#259485

    More testing is probably needed but on the face of it, it appears to work at least on the mac. Even if I end up piping data to the server this could really speed things up.
     
unityunity