Preferences file not big enough

How do you handle writing data to a users machine that needs to be bigger than 1Mb?

I looked into this before and the suggestion was to write to a db on the server, the problem with that is for large files it's too slow. Has anything moved forward in this regard, is there any other way to write more than 1Mb of data to a users computer, with their permission of course.

 

Would you mind giving more detail about what you need to do? Hard to give suggestions if we don't know what kind of data you need to store and for what purposes

 

It's an xml file, if I could compress it and store that I could fit a bunch more stuff in it but 1Mb would still be pretty limiting, and as far as I can tell you can't put binary data in a preference file.

The xml file contains a custom format for a bookmarking system that stores the current state of the 3d scene. My problem is that there can be 2,000 or more models visible and the xml file is then about 120kb in size meaning I can only store about 10 bookmarks before I run out of space.

 

Sounds like you should save a file to the local filesystem. Have a look at this thread. You can write arbitrary amounts of data to the local filesystem that way.

 

You can't do that using a webplayer, and if you're not using a webplayer, you don't have the 1MB PlayerPrefs limit anyway.

--Eric

 

Yeah this is for the webplayer.

 

Sorry, I hadn't realised this was for the webplayer.

1MB is a lot of text and you could have your own format for data and save it in PlayerPrefs as a string. You could compress and decompress the text when you save and load it which could reduce the filesize as well.

 

You can't store more than 1MB in the playerprefs and your users will already hate you for storing that 1MB into their registry

When doing a webplayer game I think you really should store the stuff serverside and use a login so they can use their stuff everywhere, not just on a single system.
Otherwise you kind of defeat the use of the webplayer if it is just a "embedded install"

 

cookies, chomp chomp chomp.. cookie monster loves cookies

 

@dreamora
ok, so going back to my original thread. If I save this data server side how do I stop it grinding the app to a halt everytime I need to send 5Mb of data to the server?

@CoherentInk
Yes it's a lot of text but not enough, check my second post for details, how do you compress a string and still save it as a string? prefs can't store binary data.

@zumwalt
I've never used cookies before, any example code anywhere?

 

I've found a couple of interesting methods in .NET that might help here, assuming they are available in mono, haven't tried yet.

System.IO.Compression

and

Convert.ToBase64String / Convert.FromBase64String

So in theory I could compress the data, convert to a string and store that in prefs.
In a crude test a 1.5Mb file compressed to a zip file became 124Kb, so this would be pretty good for me if it works.

 

i would store them in related blocks and then load them block by block.
I doubt you have 5MB of pure savegame bytedata that you need all at once, because thats a pretty massive amount given webplayers are normally in the 5 - 30 mb sizerange

if you do I would drop the support a mail to find out if the optional caching license also allows you to store webplayer generated data


as for System.IO: Unsure if that namespace can be used at all on webplayers

 

I thought about that but this is for a bookmarking system so i need to populate the bookmark menu all in one go before the user can access it, it's a lot of data because although I call them bookmarks its basically an entire scene file in the bookmark. I guess the way round that is just to store the bookmark names and then stream the actual data only when it's selected by the user, and then give them the traditional loading screen whilst the data is down loaded.

it's an option but will need a fair amount of re-writing to impliment so I'm going to see if this compression thing is possible first. If not that may be my only option.

 

I think what you need to check is what you store actually

just transform wise that would be 139k objects which you are definitely not storing.

what you should be storing is transform (what can be altered dynamically at all only) and the id of the object (a unique identifier, integer or less, by which the client can retrieve the asset)

if thats what you use, you should be reconsidering if you want to let the users place 120k objects in a scene

 

Something I remember way back from MUD programming, was that we didn't save the entire character object at one time.

We saved character stat information only when the character leveled up.
We saved character quest information only when the character started/completed a quest.
We saved character inventory once every 15 minutes.


So you break that 1 MB file into several bite size chunks, accessing and storing only the pieces you need at a specific time.

 

I'm not storing transforms or anything like that, I'm storing a list of object names and shader names.
If I have around 4,000 in a single bookmark that file comes out at 120kb.

 

and those names are dynamic?
if not you can create UUIDs from them too (-> stringtable), reduces it to 4 - 16kb

 

Now we're getting somewhere! that's a good idea. I just used the object names for readibilty, but yes that's a very sensible suggestion.

 

cool

generally anything the player can't change can be stored like that for retrieval

 

how would you store these in prefs without losing the benefit of creating them? It looks like they would still have to get stored as a string, so wouldn't that nullify the point?

 

I think I've managed to get string compression working using SharpZipLib, see this thread for more details.

http://forum.unity3d.com/viewtopic.php?p=259485#259485

More testing is probably needed but on the face of it, it appears to work at least on the mac. Even if I end up piping data to the server this could really speed things up.