Question Unity Serilization

Hello, everyone,
I'm looking for a way to save and load certain game data (Array of int[5120]) efficiently.
The basic problem is that I'm using the JsonUtility class, but it generates way too much garbage when converting the array to a string.
Is there a way I can write/read such data even without GC?

Thanks in advance

 

How much is too much? What is your limit? How does it affect your game? For example, if you save at an intermission point (ie end of level) then garbage isn‘t really an issue unless we are talking megabytes of garbage.

Any string based format will generate garbage. You may want to look into StreamWriter/Reader if you want to implement a custom format. Other alternative would be SQLite but if this array is all you need to read/write it‘s a bit overkill.

 

I have tile states (integer value) each as a block (array data). These are saved and loaded at runtime depending on the player's position. This happens every few seconds and 160 - 200kb of garbage is generated. Depending on how the garbage collector is called, the game cycle takes 2 - 6ms seconds longer.
For my kind of game (similar to terraria) this is too much.

I used SQLite and BinaryWriter/Reader class, but the result is no better. I assume that the Streamer class returns similar values, but I'll definitely try it out, thanks for that!

I saw that there is a preview version of a package called "Unity Serilization" but I can't install it due to bugs (Unity Version 2022.2.1f).
Does anyone happen to know how to use this at the current time and when it will be released?

 

If you go lower level and use a FileStream, you could go with Write. Use a fixed-size array, put chunks of your data into the array and write.

If you want to skip copying, consider if your original data needs to be integers, or if it can just be bytes.

 

Thanks for the useful information!
I'll definitely try that.

 

Be sure to implement the IO as an async background process so it won‘t interfere as much with regular gameplay. In that case garbage shouldn‘t matter as much as well, especially with the incremental garbage collector.

The stream API is async await enabled, but of course it won‘t work if your target platform is WebGL.

Btw I would expect the disk (persistent memory) access to be the determining performance factor, not in-memory garbage collection.

 

You're right. I should check the GC allocation again separately.

 

I tested again and my real problem was with the file generation and processing. After that everything went great and with the FileStream I was able to further optimize it. Thank you both for the quick support!

 

Has anyone ever tried FlatBuffers? I'd like to hear some thoughts about..

There are a few videos using it on Unity...

 

I haven't tried flat buffers. However, what's your use case? Is there a server that you need to use that is limited to the use of flat buffers? Sometimes the tool we use is just the easiest based on what sorts of formats we need to communicate across different applications.

For simplicity and default performance, I would use https://github.com/neuecc/ZeroFormatter. However, this means whatever is reading it needs to be able to support it as well.

If you need it to be human readable, I would use JSON from unity's serializer library. And if performance was a concern, I'd try the low-level API in bursted jobs where appropriate.

As a last resort, I'd just write my own solution using Unity's properties library. It doesn't take that much time for a simple implementation, and then you can modify it to your needs as they change.

 

Don't have anything in mind, just curiosity. I usually deserialize JSONs to custom Class by means of an intermediary ScriptableObject I use as a kind of scheme. Works fine enough for me. Until now I don't need to fetch anything remotely nor realtime, so no issues atm...