BinaryFormatter Serialization saved file Broken when change from .NET 3.5 to .NET 4.x

My system mac Unity version 2018.3.4f1

The save file is saved by Scripting Runtime Version .NET 3.5. But if switch the Scripting Runtime Version .NET 4.x, the file cannot loaded. It detects the format is not correct.

What I found, using BinaryFormatter Serialization to save data, use Scripting Runtime Version .NET 3.5 can only open .NET 3.5 file, while using .NET 4.x only can open .NET 4.x Serialization file.

When using hex editor to open the two files. There is many different. The main is .net 3.5 file the opening is "Assembly-CSharp", but the .net 4.x file it opening is "FAssembly-CSharp"

I have a main problem, I am using serialization to save player game data, how do I upgrade the runtime to .net 4.x, but able to deserialize the .net 3.5 file?

Unity is deprecated .net 3.5, we need to find the way to read the serialization file when upgrade to .net 4.x

 

If unity moved to a different implementation of the .net framework for 4.x support (which I bet they did), this is definitely a possibility since the assemblies are different and may have different class/struct layouts in binary format.

This is what 'version lock' is all about. If your game is already released you usually tend to version lock that release and not move to new versions.

Unity still has .net 3.5 in Unity 2018.x, it's only deprecated. Deprecated does not mean "removed", it means it's "going to be removed in later versions. Specifically it'll be removed in 2019.x. As this article states:
https://forum.unity.com/threads/net-3-5-runtime-has-been-deprecated-in-unity-2018-3.601384/

As I said, if this is an already released title, you should get on the LTS (long term support) version of Unity 2018 and stay there in .net 3.5 for this game specifically.

...

If this is an unreleased game... well, who cares if the save file is popped. Delete and start over. It's an unreleased game.

 

This is why its always best to use human readable serialized file formats.

You could also build a converter pipeline. Open a file with the old binary formatter. Convert it to human readable. Then save again with the new binary formatter. (Or leave it as human readable, because that's better).

 

Use convert pipeline is one solution, but I am not way to check is it all users have upgrade to new version? If there is a user have not upgrade to new version, or upgraded to new version but have not open game let me to convert the saved file. Once I change the .Net framework to 4.x. They will lost the save file.

Convert the save file to .net independent file first, and then wait 2 years to not update the .Net version is one solution, but not a good idea. I want to find a method, let me can load the old .Net version file on new .Net version runtime.

Any idea?

 

The general idea is that you ship versions of the game which can open both files. Attempt to open the file with the newest version of the format. If that fails, attempt again with the old format. When you save, use the new format, so everyone's saved game gets updated when next they play it.

The trick is figuring out how to get one build to be able to open both versions of the data. I've not tried something like this, but can a .NET application use a DLL compiled with an older .NET version? If they build to compatible IL this might be a possibility, I'm not sure.

Human readable formats are often both big and slow, though.

That said, they do avoid dependencies like this one, though, and they are a much safer alternative for any kind of cross-platform data.

If it's unreleased then presumably giving people a converter tool would be acceptable? It could potentially even be hosted on a server somewhere so that it's transparent to players as long as they're online.

 

Your suggest is not possible, it is Unity problem. When you using .Net 3.5, you are no way to save as .net 4.x format. When you switch to .NET 4.x, it can not backward readable to .NET 3.5 format. I hope Unity can give a suggest how to do.

 

The real question is why would you change this many things under a released game? It's usually a very bad idea.

With that said: one solution is to build an application in unity which does only the conversion (so minimal app), build with 3.5 .NET, just load the save and convert to an intermediate format. Then the updated game app can read the temp file and save in the new format.

I would really advise against human readable formats when it comes to serialized game saves. (Depends on the save file size of course) It usually is super-slow and super-resource-hungry. Reading and parsing a giant string is painful in every language where the string objects are considered immutable. Although if you think this special circumstance will not be a rare event in the future, you can consider.

 

Something important about formatting changed between 3.5 and 4.x.
The framework is now using the current culture for most operation, whereas it was using the standard one previously.
I noticed it because I am in France, and we write "2,3" instead of "2.3", all my savegames were using US format ("2.3") and the files couldn't be read anymore.

Of course, it is more likely to hit harder text serialization rather than binary serialization, but it still may be the origin of your problem.
I fixed it by forcing the use of the US culture for all the threads at the very start of the game.

 

If you want your current users to still be playing the game years down the line, you'll almost certainly need to add features and updates somewhere along the line to keep them engaged.

Also, especially where mobile apps are concerned, as new devices and versions of the OS are released, updates will sometimes be required to fix crashes with new devices. In addition, sometimes particular bugs can only be fixed by building with a newer version of Unity, if the bug is in Unity rather than your own code.

As long as proper testing is done beforehand, making changes to an already released game doesn't need to be a problem, but I wouldn't recommend it without good reason.

Edit: Also from time to time Google Play change the minimum requirements for uploading apps (including updates to existing apps), which means that fixing bugs in an older app may require a larger update than expected.

 

Mind you, when I say "unreleased", I mean that no one would have the game. It's unreleased.

As opposed to early-release or something.

 

Totally agree, this is why I talked about 'version-lock'.

And I mean version-lock of the engine, not version-lock of your product. You can still develop for it, just don't move forward on the version of the engine. LTS versions are always really good for this.

The behaviour is common in the development world. For example here in my office (enterprise software) we still use VS2013 because of version-lock. We plan to move to the latest version in the next year or so, but have been on version-lock for quite some time due to the state of our product. We don't want to introduce unknown bugs to the system.

Our games Prototype Mansion and Garden Variety Body Horror are also on version lock as well. I plan to push to LTS in the next month, but they will remain on LTS for the release of our 3rd game in that trilogy. We won't be moving to newer versions of Unity until our title following that. There's nothing we can't do in LTS currently. I mean sure we might like some features in 4.x, but we don't "need" those features. I've been putting up with .net 3.5 for years despite in other jobs I do I'm often on the latest version of .net (with the exception of this enterprise product I'm working on right now).

I use json all the time for my save files.

Of course don't load the whole file in as a string, that would definitely chew up memory. So I use StreamReader/TextReader to stream the file.

Like here I use a TextReader and StringBuilder to reduce memory consumption (and gc) while parsing in json:
https://github.com/lordofduct/space...erialization/Serialization/Json/JsonReader.cs
(of course you can't nil all of it out, memory consumption is going to happen)

And a TextWriter when writing (though there's not a whole lot of string parsing when writing... pretty straight forward):
https://github.com/lordofduct/space...erialization/Serialization/Json/JsonWriter.cs

With all that said, it still is a fatter file and slower to parse than binary. But I barely notice a difference in game. But my games aren't that massive. My save files are only a few hundred lines at most.

 

I haven't said anything about the game itself. I said this about the tool you use to develop such game. But see the detailed explanation at @lordofduct he nicely explained it while I was sleeping.

This is why I mentioned that it depends on the size of the save. Sometimes it does not worth the development time to do one way or another. So yeah it really depends what and how much you read/write and how often and in what situations.

 

Any Unity guys there? Can reply something?

 

I found that after change the framework version, it is only unable to read the "Dictionary" data, other data can success Deserialize.

 

I think it should be bugs of Unity.
The Dictionary data can success deserialize when change the .net version, if the Dictionary data is not initialize or contain data. ie. if the dictionary data cont != 0 or not initialize the "BinaryFormatter Serialization" will not broken.

Therefore, it must the bugs of Unity.

 

Has anyone found a solution to this yet?

I've tried loading in the .net 3.5 DLL to do the loading part when it fails in the 4.0 version, but haven't gotten anywhere with that.

My other solution is to find a 3rd party (de)serializer that can load in the 3.5 version and then convert it to its own format. I think this is the more likely one to actually work, but I haven't found a library yet that can handle this.

 

Not a Unity bug. It is expected behavior. It is a bug in your save system that you inadvertently included a dependency on the output of BinaryFormatter serialization never changing between .net versions.

If I were writing a solution for this here's what I would do:

1) Create a separate application with .net 3.5 which takes a command line switch directing it to old save files
-- This application reads in the old save files and then saves them in your own text format, with no dependency on .net versions (so no BinaryFormatter)
-- Make sure you force a specific culture setting when outputting things like numbers to the text file save
2) Add old save file detection to your game, either try to read the save file and check for failure, or open the save file and check for something that gives it away as an old file
3) When an old save file is detected, you launch the application from step 1
4) Check for new text format saves from the step 1 application, and when seen your game converts them to your current format
-- Make sure you force the same culture when reading the text file that you forced when saving it
-- I'd suggest your current format to not be dependent on .net versions, so maybe you use the text file format as your current save format, but that's up to you

 

How can you explain, the deserialize will success, if it has data?
I have a solution, I alway write a dummy data in to the dictionary variable. The serialization and deserialization will no problem.

 

I don't understand your question. If you're having no problem, then what was the point of this thread? I thought the point was you are having a problem.

All I was saying is the source of your problem isn't a Unity bug.