Feature Request Cross platform IO handling

We make games for PC and consoles, and writing and loading files (like save files) has always been a bit of a mess.

The console platforms generally don't support normal C# file operations (like File.Write, etc.). Instead, they have their own, special API that you use, each with their little idiosyncrasies, but all of them generally give you access to the same kind of operations - read a file, write a file, check how much space there is left, etc.

We don't do mobile dev, but I've heard that the same kind of problems exist on mobile platforms.

Could it be possible for Unity to make a unified IO system that worked across platforms without us having to re-implement the same things for each console we port to? The same way that the input system allows us to say "what's the state of the right stick?" without having to care about if we're talking about a Playstation, XBox, Switch, or possible future console system that hasn't been made yet.

Of course this system wouldn't be perfect - there's platform-specific ways you have to initialize stuff, and platform-specific ways you have to let the user know that there's not enough space to save and so on - but being able to use the same API for the things that are the same, and then only branch by platform where it mattered, would help a ton with development.

 

Not @Unity, but sounds like you're describing an abstraction layer, which sort-of already exists, via .NET's existing Serialization mechanisms. You choose your base format - JSON, XML, or Binary - and use a mix of attributes & callbacks to handle your custom serialization. Supports versioning, field-changes, type fidelity, contracts to require/validate data, all that good stuff.

You'd still have to write whatever code-behind was needed to comply with each desired platform's SDK and your own project's needs, and adjust your .NET config settings (to just get the classes you need, without extra bloat), but it's probably the most flexible/robust/maintainable approach IMO.

[Edit: My late hubby loved Girl Genius - congrats on your KS!]

 

To make it clear - our problem is not serialization or versioning or any of that at all. That's easy, and works cross-platform out of the box already. Our problem is that taking a string or an array of bytes and writing it to or reading it from a path is wildly different APIs on different console platforms.

What I'm saying is "hey, this is annoying to do and error-prone and every single game studio has to write the exact same code, if Unity could make a cross-platform API for raw data to/from file and maintain that for all the platforms that allow IO, that would help Unity users a lot"

 

Hi! Thanks for the reply.

For our last 4 games (from World to the West to Girl Genius), we've been using the same base system with some modifications. We're trying to replace it because it made some mistakes, so this might be interesting if you're looking into making something like that.

The old system ("rain saving") is pretty straight forwards:
- there's a single SaveFile with a List of SaveSlots.
- there's a single SettingsFile
- Yaml.net is used for serialization/deserialization
- All saving and loading is asynchronous
- New platforms are handled using #if definitions. For example, there's a method that writes the data to file. Each version of the method is hidden behind #if PLATFORM. So if you add a new platform, you'll immediately get compilation errors for the methods you haven't implemented. If you implement those (correctly), saving and loading will work.
- Game-specific data is handled with partial classes. So the package defines that there is a SaveDataFile with SaveDataSlots, and SaveDataSlot is partial. So the game that uses the package makes a folder with an asmref to the asmdef of the package, and creates a new partial SaveDataSlot with the game-specific data.


This has quite a few really bad drawbacks that's been causing pretty big problems;
- The system is opinionated about which files existed. This meant that adding new files would cause us to have to do a lot of refactoring. We ended up stuffing some data that really ought to have been in it's own file into a SettingsFile because we were late in production and didn't have time to refactor both the package and our game. We also couldn't choose to split each save slot into it's own file, even though that would have helped great with performance.
- The system has a hard-coded serialization framework. This is very inflexible - different serialization methods have different up- and downsides, and yaml.net has caused a lot of problems (like bad performance) that would've been easily fixed by switching to a different method, but that would require a lot of work since it's embedded into the save system.
- The system can only do asynchronous save/load. To be fair, that's what you want most of the time for save files to not drop frames, but the settings file needs to be loaded before you do anything if you want to respect the user's volume settings in the game's splash screen. So on startup, you probably want to hang the main thread until the settings file is ready. You also want to do sync file handling when doing editor tooling and in a bunch of other instances.

As a result of all of these issues, we're looking into building our own IO abstraction to get away from the system. These are the things we're working on supporting in our new system:
1: A system for pure IO. So this is taking either a string or byte[] and writing/reading it to/from a path, both synchronously and asynchronously. Paths here are all relative to the root of whatever you're writing to - so Application.persistentDataPath on PC.
2: A system that lives on top of this system that defines a serializable file. It's got data and a path, and you inject serialization and deserialization on construction. This is what would replace a SaveDataFile.
3: Platforms are handled through injection as well - there's a central place where platform implementations send in an object that handles file IO for the platform.

If you wanted to do this job for me (and other developers), I think the correct approach would be:
- Offer the equivalent of 1 as an API.
- Solve 3 for all the platforms you support. So your Switch plugin would include code to make IO on Switch, etc.
- Allow users to inject their own code for 3 on platforms you don't natively support.
- (not strictly necessary): Possibly have a version of 2. This could default to using JsonUtility to serialize files, so the same rules for serialization that Unity users are accustomed to applies, but it would need to support custom serialization formats for custom needs like better performance, partial writes, etc.