How do you store game data (damage, spell data etc)

Just wonder how everyone does this/what are the common practices.

Let's say you have an RPG with various spells that do different damage, and have various assoicated data.

1) How do you normally store this data? Do you write it to XML files, CSV files, store it in a database, something else?

2) How do you write to this data? Did you create an external program, write a GUI Editor extension for unity, do it manually?

3) How/when do you retrieve this data? Do you load it upfront, or just when that spell is used?

Just interested in everyones thoughts on this.

 

There's no universal answer, since the specifics will depend on what makes sense for your game, your data set, and the way you have that data represented at run-time. However, for simple applications, the PlayerPrefs API works well. Start with that, do whatever is easiest / makes most sense to you based on where you are now, and build from there. If you find yourself outgrowing PlayerPrefs, you can replace it with something more scalable/robust at that time.

 

Hmm wouldn't player prefs be used to store the users preferences etc? I'm guessing you must be meaning that you would use this to store for example, spells that a player has purchased?


Sorry I was really asking about storing your actual game data such as all of your spells and the damage they do etc - not specifically per player.

Let's say your game has 100 different spells, you would need somewhere to store the data for all of these, their descriptions, names, tooltips, what they do etc.

 

I'll see if I can be a bit clearer:

What I'm wondering, is the best ways to store your game data such as spells/abilities/weapons. Do you need to use XML/Databases - or is there another way in unity so that you can create each different spell and store the data directly in unity.

I mean do people make a prefab for every different spell and store data in that somehow, or some other crazy methods?

 

From what I've seen it depends on what you are comfortable with. Personally, hand-edited XML is fine for me so far but for less code-centric people having a few dozen prefabs probably is easier.

 

Sorry, didn't realize you were talking about static / design-time data. Again, though, it depends on much the same things -- how much of it there is, what you need to store (for example, for spells, you would probably have sound effects, gfx / particle systems, and other assets as well as the value data like type and amount of damage, range, etc.

You could create a game object or prefab for each that has all of that set, or you could externalize it in a database, XML/CSV/TXT file or whatever else you wanted. Again, for simpler cases, the simplest approach is probably best; create a script, drag it onto a game object, drop in the assets (sounds, particle systems, etc) and configure the value attributes through the inspector. That's certainly fine for a prototype or if you only have a limited number of such things to assemble.

If/when that starts to get unwieldy, you can look at investing the time and effort into implementing an external store, loader scripts, and authoring tools to work with your external data format if needed (since you'll then need some way to create those external data stores).

 

Thanks for the tips - I don't have a problem doing it either way as I have experience with XML, database and all that. In the end we are probably talking a few hundred of abilities/spells. Not sure if you would consider that alot.

I guess there are two things that it will come down to for me:

1) Which way is better for performance - doing it all in unity with prefabs etc means less calls to a database, but is there any performance impact? Not that the database performance would be an issue though, compared to say a webserver that gets thousands of hits per second.

2) Is it possible to extend the Unity GUI in such a way that you can write to a database/xml files from the editor and interface you create.

 

1) would depend on your implementation choices; for example, to overhead of a database query may not be desirable every time a spell is invoked; on the other hand, you might not want the additional start-up time of reading everything in up front. The answer, of course, it to measure whatever you *do* implement, and determine if it needs optimizing or balancing based on hard data.

2) absoulutely; take a look in the Resources section for the old Unite presentation on extending the editor as a good starting point.

 

For most of my projects I use a script I wrote called ConstantManager; it basically just reads every XML file I have stored in a certain directory and loads them all as individual variables when the game starts - I like XML because it's super fast and it allows for a very non-programmer friendly approach for editing the values (other people working on the game with me don't need to know where in the code I have a particular value set and used - they just need to open up the config XML file for that part of the game and read it in there).

You can do a lot of fun stuff to make it very clean; things like

Code (csharp):

1.  
2.     <SpellNode
3.           id="Fireball"
4.           castTime="2.5"
5.           manaCost="1020"
6.           damage="10,12" (the comma splits min and max values)
7.      />
8.  
9. I store these values as variables called id+attributename - so things like FireballcastTime = 2.5, FireballmanaCost = 1020. Then later I can access them by calling ConstantManager.GetFloat(spellName+"castTime");
10.  

Things like that are very easy to read and hand-edit. For some of our other XML uses though, we've built tools to allow us to edit them without touching things by hand (like our dialog engine, which has a ton of nested nodes and would be a nightmare to work with without some sort of editor).

We don't use Unity editor scripts to do any of that, but you certainly could (we like having external tools for the big things so our writers can work on dialog on machines that don't have Unity).

For your specific example I'd definitely use XML though, personally. It's not something that's going to change at run-time really, and the performance is significantly better than using a database (much easier to work with too).

Personally, my ConstantManager script reads all the XML at launch - and contains a bunch of functions like:

ConstantManager.GetFloat("stringName") that handle all the parsing for me. When an application is in heavy testing I actually do stuff like this:

Code (csharp):

1.  
2. My SpellCast Class
3.  
4. string spellID = "fireball";
5.  
6. float Damage {
7.  get { return ConstantManager.GetFloat(spellID); }
8. }
9.  

Basically I use a getter/setter that I'll eventually replace with a cached value. I like to use the getter's when testing heavily so that I can reload my XML database at runtime and have the game updated to the new values without having to stop/get back to that point (my CManager includes a forcereload function that refreshes the values). The performance hit isn't noticeable - though I do switch over to cached values when that part of my script becomes close to finalized.

 

Very nice, thanks for the tips everyone. I suspect I will end up using XML for this and look into writing a gui extension for unity to write the xml files.

 

Just wanted to revisit this.

Is there any reason why you couldn't/wouldn't use unity prefabs to store all of your spell data, without needing XML files at all?

I have a pretty flexible system that involves a prefab per spell, all using the same spell script component, each spell would also be linked to a number of different prefabs that also have specific scripts linked.

The end result could be that a spell might have 2 to 10 unique prefabs involved with it. However it would only use (and re-use) roughly 6 scripts to handle a variety of spells.

So if your game ended up with 100 spells, you could very well have 200 to 1000 prefabs just to store the data associated with spells.

Would there be any negative impact doing it this way? Because if there isn't, it's quite easy to create and maintain spells via the Unity Editor using this method.

 

If it's just you working on your game then it should be fine. One of the big advantages to XML is designers can create content without having Unity installed and without having to learn Unity's interface. You can also iterate quickly because you can create a framework that - for instance - loads some default files but then looks to an external directory to load newer data. This doesn't require you to recompile your game over and over again to tweak stuff.

 

Hmm good point about the tweaking stuff. In your final version you would want the xml files compiled in packages or something anyway wouldn't you?

I think I'll try a few out and see how it goes, because you can extend the Unity Editor veriy easily it makes things alot easier.

I know you could extend it to write to XML files but I think it's quite a bit mroe involved.

 

I have been storing quite a bit of data on disk as .txt files that I load at runtime and it's been a little nuts. Yesterday I sank all my time into realizing that when I split a string using "\n" there's still a "\r" character in there at the end of every line that I need to find and remove. Very annoying.

Long story short: I'm listening to you all talk about XML and I think that would be a better way to store my junk. Any online learning resources you would recommend? and how about the basics of loading and dealing with XML in my c# scripts in Unity?

Some links would really help me out.

Thanks!

 

If you're willing to incur the extra download size to include the System.Xml.Serialization DLL, then that's the easiest way to go: http://www.dotnetjohn.com/articles.aspx?articleid=173

P.S. you can't serilize your script objects this way, but instead you can serialize simple object representations of them and copy the values over.

 

Depends on what your goal is. If you want people to be able to essentially "mod" your game then 1 option would be to load the packaged data first and then have the game look at a specific directory for extra files. Quite a few recent RTS games have gone this route and I think it works well.

However, if you just want people to play your game and not hack around with it - then yes you would want everything packaged together.

 

Yeah I'm still debating over which way to go. Currently I have it setup so that you create an ability, you create a prefab for the Ability and attach an Ability script to it, allowing you to setup all of the values.

Then you create a number of extra prefabs for that spell for any particular effects that the spell does, like it might do damage or something, and you attach the appropriate effect script to that.

It makes it really easy to create your spells very quickly, the only problem is that one spell might involve say 10 prefabs in total, but they are using a common set of script components.

So if you had 10 spells you would have 100 prefabs in various folders that control the settings for each spell and related effects. These would probably go in your Resources folder.

I'm not sure if there is an issue with having lots of small prefabs like this, I mean they are like 6kb each, so very small, have no mesh, colliders or components like that. They do have transform components, purely because you have to.

If we were to do this via XML, we would need to create a small app or try and extend the unity GUI to basically do the same thing, but generate XML files instead of storing the data/spell values (like damage, mana used) in the prefab. On the other hand having data in XML makes it easy to do other things with (say generate a webpage of your spells/vlaues) - which is probably near impossible other.

Although I suppose it might be possible to write something in Unity that goes through all of your resources and generates XML files from their data.

 

Or you could enter that data in the prefab and let a script gather it and package the XML with it.

 

But then we might as well leave the data in the prefab and use it directly? Otherwise we would have to read in the xml again anyway.


That would be useful to generate a database or something though for other reasons.

 

You may try to use the Lightweight UnityScript XML parser: http://www.roguishness.com/unity/ by Flimgoblin.

 

Yes exactly, that is what I used it for : quickly fill a database of assets and have a one way synch. The synch becomes two way when the client is launched and the server sends it gameplay data.

It's helpful if your classes and data structures keep changing (ideally you would freeze those then tweak the gameplay but that's another discussion)

 

You are using this ?
I see it only parses, doesn't create the xml string

 

Think of your metadata as building a database. Every studio that I've worked for uses some combination of Excel / Google Sheets + JSON to manage metadata.

What we do is:

1. Store all metadata in Google Sheets, one sheet per 'table' in your game
   1. API servers run a build script to export from google sheets to CSV, convert to JSON. The JSON is delivered to the game via an API Endpoint. This allows us to update the game dynamically after launch very easily.
   2. For testing locally, we use Google2U to directly download the metadata (in JSON format) from Google Sheets. Google 2U also has 'game database' formats if you wish, I just prefer working with JSON. Working with Google 2U would also work if you aren't planning on a
2. Parse JSON into local game data models in Unity on startup. We use a master 'GameData' singleton with a collection of Dictionary<string, DataModel>() objects that can be easily referenced as needed by the game.

This technique works for any kind of game, is very scalable (small projects, just use Google2U to download locally, larger projects can pipe from google sheets into an API server).

Google Sheets works great because multiple designers can work at the same time on the metadata. JSON is great because you don't need the reliance on the massive System.XML dll.

 

A couple of very useful utilities:

Game Data Editor:
https://www.assetstore.unity3d.com/en/#!/content/24239

G2U
https://www.assetstore.unity3d.com/en/#!/content/11818

 

This is a massive necro FYI, but if you're posting for posterity then you (and everyone else) should know that we now have ScriptableObject databases to do this internally to Unity with no problem at all, and with a smaller penalty than even JSON since you don't need to include anything beyond the ScriptableObject class itself. That said, you can't easily share or allow outside editing, so it's a matter of what your particular project calls for.

 

Let's say I build my whole database using Scriptable objects as described.

case 1: I realize I need to refactor my data structure by changing a name, or even the classname of the SO itself. What happens to all the data I've stored? Is changing a field name or type easy or hard? Or migrating data in a v1 to a v2?

case 2: Somehow I make a typo... let's say I break inheritance to "ScriptableObjects" from my class. Or some other kind of weirdness. Is it possible that all the data entered can be lost or modified in a way that can only be fixed via manual? This has happened to me before with some of Unity's references. When I refactor something, linkages get broken that are impossible to find. A text file in json has no such issues, which means I'm leaning to it. Wondering if anyone has had issues like this?