Best Practice redistributing your Localized text inside your App?

Hi

I have multiple localized text files. I chose JsonUtility to serialize and parse them. All good.

Help me brainstorm:
(A) Structure - what is the recommended way to structure the data, to help redistribution?
(B) Redistribution - whats the best way to redistribute the data inside the app? Also, reaction of app systems to change in data (change in language). I'm looking on suggestion for implementation.

Some issues i would like to solve:
(1) Tight coupling between the class that needs the data and the target data:
I chose to structure the data as nested classes (branches), and public strings (leaves).

This way, when parsed, i can do this to locate a text string (data bit / leaf) using code:

string fileExists = localizedData.ui.validation.fileExists;

which fetches a string:

File with that name already exists!

This way, i get code hints (yay!), reduce the clutter when asking for data in code and get a gc-free lookup (this is of importance to me).

However this way also, the asking class needs to know of the data structure, thus there is a tight coupling between the fetching class and the data, which i dislike very much.

Some 'solutions' to (1) that i can think of:
(1A) path string to data inside Inspector, and Reflection inside the class to get to the data. Something like this inside inspector of the class:

"ui/validation/fileExists"

Then use Reflection to get the actual string.
I dislike this, because it creates garbage every time i ask for a simple string!
(1B) Using SimpleJSON. I can use string inside inspector (1A) with this. Example:

Code (CSharp):

1. var N = JSON.Parse(the_JSON_string);
2. var fileExistsString = N["ui"]["validation"]["fileExists"].Value;

EDIT: this might be cached arrays/dictionaries, which would be GC-free and exactly what i need (TODO: test).
@Bunny83 Pog?
(1C) Serialized dictionary, flat leaf structure. Leaf = text string. GC-free (good), can supply key inside inspector to remove tight coupling (good), RIP code-hints (meh), flat structure with long keys/property names to store all the data (meh).

Also i can see, for example, when adding a function to say OnValueChanged of a UI.Toggle inside an inspector, the inspector UI actually let's me 'browse' the classes, their subclasses (..), and methods. If i could use this kind of functionality to set the path to the target data string via inspector for the target class, that would be awesome. And i guess i can do that - using reflection? Aww.. then each time i ask for such string, i get garbage. Right?
I can gc-free fetch via code - since the data is strongly typed - but don't want to write the path to data inside the code!

So what you folks using in terms of structure / serializer / storage type / pattern that you came to like?

In fact that is all for now. Brainstorming (B) is still welcome, but (A) and (1) is much more important, so let's keep it simple.

 

I've done a similar thing before with this setup:

• I don't use a key for the lookup, but simply use English as a base language, this prevents displaying keys in the UI where there is no translation, but has some words (like sheep) that might cause a problem. In this specific case I would use keys (sheep:single, sheeplural)
• The base language strings are in a static class, which is distributed over multiple files by using partial classes, so all tooltips are in one file for example
• The translations are in XML files, but since the structure is very flat, I think JSON might even be better. (In my case there are many more structured files, so I just use XML everywhere.)
• There is a central (static) translation class which stores all the translations

Now, the main thing is that it is all made for dynamic updating. If the translations are updated online, a new XML is downloaded, parsed and any text will be updated directly if needed.

To do this the translations are not returned directly, but a delegate (Action<string>) is supplied when requesting a translation. So if a translation is changed (because the language code has changed or new translations are parsed), the central translation class knows which delegates to call to update the translations. This is the rough form in the central translations class not taking thread safety into account:

Code (csharp):

1.  
2. private static string currentLanguage;
3. private static Dictionary<string, Dictionary<string, string>> translations = new Dictionary<string, Dictionary<string, string>>();
4. private static Dictionary<string, List<Action<string>>> receivers = new Dictionary<string, List<Action<string>>>();
5.  
6. public static void GetTranslation(string source, Action<string> receiver)
7. {
8.    if (!receivers.ContainsKey(source)) receivers.Add(source, new List<Action<string>>());
9.    if (!receivers[source].Contains(receiver)) receivers[source].Add(receiver);
10.    if (translations.ContainsKey(currentLanguage))
11.    {
12.       if (translations[language].ContainsKey(source))
13.       {
14.          receiver(translations[language][source]);
15.          return;
16.       }
17.    }
18.    receiver(source);
19. }
20.  
21. public static void SetTranslation(string language, string source, string target)
22. {
23.    if (!translations.ContainsKey(language)) translations.Add(language, new Dictionary<string, string>());
24.    if (translations[language].ContainsKey(source))
25.    {
26.       if (target.Equals(translations[language][source])) return;
27.       translations[language].Remove(source);
28.    }
29.    translations[language].Add(source, target);
30.    if (!language.Equals(currentLanguage)) return;
31.    if (!receivers.ContainsKey(source)) return;
32.    Receive(receivers[source], target);
33. }
34.  
35. public static void SetLanguage(string language)
36. {
37.    if (language.Equals(currentLanguage)) return;
38.    currentLanguage = language;
39.    foreach (string source in receivers.Keys)
40.    {
41.       if (translations.ContainsKey(language))
42.       {
43.          if (translations[language].ContainsKey(source))
44.          {
45.             Receive(receivers[source], translations[language][source]);
46.             continue;
47.          }
48.       }
49.       Receive(receivers[source], source);
50.    }
51. }
52.  
53. private static void Receive(List<Action<string>> receivers, string translation)
54. {
55.     foreach (Action<string> receiver in receivers) receiver(translation);
56. }
57.  

You could/should create some custom classes to reduce all those generic types. For example:

Code (csharp):

1.  
2. public class ReceiverList : List<Action<string>>
3. {
4.    public void Receive(string value)
5.    {
6.       foreach (Action<string> receiver in this) receiver(value);
7.    }
8. }
9.  
10. private static Dictionary<string, ReceiverList> receivers = new Dictionary<string, ReceiverList>();
11.  

 

@jvo3dc Oh thank you for that reply! I love to see this implementation, i would never have thought of that!
Having keys as 'what you want' defined in a separate class, to which all classes have access, so they can register/inquire by and get notified via that key - very cool!

But the main point remains - your registree needs to register by that specific key - that's where you get tight coupling - between the registree and the data model - via that key inside the code (well you could say that the key is actually a proxy to get to the data, but .. well not really, you still use it inside the class code to register to the translation?).

What i was looking for was a way to 'move' knowledge about the specific key outside of the registree class. So inspector was a way to do that.. well i guess you could still do that, by -say- if your key was an enum (that way it gets exposed in inspector, and your class code can use it, not knowing 'which specific key' its using (because it was set via inspector) - removing the coupling) - like imagine you have a list of generic items and each of them will be using some data, but to determine what data - do that via inspector, not code - well that's what i wanted. But then again it can be done with your decoupling through keys - another problem though is that each time you create a new text string, you need to create a key that corresponds to it in your static class (double trouble).

Still this was a great read, and i do like the registrees and callbacks idea, thanks for that!

 

Exposing the choice in the inspector can also be done when using strings, though it is a lot easier in the enum case. You'd have to create some class with a custom drawer to do it, but it's not impossible. Actually, the number of strings will probably increase very quickly and a single enum dropdown won't be very feasible anyway. Better to work with a category and selection in that category from the start, so then you're stuck with a custom drawer anyway.

Or you could just type in the text directly and let the UI component register that with the central translation class. Then you could export the list of keys through an editor extension, which would be the base for the actual translations. It actually matches well, since in my above way the delegates are already registered, so you already know which keys are used and by who. You just have to make sure to unregister the old text when it gets changed.

 

I'm using ScriptableObjects in my framework for localization.

Each time an asset has to declare a string, instead i make it a LocalizationVar and i have a property drawer that can generate it, or let me wire with the reference an existing one if its a text shared between relevant places. The property drawer also lets me edit the text "in line" so i don't have the overhead of selecting the asset first to modify the text.

Then I have a manager with a list of all the localization assets, which it can export to json, and import the json back in which case it writes to a string property in each localization asset what the current localized text is.

That way I have 0 lookup overhead, translations are sitting there in an asset ready to be directly accessed.

For UI text I have a component that can take care of generating the localization asset and handles refreshing the text if necessary.

If you're curious, feel free to check it out there:
https://github.com/met44/uStableObject/tree/master/uStableObject.Localization

The framework is kinda for my personnal use so its not heavily documented sorry about that, but hopefully it can inspire you on how to sort out your localization issues !

 

True true, i actually thought of it, but forgot to include it in my post. i do like using ints/enums tho, strings are heavier. but might be also use-case related.

Its possible, yes, but hot damn, i just realized Bunny83's SimpleJSON is exactly what i needed, hes using strings as keys tho, and i need to use those in inspector to register by, and oh my.. loosing strong typing in code is a pain, also the possibility of error mistyping a string through the inspector exists, harder to keep control of for sure. Still better than what i had (or didn't have *chuckles*).

 

Same here, its great way to share instanced data.

Sounds great if used for even just keys, ties up with @jvo3dc's answer.

That sounds cool, so you 'create' the localization data bits for each SO item separately, then pull it all in within manager SO? How about different translations for a key? All translations are defined in each item at once?

Much appreciated. Won't lie, prob wont get to it, unless i have extra time, my system is pretty much finished. But the concept sounds super intriguing, thanks for sharing!

 

Yeah the manager stores refs to all localization SOs (although i'm working on a game which will need to bundle specific subsets of the localization data so that will become possible sometimes in the near future).

So the way it works in the end is once you've exported the JSON file with all your translation data for the default language you're creating your game in, you can copy it and fill in another language (i store Hint strings as well to help translators with details they often ask).

Finally you load the json file for that specific language, it will fill the localized value back in each SO.

If no translation is found (a key is missing) then it falls back to the default language.

 

@_met44 Sounds like a cool system for designers to fill in)

Just curious, where do you store the hints? EditorPrefs or somewhere in your SOs available only in UNITY_EDITOR?

 

Oh its very simple, there's an additionnal string in the LocalizationVar itself that gets exported to the json to make it easily accessible to translators.

Depending on how the loc var was created it gets filled in, for example if it gets created from a scriptableobject asset then it takes the words after the last - (since i usually prefix stuff with - to separate and last part is the most specific) and the variable name.

Its also possible to select the loc var and edit it, next json export the change will be reflected. And there's also a version number that gets updated whenever a locvar changes to be able to quickly spot with versionning entries that changed in the exported json file.