Get Arabic Text From JSON File in Android

Hello,
I am making a quiz game in Arabic, but the problem is when I read my JSON text the output will be like this: ?????? instead of السلام. Anyone know how to solve this problem.
This is my code:

Code (CSharp):

1. IEnumerator LoadStreamingAsset(string fileName)
2. {
3.     string result;
4.     string filePath = System.IO.Path.Combine(Application.streamingAssetsPath, fileName);
5.  
6.         WWW www = new WWW(filePath);
7.         yield return www;
8.         result = www.text;
9.  
10.     GameData loadedData = JsonUtility.FromJson<GameData>(result);
11.     Quests= loadedData.Quests;
12. }

 

You need to make sure you save the JSON file as 'UTF-8', you can do that in Uniscite/Notepad2 by changing the File->Encoding to UTF-8

 

This not solves the problem, the words still like this: ??????

 

Any other solution?

 

If you want to post your JSON file I'll have a look for you.

 

https://drive.google.com/open?id=1DuyoanP2jcPp6cnH2kJi1i8eXwl6gLJp

 

It seems to be an issue with Google Drive - when I download that file it comes down as Ansi and loses all the UTF-8 encoding.

I've hosted that file myself here: https://a.uguu.se/lEI4jwo50df6_Data.json and it works fine when I download it through Unity. Try that URL and see how you get on.

I would recommend that you encode them as UTF-8 in an application like Notepad2 and then upload them to your own web server if possible.

 

It didn't read the file when I change to UTF-8 using NotePad2 or call the URL directly.
When you tested, did you write the same code in my thread?

 

No, WWW is now deprecated so I used UnityWebRequest (https://gist.github.com/emoacht/89590d4e4571d40f9e1b)

I’ll post the code I used when I’m back at my computer tomorrow.

 

Thank you, it reads it, but now these couple lines of codes does not work anymore:

Code (CSharp):

1. using (UnityWebRequest request = UnityWebRequest.Get("https://a.uguu.se/lEI4jwo50df6_Data.json"))
2.             {
3.                 yield return request.SendWebRequest();
4.  
5.                 if (request.isNetworkError) // Error
6.                 {
7.                     Debug.Log(request.error);
8.                 }
9.                 else // Success
10.                 {
11.                     result = request.downloadHandler.text;
12.                 }
13.             }
14. // These lines does not works
15. GameData loadedData = JsonUtility.FromJson<GameData>(result);
16. Quests= loadedData.Quests

 

What type of error do you get? I think it would be better to do something like this:

Code (CSharp):

1. UnityWebRequest request = UnityWebRequest.Get("https://a.uguu.se/lEI4jwo50df6_Data.json");
2. yield return request.SendWebRequest();
3.    
4. if (request.isNetworkError) // Error
5. {
6.     Debug.Log("Error:" + request.error);
7. }
8. else // Success
9. {
10.     string result = request.downloadHandler.text;
11.     Debug.Log("Success:" + result);
12.     GameData loadedData = JsonUtility.FromJson<GameData>(result);
13.     Quests= loadedData.Quests;
14. }
15.  

Let me know what result you get from the console window.

 

Are you certain it's the data that's like that and not the display of it? Have you inspected the hex? utf-8 that's outside of the ascii set is often displayed like that in consoles. My guess is that Debug.Log simply dumps to stdout, which doesn't care about utf. My guess is the console output is emulating a linux terminal, which will display utf as garbage.

 

First, I am displaying the "result" variable using a textObject and not the debug console.
Second, I got this error on the line of this code:

Code (CSharp):

1. GameData loadedData = JsonUtility.FromJson<GameData>(result);

This is the Error Text:

Code (CSharp):

1. ArgumentException: JSON parse error: Invalid value.
2. UnityEngine.JsonUtility.FromJson[GameData] (System.String json) (at C:/buildslave/unity/build/artifacts/generated/bindings_old/common/JSONSerialize/JsonUtilityBindings.gen.cs:25)
3. GameManager+<LoadStreamingAsset>c__Iterator1.MoveNext () (at Assets/Scripts/GameManager.cs:208)
4. UnityEngine.SetupCoroutine.InvokeMoveNext (IEnumerator enumerator, IntPtr returnValueAddress) (at C:/buildslave/unity/build/Runtime/Export/Coroutines.cs:17)

And this is what the textObject display:

Code (CSharp):

1. {"Quests":[{"Quest":"السؤال رقم 1","Correct":true}]}

 

How much Arabic do you have? Does encoding it using the \uXXXX JSON escaping work?

 

If you mean to do this manually for each letter, then this impossible, because I have a lot of Arabic question.

 

Hmm... I have a theory. I see that you're getting the JSON from a web request. What's the "charset" header on that GET? Is it utf-8? I'd assume that whatever charset the GET request gets back will be honored all the way through.

 

Sorry, I think I did not get what you mean. Do I need to add something to my JSON file?

 

Doesn't your JSON file come from here? https://a.uguu.se/lEI4jwo50df6_Data.json?

How is that server configured to respond to that request? It's headers should look like this:

Content-Type: application/json; charset=utf-8

My guess is that it's currently either Content-Type is html/txt or charset is us-ascii. Did you configure apache yourself? You can find these settings in /etc/httpd/conf.d (On Linux.)

 

So I need to add this line "Content-Type: application/json; charset=utf-8" on the top of my JSON file then call it?

 

Hmm, no, you need to change how your server returns your JSON file. A web request works by sending a GET to a Webserver somewhere. A program called Apache listens for that request, processes it, and then sends a response back. The response includes headers so that the caller has information about the connection, and then the "body." Your file is located in the body and you want to look at the headers.

I think you can see the headers using request.GetResponseHeader("Content-Type")? If the headers are wrong then you may have to either configure your file server to respond differently.

 

I'm kind of confused that you both have a parse error and see ????? in your UI. Have you inspected your font to make sure that those arabic characters actually exist within it? Lots of fonts do not include the full range of utf. If you hard code that Arabic into your UI, does it display correctly?

 

I think my font support Arabic. Because the Arabic letters are displayed properly if I write:

Code (CSharp):

1. textObject.text = "السلام";

But when I read it from JSON file the text will be shown like this: ??????

 

It's outputting the text correctly so at least you know the Arabic characters are coming through correctly. It looks like the FromJSON is having problems with it though. This post (https://forum.unity.com/threads/jsonutility-fromjson-error-invalid-value.421291/) details a similar issue - you may need to remove the first 3 bytes from the string before parsing it.

e.g. Instead of:

Code (CSharp):

1. GameData loadedData = JsonUtility.FromJson<GameData>(result);

you could try this:

Code (CSharp):

1. byte[] contentData = request.downloadHandler.data;
2. string jsonString = System.Text.Encoding.UTF8.GetString(contentData, 3, contentData.Length - 3);
3. GameData loadedData = JsonUtility.FromJson<GameData>(jsonString);

 

Thank you very much, it works perfectly.