Resources.Load that worked in 5.6.6f2 no longer works in 2018.3.1f1

I have a JSON file (19kb) that contains a mapping of countries to various ISO formats. Its relative Resources file path is Data/Configs/Countries.json so I load the resource using:

Code (CSharp):

1. static CountryUtil()
2. {
3.     TextAsset countriesAsset = Resources.Load<TextAsset>("Data/Configs/Countries");
4.  
5.     CountryMasterList = JsonUtility.FromJson<CountryMasterList>(countriesAsset.text);
6. }

The asset is loaded in a static class static constructor, and, as mentioned in the title, this behavior worked in Unity 5.6.6f2. Now in Unity 2018.3.1f1, the TextAsset is null after the Resources.Load line above.

Here is a clipping of the folder structure:


I'm at a loss for why this could be happening. Interestingly, I cut the file down from roughly 1500 lines formatted to around 170 lines and the file loads without issue. This leads me to question whether something has changed in Unity regarding loading of large text files, but I cannot find any evidence of such a change. Any ideas are greatly appreciated.

Thanks for reading.

 

Ooh, that's kinda interesting. If you can pare it down to a tiny project where one text file of 1500 files fails and one succeeds, you can file a bug with Unity... they really do fix stuff like this.

Before you do that though, I would try a reimport all on the project.

 

The utility class is self-contained and doesn't have any external dependencies outside Unity, so I think I will try this approach, especially to check if it is in fact a problem with my project since it was migrated from 5.6, or a problem with Unity itself. I will report back with results.

 

I've created a new empty Unity project and the behavior is reproducible. I'll be sending a bug report to Unity.

 

hmm...I don't think anything has changed on file size. I have several large json files in a project that work fine.

Make sure your file is being saved with the proper encoding. If your json file isn't saved with the proper encoding, it doesn't work. Since you deleted some lines and saved it, it's possible your encoding changed.

It's just a thought, but I have had it happen where the json file had the wrong encoding and I had to re-save it for it to work.

 

Good thought @Brathnann . It certainly could be the BOM goofing up the JSON decode.

Please post your findings OP... enquiring minds wanna know!

BTW, if you're on a Mac, the xxd utility in Terminal is great for dumping files as hex to look for BOMs and other corruption.

 

I edit files using VS and Sublime and VS always complains when the encoding is off. I've tried reformatting the file through external tools (e.g. https://jsonformatter.curiousconcept.com) and replacing the original contents to no avail. I am on Windows.

 

In VS when you do a save as, the save button has an arrow next to it that allows save with encoding so you can choose which encoding to use. I believe I usually use Unicode codepage 65001.

But, it was just a thought, since I'm not seeing issues and I know I have a few json files larger than 1500 lines.

 

Your original post said the TextAsset itself is null. Is that correct?

Alternatively, if the text asset is coming in and present (you can print out the textAsset.text.Length to see how many characters it is), perhaps later in the file there is mal-formed JSON that used to pass an older less-strict parser, and now chokes the newer parser?

 

There are two lines of code in my snippet which is identical to the code in use. The TextAsset object itself is null, and the NullReferenceException is thrown when the text field of the TextAsset is attempted to be read from a null object.

I can confirm the JSON is not malformed as I have run it through online JSON parsers (see the link in my previous reply).

Code (CSharp):

1. static CountryUtil()
2. {
3.     // this variable evaluates to null at runtime
4.     TextAsset countriesAsset = Resources.Load<TextAsset>("Data/Configs/Countries");
5.     // countriesAsset is null here, so attempting to reference countriesAsset.text throws a NullReferenceException
6.     CountryMasterList = JsonUtility.FromJson<CountryMasterList>(countriesAsset.text);
7. }

 

Interesting update: I checked the encoding setting as @Brathnann suggested. It seems something is forcing "Western European (Windows) - Codepage 1252" encoding. I saved it using standard Unicode encoding (Codepage 65001), and, interestingly enough, the file preview in the Inspector correctly rendered a preview of the JSON text. When I ran my project in the Editor, the encoding was reverted back to Codepage 1252 and I continued to encounter the same issue as in the OP.

 

Yeah, the code preview doesn't work if the encoding is wrong I have discovered, which is usually the first sign for me at work that the json wasn't encoded correctly.

However, I have not had it ever revert the encoding or switch it automatically on me, which is certainly what is creating your error. I'm not certain what would be changing it.

 

So I seem to have solved it by creating a new JSON file with the correct UTF-8 encoding from the old one, then deleting the old one and renaming the new one. The encoding seems to be sticking to UTF-8 now and the issue is resolved. How strange.

 

I've seen issues years ago where Microsoft Visual Studio tries to be clever: once it has seen a file added to one of its projects, it saves the encoding setting separately somewhere, and then restores it no matter what you do to it. This might be what you are seeing.

 

For the benefit of anyone else encountering a similar issue, here's the response from a Unity QA team member about my Unity bug report: