Copying to and reading WebGL persistentDataPath files

Hey everyone,

I'm developing a game where I'd like to enable database connectivity. My StreamingAssets folder contains database credential files that I'll need the game to read. More specifically, I need to point my connection to a directory where these credentials exist. This Unity documentation says WebGL and Android cannot read StreamingAsset files directly. I have a solution that works great on Android, which is to copy these files to the persistentDataPath. However, this solution does not work for my WebGL build. My code looks something this. I run this co-routine for each file I need to copy from the streamingAssetsPath to the persistentDataPath:

Code (CSharp):

1. private IEnumerator writeFile(string fileName){
2.     string filePath = Path.Combine(Application.streamingAssetsPath, fileName);
3.     string savePath = Path.Combine(Application.persistentDataPath, fileName);
4.  
5.     UnityWebRequest uwr = UnityWebRequest.Get(filePath);
6.     DownloadHandlerFile dhf = new DownloadHandlerFile(savePath);
7.     dhf.removeFileOnAbort = true;
8.     uwr.downloadHandler = dhf;
9.     yield return uwr.SendWebRequest();
10.  
11.     dhf.Dispose();
12.     uwr.Dispose();
13. }

I have code later that attempts to read the files from the persistenDataPath, just as a test in WebGL. But nothing is printed. I'm also not able to find the files in idbfs via the browser inspector. This makes me think the database credential files aren't actually being written to the persistentDataPath.

Code (CSharp):

1.     StreamReader reader = new StreamReader(Path.Combine(Application.persistentDataPath, "README"));
2.     string s = reader.ReadToEnd().ToString();
3.     Debug.Log(s);
4.     reader.Close();

I don't know what I'm missing, or if I'm even pursuing a strategy that will work for WebGL. But I suspect it is possible because I am able to save player progress as files in the persistentDataPath. Is there another way of copying over files for my WebGL build? Maybe something like creating a bytestream from these files and then writing them? Or should I be attempting an entirely different strategy?

Thanks,
NickR2600

 

Hi Jukka. Good news and bad news. Chrome's Network tab reports a status call of 200 for all these assets. I can also see the content of the files via the Preview.

Here's the bad news. The files do display as keys under the Application tab, but I suspect they're blank based on this Value summary:

Code (CSharp):

1. {timestamp: Mon Oct 19 2020 16:54:42 GMT-0400 (Eastern Daylight Time), mode: 33206, contents: Uint8Array(0)}
2.     contents: Uint8Array(0)
3.         buffer: ArrayBuffer(0)
4.         byteLength: 0
5.         byteOffset: 0
6.         length: 0
7.         Symbol(Symbol.toStringTag): undefined
8.     mode: 33206
9.     timestamp: Mon Oct 19 2020 16:54:42 GMT-0400 (Eastern Daylight Time)
10.         No properties

One of these files is a simple README text file. When I attempt to print its contents, it's as if the file contains nothing. The program crashes a few lines later just as I'm about the open the database connection. It produces this error, with <domain> replaced of course:

Code (CSharp):

1. DXNLE-YBWWY-AR74T-WMD99-77VRA:16 exception thrown: RuntimeError: memory access out of bounds,RuntimeError: memory access out of bounds
2.     at <anonymous>:wasm-function[17973]:0x7fd669
3.     at dynCall_vii (<anonymous>:wasm-function[58221]:0x1256425)
4.     at Object.dynCall_vii (blob:https://www.<domain>.com/9034d81f-23e4-4ce8-b94a-2f70cc1677fe:2:487465)
5.     at invoke_vii (blob:https://www.<domain>.com/9034d81f-23e4-4ce8-b94a-2f70cc1677fe:2:368195)
6.     at <anonymous>:wasm-function[55273]:0x10e1e94
7.     at <anonymous>:wasm-function[55272]:0x10e1b75
8.     at <anonymous>:wasm-function[17866]:0x7ea6c2
9.     at <anonymous>:wasm-function[48753]:0xfc8f96
10.     at <anonymous>:wasm-function[53323]:0x10874fe
11.     at dynCall_iiiii (<anonymous>:wasm-function[58192]:0x1255ffc)
12.     at Object.dynCall_iiiii (blob:https://www.<domain>.com/9034d81f-23e4-4ce8-b94a-2f70cc1677fe:2:470461)
13.     at invoke_iiiii (blob:https://www.<domain>.com/9034d81f-23e4-4ce8-b94a-2f70cc1677fe:2:340409)
14.     at <anonymous>:wasm-function[51108]:0x103366f
15.     at <anonymous>:wasm-function[51531]:0x10446bb
16.     at <anonymous>:wasm-function[2884]:0x16796f
17.     at <anonymous>:wasm-function[2883]:0x1678db
18.     at <anonymous>:wasm-function[5417]:0x221d47
19.     at <anonymous>:wasm-function[5415]:0x221a57
20.     at <anonymous>:wasm-function[5419]:0x221f50
21.     at <anonymous>:wasm-function[5836]:0x25439f
22.     at <anonymous>:wasm-function[8734]:0x3931c4
23.     at <anonymous>:wasm-function[8043]:0x349f0c
24.     at <anonymous>:wasm-function[8043]:0x349f21
25.     at <anonymous>:wasm-function[8037]:0x3492d1
26.     at <anonymous>:wasm-function[8032]:0x347c26
27.     at dynCall_v (<anonymous>:wasm-function[58213]:0x125638a)
28.     at Object.dynCall_v (blob:https://www.<domain>.com/9034d81f-23e4-4ce8-b94a-2f70cc1677fe:2:484357)
29.     at browserIterationFunc (blob:https://www.<domain>.com/9034d81f-23e4-4ce8-b94a-2f70cc1677fe:2:129474)
30.     at Object.runIter (blob:https://www.<domain>.com/9034d81f-23e4-4ce8-b94a-2f70cc1677fe:2:132547)
31.     at Browser_mainLoop_runner (blob:https://www.<domain>.com/9034d81f-23e4-4ce8-b94a-2f70cc1677fe:2:131009)

Could it be that I'm successfully pulling the files, but just not writing them to idbfs correctly? What could I change?

 

This was helpful because it got me thinking about DownloadHandlerBuffer. Depending where in the code I try to read the README file from, sometimes I get the data, sometimes I get no data. That made me think it was a concurrency issue, so I tried to rejigger the coroutines. But even more useful was DownloadHandlerBuffer. I could see the data was getting pulled, but for whatever reason wasn't getting written to the file. So I instead wrote the data using a FileStream:

Code (CSharp):

1. UnityWebRequest uwr = new UnityWebRequest(filePath);
2. uwr.downloadHandler = new DownloadHandlerBuffer();
3. yield return uwr.SendWebRequest();
4.  
5. using (FileStream fs = File.Create(savePath)){
6.     byte[] info = uwr.downloadHandler.data;
7.     fs.Write(info, 0, info.Length);
8. }

This works to create the files. But I'm still getting a similar error as before when I try opening the database connection:

Code (CSharp):

1. DXNLE-YBWWY-AR74T-WMD99-77VRA:16
2. exception thrown: RuntimeError: memory access out of bounds,RuntimeError: memory access out of bounds
3.     at <anonymous>:wasm-function[19888]:0x818a5c
4.     at dynCall_vii (<anonymous>:wasm-function[58230]:0x12565dd)
5.     at Object.dynCall_vii (blob:https://www.<domain>.com/406fc847-ac12-45f0-b1ba-132bd48c39c1:2:487465)
6.     at invoke_vii (blob:https://www.<domain>.com/406fc847-ac12-45f0-b1ba-132bd48c39c1:2:368195)
7.     at <anonymous>:wasm-function[57277]:0x11a20a3
8.     at <anonymous>:wasm-function[57276]:0x11a1d84
9.     at <anonymous>:wasm-function[20070]:0x843ab5
10.     at <anonymous>:wasm-function[52891]:0x10e1afc
11.     at <anonymous>:wasm-function[18182]:0x7ef934
12.     at dynCall_iiiii (<anonymous>:wasm-function[58201]:0x12561b4)
13.     at Object.dynCall_iiiii (blob:https://www.<domain>.com/406fc847-ac12-45f0-b1ba-132bd48c39c1:2:470461)
14.     at invoke_iiiii (blob:https://www.<domain>.com/406fc847-ac12-45f0-b1ba-132bd48c39c1:2:340409)
15.     at <anonymous>:wasm-function[55247]:0x114c262
16.     at <anonymous>:wasm-function[55670]:0x115d2af
17.     at <anonymous>:wasm-function[2884]:0x167902
18.     at <anonymous>:wasm-function[2883]:0x16786e
19.     at <anonymous>:wasm-function[5417]:0x221d0f
20.     at <anonymous>:wasm-function[5415]:0x221a1f
21.     at <anonymous>:wasm-function[5419]:0x221f18
22.     at <anonymous>:wasm-function[5836]:0x254367
23.     at <anonymous>:wasm-function[8734]:0x393197
24.     at <anonymous>:wasm-function[8043]:0x349eda
25.     at <anonymous>:wasm-function[8043]:0x349eef
26.     at <anonymous>:wasm-function[8037]:0x34929f
27.     at <anonymous>:wasm-function[8032]:0x347bf4
28.     at dynCall_v (<anonymous>:wasm-function[58222]:0x1256541)
29.     at Object.dynCall_v (blob:https://www.<domain>.com/406fc847-ac12-45f0-b1ba-132bd48c39c1:2:484357)
30.     at browserIterationFunc (blob:https://www.<domain>.com/406fc847-ac12-45f0-b1ba-132bd48c39c1:2:129474)
31.     at Object.runIter (blob:https://www.<domain>.com/406fc847-ac12-45f0-b1ba-132bd48c39c1:2:132547)
32.     at Browser_mainLoop_runner (blob:https://www.<domain>.com/406fc847-ac12-45f0-b1ba-132bd48c39c1:2:131009)

I haven't got a clue how to debug this. I suspect the issue is on the WebGL side, since the database connection works perfectly fine on Android. Any ideas?

 

Thanks, Jukka. Trying a Development build sounds like a good idea. Yes, the issue occurs consistently and predictably on Chrome, Firefox, and Edge, right at the line of code where I attempt to open the database connection. It occurs regardless of if I have the browser's Developer tools open. This is what I'm using:

• Chrome 86.0.4240.111
• Firefox 78.3.1esr (32-bit)
• Edge 44.18362.449.0
• Unity 2018.3.7f1
• oracle.manageddataaccess.core.2.19.80 (DLL from Oracle)
  
• system.diagnostics.performancecounter.4.7.0 (DLL from Microsoft)

You mentioned in this thread that multithreaded C# is not supported properly in WebGL. I wonder if the Oracle database connection uses of one of these unsupported multithreading aspects? I'll see what I can dig up on Oracle's database connectivity. Would a Developer build reveal this? If this is the issue, does Unity have an estimate when it will be fixed? And maybe this is a crazy question, but is there a way to force someone else's DLL to not use an unsupported multithread feature?

 

This thread indicates Unity 2019.1.0 might allow multi-threading. Although I can't find the option in Project Settings. I try manually adding PlayerSettings.WebGL.threadsSupport = true; after importing UnityEngine, but that only produces an error when I go to build.

 

I can't find Oracle mentioning platforms like WebGL or Android explicitly. But since I have it working on Android, I'd assume WebGL support is theoretically possible as long as all the relevant .NET libraries are also supported?

Ugg... I'm unable to create a WebGL Development build. I get this confusing error:

Code (CSharp):

1. Failed running "C:\Program Files\Unity\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten_Win\python\2.7.5.3_64bit\python.exe" -E "C:\Program Files\Unity\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emcc" @"C:\Users\<user>\Documents\<folder>\FirstRunable\Unity\<Game>\Assets\..\Temp\emcc_arguments.resp"
2.  
3. stdout:
4. stderr:warning: unexpected number of arguments 1 in call to '__cxa_pure_virtual', should be 0warning: unexpected number of arguments 2 in call to '_ZN6il2cpp6icalls8mscorlib6System6String22RedirectToCreateStringEv', should be 0warning: unexpected number of arguments 4 in call to '_ZN6il2cpp6icalls8mscorlib6System6String22RedirectToCreateStringEv', should be 0warning: unexpected number of arguments 2 in call to '_ZN6il2cpp6icalls8mscorlib6System6String22RedirectToCreateStringEv', should be 0warning: unexpected number of arguments 4 in call to '_ZN6il2cpp6icalls8mscorlib6System6String22RedirectToCreateStringEv', should be 0warning: unexpected number of arguments 5 in call to '_ZN6il2cpp6icalls8mscorlib6System6String22RedirectToCreateStringEv', should be 0warning: unexpected number of arguments 4 in call to '_ZN6il2cpp6icalls8mscorlib6System6String22RedirectToCreateStringEv', should be 0warning: unexpected number of arguments 2 in call to '_ZN6il2cpp6icalls8mscorlib6System6String22RedirectToCreateStringEv', should be 0warning: unexpected number of arguments 3 in call to '_ZN6il2cpp6icalls8mscorlib6System6String22RedirectToCreateStringEv', should be 0error: unresolved symbol: _ZN6il2cpp2os9Semaphore11GetOSHandleEvAborting compilation due to previous errors | undefinedTraceback (most recent call last):  File "C:\Program Files\Unity\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emcc.py", line 3063, in <module>    sys.exit(run())  File "C:\Program Files\Unity\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emcc.py", line 1780, in run    final = shared.Building.emscripten(final, append_ext=False, extra_args=extra_args)  File "C:\Program Files\Unity\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\tools\shared.py", line 2274, in emscripten    emscripten._main(cmdline)  File "C:\Program Files\Unity\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten.py", line 2233, in _main    return temp_files.run_and_clean(lambda: main(  File "C:\Program Files\Unity\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\tools\tempfiles.py", line 93, in run_and_clean    return func()  File "C:\Program Files\Unity\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten.py", line 2238, in <lambda>    DEBUG=DEBUG,  File "C:\Program Files\Unity\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten.py", line 2164, in main    temp_files=temp_files, DEBUG=DEBUG)  File "C:\Program Files\Unity\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten.py", line 86, in emscript    glue, forwarded_data = compiler_glue(metadata, libraries, compiler_engine, temp_files, DEBUG)  File "C:\Program Files\Unity\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten.py", line 218, in compiler_glue    glue, forwarded_data = compile_settings(compiler_engine, libraries, temp_files)  File "C:\Program Files\Unity\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\emscripten.py", line 541, in compile_settings    cwd=path_from_root('src'), error_limit=300)  File "C:\Program Files\Unity\Editor\Data\PlaybackEngines\WebGLSupport\BuildTools\Emscripten\tools\jsrun.py", line 132, in run_js    raise Exception('Expected the command ' + str(command) + ' to finish with return code ' + str(assert_returncode) + ', but it returned with code ' + str(proc.returncode) + ' instead! Output: ' + str(ret)[:error_limit])Exception: Expected the command ['C:/Program Files/Unity/Editor/Data\\Tools\\nodejs\\node.exe', '--stack_size=8192', '--max-old-space-size=4096', 'C:\\Program Files\\Unity\\Editor\\Data\\PlaybackEngines\\WebGLSupport\\BuildTools\\Emscripten\\src\\compiler.js', 'C:\\Users\\nristucc\\AppData\\Local\\Temp\\tmpfsygi8.txt', 'C:\\Program Files\\Unity\\Editor\\Data\\PlaybackEngines\\WebGLSupport\\BuildTools\\lib\\Audio.js', 'C:\\Program Files\\Unity\\Editor\\Data\\PlaybackEngines\\WebGLSupport\\BuildTools\\lib\\Cursor.js', 'C:\\Program Files\\Unity\\Editor\\Data\\PlaybackEngines\\WebGLSupport\\BuildTools\\lib\\Eval.js', 'C:\\Program Files\\Unity\\Editor\\Data\\PlaybackEngines\\WebGLSupport\\BuildTools\\lib\\FileSystem.js', 'C:\\Program Files\\Unity\\Editor\\Data\\PlaybackEngines\\WebGLSupport\\BuildTools\\lib\\Logging.js', 'C:\\Program Files\\Unity\\Editor\\Data\\PlaybackEngines\\WebGLSupport\\BuildTools\\lib\\Profiler.js', 'C:\\Program Files\\Unity\\Editor\\Data\\PlaybackEngines\\WebGLSupport\\BuildTools\\lib\\SystemInfo.js', 'C:\\Program Files\\Unity\\Editor\\Data\\PlaybackEngines\\WebGLSupport\\BuildTools\\lib\\UnetWebSocket.js', 'C:\\Program Files\\Unity\\Editor\\Data\\PlaybackEngines\\WebGLSupport\\BuildTools\\lib\\Video.js', 'C:\\Program Files\\Unity\\Editor\\Data\\PlaybackEngines\\WebGLSupport\\BuildTools\\lib\\WebCam.js', 'C:\\Program Files\\Unity\\Editor\\Data\\PlaybackEngines\\WebGLSupport\\BuildTools\\lib\\WebRequest.js', 'C:\\Users\\<user>\\Documents\\<folder>\\FirstRunable\\Unity\\Game\\Assets\\Plugins\\WebGL\\OpenNewTab.jslib', 'C:\\Program Files\\Unity\\Editor\\Data\\PlaybackEngines\\WebGLSupport\\BuildTools\\Emscripten\\src\\library_pthread_stub.js'] to finish with return code 0, but it returned with code 1 instead! Output: // The Module object: Our interface to the outside world. We import// and export values on it. There are various ways Module can be used:// 1. Not defined. We create it here// 2. A function parameter, function(Module) { ..generated code.. }// 3. pre-run appended it, var Module = {}; ..generated
5. UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr)
6.  

 

I've given up on creating a WebGL development build and connecting to the Oracle database via credential wallet. I'm having much more luck connecting to the database via REST on WebGL and Android.