Search Unity

Question Send byte array from js to Unity

Discussion in 'Web' started by Marks4, Mar 14, 2022.

  1. Marks4

    Marks4

    Joined:
    Feb 25, 2018
    Posts:
    536
    @brendanduncan_u3d Is it possible to send a byte array from jslib to Unity? I saw this SO post, but I would like to skip the conversion step if possible. It takes more time to convert from a string to a byte array. It can even be a shared byte array from the heap. For example, I saw this forum post with a shared float array that was declared in Unity, but the problem is that I don't know the size of the array at the time of declaration. I want to malloc this array in the jslib and use it in Unity directly, without the conversion step from the SO post. Is it possible?
     
  2. brendanduncan_u3d

    brendanduncan_u3d

    Unity Technologies

    Joined:
    Jul 30, 2019
    Posts:
    434
    _malloc allocates memory on the shared WASM heap. If you have JS specific memory, like a string, you do need to copy it to the shared WASM heap to be accessible from C#/C++. There's the function stringToUTF8 to convert a string to byte array.
     
  3. Marks4

    Marks4

    Joined:
    Feb 25, 2018
    Posts:
    536
    This is not what I meant. I found a solution to my issue, and what I'm doing is many, many times faster than the answer on SO. I can transfer byte arrays over 20MB from the js context to Unity context in less than a second instead of taking several seconds now. I'll post my solution here in case anyone wants to do this as well.

    The secret is to have a method to instantiate the byte array and call it from the js context. The method that instantiates the array then calls the js method that actually fills the byte array. It's a 2 step process, because if you try to do it with one function like the SO post says, it doesn't work, and you have to use a string as an intermediary form(which is very slow).

    C# side
    Code (CSharp):
    1. [DllImport("__Internal", EntryPoint = "getByteArrayTmp")]
    2. private static extern void getByteArrayTmp(Action<int> instantiateByteArrayCallback);
    3. [DllImport("__Internal", EntryPoint = "getByteArray")]
    4. private static extern void getByteArray(Action callback, byte[] byteArray);
    5.  
    6. private static event Action callbackEvent;
    7.  
    8. public static byte[] byteArray;
    9.  
    10. [MonoPInvokeCallback(typeof(Action))]
    11. private static void callback() {
    12.     callbackEvent?.Invoke();
    13.     callbackEvent = null;
    14. }
    15.  
    16. [MonoPInvokeCallback(typeof(Action<int>))]
    17. private static void instantiateByteArrayCallback(int size) {
    18.     byteArray = new byte[size];
    19.     getByteArray(callback, byteArray);
    20. }
    The method that is actually called by the user is

    Code (CSharp):
    1. public static void getByteArray(Action callback) {
    2.     #if UNITY_WEBGL && !UNITY_EDITOR
    3.         callbackEvent = null;
    4.         callbackEvent += callback;
    5.         getByteArrayTmp(instantiateByteArrayCallback);
    6.     #endif
    7. }
    On the js side of things
    Code (JavaScript):
    1. getByteArrayTmp: function(instantiateByteArrayCallback) {
    2.     const byteArray = new Uint8Array([21, 31]);//example
    3.     Module["yourplugin"].byteArray = byteArray;
    4.     Module.dynCall_vi(instantiateByteArrayCallback, byteArray.length);
    5.      
    6. },
    7. getByteArray: function(callback, byteArray) {
    8.     Module.HEAPU8.set(Module["yourplugin"].byteArray, byteArray);
    9.     Module.dynCall_v(callback);
    10. },
    @brendanduncan_u3d if you know of an easier solution please let me know, but this works already :).
     
  4. GroovyKoala

    GroovyKoala

    Joined:
    Feb 16, 2022
    Posts:
    23
    Hei @Marks4 which version of Unity are you using? As as I know, now it's not necessary to type Module in order to call dynCall method.

    This is the code I've been using to send the UA data to Unity.
    Code (JavaScript):
    1. sendData:function(callback, data) {
    2.     const buffer = _malloc(data.length * data.BYTES_PER_ELEMENT);
    3.     HEAPU8.set(data, buffer);
    4.     dynCall('vii', callback, [buffer, data.length]);
    5.     _free(buffer);
    6. }
    But I need to alloc memory and afterwards, delete the buffer.
     
  5. Marks4

    Marks4

    Joined:
    Feb 25, 2018
    Posts:
    536
    Can you show me the C# side of it? What method on C# gets the buffer and data.length? Is it a public byte array?

    I only type Module because I like to know where the methods come from, even though it's unnecessary and works without it. It's the same reason I use
    Code (Javascript):
    1. window.devicePixelRatio
    and not just
    Code (Javascript):
    1. devicePixelRatio
    for example.
     
  6. brendanduncan_u3d

    brendanduncan_u3d

    Unity Technologies

    Joined:
    Jul 30, 2019
    Posts:
    434
    Looking at it more closely, I'm not sure IL2CPP lets you get arbitrary byte[] data that was _malloc'd in JS. You can get a C# string from JS _malloc data, which IL2CPP takes care of, but I'm not seeing a way to get byte[] from JS. When IL2CPP generates the code to return a byte[], the array always has a length of 1. So, allocating the data from C# and passing the pointer to JS might be the best method for shared data. I'll ask around to see if getting byte[] data from JS into C# is possible.
     
  7. Marks4

    Marks4

    Joined:
    Feb 25, 2018
    Posts:
    536
    @brendanduncan_u3d That's why my method takes 2 steps. First I create the array in js and save in the js context, but send the byte array length information to C#. Then on the C# side I instantiate the array and pass the pointer to JS, and finally I set the memory in JS.

    What about @GroovyKoala 's solution? He didn't show the C# side, but it looks like he's sending the byte array directly?
     
  8. GroovyKoala

    GroovyKoala

    Joined:
    Feb 16, 2022
    Posts:
    23
    Hi @Marks4 ! sure this is the C# side.

    First the method callback

    Code (CSharp):
    1. [DllImport("__Internal")]
    2. private static extern void init(Action<byte[], int> ReceiveArrayData);
    3.  
    4. [AOT.MonoPInvokeCallback(typeof(Action<byte[], int>))]
    5. private static void OnByteArray(
    6. [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.U1, SizeParamIndex = 1)] byte[] data, int length)
    7. {
    8.     if (data != null)
    9.     {
    10.         // process your data
    11.     }
    12. }
    Note hat the sendData from the Js library, sends an Uint8ClampedArray typed array (represents an array of 8-bit unsigned integers clamped to 0-255), so it needs to be marshalled as described above.

    Also, when the init is called, it passes the ReceiveArrayData arg with OnByteArray
    Code (CSharp):
    1. public void Initialize()
    2. {
    3.     init(OnByteArray);
    4. }
    Finally in the Js lib, this is way I'm storing the callback for later use

    Code (JavaScript):
    1. $utils: {
    2.     onByteArray: {},
    3. },
    4.  
    5. init: function (onByteArray) {
    6.     utils.onByteArray = onByteArray;
    7. },
     
    poprev-3d and Marks4 like this.
  9. Marks4

    Marks4

    Joined:
    Feb 25, 2018
    Posts:
    536
    Amazing, thank you for sharing your method. It's still a 2 step process like mine, but it has the advantage(or disadvantage depending on the use case) of not requiring a public byte array!

    @GroovyKoala Just one questions regarding the length parameter. Why do you send the length to C#? If you can marshal the byte array, you can simply get the byte array's length in C# with "mybytearray.lengh", no?
     
    GroovyKoala likes this.
  10. GroovyKoala

    GroovyKoala

    Joined:
    Feb 16, 2022
    Posts:
    23
    Yep, it's a 2 step process. And you're right @Marks4, it can be accessed with the array length, but i like to pass the length from the js lib.
     
    Marks4 likes this.
  11. Marks4

    Marks4

    Joined:
    Feb 25, 2018
    Posts:
    536
    @GroovyKoala @brendanduncan_u3d I just noticed one thing that I never used before. This $utils. What's the purpose of the "$"? It makes a scoped variable in that jslib? Or can $utils be seen from other jslibs as well? So you use it like this?

    Code (Javascript):
    1. mergeInto(LibraryManager.library, {
    2.     $utils: {
    3.         onByteArray: {},
    4.     },
    5.    
    6.     init: function (onByteArray) {
    7.         utils.onByteArray = onByteArray;
    8.     },
    9. });
    Normally I'd use a .jspre file for the utils. I didn't know you could do it in the jslib.
     
  12. GroovyKoala

    GroovyKoala

    Joined:
    Feb 16, 2022
    Posts:
    23
    @Marks4 that's another interesting question, I'm using it to encapsulate member like variables to be called in this scope.
    But actually I haven't found solid documentation to explain the use of the "$" (not even in the emscripten site) and I've seen it a lot in jslibs.

    Not sure if it's exposed to the other libs as well, there's a way to find out.

    Cheers
     
    Marks4 likes this.
  13. Marks4

    Marks4

    Joined:
    Feb 25, 2018
    Posts:
    536
    This $ variable might be a Unity thing only. @brendanduncan_u3d Can you shed some light on the matter? Thanks!
     
  14. brendanduncan_u3d

    brendanduncan_u3d

    Unity Technologies

    Joined:
    Jul 30, 2019
    Posts:
    434
    The $ is an Emscripten thing, and like many Emscripten things, poorly documented. It is mentioned in passing in https://emscripten.org/docs/porting/connecting_cpp_and_javascript/Interacting-with-code.html,
    at the bottom of the JavaScript limits in library files section, in the Note block, where it says "Keys starting with $ have the $ stripped and no underscore added."

    In the words of the great Douglas Adams:

    It was on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard.'
     
    GroovyKoala and Marks4 like this.
  15. Marks4

    Marks4

    Joined:
    Feb 25, 2018
    Posts:
    536
    @brendanduncan_u3d is this correct?

    Code (Javascript):
    1. mergeInto(LibraryManager.library, {
    2.     $utils: {
    3.         onByteArray: {},
    4.     },
    5.  
    6.     init: function (onByteArray) {
    7.         utils.onByteArray = onByteArray;
    8.     },
    9. });
    On Unity 2019, I get an error: utils is not defined. How is the correct way to use this $ sign?
     
  16. brendanduncan_u3d

    brendanduncan_u3d

    Unity Technologies

    Joined:
    Jul 30, 2019
    Posts:
    434
    The code stripping can only automatically detect usage from C. In this case, utils is not getting detected as being used, and is getting stripped as unused. You need to add it to the __deps of init.

    Code (JavaScript):
    1. init__deps: ["$utils"],
    2. init: function...
     
    GroovyKoala likes this.
  17. GroovyKoala

    GroovyKoala

    Joined:
    Feb 16, 2022
    Posts:
    23
    That makes sense using __deps in that way.

    I also used it the other way:

    Code (JavaScript):
    1. const myLib = {
    2.  
    3.     $utils: {
    4.         onByteArray: {}
    5.     }
    6. };
    7.  
    8. autoAddDeps(myLib, '$utils');
    9. mergeInto(LibraryManager.library, myLib);
     
    Last edited: Mar 30, 2022
    brendanduncan_u3d and Marks4 like this.
  18. poprev-3d

    poprev-3d

    Joined:
    Apr 12, 2019
    Posts:
    71
    Thanks @GroovyKoala, this is of great assistance !!

    I'll use this from now on instead of SendMessage because this is much more efficient and evitates any Serialization/Deserialization process.

    For anyone coming after, here is a few tricks I discovered along the way:

    1) You can also efficiently pass to Unity some float[] or int[] this way using "HEAPF32.set"/"HEAP32.set":

    In JsLib:
    Code (JavaScript):
    1.  
    2. const myLib = {
    3. $utils: {
    4.     onArrayFloat: null,
    5. },
    6.  
    7. $sendDataFloat: function(floatArray) {
    8.     const buffer = _malloc(floatArray.length * floatArray.BYTES_PER_ELEMENT);
    9.     HEAPF32.set(floatArray, buffer >> 2);
    10.     dynCall('vii', utils.onArrayFloat, [buffer, floatArray.length]);
    11.     _free(buffer);
    12. },
    13. init: function (onByteArrayFloat) {
    14.     utils.onArrayFloat= onArrayFloat;
    15.     window.unitySendDataFloat = sendDataFloat;
    16. },
    17.  
    18. };
    19. autoAddDeps(myLib, "$utils");
    20. autoAddDeps(myLib, "$sendDataFloat");
    21.  
    22. mergeInto(LibraryManager.library, myLib);
    23.  
    In your Unity C# code:

    Code (CSharp):
    1.  
    2.  
    3. public class PocArrays : MonoBehaviour
    4. {
    5.     [DllImport("__Internal")]
    6.     public static extern void init(Action<float[], int> ReceiveFloatArrayData);
    7.  
    8.     [AOT.MonoPInvokeCallback(typeof(Action<float[], int>))]
    9.     public static void OnFloatArray([MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.R4, SizeParamIndex = 1)] float[] floats, int floatsLength
    10.         )
    11.     {
    12.         if (floats != null )
    13.         {
    14.             foreach (var item in floats)
    15.             {
    16.                 Debug.Log("float element is " + item);
    17.             }
    18.         }
    19.     }
    20.  
    21.     void Awake()
    22.     {
    23.         init(OnFloatArray);
    24.     }
    25. }
    26.  
    27.  
    28.  
    In your js code:
    Code (JavaScript):
    1.  
    2. window.unitySendData(new Float32Array([755.188,18.255, 45.588]));
    3.  
    2) You can pass as many arguments as you want using dyncall. When adding arguments, make sure you update the part with "dynCall('viiiiiiii'". v means void (the return type) and all the "i" are the types of arguments. For example, when adding an int (which is a pointer to an array in our case), add an "i" at the end of the string. All of the supported types are available here: https://www.dyncall.org/docs/manual/manualse4.html

    Also, make sure you add the proper marshaling types on the C# side:

    Code (CSharp):
    1.  
    2. [AOT.MonoPInvokeCallback(typeof(Action<float[], int, int[], int, byte[], int>))]
    3.     public static void OnFloatArray(
    4.         [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.R4, SizeParamIndex = 1)] float[] myFloats, int floatsLength,
    5.         [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.I4, SizeParamIndex = 3)] int[] myInts, int intsLength,
    6.         [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.U1, SizeParamIndex = 5)] byte[] myBytes, int bytesLength
    7.  
    8.         )
    9.     {
    10.         // Directly use myFloats / myInts / myBytes
    11.     }
    12.  
    In JsLib

    Code (JavaScript):
    1.  
    2. // Note the iiiiii (because there is 6 int (3 being pointers) as an argument)
    3. dynCall('viiiiii', utils.onReceiveMyArrays, [aBuffer, a.length, bBuffer, b.length, cBuffer, c.length]);
    4.  
    In your Js
    Passing float[], int[], string as byte[] to Unity in a single call.

    Code (JavaScript):
    1. window.unitySendData(new Float32Array([1.188,1.256]), new Int32Array([155,1, 488]), new TextEncoder().encode("abcd"));



    3) To call the JsLib functions directly from javascript, I declared the function I want to use in the window scope, which is not especially good practice as it messing up the windows scope. As mentionned by @brendanduncan_u3d, to solve this, you may also declare an "exposedFunctions" object in the Module object which can be accessed from the JsLib scope and directly call it from the unityInstance object that you have stored in your html. In this "exposedFunctions" object you may put all of the functions you want to expose to javascript. It still requires to be initialized in the init function though.

    In JsLib
    Code (JavaScript):
    1.  
    2. ...
    3. $exposedFunctions: {
    4.     sendData: null,
    5. },
    6.  
    7. init: function () {
    8.     Module.exposedFunctions = exposedFunctions;
    9.     exposedFunctions.sendData = sendData;
    10. },
    11. ...
    12. autoAddDeps(myLib, "$exposedFunctions");
    13. ...
    14.  
    In Js
    Code (JavaScript):
    1.  
    2. unityInstance.Module.exposedFunctions.sendData(...)
    3.  
     
    Last edited: Mar 31, 2022
    GroovyKoala and Marks4 like this.
  19. brendanduncan_u3d

    brendanduncan_u3d

    Unity Technologies

    Joined:
    Jul 30, 2019
    Posts:
    434
    @poprev-3d The Module object is often used instead of window for putting things into a global scope. Keeps things a bit more organized.
     
  20. poprev-3d

    poprev-3d

    Joined:
    Apr 12, 2019
    Posts:
    71
    @brendanduncan_u3d thank you! I also thought this but for some strange reason, i cannot find my jsLibs functions in the Module object. Are those supposed to be at the root of the Module object or somewhere else? Or is there some specific syntax to embed those as its not automatic ?
     
    Last edited: Mar 31, 2022
  21. brendanduncan_u3d

    brendanduncan_u3d

    Unity Technologies

    Joined:
    Jul 30, 2019
    Posts:
    434
    You'll need to add what you want to Module, in the same you did to window. You can use window too, whatever works for you.
     
  22. poprev-3d

    poprev-3d

    Joined:
    Apr 12, 2019
    Posts:
    71
    @brendanduncan_u3d thank you i didn't know you could directly access the Module for the jslib file. For those coming after i updated my answer above
     
  23. Marks4

    Marks4

    Joined:
    Feb 25, 2018
    Posts:
    536
    Quick question. When you send the data to Unity, is there no risk of it becoming invalid because of the _free(buffer) on the js side?

    For example

    Code (CSharp):
    1. public float[] floatArray;
    2. [AOT.MonoPInvokeCallback(typeof(Action<float[], int, int[], int, byte[], int>))]
    3.     public static void OnFloatArray(
    4.         [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.R4, SizeParamIndex = 1)] float[] myFloats, int floatsLength,
    5.         [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.I4, SizeParamIndex = 3)] int[] myInts, int intsLength,
    6.         [MarshalAs(UnmanagedType.LPArray, ArraySubType = UnmanagedType.U1, SizeParamIndex = 5)] byte[] myBytes, int bytesLength
    7.         )
    8.     {
    9.        floatArray = myFloats;//this is great, I'm storing a reference to myFloats, but on the js side there's a _free(buffer);. What happens here?
    10.     }
     
  24. poprev-3d

    poprev-3d

    Joined:
    Apr 12, 2019
    Posts:
    71
    @Marks4 I thought this as well but in practice, "free" doesn't seem to free the memory on the Unity side. I feel like the data is copied when transferred to Unity. Would be glad if someone knew exactly what is happening here.
     
    Marks4 likes this.
  25. Marks4

    Marks4

    Joined:
    Feb 25, 2018
    Posts:
    536
    If the data is being copied my method is still better because it's using the same shared memory, so it should be faster.

    @brendanduncan_u3d @GroovyKoala Do you know what's happening? Is the marshalling copying the data or is it still a reference? And if it's a reference, is it still safe to use free on my example and why?
     
  26. GroovyKoala

    GroovyKoala

    Joined:
    Feb 16, 2022
    Posts:
    23
    As @poprev-3d points out, it appears to be copying the data in the C# env. There's something written in the documentation WebGL: Interacting with browser scripting

    And since the string is a pointer to a value in the heap, it will be free automatically if it's a return value, like in the example. But not sure if only works for string or any other value that's is returned inside the method.

    On the emscripten side, you can still manipulate memory

    But to answer your question, I think Marshalling will convert and copy the data, but again I'm not sure of this assumption. Will look into more detail
     
    poprev-3d and Marks4 like this.