Calling native code from Burst optimized jobs

I need to call a native code function in a Burst optimized job

Code (CSharp):

1.        
2. [ComputeJobOptimization]
3. public struct TreeRefitJob : IJob
4. {
5.     [NativeDisableUnsafePtrRestriction]
6.     public void* treePtr;
7.  
8.     [NativeDisableUnsafePtrRestriction]
9.     public void* positionsPtr;
10.  
11.     [NativeDisableUnsafePtrRestriction]
12.     public void* trianglesPtr;
13.  
14.     public int trianglesCount;
15.  
16.     public void Execute()
17.     {
18.         DbvtWrapper.DbvtRefitTriangles(treePtr, positionsPtr, trianglesPtr, trianglesCount);
19.  
20.         //DO SOME HEAVY MATH HERE
21.     }
22.  
23. }
24.  

but I am getting the following error:

Code (CSharp):

1. Unexpected exception Burst.Compiler.IL.CompilerException: Missing external function `name: DefKit.DbvtWrapper::DbvtRefitTriangles` => `method: System.Void DefKit.DbvtWrapper::DbvtRefitTriangles(System.Void*,System.Void*,System.Void*,System.Int32). Reason: Function pointer cannot be IntPtr.Zero`
2.  
3.   at Burst.Compiler.IL.Jit.JitCompiler.CompileMethod (Mono.Cecil.MethodReference methodReference, Burst.Compiler.IL.Jit.JitOptions jitOptions) [0x003e8] in <ff35cf56a4e14266be5fb4c5a298e1c8>:0
4.   at Burst.Compiler.IL.Jit.JitCompilerService.Compile (Burst.Compiler.IL.Jit.JitCompilerService+CompileJob job) [0x002a6] in <ff35cf56a4e14266be5fb4c5a298e1c8>:0
5.  
6. While compiling job: System.Void Unity.Jobs.IJobExtensions/JobStruct`1<DefKit.DbvtTree/TreeRefitJob>::Execute(T&,System.IntPtr,System.IntPtr,Unity.Jobs.LowLevel.Unsafe.JobRanges&,System.Int32)

 

I don't think Burst supports calls into native yet (among a lot of other features it should support eventually). You may want to split the calls into a 2 job chain:
* A non-Burst job that calls your native code.
* A burst-enabled job that has your HEAVY MATH, dependent on the first job.

Hopefully that works.

 

Thanks @recursive, I know that I can split the job in two but then I am getting a performance hit from transferring the temporary data between jobs. Maybe in the above example it is not that visible but for sure that might be a case.

 

yes, but you need to store and re-read data from job to job, which requires slow memory accesses.
On the GPUs this is might be a performance bottleneck.

 

June 2020... is it supported yet?

I think this would allow us to run "Environment.TickCount" within a burst compiled function.

 

You can do this with Function pointers: https://docs.unity3d.com/Packages/com.unity.burst@1.3/manual/index.html#function-pointers

 

How exactly is this done? I can't seem to get it to work with function pointers.

 

I'm getting these errors at build time on Android (on Unity 2022.3.10)

Code (CSharp):

1. Library/Bee/artifacts/Android/AsyncPluginsFromLinker: ld: error: undefined symbol: TestScreenAABB
2. UnityEngine.GUIUtility:ProcessEvent (int,intptr,bool&) (at /Users/bokken/build/output/unity/unity/Modules/IMGUI/GUIUtility.cs:190)
3.  

Code (CSharp):

1. Library/Bee/artifacts/Android/AsyncPluginsFromLinker: Burst internal compiler error: Burst.Compiler.IL.Aot.AotLinkerException: Non 0 exit code | The native link step failed. Check previous exception in the log - linker command line : "/Applications/Unity/Hub/Editor/2022.3.10f1/PlaybackEngines/AndroidPlayer/NDK/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang '@/var/folders/tt/q4189z2s0xs3bdkr2y_3_dz80000gn/T/tmp5b4f49f8.tmp'"
2. UnityEngine.GUIUtility:ProcessEvent (int,intptr,bool&) (at /Users/bokken/build/output/unity/unity/Modules/IMGUI/GUIUtility.cs:190)
3.  

Code (CSharp):

1. Building Library/Bee/artifacts/Android/AsyncPluginsFromLinker failed with output:
2. UnityEditor.Build.BuildFailedException: Burst compiler (1.8.8) failed running
3.  
4. stdout:
5. Starting 1 library requests
6. Error: ld: error: undefined symbol: TestScreenAABB
7. >>> referenced by CPUDrivenRenderer.cs:240 (/Users/joshuawilde/ProjectGamma/Assets/Scripts/Rendering/CPUDrivenRenderer.cs:240)
8. >>>               /Users/joshuawilde/ProjectGamma/Temp/Burst/burst-aotybvypi50.0hy/lib_burst_generated_part_593183a4a0e0e14bbf8e3a4b4265c375.o:(Unity.Jobs.IJobExtensions.JobStruct`1<CPUDrivenRenderer.CPUDrivenRendererCullingJob>.Execute(ref CPUDrivenRenderer.CPUDrivenRendererCullingJob data, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, ref Unity.Jobs.LowLevel.Unsafe.JobRanges ranges, int jobIndex) -> void_ed23eb2e88e7599b222ed770d92c1a02 from UnityEngine.CoreModule, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null)
9. clang: error: linker command failed with exit code 1 (use -v to see invocation)
10.  
11. Error: Burst internal compiler error: Burst.Compiler.IL.Aot.AotLinkerException: Non 0 exit code | The native link step failed. Check previous exception in the log - linker command line : "/Applications/Unity/Hub/Editor/2022.3.10f1/PlaybackEngines/AndroidPlayer/NDK/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang '@/var/folders/tt/q4189z2s0xs3bdkr2y_3_dz80000gn/T/tmp5b4f49f8.tmp'"
12.  
13.  
14.  
15. ld: error: undefined symbol: TestScreenAABB
16. >>> referenced by CPUDrivenRenderer.cs:240 (/Users/joshuawilde/ProjectGamma/Assets/Scripts/Rendering/CPUDrivenRenderer.cs:240)
17. >>>               /Users/joshuawilde/ProjectGamma/Temp/Burst/burst-aotybvypi50.0hy/lib_burst_generated_part_593183a4a0e0e14bbf8e3a4b4265c375.o:(Unity.Jobs.IJobExtensions.JobStruct`1<CPUDrivenRenderer.CPUDrivenRendererCullingJob>.Execute(ref CPUDrivenRenderer.CPUDrivenRendererCullingJob data, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, ref Unity.Jobs.LowLevel.Unsafe.JobRanges ranges, int jobIndex) -> void_ed23eb2e88e7599b222ed770d92c1a02 from UnityEngine.CoreModule, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null)
18. clang: error: linker command failed with exit code 1 (use -v to see invocation)
19.  
20.   at Burst.Compiler.IL.Aot.AotNativeLinkBase.RunNativeLinkerTool (System.String command, Burst.Compiler.IL.Helpers.CommandLineArguments arguments, System.String errorMessage, System.String commandType, System.String workingDirectory, Burst.Compiler.IL.Aot.AotNativeLinkBase+AotNativeLinkerConsoleOutput consoleOutput) [0x000f8] in <772865cdae2d4bff9c305a06338cabcf>:0
21.   at Burst.Compiler.IL.Aot.AotNativeLinkAndroid.Link (Burst.Backend.TargetCpu targetCpu, System.Collections.Generic.List`1[T] inputFiles, System.String outputFile, System.Boolean enableDebugInfo) [0x000cd] in <772865cdae2d4bff9c305a06338cabcf>:0
22.   at Burst.Compiler.IL.Aot.AotCompiler.Link (System.Collections.Generic.List`1[T] groups, System.String nameSuffix, Burst.Compiler.IL.Aot.AotCompilerOptions compilerOptions, System.IO.TextWriter consoleOut, System.IO.TextWriter consoleError) [0x0023f] in <772865cdae2d4bff9c305a06338cabcf>:0
23.   at Burst.Compiler.IL.Server.LibraryCompiler.LinkAndFinalize (Burst.Compiler.IL.Server.CompilationJob request, Burst.Compiler.IL.Server.SharedLibraryCompilationState sharedState, System.Int32 methodGroupIndex, Burst.Compiler.IL.NativeCompiler nativeCompiler, Burst.Compiler.IL.Aot.AotCompilerOptions defaultOptions, Burst.Compiler.IL.Aot.AotModuleGroup aotModuleGroup) [0x007be] in <772865cdae2d4bff9c305a06338cabcf>:0
24. stderr:
25.  
26.   at Unity.Burst.Editor.BurstAotCompiler+BclRunner.RunProgram (UnityEditor.Utils.Program p, System.String exe, System.String args, System.String workingDirectory, UnityEditor.Scripting.Compilers.CompilerOutputParserBase parser) [0x001f7] in ./Library/PackageCache/com.unity.burst@1.8.8/Editor/BurstAotCompiler.cs:1710
27.   at Unity.Burst.Editor.BurstAotCompiler+BclRunner.RunManagedProgram (System.String exe, System.String args, System.String workingDirectory, UnityEditor.Scripting.Compilers.CompilerOutputParserBase parser) [0x0003f] in ./Library/PackageCache/com.unity.burst@1.8.8/Editor/BurstAotCompiler.cs:1587
28.   at Unity.Burst.Editor.BurstAotCompiler+BclRunner.RunManagedProgram (System.String exe, System.String args, UnityEditor.Scripting.Compilers.CompilerOutputParserBase parser) [0x00000] in ./Library/PackageCache/com.unity.burst@1.8.8/Editor/BurstAotCompiler.cs:1562
29.   at Unity.Burst.Editor.BurstAotCompiler.OnPostBuildPlayerScriptDLLsImpl (Unity.Burst.Editor.BurstAotCompiler+BurstAOTSettings settings, UnityEditor.Compilation.Assembly[] playerAssemblies) [0x00914] in ./Library/PackageCache/com.unity.burst@1.8.8/Editor/BurstAotCompiler.cs:710
30.   at Unity.Burst.Editor.BurstAOTCompilerPostprocessor.DoGenerate (UnityEditor.Compilation.Assembly[] assemblies) [0x00013] in ./Library/PackageCache/com.unity.burst@1.8.8/Editor/BurstAotCompiler.cs:270
31.   at Unity.Burst.Editor.BurstAOTCompilerPostprocessor.GenerateNativePluginsForAssemblies (UnityEditor.Build.IGenerateNativePluginsForAssemblies+GenerateArgs args) [0x0007d] in ./Library/PackageCache/com.unity.burst@1.8.8/Editor/BurstAotCompiler.cs:179
32.   at UnityEditor.Modules.BeeBuildPostprocessor.GenerateNativePluginsForAssemblies (PlayerBuildProgramLibrary.Data.GenerateNativePluginsForAssembliesArgs args) [0x00050] in /Users/bokken/build/output/unity/unity/Editor/Mono/Modules/BeeBuildPostprocessor.cs:605
33.   at Bee.BeeDriver.BuildRequest+<>c__DisplayClass63_0`1[T].<RegisterRPCCallback>b__0 (System.Object o) [0x00000] in /Users/bokken/build/output/unity/unity/Tools/Bee/Bee.BeeDriver2/BuildRequest.cs:66
34.   at Bee.BeeDriver.BeeDriver_RunBackend+<>c__DisplayClass1_0.<ProcessRPCRequest>b__0 () [0x00000] in /Users/bokken/build/output/unity/unity/Tools/Bee/Bee.BeeDriver2/BeeDriver_RunBackend.cs:88
35.   at System.Threading.Tasks.Task.InnerInvoke () [0x0000f] in <8f2b770c4a1b401a99ba9ed80a5c09b9>:0
36.   at System.Threading.Tasks.Task.Execute () [0x00000] in <8f2b770c4a1b401a99ba9ed80a5c09b9>:0
37. --- End of stack trace from previous location where exception was thrown ---
38.  
39.   at Bee.BeeDriver.BeeDriver_RunBackend.ProcessRPCRequest (Bee.BinLog.RPCActionMessage msg, Bee.BeeDriver.BuildRequest+RPCCallback rpcCallback, IPCConnection ipcConnection, System.Threading.Tasks.Task writePipeConnectionTask, Bee.BeeDriver.InternalState state) [0x000ba] in /Users/bokken/build/output/unity/unity/Tools/Bee/Bee.BeeDriver2/BeeDriver_RunBackend.cs:88
40. UnityEngine.GUIUtility:ProcessEvent (int,intptr,bool&) (at /Users/bokken/build/output/unity/unity/Modules/IMGUI/GUIUtility.cs:190)
41.  

The function I have is called TestScreenAABB.

Here is the C# declaration :

Code (CSharp):

1. public class OcclusionCullingPlugin {
2.  
3. #if UNITY_EDITOR
4.     private const string dllName = "SoftwareOcclusionCulling";
5. #else
6. private const string dllName = "__Internal";
7. #endif
8.  
9.     [DllImport(dllName)]
10.     public static extern bool TestScreenAABB(float minX, float minY, float maxX, float maxY, float w);
11.  
12. ...
13.  
14. }

and the c code :

Code (CSharp):

1. extern "C" {
2.  
3. bool TestScreenAABB(float minX, float minY, float maxX, float maxY, float w) {
4.         return true;
5. }
6.  
7. }

and I call it in the job like so :

Code (CSharp):

1. bool isVisibleFromOcclusionCulling = OcclusionCullingPlugin.TestScreenAABB(min.x, min.y, max.x, max.y, max.w);
2.  

If I call the function outside of a burst job it works fine, but inside a burst job and it gives me this linker error.

 

any idea @MarcoPersson ?

 

Does it matter if the plugin is from a cpp file, vs an actual android .so lib?

I wouldn't think so...

I tried with the latest burst package, it's still not working. Also if I try the function pointer option, I get runtime null pointer errors.

 

Can you please send me the sample project you made @MarcoPersson ? I will try to build it on my machine. I'm testing in a super basic project and it's still not working. Function pointer nor dllimport call.

 

I've tried everything I can think of (static lib, dynamic lib, cpp files, using functton pointers, not using function pointers, tried on both Unity 2022 and Unity 2023, I'm on the latest Burst).

I have no idea why it's not working, and I would VERY MUCH appreciate a sample project where this does work.

 

Thanks, but that's not it. I tried with all different combinations of that and it's still not working. Have you explicitly tested on Android in a build?

 

If I use a static lib ".a" and "__Internal" it will just not build at all. If I use a dynamic lib ".so" and "SoftwareOcclusionCulling", then it builds but then has a null pointer exception at runtime, saying "unable to load plugin", even though code that isn't in jobs can still access those same plugin functions just fine.