Search Unity

  1. Unity 2020.1 has been released.
    Dismiss Notice
  2. We are looking for feedback on the experimental Unity Safe Mode which is aiming to help you resolve compilation errors faster during project startup.
    Dismiss Notice
  3. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

Scheduling jobs from NOT a MainThread

Discussion in 'Data Oriented Technology Stack' started by korzen303, Oct 16, 2018.

  1. korzen303

    korzen303

    Joined:
    Oct 2, 2012
    Posts:
    175
    Hi,

    I have got a large code base of jobified and Burst compiled physics simulation, which is scheduled from the main thread.

    However, now I need to support a use case where I have got a native thread spawned and owned by haptics hardware API, which should schedule all the calculations in sync. It runs at more or less constant rate of 1kHz, which gives less than 1ms to do the physics calculation and send the resulting force-feedback vector to the haptic device (so called "hapitc frame").

    @Joachim_Ante , @xoofx I would like very much to reuse my current jobified code, which does the job very well and took me a lot of time to develop. I don't want to schedule/complete jobs from multiple threads, just be able to do this from non-mainthread. I don't need advanced profiler or debugger support.

    I think this is not that unusual use case as it is quite common while working with hardware, for example, with automotive sensors. Currently, such use cases prevent the use of JobSystem and Burst, which are great tools

    Thank you
     
    Roni92pl likes this.
  2. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,053
    So there is a native haptic thread in C++ plugin?
    And it calls into C# from that C++ code?

    And from that C# callback on a custom C++ thread, you want to be able to schedule a C# job?
    Do you only want to schedule it or do you also want to call JobHandle.Complete() ?
     
  3. korzen303

    korzen303

    Joined:
    Oct 2, 2012
    Posts:
    175
    Hi @Joachim_Ante and thanks a lot for a quick response!

    Yes, exactly. I would like to schedule jobs as well as complete them from C# callback(s) on a custom C++ thread.
     
  4. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,053
    Its theretically enable to do that for us. Unfortunately the safety system which is always on would break if we do that because it is a deterministic system that does all checks on the main thread before the job is scheduled etc.

    So its a good chunk of work and we are not planning to do that any time soon unless there is a significant demand for it.

    That said, burst has an API to compile a delegate directly that you might be able to use to get better C# perf. And you could this way run existing job code directly on that C# callback thread by making your own scheduling method that just runs in on that C# callback thread.
     
    Roni92pl likes this.
  5. korzen303

    korzen303

    Joined:
    Oct 2, 2012
    Posts:
    175
    I see @Joachim_Ante . Having a fully deterministic simulation would be great indeed. However, for, now, I am breaking determinism anyways by using [NativeDisableUnsafePtrRestriction] to call native code or [NativeDisableParallelForRestriction] to atomically write to other indices during constraints solving, which is typical in physics simulation.

    Taking these under consideration, would it be possible to make a "non-deterministic" job scheduling, which would omit all the main-thread safety checks, in a similar spirit how the aforementioned attributes work?

    Regarding Burst would it be possible to get a small code snippet to manually compile and invoke the job? The Burst documentation does not mention this.

    Summarizing, job scheduling is more important to me as I have got a quite complex jobified physics solver, which already schedules hundreds of jobs from different sub-systems each frame (collisions, deformation, heat propagation, logic). Burst is undoubtedly great piece of software but, in the worst case, with some extra work, I could port the performance-critical algorithms back to native C++, just the way I did it for last couple of years...

    Thanks
     
  6. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,053
    I just want to point out that the most successful commercial physics engine out there is in fact deterministic and this is a perfectly solvable problem... Can't share any code right now. But making deterministic physics is very achievable without performance loss. And thats our intention for what we want to include in the Unity ECS physics solution.

    Check out the Unity.Burst namespace. There is a compile delegate function there somewhere. Sorry for the brevity. Too much stuff to do right now preparing for Unite...
     
    pvloon likes this.
  7. Roni92pl

    Roni92pl

    Joined:
    Jun 2, 2015
    Posts:
    328
    +1 on this feature very much. If safety system is biggest issue, maybe you could do it without it at first(unsafe) and we would use it at own risk, I mean we're not all children playing in sandbox :)
     
  8. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,053
    The issue is really that its deeply integrated and cutting a codepath to turn it off in a subset of the jobs depending on where they are scheduled is actual work. If we do that work i'd rather do it right and make it safe.

    I know we are not children... In our community we are professionals & enthusiasts with an incredible amount of skills all across...

    But guess what. I think I am somewhat knowledgable in the field of multithreading code. And i can't possibly imagine writing the amount of jobified code I am writing for features on top of ECS without a safety system. We had so some many freaking examples in our C++ code of just plain wrong jobified code. This is freaking hard to get right. Not having a safety system and writing large amounts of jobified code is quite simply not a responsible thing to do. On the flip side i know for sure of bazillion little issues the safety system found for me that would simply be shipped and the one who cries at the end is the game developer because of a S*** metacritic rating because the executable crashes in very hard to reproduce cases...

    So in your choice of words... A safety system for jobs is in fact an adult thing to do. Made by and for Adults, who have been there and know why it's valuable in the long run.
     
    Last edited: Oct 18, 2018
    Kender, June1111, FROS7 and 3 others like this.
  9. eizenhorn

    eizenhorn

    Joined:
    Oct 17, 2016
    Posts:
    2,034
    Disagree. We are more a childs, than other, because we make games :D If you lost child inside - you can't create games with soul :p
     
    Last edited: Oct 18, 2018
  10. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,053
    Good point :)
     
  11. Lurking-Ninja

    Lurking-Ninja

    Joined:
    Jan 20, 2015
    Posts:
    5,457
    You should see my face when I get a new toy to play around (like ECS, Jobs, new .NET version, etc)... I drop all my matchboxes immediately and jump in the sandbox with drawn toy shovel and bucket, head first.
     
    Kender, M_R and FROS7 like this.
  12. korzen303

    korzen303

    Joined:
    Oct 2, 2012
    Posts:
    175
    Hi @Joachim_Ante,
    I have been developing a workaround for a while. I have modified my physics solver to be scheduled at 1kHz from the haptics native thread. The solver loop runs a bunch of math heavy functions, which I am porting from Jobs/Burst back to native C/C++. However, it would be great not to double the work and later maintain quite complex algorithms in both HPC# and C/C++.

    I have tried calling IJob.Execute() from my native callback and it worked fine. Unfortunately, it is not Burst accelerated. The IJobExtensions.Run(T), although the docs say "Perform the job's Execute method immediately on the same thread.", gives exception saying that it has to be called from the MainThread.

    I can Burst compile a delegate but I have got no idea how to combine all this with IJob
    Code (CSharp):
    1.          
    2. public delegate void ExecuteJob();
    3. (...)
    4. ExecuteJob exec = BurstCompiler.CompileDelegate<ExecuteJob>(what should go here???);
    So going back to your quoted answer from few weeks ago, could you please provide some code snippet of my own scheduling method? I would appreciate this very much.

    Thanks!
     
  13. korzen303

    korzen303

    Joined:
    Oct 2, 2012
    Posts:
    175
    Hi @Joachim_Ante,

    to quickly recap, I have got a heavily jobified surgical simulation system implemented using HPC (JobSystem + Burst, no ECS). It works great with VR controllers input and I am very happy with it. Please have a quick look:


    However, now I have got customers, who would like to use my simulator with some more serious, haptic-enabled devices, such as Phantom Omni pictured below. In order to do this, I need to be able to run my simulation physics update at more or less constant rate of 1000Hz to generate realistic and smooth force-feedback.

    Unfortunately, it is currently not possible, as Jobs can only be scheduled from Unity's main-thread. I have started porting a lot of performance critical code back to C/C++ to overcome this issue but it is a lot of code and maintaining two math-heavy versions of the same code is rather painful.

    I just wanted to ask whether something has changed over the last year in the topic of giving developers possibility to overcome the safety-system and be able to use JobSystem and Burst-compiled jobs from non-main thread?

    Thank you



     
    e199, florianhanke and wobes like this.
  14. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,053
    It is not clear to me why running the simulation at a 1000Hz rate for haptic feedback necessitates a controlling thread.

    Assuming your rendering happens at 60 FPS you could schedule 15 chained sets of jobs that update the simulation. IJobParallelForDeferred is necessary to do this assuming that some of batch sizes aren't known at schedule time anymore. This is how Unity.Physics works. There is no sync point on the main thread.

    Truly... The concept of a thread is absolutely not necessary for what you are doing it simply creates unnecessary complexities. It's just a different way of looking at the problem.
     
  15. korzen303

    korzen303

    Joined:
    Oct 2, 2012
    Posts:
    175
    Hi @Joachim_Ante, thanks for a quick reply and apologies for delay on my side.

    The human haptic perception operates at a far higher rate than our visual system (around 30 times faster).
    It is commonly accepted that the required refresh rate to provide realistic force feedback to be at least 1,000 Hz.*

    I have already tried what you have described (scheduling 15 jobs inside the graphics frame). This makes sens assuming that I could have the wait period between each subsequent job launch/completion of roughly 1ms. However, in practice, this does not work that well with the hardware, the force feedback is not that smooth and I can feel a small jitter on the device.

    To solve this I would need to rather precisely match the hardware timing (its called haptic frame).
    Ideally, I would like to to launch burst jobs from the native thread, which is controlled by the haptic device drivers.

    I think such use cases are quite common when dealing with timing-dependent hardware. For example, I had similar issues in my previous job in the automotive sector while working with car sensors. If I didn't "tick" them in a given time-frame they were going into error mode.

    I am fully aware that Unity is a game engine and not real-time operating system but having a possibility to use such a great tools as JobSystem and Burst for such cases, without the hassle of native code implementations, would be great and enable many more interesting applications in academia and industry.

    Thanks!

    * “The Role of Haptics in Medical Training Simulators: A Survey of the State of the Art”, T. Coles & N. John, 2010
     
  16. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,053
    I recommend sharing the code and using BurstCompiler.CompileFunctionPointer for the time being. It probably requires 19.3, it has a lot of limitations

    * safety system gone
    * Any exception will currently crash (Will be fixed)
    * Possible aliasing bugs at the moment (Will be fixed)

    That said, it would allow you to share code. I would recommend using normal system / C# job style approach inside of Unity for best development iteration and then have a wrapper around it with BurstCompiler.CompileFunctionPointer essentially executing the jobs yourself from the dedicated hardware thread. It should even be possible to execute the FunctionPointer you get back directly from C++ without any extra interop.
     
    recursive and Seb-1814 like this.
  17. korzen303

    korzen303

    Joined:
    Oct 2, 2012
    Posts:
    175
    Thanks again for taking time to help me out and quick response.
    Yes, that would be already very helpful. I could use some other job scheduler to launch the Burst compiled jobs.

    I have found some more info on BurstCompiler.CompileFunctionPointer<T> but, given it's description in the 1.1.1 release notes and some example Burst 1.1.1 new release , I am a bit confused how can I achieve this:
    Here is some code sample from my repo with JobSystem/Burst examples examples
    https://github.com/korzen/Unity3D-JobsSystemAndBurstSamples

    I would be very grateful for some snippet on how to use BurstCompiler.CompileFunctionPointer<T> to launch this job from outside the JobSystem

    Code (CSharp):
    1. [BurstCompile]
    2. struct OceanJob : IJob
    3. {
    4.     public NativeArray<Vector3> vertices;
    5.  
    6.     public float time, rippleStrength, scale;
    7.  
    8.     public void Execute()
    9.     {
    10.         for (int i = 0; i < vertices.Length; i++)
    11.         {
    12.             float3 vertex = vertices[i];
    13.             float2 offset = new float2(vertex.x, vertex.z);
    14.             float ripple = math.sin(time + math.length(offset));
    15.             vertex.y = ripple * rippleStrength;
    16.             vertices[i] = vertex;
    17.         }
    18.     }
    19. }
    Thank you
     
  18. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,053
    See the attached cs file for example code.
     

    Attached Files:

    MostHated likes this.
  19. korzen303

    korzen303

    Joined:
    Oct 2, 2012
    Posts:
    175
    Thank you Joachim! Have a good weekend
     
    MostHated likes this.
  20. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    7,289
    To be honest, if you trying to control/monitor hardware at such high frequency, you should be using relevant hardware, which does high frequency controls. I wouldn't be calling Desktop as good tool for this nature. It has lots of own internal overheads and risk of instability at such high rate.

    Unity should be used more to do visualization HMI, rather than direct hardware control.
    I am just not convinced upon using chosen approach, as for surgical precision solution.
     
  21. korzen303

    korzen303

    Joined:
    Oct 2, 2012
    Posts:
    175
    Hi @Joachim_Ante I have managed to convert some of my jobs to FunctionPointers and successfully schedule them from a native C++ thread. Thanks a lot for helping out!

    Any chances of adding Job System to this equation? In other words, after having a callback from native thread to C# I would like to split the workload and schedule/complete bunch of jobs from within this callback.

    If not could you please you recommend some alternative C/C++ or C# job/task system to use instead of Unity's built-in?

    Thanks

    PS: Just for the reference: a guide how to use Burst-compiled functions directly from C# can now be found in the offical docs https://docs.unity3d.com/Packages/com.unity.burst@1.2/manual/index.html
     
unityunity