Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. We have updated the language to the Editor Terms based on feedback from our employees and community. Learn more.
    Dismiss Notice

Why does a Unity program that does very little use 80 threads?

Discussion in 'Editor & General Support' started by Arowx, May 22, 2022.

  1. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Just noticed that a Unity test program using threads was actually using 89 threads when I was only programming it to use 8 and then tested another Unity simple program I made earlier and noticed that it used 80 threads why so many?
     
  2. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,594
    Do you have CPU with 40+ cores?
    Or do you mean processes instead threads.

    Please provide code.
     
  3. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    20,185
    Game engines have a lot of tasks that they perform behind the scenes. Before I started using Unity I was reading up on how to create my own and it was common to spin up threads for just about everything. Reading and writing to files, input devices and networking sockets, playing audio clips, etc was all best performed in threads.
     
    Last edited: May 22, 2022
    Not_Sure and angrypenguin like this.
  4. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Task Manager Threads.
     
  5. DragonCoder

    DragonCoder

    Joined:
    Jul 3, 2015
    Posts:
    1,493
    How do you see threads in Task Manager?
    You mean the separate MS Programm Process Explorer?
    Task manager shows processes and while the UNITY Editor seems to load a couple of repeated things like "Console Window Host" and "Unity Shader Compiler", a built game only has one game process and that crash-watcher utility.

    If you do actually mean threads, it does not make that much sense to limit all threads to exactly the CPUs native supported number, because creating threads takes time so you may get more performance if you launch more threads and let the thread scheduler do its job.

    Still would be surprised if it is actually 80. Show some screens please :)

    Don't forget that Thread-Pooling is a thing!
     
  6. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,594
    I think OP means this threads

    upload_2022-5-23_3-36-22.png
     
    DragonCoder likes this.
  7. runner78

    runner78

    Joined:
    Mar 14, 2015
    Posts:
    763
    I do not know if this is part of the process, but e.g. C# thread pool also reserves some threads (I think 25 by default), then the default threads from the engine (worker threads per logical core, render thread, amin thread), in addition If editor, tools, etc. comes there, some can already come together.
     
    DragonCoder likes this.
  8. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Details in Task Manager - optional threads column.

    Not Unity.exe, YourGame.exe even a simple empty none DOTS game project has 80 threads.

    Is there a breakdown of what these threads are for and is Unity modular enough to reduce this count for smaller exes?
     
  9. hippocoder

    hippocoder

    Digital Ape Moderator

    Joined:
    Apr 11, 2010
    Posts:
    29,723
    First time I've seen complaining about parallel execution! If you can demonstrate this reduces Unity's perf then please do.
     
  10. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Well I was working on a threaded way of boosting an algorithms performance and was getting some very odd results where a few more threads would boost performance then adding more threads would reduce performance.

    Could Unity's 80 threads log jam the threading system?

    Googled it and... Threads use:
    1. Memory in the form of stack space 2MB a thread.
    2. Scheduling overheads that consume CPU cycles.
    So they are sucking up memory and CPU time before we even consider that they could impact my programs performance. e.g. my 8-16 threads could be competing with 80 threads and this could impact the performance gains achievable.

    So every time the Scheduler runs a Unity thread's 2MB of stack will be moved through the cache if you do this 80 times then unless you have a very large cache and core CPU your threads will be displaced and have to go back to main memory.

    Surely this would negatively impact performance of anyone trying to use Threading with Unity?
    Would this also slow down Jobs or are some of these 80 threads Jobs threads and even so 80 seems excessive and prone to cache thrashing reducing performance?

    PS What CPUs are Unity developing on as even the 5800X3D only has 96 MB of L3 Cache, are there any CPUs with 160 MB cache?

    A: Ryzen Threadripper PRO 5995WX has 256 MB L3 Cache!!
     
    Last edited: May 23, 2022
  11. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    10,594
    This is hypothetical. Never heard of such real case scenario.

    The more real problem is, when you have some application running, like vide recorder, which uses own CPU core's thread, while game may want to use that core's thread too.
    But that can be the case, assuming game is capable utilising at full capacity all CPU cores.

    Still, it is a bit iffy, if and how it would affect idę recorder as an example.

    You could always tell game, not to use all cpu core's threads. I.e. 8 cores is 16 threads, let game use 7 cores, or 15 threads only. Then you have spare thread to run anything in background. Again, that would be the case, if all threads are busy at full capacity.
     
  12. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Process Explorer shows the actual threads and

    50 - UnityPlayer.dll
    20 - amdxx64.dll!boost
    3 - UnityPlayer.dll!UnityMain
    2 - amdilhk64.dll
    1 - GameAssembly.dd
    2 - ntdll.dll!TpReleaseCleanupGroupMembers
    1 - Project.exe
    1 - combase.dll

    But only about 20 appear active...

    Active threads
    1 - amdxx64.dll!boost
    1 - Project.exe
    3 - UnityPlayer.dll!UnityMain
    15 - UnityPlayer.dll
    1* - ntdll.dll!TpReleaseCleanupGroupMembers - occasionally

    So why does Unity have 60 dormant threads taking up 2 Mb memory and scheduling CPU time each?
     
  13. runner78

    runner78

    Joined:
    Mar 14, 2015
    Posts:
    763
    80 thread is not so much, I have some programs that use the same amount (e.g. PHP storm, and probably other intellij based IDEs like Rider)
    I recently seen a program in my Process explorer that had over 250 threads.

    Unity starts always the worker threads for the Job system, and don't forget the C# Thread pool. It is better to start and keep threads than to constantly start and end them. That's why C# also uses a thread pool for System.Task.
     
  14. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    How many are active and how many dormant (Process Explorere -> Propteries -> Threads, active threads have additional information CPU/Cycles)?
     
  15. Ryiah

    Ryiah

    Joined:
    Oct 11, 2012
    Posts:
    20,185
    What you're seeing is completely normal and why most games only actively use a couple of threads. The number of threads is not the culprit though. Modern OSes can handle thousands of them and many applications have dozens if not hundreds. Windows 10's calculator is sitting at 56.

    What you've encountered is overhead. Where the overhead is coming from though we're not going to be able to tell without more in-depth details and ideally source access. You mentioned the JobSystem. Are you using ECS? Burst?
     
    Last edited: May 23, 2022
  16. neoshaman

    neoshaman

    Joined:
    Feb 11, 2011
    Posts:
    6,469
    If you are concern about that, maybe you should stop using unity and roll your own engine?
     
  17. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,518
    Not a great reason from technical point of view, but it's that way due to practicality. Different subsystems like to spawn their own little thread pools instead of sharing the threads as it's just easier that way. That makes you end up with way more threads than your machine has cores. Given 8 virtual cores, Unity will typically spawn:

    • 8 job queue threads;
    • 8 enlighten worker threads;
    • 8 asset gc threads (the thing that unloads unused objects from Resources.UnloadUnusedObjects, I think it's spawned on demand);
    • several audio worker threads (not 100% sure how many);
    • C# thread pool threads (spawned on demand as you queue work onto the threadpool);
    • Dedicated scene deserializer/loader thread;
    • Dedicated object loading thread;
    • Dedicated object destroying thread;
    • Dedicated GfxDevice worker thread;
    • Dedicated C# finalizer thread;
    • Dedicated profiler thread (only in development builds)

    Now, ideally we'd make different subsystems share threads. That's easier said than done:

    • JobQueue requires super fast scheduling algorithm as it is expected to support thousands of jobs every frame;
    • Enlighten threads come from middleware, which we don't have much control over;
    • Asset GC threads are forbidden from allocating memory, taking locks or doing anything else that's not "walking through memory". Resources.UnloadUnusedObjects suspends all the threads, which could be in middle of memory allocation, so if these threads allocated, they could result in a deadlock;
    • Audio worker threads are required to be super low latency to not drop audio samples;
    • C# thread pool is expected to behave similarly to .NET, where you can queue many long running tasks and threadpool is expected to spawn more threads if it runs out.

    There are other things also spawning threads that are outside of our control:
    • Graphics driver;
    • OS thread pool.

    We haven't seen any evidence of this affecting performance negatively (other than slightly higher memory usage), so it was never prioritized to be "resolved".

    Also, it's more like 1 MB of memory per thread, rather than 2 MB.

    P. S. You can see what threads Unity spawns by attaching a debugger to it, load our symbols from https://symbolserver.unity3d.com and then open the parallel stacks window:

    upload_2022-5-23_17-52-38.png
     
  18. runner78

    runner78

    Joined:
    Mar 14, 2015
    Posts:
    763
    So far i know, 64-bit C# application C# has an 4 MB stack size, is this also true for Unity's Mono fork?
     
  19. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,518
    This is all very platform specific, but on Windows, default stack size is specified when building the executable. On Windows, that defaults to 1 MB on both 32-bit and 64-bit: https://docs.microsoft.com/en-us/cpp/build/reference/stack-stack-allocations?view=msvc-170. Note that the 1 MB is reserved memory. It doesn't get committed until each page gets touched.

    Threads that don't explicitly set a stack size (most don't - I believe we make the stack smaller for audio threads, though), use the default 1 MB.

    This has nothing to do with .NET vs Mono. You can check with this code on Windows:

    Code (csharp):
    1. [DllImport("kernelbase.dll")]
    2. static extern void GetCurrentThreadStackLimits(out IntPtr low, out IntPtr high);
    3.  
    4. static void CheckStackSize()
    5. {
    6.     GetCurrentThreadStackLimits(out var low, out var high);
    7.     Debug.Log($"Stack size: {high.ToInt64() - low.ToInt64()} bytes");
    8. }
     
    MadeFromPolygons and Ryiah like this.
  20. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    40 x slower using Unity?

    Re-wrote the program as a C# .Net6 console and it zips along processing the data in 3 minutes. Running under Unity Mono and IL2CPP the process took > 2 hours with very strange thread scaling performance issues.

    I'm using Tasks to multi-thread the data processing anyone else finding slow Threading/Task issues in Unity?

    PS even the console version uses 22 threads (16 to process the data and 6 to run the console/program).
     
    Last edited: May 24, 2022
  21. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,518
    Sounds like an unrelated issue that might better be suited as a bug report (or a post in the scripting forum at the very least).
     
    JoNax97 and MadeFromPolygons like this.
  22. DragonCoder

    DragonCoder

    Joined:
    Jul 3, 2015
    Posts:
    1,493
    What happens if you run your separate program and the Unity variant at the same time so the CPU has to support all the threads?
    Also have you tried to profile in Unity where the time gets spent?
    And you did a build in Unity, not started in editor, I hope.

    Edit: At this difference non-parallelized code in Unity should still be faster. Undo the threading to confirm that and then profile what's slower than in your dedicated appliction.
     
    Last edited: May 25, 2022
    MadeFromPolygons likes this.
  23. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    It is the underlying reason that started this thread.

    It looks like Tasks perform totally different under Unity compared to .Net with a x40 slowdown in performance both when built to IL2CPP and Mono in this case.

    What algorithms would be good to test as Tasks for comparison between Unity and .Net?

    Maybe hashing, compression, decompression, n-body any of these GitHub - nxrighthere/BurstBenchmarks: Benchmarking Burst/IL2CPP against GCC/Clang machine code?
     
  24. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    @Tautvydas-Zilys On the issue of total threads what if Unity could analyse a games actual thread load and reduce the number based on this data or give the option of thread pool sizes to developers e.g in Build Options.
     
  25. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,518
    Doesn't matter. That is an unrelated issue and should be discussed elsewhere.

    There's no point of doing that. This has never came up in performance profiling as an issue.
     
    angrypenguin likes this.
  26. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    So you have tested this thoroughly especially with DOTS and Jobs being core features of Unity?
     
  27. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    OK I'm going to explore this a bit more and raise a new thread if I make any progress in working out what is going on.
     
  28. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,518
    Yes, as I said before, us spawning a lot of threads was never identified as a performance issue. And we profile Unity all the time.

    There were some issues in the past where we spawned too many job threads, and job system wasn't able to keep up but that is also a separate issue that has little to do with this.
     
    Ryiah, angrypenguin and Arowx like this.
  29. DragonCoder

    DragonCoder

    Joined:
    Jul 3, 2015
    Posts:
    1,493
    Why not use your algorithm in question? "de threadify" it and see what the performance is like and what the profiler shows then.
    That's to make sure it is actually threading overhead.
     
    Last edited: May 26, 2022
    Ryiah likes this.
  30. runner78

    runner78

    Joined:
    Mar 14, 2015
    Posts:
    763
    First, make sure your test doesn't contain any errors.
    Secondly, profile what exactly is taking up so much time.
    Without reference code from your side we don't know exactly what you are doing.

    I once did a test, .NET 5 vs Unity mono. I wanted to test the difference in the garbage collector.
    The result being that .NET 5 had no problems with my test, was fast and hardly used any memory. But for the old Mono GC it was a worst case scenario. It was slow and the memory consumption increased until Unity crashed.

    Check if your code has a GC problem.
     
    Ryiah and DragonCoder like this.
  31. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    It sounds like Unity has an underlying problem working with Async Tasks...

    Unity and .NET, what’s next? | Unity Blog
     
  32. DragonCoder

    DragonCoder

    Joined:
    Jul 3, 2015
    Posts:
    1,493
    Maybe? Maybe not? That they seek to adapt more of the performance improvements and language features of .NET does not really prove that.
    Please be so kind and provide some minimum example that showcases the issue. Then unity staff can investigate better and we as users know what exactly to avoid or where to search when encountering unexpected performance loss.

    Also I remember to have read that it's a general recommendation to not use async for high performance computing etc. Its intention is enabling a more readable code/architecture but like most of the time, comfort comes with performance overhead.. It's like one of the rules of computing sadly :(

    If you want pure performance in Unity: Burst jobs are almost always the thing to use. You can beat regular non-Unity C# by far most of the time and the system does a good job at, let's say, forcing you to avoid race conditions and deadlocks etc. which are the main hassle of traditional multithreading.
    It only gets tricky when you need large amounts of data back on the main thread very often to use with the Unity api, but that does not sound like your usecase at all.
     
    Last edited: May 27, 2022
    lmbarns and Ryiah like this.
  33. BIGTIMEMASTER

    BIGTIMEMASTER

    Joined:
    Jun 1, 2017
    Posts:
    5,181
    what sort of games are you guys making?

    are these sorts of questions coming up in your day-to-day development? Do you have to manage what is happening at happening at hardware level?
     
    Ryiah likes this.
  34. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Tried Jobs based processing with surprisingly terrible results compared to Tasks in .Net.

    Every approach I tried within Unity would take hours to process data that Tasks in .Net could process in minutes.

    And all I'm doing is SHA256 hash.

    I Just think that a larger width and deeper cycle algorithm like SHA256 seems to be outside the bounds of what the Job system is capable of doing?

    Or there are fundamental bugs in the Jobs systems* that emerge when running SHA256?

    Try it yourself process a lot of data via SHA256 hint: the mining algorithm in Bitcion uses a 4 byte value (ulong) to try and hit below a target hash value so only 4 billion SHA256 cycles needed.

    Single Threaded
    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using System.Security.Cryptography;
    4. using System;
    5. using UnityEngine;
    6. using TMPro;
    7. using System.Runtime.CompilerServices;
    8.  
    9. public class BitcoinTarget : MonoBehaviour
    10. {
    11.     // Cycle throught bitcoin Nonces trying to hit under a target value
    12.  
    13.     public SHA256 sha;
    14.     public byte[] hash0;
    15.     public byte[] hash1;
    16.  
    17.     public uint n = 0;
    18.  
    19.     public float startTime;
    20.     public float endTime = 0f;
    21.  
    22.     public static byte[] minHash = { //0x00, 0x00, 0x00, 0x00,  0x00, 0x00, 0x00, 0x00,  0x00, 0x00, 0x00, 0x00,  0x00, 0x00, 0x00, 0x00,
    23.                            0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,
    24.                            0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF
    25.                          };
    26.  
    27.     public TextMeshProUGUI text;
    28.  
    29.     void Start()
    30.     {
    31.  
    32.         byte[] rbytes = new byte[32];      
    33.  
    34.         for (int i = 0; i < rbytes.Length; i++)
    35.         {
    36.             rbytes[i] = (byte)(UnityEngine.Random.Range(0, 256));
    37.         }
    38.  
    39.         sha = new SHA256Managed();
    40.         hash0 = sha.ComputeHash(rbytes);
    41.  
    42.         startTime = Time.realtimeSinceStartup;
    43.     }
    44.  
    45.  
    46.     // Update is called once per frame
    47.     void Update()
    48.     {
    49.         byte[] data = new byte[36];
    50.  
    51.         Array.Copy(hash0, data, hash0.Length);
    52.  
    53.         float t = Time.realtimeSinceStartup;
    54.  
    55.         float nextFrame = t + 0.05f;
    56.  
    57.         byte[] nbytes;
    58.  
    59.         while (n < uint.MaxValue && Time.realtimeSinceStartup < nextFrame)
    60.         {
    61.             nbytes = BitConverter.GetBytes(n);
    62.             //Debug.Log("NBytes " + nbytes.Length + " Data " + data.Length + " D-N " + (data.Length - nbytes.Length));
    63.             Array.Copy(nbytes, 0, data, data.Length - nbytes.Length, nbytes.Length);
    64.             hash1 = sha.ComputeHash(data);
    65.  
    66.             if (HashLessThan(hash1, minHash))
    67.             {
    68.                 minHash = hash1;
    69.             }
    70.             n++;
    71.         }
    72.  
    73.         if (n == uint.MaxValue)
    74.         {
    75.             if (endTime > 0f)
    76.             {
    77.                 endTime = Time.realtimeSinceStartup;
    78.  
    79.                 Debug.Log("Completed total time = "+(endTime - startTime));
    80.             }
    81.         }
    82.  
    83.         float et = Time.realtimeSinceStartup - startTime;
    84.         if (endTime > 0f) et = endTime - startTime;
    85.         float percentage = (float)n / (float)uint.MaxValue;
    86.         float eta = et * 1f / percentage;
    87.  
    88.         TimeSpan etTime = TimeSpan.FromSeconds(et);
    89.         TimeSpan etaTime = TimeSpan.FromSeconds(eta);
    90.  
    91.         text.text = n.ToString("N0") + "\n"
    92.             + percentage.ToString("00.00%")
    93.             + "\nET  " + etTime.ToString(@"hh\:mm\:ss\:ff")
    94.             + "\nETA " + etaTime.ToString(@"hh\:mm\:ss\:ff")
    95.             + "\nhs  " + ((float)n / et).ToString("N0")
    96.             + "\nMin [" + (System.BitConverter.ToString(minHash).Replace("-", "")) + "]"
    97.         ;
    98.     }
    99.  
    100.     [MethodImpl(MethodImplOptions.AggressiveInlining)]
    101.     public static bool HashLessThan(byte[] a, byte[] b)
    102.     {
    103.         int i = 0;
    104.  
    105.         if (a.Length == b.Length)
    106.         {
    107.             while (i < a.Length)
    108.             {
    109.                 if (a[i] < b[i]) return true;
    110.                 else if (a[i] > b[i]) return false;
    111.                 i++;
    112.             }
    113.         }
    114.  
    115.         /** else Debug.LogWarning("Hash A ["+ System.BitConverter.ToString(a).Replace("-", "") + "]" +
    116.             a.Length + " not eqaul in length to [" +
    117.             System.BitConverter.ToString(a).Replace("-", "") + "] " +
    118.             b.Length);
    119.         */
    120.  
    121.         // Equal
    122.         return false;
    123.     }
    124. }
    125.  
    VS

    IJobParallelFor
    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using System.Security.Cryptography;
    4. using System;
    5. using UnityEngine;
    6. using TMPro;
    7. using Unity.Jobs;
    8. using Unity.Collections;
    9. //using Unity.Burst;
    10.  
    11. public class BitcoinTargetParallel : MonoBehaviour
    12. {
    13.     // Cycle throught bitcoin Nonces trying to hit under a target value
    14.  
    15.     //public SHA256[] sha;
    16.     public byte[] hash0;
    17.     public byte[] hash1;
    18.  
    19.     public uint n = 0;
    20.  
    21.     public float startTime;
    22.     public float endTime = 0f;
    23.  
    24.     public TextMeshProUGUI text;
    25.  
    26.     //[BurstCompile] //incompatible with managed 256
    27.     struct BitcoinTargetJob : IJobParallelFor
    28.     {
    29.         [ReadOnly]
    30.         public NativeArray<byte> hash;
    31.  
    32.         [ReadOnly]
    33.         public NativeArray<uint> nonces;
    34.  
    35.         public void Execute(int i)
    36.         {
    37.             SHA256 sha = new SHA256Managed();
    38.             byte[] data = new byte[36];
    39.             uint n = nonces[i];
    40.             byte[] nbytes = BitConverter.GetBytes(n);
    41.             byte[] hash1;
    42.  
    43.             //Debug.Log("NBytes " + nbytes.Length + " Data " + data.Length + " D-N " + (data.Length - nbytes.Length));
    44.             Array.Copy(nbytes, 0, data, data.Length - nbytes.Length, nbytes.Length);
    45.             hash1 = sha.ComputeHash(data);      
    46.         }
    47.     }
    48.  
    49.  
    50.     void Start()
    51.     {
    52.    
    53.         byte[] rbytes = new byte[32];        
    54.    
    55.         for (int i = 0; i < rbytes.Length; i++)
    56.         {
    57.             rbytes[i] = (byte)(UnityEngine.Random.Range(0, 256));
    58.         }
    59.  
    60.         SHA256 sha = new SHA256Managed();
    61.         hash0 = sha.ComputeHash(rbytes);
    62.  
    63.         startTime = Time.realtimeSinceStartup;
    64.     }
    65.  
    66.  
    67.     // Update is called once per frame
    68.     void Update()
    69.     {  
    70.         byte[] data = new byte[36];
    71.  
    72.         Array.Copy(hash0, data, hash0.Length);
    73.  
    74.         float t = Time.realtimeSinceStartup;
    75.  
    76.         float nextFrame = t + 0.016f;
    77.  
    78.         const int innnerloop = 128;
    79.         const int step = 4 * innnerloop;
    80.    
    81.         while (n < uint.MaxValue && Time.realtimeSinceStartup < nextFrame)  
    82.         {
    83.             //nbytes = BitConverter.GetBytes(n);
    84.             //Debug.Log("NBytes " + nbytes.Length + " Data " + data.Length + " D-N " + (data.Length - nbytes.Length));
    85.             //Array.Copy(nbytes, 0, data, data.Length - nbytes.Length, nbytes.Length);
    86.             //hash1 = sha.ComputeHash(data);
    87.  
    88.             var hash0 = new NativeArray<byte>(32, Allocator.Persistent);
    89.             var ns = new NativeArray<uint>(step, Allocator.Persistent);
    90.  
    91.             for (uint i = n; i < n + step; i++)
    92.             {
    93.                 ns[(int)(i - n)] = n;
    94.             }
    95.  
    96.             var job = new BitcoinTargetJob()
    97.             {
    98.                 nonces = ns,
    99.                 hash = hash0
    100.             };
    101.  
    102.             JobHandle jobHandle = job.Schedule(step, innnerloop);
    103.             jobHandle.Complete();
    104.  
    105.             hash0.Dispose();
    106.             ns.Dispose();
    107.  
    108.             n+=step;
    109.         }
    110.  
    111.         if (n == uint.MaxValue)
    112.         {
    113.             if (endTime > 0f)
    114.             {
    115.                 endTime = Time.realtimeSinceStartup;
    116.  
    117.                 Debug.Log("Completed total time = "+(endTime - startTime));
    118.             }
    119.         }
    120.  
    121.         float et = Time.realtimeSinceStartup - startTime;
    122.         if (endTime > 0f) et = endTime - startTime;
    123.         float percentage = (float)n / (float)uint.MaxValue;
    124.         float eta = et * 1f / percentage;
    125.  
    126.         TimeSpan etTime = TimeSpan.FromSeconds(et);
    127.         TimeSpan etaTime = TimeSpan.FromSeconds(eta);
    128.  
    129.         text.text = n.ToString("N0") + "\n"
    130.             + percentage.ToString("00.00%")
    131.             + "\nET " + etTime.ToString(@"hh\:mm\:ss")
    132.             + "\nETA " + etaTime.ToString(@"hh\:mm\:ss")
    133.             + "\nhs " + ((float)n / et).ToString("N0");
    134.             ;
    135.     }
    136. }
    137.  
    A modern CPU can easily do this in a few minutes but not from within Unity Jobs multi-threading.

    Most modern CPUs have SIMD SHA256 insturctions as well so maybe even faster if you could use those or if they are used by C# SHA256 class.

    *As terrible results were also found while trying Tasks within Unity maybe there is a Multi-threaded problem in Unity, see previous post with a llink to Blog post where Unity admits there are issues with Async operations in Unity.
     
    Last edited: May 28, 2022
  35. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,064
    They are questions that come up when working around some Unity bullshit, which is what people who use Unity think game development is.
     
  36. DragonCoder

    DragonCoder

    Joined:
    Jul 3, 2015
    Posts:
    1,493
    Thank you for posting your code! That gets us a good step forward :)

    Well, you should try to find a way to enable BurstCompile because otherwise there is naturally no acceleration from the Burst compiler at all and the jobs execute regular C#.

    I just found this, but honestly not sure how to use: https://docs.unity3d.com/Packages/c...Burst.Intrinsics.Arm.Neon.vsha256h2q_u32.html

    This is a very suboptimal pattern! Do not Complete() right after scheduling. Dispatching to the Job-worker threads has a slight overhead, therefore if a job has not been dispatched yet when Complete() is called (which will always be the case if Complete() is right away the next method), the system will just execute the job on the spot on the main thread.
    You need to let the system do other things before you call Complete(). One way to do it (in games at least), is to always complete at the beginning of the next frame what has been scheduled the last frame.

    So effectively your current IJobParallelFor implementation is a single-thread, regular Unity C# implementation, plus the Job creation overhead.

    One gotta get a bit familiar with the Burst compiler and Job system. Just like you had to familiarize yourself with Tasks\threads at some point.


    EDIT:
    Also, you should try to pre-allocate one or two NativeArrays and then pass indices on it around. Allocating and disposing so many arrays inside a loop is a terrible idea (and would likely be with any programming language or compiler). Those are fairly sure a significant spike if you look into the profiler.
    Same with the managed memory allocation inside the iJob (which naturally would not be allowed if you'd Burst-Compile).
    Furthermore take a look at Allocator.Temp and Allocator.TempJob since Allocator.Persistent is clearly meant for long term memory and the docs explicitly say "Don’t use Persistent where performance is essential.": https://docs.unity3d.com/2022.2/Documentation/Manual/JobSystemNativeContainer.html
    Alternatively, create a pool of BitcoinTargetJob instances and give those the ownership over the persistent memory to avoid allocating anything later in a loop at all. Note that NativeArrays are structs that hold a pointer to the real memory (similar to managed ones).

    So it's clear you do not know the ins and outs of how to achieve computing performance in Unity. That's fine. It is tricky if you don't invest a couple days on tutorials and experimenting. You already have knowledge of other tools to achieve what you want. So use those, but do not assume right away that there are performance bugs of this magnitude in Unity please...

    Btw. computing Bitcoin in a game engine? Really? Fairly sure the off the shelf solutions are optimized on assembler level.
    Albeit it would be interesting to compare a well done Burst solution (which would require a burst compiled sha256 algo (with or without manually optimized SIMD)) with C++ for example. Fear at least I do not have the time for that tho xP


    Please explain how one gets to compute Bitcoins from running into an issue while developing games :p
     
    Last edited: May 28, 2022
    PraetorBlue likes this.
  37. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Odd these are in every example of Jobs I've seen, yet no mention of offsetting Complete() to next frame or any mechanism to achieve this in the API?

    Or are we then back to the Async problem Unity has built into it's Mono legacy?

    Tried using Temp or TempJob and got errors about Jobs running longer than 4 frames?
    It kind of depends where the performance is needed with larger Jobs it can be better to let them run longer so the overheads of launching more Jobs does not grow too large.

    In this case the SHA256 algorithms internal requirements are probably larger (wider and deeper) than most of the algorithms that the Jobs system was designed for.

    Intel® SHA Extensions or AMD64 Architecture Programmer’s Manual, Volume 4: 128-Bit and 256-Bit Media Instructions

    SHA256 is already hardware optimised via dedicated SIMD instructions. Burst would just need to compile any SHA256 Algorithm to these SIMD instructions if they are not already used by .Net.
     
  38. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Great critical feedback but can you write/adapt a better Jobs based BitcionTarget class example?
     
  39. DragonCoder

    DragonCoder

    Joined:
    Jul 3, 2015
    Posts:
    1,493
    It is on one of the main pages one should stumble across:
    https://docs.unity3d.com/ScriptReference/Unity.Jobs.IJob.html
    But you are right, they really should add that comment in all examples where they call Complete() for simplicities sake.
    Or at very least here under "optimizations": https://docs.unity3d.com/Packages/com.unity.burst@0.2-preview.20/manual/index.html

    You are doing computing here and not a game. Therefore you should probably do everything in one frame..

    Definitely not. One (I) can do procedural terrain generation with burst jobs. The Sha256 algo is not thaaat huge: https://github.com/B-Con/crypto-algorithms/blob/master/sha256.c
    It does require some memory though and that's why the implementation in the cryptography library is a managed object *.
    The only inherent limitation of the Burst compiler is that it does not support full OOP (stuff like inheritance).

    * speaking of which, creating a new instance of that with every job is likely another culprit.
     
  40. DragonCoder

    DragonCoder

    Joined:
    Jul 3, 2015
    Posts:
    1,493
    I added this link in an edit to my post: https://docs.unity3d.com/Packages/c....Burst.Intrinsics.Arm.Neon.vsha256hq_u32.html
    There is something that has sha256 in its name, but admittedly no clue how to use that (and it might only be available on ARM cpus). Documentation is terrible.
    It just is not a common usecase though...

    Btw. what exactly is your goal?
    Bitcoin is pointless on PC CPUs even with assembler optimized by true experts in that field. It's dominated by ASICS where the algorithm has been physically programmed directly in silicon.
     
  41. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Intel® SHA Extensions or AMD64 Architecture Programmer’s Manual, Volume 4: 128-Bit and 256-Bit Media Instructions

    SHA256 is already hardware optimised via dedicated SIMD instructions. Burst would just need to compile any SHA256 Algorithm to these SIMD instructions if they are not already used by .Net.
     
  42. DragonCoder

    DragonCoder

    Joined:
    Jul 3, 2015
    Posts:
    1,493
    They probably are used by that .Net library as their developers had a greater motivation to make use of that acceleration than Burst developers where it's an edge case not very relevant to Unity or games...
    There are really a large number of CPU accelerated commands out there nowadays.
     
  43. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    So why don't Examples use LateUpdate() or Thread.Sleep(frameInterval) to Complete() Jobs?
     
  44. AcidArrow

    AcidArrow

    Joined:
    May 20, 2010
    Posts:
    11,064
    Yeah, I'm also confused on why the manual page that is supposed to tell us how to use things shows the wrong thing to do, but has a large comment saying "don't do this, this is bad". How about showing a better pattern to use?
     
    Arowx likes this.
  45. DragonCoder

    DragonCoder

    Joined:
    Jul 3, 2015
    Posts:
    1,493
    Thread.Sleep would be a rather bad idea since you stop the main thread with it and you cannot foresee how long your jobs would take accurately.
    The main idea behind jobs is: Schedule them as early as possible and Complete() the latest possible point when you need the result. This on its own is fairly logical I'd say as it simply maximizes the time the threads can work on the job and thus reduces the likelyhood that the main thread needs to wait during a Complete() call (which would result in lag).

    I strongly assume they showcase the direct call of Complete() for simplicity sake, as I said.

    What I often use when I do not need the result next frame but want to let something run across as many frames as the Job-Schedulers sees fit, is a coroutine which polls on jobhandle.IsCompleted(). Naturally that aims more at a game context where you execute frames. Also don't forget to call Complete() right after, because even if IsCompleted() returned true, it's still required to call Complete() so resources are given free, however the runtime error will tell you that.

    We are getting lost in details now however.
     
    Last edited: May 28, 2022
  46. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Tried a more parallel attempt with an array of jobs that cycle through a range of values (innerloop) so no need for arrays just to store a basic value.
    • Removes the Shedule then Complete problem of launching and closing in the same Update().
    • Improved the min hash testing and feedback to show progress or in this case lack of progress.
    This definitely shows up on Task Manager as multi-threaded on my 8 core 16 thread cpu with it set to run 16 threads.

    But running it the ETA timer and progress percentage even on 16 threads indicates that it would take 7-8 hours to hash 4 billion times as opposed to the 3-4 minutes using .net.

    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using System.Security.Cryptography;
    4. using System;
    5. using UnityEngine;
    6. using TMPro;
    7. using Unity.Jobs;
    8. using Unity.Collections;
    9. using System.Threading;
    10. using System.Runtime.CompilerServices;
    11.  
    12. //using Unity.Burst;
    13.  
    14. public class BitcoinTargetParallel : MonoBehaviour
    15. {
    16.     // Cycle throught bitcoin Nonces trying to hit under a target value
    17.  
    18.     //public SHA256[] sha;
    19.     public byte[] hash0;
    20.     public byte[] hash1;
    21.  
    22.     public ulong n = 0;
    23.     public ulong hashCounter = 0;
    24.  
    25.  
    26.     public float startTime;
    27.     public float endTime = 0f;
    28.  
    29.     public static byte[] minHash = { //0x00, 0x00, 0x00, 0x00,  0x00, 0x00, 0x00, 0x00,  0x00, 0x00, 0x00, 0x00,  0x00, 0x00, 0x00, 0x00,
    30.                            0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,
    31.                            0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF
    32.                          };
    33.  
    34.  
    35.     public TextMeshProUGUI text;
    36.  
    37.     JobHandle[] jobHandles;
    38.     IJob[] jobs;
    39.     NativeArray<byte>[] minhashes;
    40.     NativeArray<byte>[] hash0s;  
    41.  
    42.     const int threads = 16;
    43.  
    44.     //[BurstCompile] //incompatible with managed 256
    45.     struct BitcoinTargetJob : IJob
    46.     {
    47.         [ReadOnly]
    48.         public NativeArray<byte> hash;
    49.  
    50.         [ReadOnly]
    51.         public ulong nonceMin;
    52.         public ulong nonceMax;
    53.      
    54.         public NativeArray<byte> minhash;
    55.  
    56.         //public SHA256 sha;
    57.  
    58.         public void Execute()
    59.         {
    60.          
    61.             byte[] data = new byte[36];
    62.             Array.Copy(hash.ToArray(), data, hash.Length);
    63.  
    64.             SHA256 sha = SHA256.Create();
    65.  
    66.             for (ulong n = nonceMin; n < nonceMax; n++)
    67.             {
    68.                 byte[] nbytes = BitConverter.GetBytes((uint)n);
    69.                 byte[] hash1;
    70.              
    71.                 //Debug.Log("NBytes " + nbytes.Length + " Data " + data.Length + " D-N " + (data.Length - nbytes.Length));
    72.                 Array.Copy(nbytes, 0, data, data.Length - nbytes.Length, nbytes.Length);
    73.                 hash1 = sha.ComputeHash(data);
    74.  
    75.                 if (HashLessThan(hash1, minhash.ToArray()))
    76.                 {
    77.                     minhash.CopyFrom(hash1);
    78.                 }
    79.             }
    80.         }
    81.     }
    82.  
    83.  
    84.     void Start()
    85.     {      
    86.         byte[] rbytes = new byte[32];            
    87.      
    88.         for (int i = 0; i < rbytes.Length; i++)
    89.         {
    90.             rbytes[i] = (byte)(UnityEngine.Random.Range(0, 256));
    91.         }
    92.  
    93.         SHA256 sha = new SHA256Managed();
    94.         hash0 = sha.ComputeHash(rbytes);
    95.  
    96.         startTime = Time.realtimeSinceStartup;
    97.  
    98.         jobHandles = new JobHandle[threads];
    99.         jobs = new IJob[threads];
    100.         minhashes = new NativeArray<byte>[threads];
    101.         hash0s = new NativeArray<byte>[threads];      
    102.  
    103.         for (int i = 0; i < minhashes.Length; i++)
    104.         {
    105.             minhashes[i] = new NativeArray<byte>(32, Allocator.TempJob);
    106.             minhashes[i].CopyFrom(minHash);
    107.             hash0s[i] = new NativeArray<byte>(36, Allocator.TempJob);          
    108.         }
    109.     }
    110.  
    111.  
    112.     void Update()
    113.     {      
    114.         byte[] data = new byte[36];
    115.  
    116.         Array.Copy(hash0, data, hash0.Length);
    117.  
    118.         float t = Time.realtimeSinceStartup;
    119.  
    120.         float nextFrame = t + 0.01f;
    121.  
    122.         const int innnerloop = 10000;              
    123.  
    124.         if (hashCounter < uint.MaxValue)      
    125.         {          
    126.             for (int i = 0; i < jobHandles.Length; i++)
    127.             {
    128.                 if (jobHandles[i].IsCompleted) // check for completion before relaunching
    129.                 {
    130.                     hashCounter += innnerloop;
    131.  
    132.                     jobHandles[i].Complete();
    133.  
    134.                     if (HashLessThan(minhashes[i].ToArray(), minHash))
    135.                     {
    136.                         minHash = minhashes[i].ToArray();
    137.                     }
    138.  
    139.                     hash0s[i].Dispose();
    140.                     minhashes[i].Dispose();
    141.  
    142.                     hash0s[i] = new NativeArray<byte>(36, Allocator.TempJob);
    143.                     minhashes[i] = new NativeArray<byte>(32, Allocator.TempJob);
    144.                  
    145.                     minhashes[i].CopyFrom(minHash);
    146.                     hash0s[i].CopyFrom(data);
    147.  
    148.                     var job = new BitcoinTargetJob()
    149.                     {                      
    150.                         nonceMin = n,
    151.                         nonceMax = n + innnerloop,
    152.                         hash = hash0s[i],
    153.                         minhash = minhashes[i]
    154.                     };
    155.  
    156.                     jobs[i] = job;
    157.  
    158.                     jobHandles[i] = job.Schedule();
    159.  
    160.                     n += innnerloop;
    161.                 }
    162.             }
    163.  
    164.             int sleepTimeMs = (int)((nextFrame - Time.realtimeSinceStartup) * 1000f);
    165.  
    166.             if (sleepTimeMs > 0)
    167.                 Thread.Sleep(sleepTimeMs);
    168.         }
    169.  
    170.         if (n == uint.MaxValue)
    171.         {
    172.             if (endTime > 0f)
    173.             {
    174.                 for (int i = 0; i < threads; i++)
    175.                 {
    176.                     hash0s[i].Dispose();
    177.                     minhashes[i].Dispose();
    178.                 }
    179.          
    180.                 endTime = Time.realtimeSinceStartup;
    181.  
    182.                 Debug.Log("Completed total time = "+(endTime - startTime));
    183.             }
    184.         }
    185.  
    186.         float et = Time.realtimeSinceStartup - startTime;
    187.         if (endTime > 0f) et = endTime - startTime;
    188.         float percentage = (float)hashCounter / (float)uint.MaxValue;
    189.         float eta = et * 1f / percentage;
    190.  
    191.         TimeSpan etTime = TimeSpan.FromSeconds(et);
    192.         TimeSpan etaTime = TimeSpan.FromSeconds(eta);
    193.  
    194.         text.text = n.ToString("N0") + "\n"
    195.             + percentage.ToString("00.00%")
    196.             + "\nET  " + etTime.ToString(@"hh\:mm\:ss\:ff")
    197.             + "\nETA " + etaTime.ToString(@"hh\:mm\:ss\:ff")
    198.             + "\nhs  " + ((float)n / et).ToString("N0")
    199.             + "\nMin [" + (System.BitConverter.ToString(minHash).Replace("-", "")) + "]"
    200.         ;
    201.     }
    202.  
    203.     [MethodImpl(MethodImplOptions.AggressiveInlining)]
    204.     public static bool HashLessThan(byte[] a, byte[] b)
    205.     {
    206.         int i = 0;
    207.  
    208.         if (a.Length == b.Length)
    209.         {
    210.             while (i < a.Length)
    211.             {
    212.                 if (a[i] < b[i]) return true;
    213.                 else if (a[i] > b[i]) return false;
    214.                 i++;
    215.             }
    216.         }
    217.  
    218.         // Equal
    219.         return false;
    220.     }
    221. }
    222.  
     
  47. DragonCoder

    DragonCoder

    Joined:
    Jul 3, 2015
    Posts:
    1,493
    The constant re-creation of those managedSHA256 instances could be the culprit. And you are still allocating and disposing memmory in the tight loop (even if TempJob is faster).
    Have you tried profiling?

    Know what? I'll take this as a challenge and will experiment a bit later today. Am curious myself what's possible xP
     
    Last edited: May 29, 2022
    Arowx and JoNax97 like this.
  48. DragonCoder

    DragonCoder

    Joined:
    Jul 3, 2015
    Posts:
    1,493
    Well well!
    This certainly was a fun challenge and boy was it successful:

    upload_2022-5-30_1-21-41.png

    One minute and a half now on my 12 Core 3900X which required ~4 hours on @Arowx 's original code.


    Main lesson:
    - Memory allocation is the culprit!
    Your inner loop (with 10000 iterations) was allocating 64 MB total every time!

    First of all, avoid ToArray(), that makes a whole copy of the data.
    Solely those two tiny changes:
    upload_2022-5-30_1-44-41.png
    Reduce the time in your code from 4 hours to 2 and 40 minutes!

    The sha256managed (as well as its "non managed" sha256CNG which you can by the way use if you switch the .net environment in the project preferences) also does memory allocations because it returns the hash as a new variable.

    So I had no choice but to take the code from here: https://www.programmingalgorithms.com/algorithm/sha256/
    Using it directly like that (with some new uint[] in the inner loop) reduced computation time to about 55 minutes.
    Adapting it to use NativeArray only and getting rid of all remaining allocations eventually reduced it to 21 minutes.

    Turning on the Burst compiler finally results in one and a half minutes (without any attempts at manual SIMD support).

    Also no really big shenanigans with the jobs needed by the way and neither any thread.sleep. Just start in one frame and complete in the next. However best results were achieved with an inner loop of 100 000 which probably give the jobs more time to ramp up (10 000 resulted in ~2 minutes).
    Calling Complete() right after Schedule has the expected effect of dropping performance to "use-job-threads"==disabled mode.

    Now to actually return to the topic of this thread:
    More or less surprisingly, even extra threads with load do not have a huge influence.
    I see almost no difference between 24 parallel jobs (that's thebmax my 12core +Hyperthreading could manage) vs 48 parallel jobs.

    Note you should install the full jobs package with com.unity.jobs so you can comfortably disable the safety meassures of Burst and the job system as those cost 20-30% performance (but would be disabled in a build).

    Here the code (you might want to put the Sha256 stuff in a separate file):

    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using System.Security.Cryptography;
    4. using System;
    5. using UnityEngine;
    6. using TMPro;
    7. using Unity.Jobs;
    8. using Unity.Collections;
    9. using System.Threading;
    10. using System.Runtime.CompilerServices;
    11. using Unity.Burst;
    12. using Unity.Mathematics;
    13.  
    14. public class BitcoinTargetParallel: MonoBehaviour
    15. {
    16.     // Cycle throught bitcoin Nonces trying to hit under a target value
    17.  
    18.     [SerializeField] int threads = 36;
    19.     [SerializeField] ulong innnerloop = 10000;
    20.  
    21.     [Header("Info")]
    22.  
    23.     public byte[] hash0;
    24.     public byte[] hash_out;
    25.  
    26.     public ulong n = 0;
    27.     public ulong hashCounter = 0;
    28.  
    29.     public float startTime;
    30.     public float endTime = 0f;
    31.  
    32.     public NativeArray<byte> minHash;
    33.  
    34.     public TextMeshProUGUI text;
    35.  
    36.     JobHandle[] jobHandles;
    37.     BitcoinTargetJob[] jobs;
    38.  
    39.  
    40.     [BurstCompile]
    41.     struct BitcoinTargetJob : IJob
    42.     {
    43.         [ReadOnly]
    44.         public NativeArray<byte> given_hash;
    45.         public NativeArray<byte> minhash;
    46.         public NativeArray<byte> data_to_hash;
    47.         public NativeArray<byte> hash_out;
    48.  
    49.         NativeArray<uint> nonce_converter;
    50.  
    51.         CustomSHA256 sha256;
    52.  
    53.         [ReadOnly]
    54.         public ulong nonceMin;
    55.         [ReadOnly]
    56.         public ulong nonceMax;
    57.  
    58.         static public BitcoinTargetJob Create() // using this pattern because structs may not have parameetrless constructors
    59.         {
    60.             return new BitcoinTargetJob()
    61.             {
    62.                 given_hash = new NativeArray<byte>(36, Allocator.Persistent),
    63.                 minhash = new NativeArray<byte>(32, Allocator.Persistent),
    64.                 data_to_hash = new NativeArray<byte>(36, Allocator.Persistent),
    65.                 hash_out = new NativeArray<byte>(32, Allocator.Persistent),
    66.  
    67.                 nonce_converter = new NativeArray<uint>(1, Allocator.Persistent),
    68.  
    69.                 sha256 = CustomSHA256.Create(),
    70.  
    71.                 // init values only
    72.                 nonceMin = 0,
    73.                 nonceMax = 0
    74.             };
    75.         }
    76.  
    77.         public void Dispose()
    78.         {
    79.             if (!given_hash.IsCreated)
    80.                 return;
    81.             given_hash.Dispose();
    82.             minhash.Dispose();
    83.             data_to_hash.Dispose();
    84.             hash_out.Dispose();
    85.  
    86.             nonce_converter.Dispose();
    87.  
    88.             sha256.Dispose();
    89.         }
    90.  
    91.         public void Execute()
    92.         {
    93.             NativeArray<byte>.Copy(given_hash, data_to_hash);
    94.  
    95.             for (ulong n = nonceMin; n < nonceMax; n++)
    96.             {
    97.                 nonce_converter[0] = (uint)n;
    98.  
    99.                 // convert the uint to a bytes array
    100.                 NativeArray<byte> nbytes = nonce_converter.Reinterpret<byte>(4);
    101.  
    102.                 // copy it to the end of data_to_hash
    103.                 NativeArray<byte>.Copy(nbytes, 0, data_to_hash, data_to_hash.Length - nbytes.Length, nbytes.Length);
    104.  
    105.                 sha256.ComputeHash(data_to_hash, hash_out);
    106.  
    107.                 if (HashLessThan(hash_out, minhash))
    108.                 {
    109.                     minhash.CopyFrom(hash_out);
    110.                 }
    111.             }
    112.  
    113.         }
    114.  
    115.  
    116.         [BurstCompile]
    117.         struct CustomSHA256
    118.         {
    119.             [ReadOnly]
    120.             private NativeArray<uint> k;
    121.  
    122.             private NativeArray<byte> data;
    123.             private NativeArray<uint> bitlen;
    124.             private NativeArray<uint> state;
    125.             private NativeArray<uint> local_m;
    126.             private int datalen;
    127.  
    128.             public static CustomSHA256 Create()
    129.             {
    130.                 return new CustomSHA256()
    131.                 {
    132.                     data = new NativeArray<byte>(64, Allocator.Persistent),
    133.                     bitlen = new NativeArray<uint>(2, Allocator.Persistent),
    134.                     state = new NativeArray<uint>(8, Allocator.Persistent),
    135.                     local_m = new NativeArray<uint>(64, Allocator.Persistent),
    136.                     datalen = 0,
    137.  
    138.                     k = new NativeArray<uint>(new uint[] {
    139.                     0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5,0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5,
    140.                     0xd807aa98,0x12835b01,0x243185be,0x550c7dc3,0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174,
    141.                     0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc,0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da,
    142.                     0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7,0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967,
    143.                     0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13,0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85,
    144.                     0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3,0xd192e819,0xd6990624,0xf40e3585,0x106aa070,
    145.                     0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5,0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3,
    146.                     0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208,0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2
    147.                     }, Allocator.Persistent)
    148.                 };
    149.             }
    150.  
    151.             public void Dispose()
    152.             {
    153.                 data.Dispose();
    154.                 bitlen.Dispose();
    155.                 state.Dispose();
    156.                 local_m.Dispose();
    157.                 k.Dispose();
    158.             }
    159.  
    160.             public void ComputeHash(NativeArray<byte> input_data, NativeArray<byte> hash_out)
    161.             {
    162.                 // reset
    163.                 datalen = 0;
    164.                 bitlen[0] = 0;
    165.                 bitlen[1] = 0;
    166.                 state[0] = 0x6a09e667;
    167.                 state[1] = 0xbb67ae85;
    168.                 state[2] = 0x3c6ef372;
    169.                 state[3] = 0xa54ff53a;
    170.                 state[4] = 0x510e527f;
    171.                 state[5] = 0x9b05688c;
    172.                 state[6] = 0x1f83d9ab;
    173.                 state[7] = 0x5be0cd19;
    174.  
    175.                 // update
    176.                 int len = input_data.Length;
    177.                 for (int i = 0; i < len; ++i)
    178.                 {
    179.                     data[datalen] = input_data[i];
    180.                     datalen++;
    181.  
    182.                     if (datalen == 64)
    183.                     {
    184.                         SHA256Transform();
    185.  
    186.                         uint a = bitlen[0];
    187.                         uint b = bitlen[1];
    188.                         DBL_INT_ADD(ref a, ref b, 512);
    189.                         bitlen[0] = a;
    190.                         bitlen[1] = b;
    191.                         // Code above replaces this:
    192.                         // DBL_INT_ADD(ref bitlen[0], ref bitlen[1], 512);
    193.  
    194.                         datalen = 0;
    195.                     }
    196.                 }
    197.  
    198.                 Finalize(hash_out);
    199.             }
    200.  
    201.             void Finalize(NativeArray<byte> hash_out)
    202.             {
    203.                 int i = datalen;
    204.  
    205.                 if (datalen < 56)
    206.                 {
    207.                     data[i++] = 0x80;
    208.  
    209.                     while (i < 56)
    210.                         data[i++] = 0x00;
    211.                 }
    212.                 else
    213.                 {
    214.                     data[i++] = 0x80;
    215.  
    216.                     while (i < 64)
    217.                         data[i++] = 0x00;
    218.  
    219.                     SHA256Transform();
    220.                 }
    221.  
    222.  
    223.                 uint a = bitlen[0];
    224.                 uint b = bitlen[1];
    225.                 DBL_INT_ADD(ref a, ref b, (uint)datalen * 8);
    226.                 bitlen[0] = a;
    227.                 bitlen[1] = b;
    228.                 // Code above repalces this:
    229.                 // DBL_INT_ADD(ref bitlen[0], ref bitlen[1], datalen * 8);
    230.  
    231.  
    232.                 data[63] = (byte)(bitlen[0]);
    233.                 data[62] = (byte)(bitlen[0] >> 8);
    234.                 data[61] = (byte)(bitlen[0] >> 16);
    235.                 data[60] = (byte)(bitlen[0] >> 24);
    236.                 data[59] = (byte)(bitlen[1]);
    237.                 data[58] = (byte)(bitlen[1] >> 8);
    238.                 data[57] = (byte)(bitlen[1] >> 16);
    239.                 data[56] = (byte)(bitlen[1] >> 24);
    240.                 SHA256Transform();
    241.  
    242.                 for (i = 0; i < 4; ++i)
    243.                 {
    244.                     hash_out[i] = (byte)(((state[0]) >> (int)(24 - i * 8)) & 0x000000ff);
    245.                     hash_out[i + 4] = (byte)(((state[1]) >> (int)(24 - i * 8)) & 0x000000ff);
    246.                     hash_out[i + 8] = (byte)(((state[2]) >> (int)(24 - i * 8)) & 0x000000ff);
    247.                     hash_out[i + 12] = (byte)((state[3] >> (int)(24 - i * 8)) & 0x000000ff);
    248.                     hash_out[i + 16] = (byte)((state[4] >> (int)(24 - i * 8)) & 0x000000ff);
    249.                     hash_out[i + 20] = (byte)((state[5] >> (int)(24 - i * 8)) & 0x000000ff);
    250.                     hash_out[i + 24] = (byte)((state[6] >> (int)(24 - i * 8)) & 0x000000ff);
    251.                     hash_out[i + 28] = (byte)((state[7] >> (int)(24 - i * 8)) & 0x000000ff);
    252.                 }
    253.             }
    254.  
    255.             void SHA256Transform()
    256.             {
    257.                 uint a, b, c, d, e, f, g, h, t1, t2;
    258.                 int i, j;
    259.  
    260.                 for (i = 0, j = 0; i < 16; ++i, j += 4)
    261.                     local_m[i] = (uint)((data[j] << 24) | (data[j + 1] << 16) | (data[j + 2] << 8) | (data[j + 3]));
    262.  
    263.                 for (; i < 64; ++i)
    264.                     local_m[i] = SIG1(local_m[i - 2]) + local_m[i - 7] + SIG0(local_m[i - 15]) + local_m[i - 16];
    265.  
    266.                 a = state[0];
    267.                 b = state[1];
    268.                 c = state[2];
    269.                 d = state[3];
    270.                 e = state[4];
    271.                 f = state[5];
    272.                 g = state[6];
    273.                 h = state[7];
    274.  
    275.                 for (i = 0; i < 64; ++i)
    276.                 {
    277.                     t1 = h + EP1(e) + CH(e, f, g) + k[i] + local_m[i];
    278.                     t2 = EP0(a) + MAJ(a, b, c);
    279.                     h = g;
    280.                     g = f;
    281.                     f = e;
    282.                     e = d + t1;
    283.                     d = c;
    284.                     c = b;
    285.                     b = a;
    286.                     a = t1 + t2;
    287.                 }
    288.  
    289.                 state[0] += a;
    290.                 state[1] += b;
    291.                 state[2] += c;
    292.                 state[3] += d;
    293.                 state[4] += e;
    294.                 state[5] += f;
    295.                 state[6] += g;
    296.                 state[7] += h;
    297.             }
    298.  
    299.             static void DBL_INT_ADD(ref uint a, ref uint b, uint c)
    300.             {
    301.                 if (a > 0xffffffff - c) ++b; a += c;
    302.             }
    303.  
    304.             static uint ROTLEFT(uint a, byte b)
    305.             {
    306.                 return ((a << b) | (a >> (32 - b)));
    307.             }
    308.  
    309.             static uint ROTRIGHT(uint a, byte b)
    310.             {
    311.                 return (((a) >> (b)) | ((a) << (32 - (b))));
    312.             }
    313.  
    314.             static uint CH(uint x, uint y, uint z)
    315.             {
    316.                 return (((x) & (y)) ^ (~(x) & (z)));
    317.             }
    318.  
    319.             static uint MAJ(uint x, uint y, uint z)
    320.             {
    321.                 return (((x) & (y)) ^ ((x) & (z)) ^ ((y) & (z)));
    322.             }
    323.  
    324.             static uint EP0(uint x)
    325.             {
    326.                 return (ROTRIGHT(x, 2) ^ ROTRIGHT(x, 13) ^ ROTRIGHT(x, 22));
    327.             }
    328.  
    329.             static uint EP1(uint x)
    330.             {
    331.                 return (ROTRIGHT(x, 6) ^ ROTRIGHT(x, 11) ^ ROTRIGHT(x, 25));
    332.             }
    333.  
    334.             static uint SIG0(uint x)
    335.             {
    336.                 return (ROTRIGHT(x, 7) ^ ROTRIGHT(x, 18) ^ ((x) >> 3));
    337.             }
    338.  
    339.             static uint SIG1(uint x)
    340.             {
    341.                 return (ROTRIGHT(x, 17) ^ ROTRIGHT(x, 19) ^ ((x) >> 10));
    342.             }
    343.         }
    344.     }
    345.  
    346.     private void OnApplicationQuit()
    347.     {
    348.         foreach (var job_handle in jobHandles)
    349.             job_handle.Complete();
    350.  
    351.         foreach (var job in jobs)
    352.             job.Dispose();
    353.  
    354.         minHash.Dispose();
    355.     }
    356.  
    357.     void Start()
    358.     {
    359.         byte[] rbytes = new byte[32];
    360.  
    361.         for (int i = 0; i < rbytes.Length; i++)
    362.         {
    363.             rbytes[i] = (byte)(UnityEngine.Random.Range(0, 256));
    364.         }
    365.  
    366.         minHash = new NativeArray<byte>(new byte[]
    367.         { //0x00, 0x00, 0x00, 0x00,  0x00, 0x00, 0x00, 0x00,  0x00, 0x00, 0x00, 0x00,  0x00, 0x00, 0x00, 0x00,
    368.             0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,
    369.             0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF
    370.         }, Allocator.Persistent);
    371.  
    372.         SHA256 sha = new SHA256Managed();
    373.         hash0 = sha.ComputeHash(rbytes);
    374.  
    375.         startTime = Time.realtimeSinceStartup;
    376.  
    377.         jobHandles = new JobHandle[threads];
    378.         jobs = new BitcoinTargetJob[threads];
    379.         for (int job_id = 0; job_id < threads; job_id++)
    380.             jobs[job_id] = BitcoinTargetJob.Create();
    381.  
    382.         InvokeRepeating("UpdateText", 0.25f, 0.25f);
    383.     }
    384.  
    385.  
    386.     private bool first_jobs_started = false;
    387.  
    388.     void Update()
    389.     {
    390.         byte[] data = new byte[36];
    391.         Array.Copy(hash0, data, hash0.Length); // To check: Is it intended that hash0 has only 32 byte and data 36?
    392.  
    393.         if (hashCounter < uint.MaxValue)
    394.         {
    395.             for (int job_id = 0; job_id < jobs.Length; job_id++)
    396.             {
    397.                 hashCounter += innnerloop;
    398.  
    399.                 jobHandles[job_id].Complete();
    400.  
    401.                 BitcoinTargetJob job = jobs[job_id];
    402.  
    403.                 if (first_jobs_started)
    404.                 {
    405.                     if (HashLessThan(job.minhash, minHash))
    406.                     {
    407.                         Debug.Log("New min hash!");
    408.                         minHash.CopyFrom(job.minhash);
    409.                     }
    410.                 }
    411.  
    412.                 job.minhash.CopyFrom(minHash);
    413.                 job.given_hash.CopyFrom(data);
    414.  
    415.                 job.nonceMin = n;
    416.                 job.nonceMax = n + innnerloop;
    417.  
    418.                 jobs[job_id] = job;
    419.  
    420.                 jobHandles[job_id] = job.Schedule();
    421.  
    422.                 n += innnerloop;
    423.             }
    424.  
    425.             first_jobs_started = true;
    426.         }
    427.  
    428.         JobHandle.ScheduleBatchedJobs();
    429.  
    430.  
    431.         if (n >= uint.MaxValue)
    432.         {
    433.             if (endTime <= 0f)
    434.             {
    435.                 endTime = Time.realtimeSinceStartup;
    436.  
    437.                 Debug.Log("Completed total time = " + (endTime - startTime));
    438.             }
    439.         }
    440.     }
    441.  
    442.  
    443.     void UpdateText()
    444.     {
    445.         float et = Time.realtimeSinceStartup - startTime;
    446.         if (endTime > 0f) et = endTime - startTime;
    447.         float percentage = (float)hashCounter / (float)uint.MaxValue;
    448.         float eta = et * 1f / percentage;
    449.  
    450.         TimeSpan etTime = TimeSpan.FromSeconds(et);
    451.         TimeSpan etaTime = TimeSpan.FromSeconds(eta);
    452.  
    453.         text.text = n.ToString("N0") + "\n"
    454.             + percentage.ToString("00.00%")
    455.             + "\nET  " + etTime.ToString(@"hh\:mm\:ss\:ff")
    456.             + "\nETA " + etaTime.ToString(@"hh\:mm\:ss\:ff")
    457.             + "\nhs  " + ((float)n / et).ToString("N0")
    458.             + "\nMin [" + (System.BitConverter.ToString(minHash.ToArray()).Replace("-", "")) + "]"
    459.         ;
    460.     }
    461.  
    462.     [MethodImpl(MethodImplOptions.AggressiveInlining)]
    463.     public static bool HashLessThan(NativeArray<byte> a, NativeArray<byte> b)
    464.     {
    465.         int i = 0;
    466.  
    467.         if (a.Length == b.Length)
    468.         {
    469.             while (i < a.Length)
    470.             {
    471.                 if (a[i] < b[i]) return true;
    472.                 else if (a[i] > b[i]) return false;
    473.                 i++;
    474.             }
    475.         }
    476.  
    477.         // Equal
    478.         return false;
    479.     }
    480.  
    481. }
     

    Attached Files:

    Last edited: May 30, 2022
    Arowx and Ryiah like this.
  49. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Nice work...
    IL2CPP Build 3-4 minutes.
    Mono Build 4-5 minutes.​

    But no need to write your own SHA256 when you take a Task based approach on .Net
    .Net Task Build 2-3 minutes.​

    Test it yourself built in VS2020 as a .Net Console App

    Code (CSharp):
    1. // See https://aka.ms/new-console-template for more information
    2. using System.Security.Cryptography;
    3. using System.Runtime.CompilerServices;
    4. using System;
    5.  
    6. Console.WriteLine("BitHasher starting...");
    7.  
    8. BitcoinTargetThreaded1 bitHasher = new BitcoinTargetThreaded1();
    9.  
    10. while(bitHasher.isRunning)
    11. {
    12.     bitHasher.Run();
    13.     if (bitHasher.isRunning) Thread.Sleep(500);
    14. }
    15.  
    16. Console.WriteLine("...ended");
    17.  
    18. /*
    19. using System.Collections;
    20. using System.Collections.Generic;
    21. using System.Threading;
    22. using System.Threading.Tasks;
    23. using UnityEngine;
    24. using TMPro;
    25. */
    26.  
    27. public class BitcoinTargetThreaded1
    28. {
    29.     // Cycle throught bitcoin Nonces trying to hit under a target value
    30.  
    31.     public byte[] hash0;
    32.     public byte[] hash1;
    33.  
    34.     public HashItData[] hashItDatas;
    35.  
    36.     public static byte[] minHash = { //0x00, 0x00, 0x00, 0x00,  0x00, 0x00, 0x00, 0x00,  0x00, 0x00, 0x00, 0x00,  0x00, 0x00, 0x00, 0x00,
    37.                            0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,
    38.                            0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF,  0xFF, 0xFF, 0xFF, 0xFF
    39.                          };
    40.  
    41.  
    42.     public static long n = 0;
    43.     public long hashCounter = 0;
    44.  
    45.     public long startTime;
    46.     public long endTime = 0;
    47.  
    48.     //public TextMeshProUGUI text;
    49.  
    50.     public int step = 10000000; //1024 * 16; //4096 * 2;
    51.     public int threads = 16;
    52.  
    53.     public bool isRunning = true;
    54.  
    55.     public class HashItData
    56.     {
    57.         public byte index;
    58.         public byte[] data;
    59.         public SHA256 sha256;
    60.         public long hn;
    61.         public byte[] hash;
    62.     }
    63.  
    64.     public byte[] HashIt(HashItData hashItdata)
    65.     {
    66.         HashItData hData = hashItdata;
    67.  
    68.         long n0 = hData.hn;
    69.         long n1 = n0 + (long)step;
    70.  
    71.         if (n1 > uint.MaxValue) n1 = uint.MaxValue;
    72.  
    73.         //Debug.Log("HashIt range " + n0 + " to " + n1 + " on thread " + Thread.CurrentThread.ManagedThreadId);
    74.  
    75.         byte[] minHash = hData.hash;
    76.         byte[] hash1;
    77.         byte[] nbytes;
    78.         byte[] data = new byte[36];
    79.  
    80.         Array.Copy(hData.data, data, hData.data.Length);
    81.  
    82.         SHA256 sha = hData.sha256;
    83.  
    84.         for (long nn = n0; nn < n1; nn++)
    85.         {
    86.             //nbytes = BitConverter.GetBytes((uint)nn);
    87.  
    88.             //Array.Copy(nbytes, 0, data, 32, 4);
    89.  
    90.             //Debug.Log("NBytes " + nbytes.Length + " Data " + data.Length + " D-N " + (data.Length - nbytes.Length));
    91.             //Array.Copy(nbytes, 0, data, data.Length - nbytes.Length, nbytes.Length);
    92.  
    93.             data[32] = (byte)(nn | 255);
    94.             data[33] = (byte)(nn >> 8 | 255);
    95.             data[34] = (byte)(nn >> 16 | 255);
    96.             data[35] = (byte)(nn >> 24 | 255);
    97.  
    98.             //sha.Initialize();
    99.  
    100.             hash1 = sha.ComputeHash(data);
    101.  
    102.             // check is hash less than min
    103.             if (HashLessThan(hash1, minHash))
    104.             {
    105.                 //Debug.Log("min thread "+hData.index+" [" + System.BitConverter.ToString(hash1).Replace("-", "") + "]");
    106.                 minHash = hash1;
    107.             }
    108.         }
    109.  
    110.         //sha.Dispose();
    111.  
    112.         return minHash;
    113.     }
    114.  
    115.     [MethodImpl(MethodImplOptions.AggressiveInlining)]
    116.     public bool HashLessThan(byte[] a, byte[] b)
    117.     {
    118.         int i = 0;
    119.  
    120.         if (a.Length == b.Length)
    121.         {
    122.             while (i < a.Length)
    123.             {
    124.                 if (a[i] < b[i]) return true;
    125.                 else if (a[i] > b[i]) return false;
    126.                 i++;
    127.             }
    128.         }
    129.  
    130.         return false;
    131.     }
    132.  
    133.     public Task<byte[]>[] tasks;
    134.  
    135.     public BitcoinTargetThreaded1()
    136.     {
    137.  
    138.         Console.WriteLine("Min Hash [" + System.BitConverter.ToString(minHash).Replace("-", "") + "]");
    139.  
    140.         byte[] rbytes = //new byte[32];
    141.             {
    142.                 0xE9,0x84,0x9C,0xDF, 0x53,0x13,0x45,0x2F,
    143.                 0x78,0x7D,0xCF,0x31, 0x62,0xD3,0x36,0xF3,
    144.                 0xC1,0x8E,0xAB,0x79, 0x16,0x5F,0xBB,0x65,
    145.                 0x78,0x26,0x7B,0x49, 0x67,0x4B,0xA9,0xF3
    146.             };
    147.  
    148.         /** for (int i = 0; i < rbytes.Length; i++)
    149.         {
    150.             rbytes[i] = (byte)(UnityEngine.Random.Range(0, 256));
    151.         }
    152.         */
    153.  
    154.         SHA256 sha = SHA256.Create();
    155.         hash0 = sha.ComputeHash(rbytes);
    156.         sha.Clear();
    157.  
    158.         Console.WriteLine("Start Hash [" + System.BitConverter.ToString(hash0).Replace("-", "") + "]");
    159.  
    160.         startTime = DateTime.Now.Ticks;
    161.  
    162.         hashItDatas = new HashItData[threads];
    163.  
    164.         for (int i = 0; i < hashItDatas.Length; i++)
    165.         {
    166.             hashItDatas[i] = new HashItData();
    167.             hashItDatas[i].sha256 = new SHA256Managed();
    168.             //Console.WriteLine("SHA256 [" + i + "]  can transform multiple blocks " + (hashItDatas[i].sha256.CanTransformMultipleBlocks ? "true" : "false"));
    169.             hashItDatas[i].hash = minHash;
    170.             hashItDatas[i].index = (byte)i;
    171.         }
    172.  
    173.         // Start tasks
    174.         tasks = new Task<byte[]>[hashItDatas.Length];
    175.  
    176.         byte[] data = new byte[36];
    177.  
    178.         Array.Copy(hash0, data, hash0.Length);
    179.  
    180.         for (int i = 0; i < hashItDatas.Length; i++)
    181.         {
    182.             if (n <= uint.MaxValue)
    183.             {
    184.                 hashItDatas[i].hn = n;
    185.                 hashItDatas[i].data = data;
    186.  
    187.                 HashItData hData = hashItDatas[i];
    188.  
    189.                 tasks[i] = Task<byte[]>.Factory.StartNew(() => HashIt(hData), TaskCreationOptions.LongRunning);
    190.  
    191.                 n += (long)step;
    192.             }
    193.         }
    194.     }
    195.  
    196.     // Update is called once per frame
    197.     public void Run()
    198.     {
    199.         bool newUpdate = false;
    200.  
    201.         if (isRunning)
    202.         {
    203.             byte[] data = new byte[36];
    204.  
    205.             Array.Copy(hash0, data, hash0.Length);
    206.  
    207.             //long t = DateTime.Now.Ticks;
    208.  
    209.             //long nextFrame = t + TimeSpan.TicksPerSecond;
    210.  
    211.             int i = 0;      
    212.  
    213.             //while (n < uint.MaxValue && DateTime.Now.Millisecond < nextFrame)
    214.             //{
    215.             if (hashCounter < uint.MaxValue)
    216.             {
    217.                 for (i = 0; i < hashItDatas.Length; i++)
    218.                 {
    219.                     if (tasks[i].IsCompletedSuccessfully)
    220.                     {
    221.                         newUpdate = true;
    222.                         hashCounter += step;
    223.  
    224.                         byte[] hashData = tasks[i].Result;
    225.  
    226.                         if (hashData != null)
    227.                         {
    228.                             hashItDatas[i].hash = hashData;
    229.  
    230.                             if (HashLessThan(hashData, minHash))
    231.                             {                          
    232.                                 minHash = hashData;
    233.                                 //Console.WriteLine("Min Hash " + System.BitConverter.ToString(minHash).Replace("-", ""));
    234.                             }
    235.                         }
    236.  
    237.                         if (n <= uint.MaxValue)
    238.                         {
    239.                             hashItDatas[i].hn = n;
    240.                             hashItDatas[i].data = data;
    241.  
    242.                             HashItData hData = hashItDatas[i];
    243.  
    244.                             tasks[i] = Task<byte[]>.Factory.StartNew(() => HashIt(hData));//, TaskCreationOptions.LongRunning);                                    
    245.                         }
    246.  
    247.                         n += (long)step;
    248.                     }
    249.                 }          
    250.             }
    251.  
    252.             if (hashCounter >= uint.MaxValue)
    253.             {
    254.                 hashCounter = uint.MaxValue;
    255.  
    256.                 if (endTime == 0f)
    257.                 {
    258.                     isRunning = false;
    259.                     endTime = DateTime.Now.Ticks;
    260.  
    261.                     Console.WriteLine("Completed total time = " + ((float)(endTime - startTime)/TimeSpan.TicksPerSecond));
    262.                     Console.WriteLine("Min Hash [" + System.BitConverter.ToString(minHash).Replace("-", "") + "]");
    263.                 }
    264.             }
    265.  
    266.             if (newUpdate)
    267.             {
    268.                 long et = DateTime.Now.Ticks - startTime;
    269.                 if (endTime > 0f) et = endTime - startTime;
    270.                 float percentage = 0f;
    271.                 string etaStr = "--:--:--:--";
    272.                 string etStr = etaStr;
    273.                 string hsStr = "-,---,---";
    274.  
    275.                 if (hashCounter > 0)
    276.                 {
    277.                     percentage = (float)hashCounter / (float)uint.MaxValue;
    278.                     long eta = (long)(et * 1f / percentage);
    279.  
    280.                     TimeSpan etTime = TimeSpan.FromTicks(et);
    281.                     TimeSpan etaTime = TimeSpan.FromTicks(eta);
    282.                     etStr = etTime.ToString(@"hh\:mm\:ss\:ff");
    283.                     etaStr = etaTime.ToString(@"hh\:mm\:ss\:ff");
    284.                     hsStr = ((float)hashCounter / ((float)et / (float)TimeSpan.TicksPerSecond)).ToString("N0");
    285.                 }
    286.  
    287.                 string text = hashCounter.ToString("N0") + "\n"
    288.                     + percentage.ToString("00.00%")
    289.                     + "\nET  " + etStr
    290.                     + "\nETA " + etaStr
    291.                     + "\nhs  " + hsStr
    292.                     + "\nMin [" + (System.BitConverter.ToString(minHash).Replace("-", "")) + "]"
    293.                 ;            
    294.  
    295.                 Console.Clear();            
    296.                 Console.WriteLine(text);
    297.              
    298.             }
    299.         }
    300.     }
    301. }
    302.  
    So the question is why are Tasks/Threads so slow in Unity compared to .Net?
     
    Last edited: May 30, 2022
  50. Tautvydas-Zilys

    Tautvydas-Zilys

    Unity Technologies

    Joined:
    Jul 25, 2013
    Posts:
    10,518
    I suspect it's the hashing that's slower, not threads.