Search Unity

Can this algorithm be done with the Jobs System?

Discussion in 'Entity Component System' started by konstantin_lozev, Apr 26, 2019.

  1. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    I am prototyping a game in Unity. It involves building a randomly generated grid. The first phase is the most resource-consuming and involves creating an initial "tree" grid of nodes.
    I build the initial tree by "spawning" nodes (I am not really spawning GameObjects, but only generating random positions and comparing to existing positions positions, I use "spawning" below just to be clearer and more graphical). I spawn the nodes randomly on a unit sphere from their parent and discarding them if they are too close to any other node, generating another random on unit sphere position, gradually relaxing the distance threshold.
    As the spawning happens, I put the nodes in a flat array to operate on them later.
    It is important that each new spawn has to check its distance to all existing spawns, including the spawns that have just occurred at the previous iteration.
    To explain better, If I have one parent that has to spawn 2 children, when the first child spawns, it checks its distance only with regard to the parent, but when the second child spawns, it checks its distance to both the parent and the first child.
    However, it is not important whether the first child is spawned before the second one, as long as they are checked against each other by the one that is spawning the later.
    I have read about the Unity's C# Job system and I have doubts whether the above algorithm could be implemented with the IJobParallelFor, because all parallel iterations (corresponding to each child that should be spawned) should be able to write and read the array of existing "accepted" spawn positions, while the other iterations are also doing so. As I wrote, for the proper operation of the algorithm, it does not matter in which order they write and read from the array of existing "accepted" spawn positions.
    It would be really great if I could parallelise that process, since the generation of the "tree" grid takes around 7-8 seconds for a grid of 1000 nodes and more importantly freezes the game in the meantime.
    Is there a hope?
     
  2. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    I'm very surprised it takes 7-8 seconds to do this as all you need is a distance check between two Vector3's. Have you deep profiled this to see what is causing the problem?

    I suspect that your distance threshold is too large and the algorithm is spending lots of time trying to grow outward.

    Have you considered a different generation approach, e.g. you could randomly generate a point offset by a rotation along a vector from the original point, then limit future random rotations to be > 45* degrees away from any existing points therefore ensuring the minimum distance is maintained and the node tree will grow with minimal pruning needed.

    *The angle could be any number of degrees that at one unit matches the minimum distance threshold you have set.
     
    Last edited: Apr 26, 2019
  3. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    I am not sure whether I understood that, sorry. What do you mean by "rotation along a vector from the original point"? I cannot visualise that...
    I have been thinking that my approach is not a very smart one and many random on unit shpere generations go to waste.
    I actually managed to put the generation down from 15 sec to 7 sec simply by comparing thr sqrMagnitude instead of the distance.
    I also probably start with a too high threshold.
    But what about the job system? Would I be able to speed up things that way?
     
  4. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    You are generating a new random point in a sphere however worst case is the half that space would be too close to your last point(s) and as your point cloud grows even more of that space becomes blocked.

    If you moved that sphere away from your original point, so that the old point was outside of the sphere this could guarantee that early expansion would never be too close to the last point.

    Or if you use Random.OnUnitSphere, you are guaranteed the point is not within range of it's parent point, you can then randomize this value by scaling the vector a random amount e.g. 1.0 to 2.0

    All you are doing is a distance calculation between all the existing points and a new point, the hierarchy does not matter to the testing aspect of the program batching and jobifying this would massively speed things up, however I don't think it will solve what I think is an underlying problem with your algorithm.

    So please profile your code and/or share it with the community so we can help you to the best of our abilities.

    Idea: If you think of this as a plant growing then you could use the growth vectors like branches and generate child angle offsets e.g. take the parents offset vector or branch and use it to make the next child offset with a limited rotation and safe scale (< limit).
     
    Last edited: Apr 26, 2019
  5. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    Here is the zip with the Deep Profiler data https://drive.google.com/file/d/1ZAOcw8gzAve4BoDel8m1HVkXFD1ACbPx/view?usp=sharing
    Here is my code:
    Code (CSharp):
    1. while (foundNodePos == false)
    2.                 {
    3.                     foundNodePos = true;
    4.                     float minDist = minDistNode * scaleGrid - extraDecrease;
    5.                     for (int i = 0; i < allLvlArrayNum; i++)
    6.                     {
    7.                         //Discards the new node position if it is too close to other existing nodes. This might make it difficult to put into a job, since there is a dependency
    8.                         Vector3 newNodeToAllNodes = NodePositions[i] - newNodePos;
    9.                         if (Vector3.SqrMagnitude(newNodeToAllNodes) < minDist*minDist)
    10.                         {
    11.                             foundNodePos = false;
    12.                             newNodePos = NodePositions[parentNodeNum] + Random.onUnitSphere * lineLength * scaleGrid;
    13.                             //i = allLvlArrayNum;
    14.                             break;
    15.                         }
    16.                     }
    17.                     randomLoops++;
    18.                     if (randomLoops > maxRandomLoops)
    19.                     {
    20.                         extraDecrease += (extraDecreaseStep * scaleGrid);
    21.                         randomLoops = 0;
    22.                     }
    23.                     //Debug.Log ("foundNodePos="+foundNodePos);
    24.                 }
    And here are the 3 most resource intensive operations, from what I could gather:
    https://i.imgur.com/gnGY5la.jpg

    https://i.imgur.com/H3ujuAn.jpg

    https://i.imgur.com/UM9rywu.jpg

    I agree that most of the loops go to waste...
     
  6. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,445
    not jobs related, but some micro-optimizations,
    - can calculate this outside for-loop minDist*minDist, same for lineLength * scaleGrid
    - and looks like this can be outside while-loop minDistNode * scaleGrid
    - use floats or vector3 components directly (v.x = v.x*some, v.y = v.y*some, v.z = v.z*some, is faster than v = v*some)
    - test calculating SqrMagnitude manually from floats or vector components http://answers.unity.com/answers/384974/view.html
    - could move Vector3 newNodeToAllNodes; outside while loop, then just assign xyz values to it in loop

    in any case,
    could search for some algorithm that evenly places points on sphere surface,
    they work without having to loop all points and checking distances (which gets slower the more points you have)
     
  7. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    Thanks, I agree there was no need to recompute some of that in the for loop or the while loop. I tried with the changed text, but I am again at 7sec. It is the algorithm that discards too many random on unit spheres (6 million for 350 nodes).
    Arowx above suggested another approach by moving the center of the OnUnitSphere outside the parent node and away from existing nodes.
     
  8. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    4,271
    Trying to use IJobParallelFor is NOT the first step in optimizing your code. There's a bunch of other optimizations and restructurings that are going to get you to your goal faster. But first, let's take a step back and look at your real problem:

    The algorithm is freezing the game for 7 seconds.

    There's two ways to solve this:
    1) Make the algorithm waaaaaaay faster
    2) Make the algorithm not freeze the game while it is running

    I'm willing to walk you through a bunch of tips and tricks to get this thing under control, but I'll need you to answer a couple of questions just to make sure this is feasible within your game's constraints.

    A) Assuming that it is possible for this algorithm to run across multiple frames without freezing the game, does this algorithm need to run in a single frame?

    B) How long (milliseconds or frame count) can you afford this algorithm to run given the same assumption as A?

    C) Would you be willing to upload a more complete version of this algorithm which includes the outer loops and all variables this algorithm uses?
     
  9. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    No, I can show the player some tutorial tips in the meantime
    2-3 seconds ideally
    Here is a shorter version of the CreateLevel() function
    Code (CSharp):
    1. //parentNode is the GameObject of the parent that has to spawn the children
    2. //childrenNum is the child index that the parent has to spawn at this iteration of this function
    3. void CreateLevel(GameObject parentNode, int childrenNum)
    4.     {
    5.         //allLvlArrayNum is the int that describes the index in the GameObject[] AllLvlArray that when the grid is finished should contain all nodes;
    6.         //allLvlArrayLength is the int that describes the maximum number of nodes allowed in the grid;
    7.         if (allLvlArrayNum < allArrayLength)
    8.         {
    9.             //We get the index of the parent node in order to find its position in the array that holds the node positions
    10.             parentNodeNum = int.Parse(parentNode.name);
    11.             //We generate a random location around the parent node float lineLength = 0.5f;
    12.             newNodePos = NodePositions[parentNodeNum] + Random.onUnitSphere * lineLength;
    13.             //This is sort of a safety fuse system to enable the while loop to actually complete, that compares the current loop number with the int maxRandomLoops = 30; and if we reached maxRandomLoops, relaxes the threshold by float extraDecreaseStep = 0.1f;
    14.             randomLoops = 0;
    15.             //Everytime that the safety fuse system reaches the maxRandomLoops = 30 the extraDecrease is increased by float extraDecreaseStep = 0.1f;
    16.             extraDecrease = 0;
    17.             while (foundNodePos == false)
    18.             {
    19.                 //At each new while loop iteration we set the condition for exiting the while loop to true and if the potential position for the new child node does not pass the threshold (i.e. is too close to other existing nodes), we set the foundNodePos = false; thus triggering another iteration of the while loop
    20.                 foundNodePos = true;
    21.                 //the minimum distance starts probably too high at float minDistNode = 1f; which is double the radius of the Random.onUnitSphere (for reference float lineLength = 0.5f;) and every 30 loops gets decreased by float extraDecreaseStep = 0.1f; Maybe a potential optimisation could be to have faster extraDecreaseStep first and slower later
    22.                 float minDist = minDistNode - extraDecrease;
    23.                 //we calculate the square of minDist for faster comparison with the distance between the potential position of the child node and the positions of all existing nodes
    24.                 float minDistSq = minDist * minDist;
    25.                 //check the distance to all existing nodes allLvlArrayNum in a for loop.
    26.                 for (int i = 0; i < allLvlArrayNum; i++)
    27.                 {
    28.                     //Discards the new node position if it is too close to other existing nodes.
    29.                     Vector3 newNodeToAllNodes = NodePositions[i] - newNodePos;
    30.                     if (Vector3.SqrMagnitude(newNodeToAllNodes) < minDistSq)
    31.                     {
    32.                         //Discards the current potential position for the child node, preparing the next interation of the while loop
    33.                         foundNodePos = false;
    34.                         //Generates a new potential child node position for the next while loop iteration and breaks from the for loop
    35.                         newNodePos = NodePositions[parentNodeNum] + Random.onUnitSphere * lineLength;
    36.                         break;
    37.                     }
    38.                 }
    39.                 randomLoops++;
    40.                 //If no new position was found with 30 iterations, relax the threshold by float extraDecreaseStep = 0.1f;
    41.                 if (randomLoops > maxRandomLoops)
    42.                 {
    43.                     extraDecrease += (extraDecreaseStep);
    44.                     randomLoops = 0;
    45.                 }
    46.            }
    47.             //Instantiates and sets up (orients and scales) the new node,
    48.             AllLvlArray[allLvlArrayNum] = Instantiate(node, newNodePos, Quaternion.identity, grid);
    49.             AllLvlArray[allLvlArrayNum].name = allLvlArrayNum.ToString();
    50.             //Immediately fills in the new confirmed child node position into the array of node position so that the next child will check its potential position against that new position too. I think this might be problematic for a IJobParallelFor, since if all children's CreateLevel functions run in parallel, they would have to read and write to NodePositions[] simultaneously
    51.             NodePositions[allLvlArrayNum] = newNodePos;
    52.             //k=linksContainerArray[i,j] describes which adjacent node number k is at which link number j coming from the current node i
    53.             // the 0 link (j=0) of each node always refers to its parent node
    54.             linksContainerArray[allLvlArrayNum, 0] = parentNodeNum;
    55.             // The j = childrenNum + 1 refers to the current child that was just spawned
    56.             linksContainerArray[parentNodeNum, childrenNum + 1] = allLvlArrayNum;
    57.             //increments int allLvlArrayNum in order to prepare the next iteration of CreateLevel();
    58.             allLvlArrayNum++;
    59.             //Sets the bool foundNodePos in order to enable the while loop for the next iteration of CreateLevel();
    60.             foundNodePos = false;
    61.         }
    62.     }
    And here is the function that creates the successive tree levels (I put only 3 levels, but you get the idea):
    Code (CSharp):
    1.     void CreateGrid()
    2.     {
    3.         //Creates the central node, it cannot use the CreateLevel() function, since it has no parents
    4.         CreateCenterNode();
    5.         //I think this will be difficult to parallelise since we want in the creation of each level the new child nodes to consider also already created nodes from the same CreateLevel() function
    6.         //We don't use fully nested for loops since this way we are building the tree gradually level by level, which makes it more evenly spread
    7.         //Level 1 - Node # = linksPerNode
    8.         for (int i = 0; i < linksPerNode; i++)
    9.         {
    10.             //Builds the first level directly off the central node as a parent
    11.             CreateLevel(AllLvlArray[0], i);
    12.         }
    13.         //Level 2 - Node # = linksPerNode^2
    14.         if (branchingLevels >= 2 && allLvlArrayNum < allArrayLength)
    15.         {
    16.             for (int i = 0; i < linksPerNode; i++)
    17.             {
    18.                 for (int j = 0; j < linksPerNode; j++)
    19.                 {
    20.                     //Creates the second level by finding all children from level 1 and assigning them as a parent  for the creation of level 2
    21.                     CreateLevel(AllLvlArray[linksContainerArray[0, i + 1]], j);
    22.                 }
    23.             }
    24.         }
    25.         //Level 3 - Node # = linksPerNode^3
    26.         if (branchingLevels >= 3 && allLvlArrayNum < allArrayLength)
    27.         {
    28.             for (int i = 0; i < linksPerNode; i++)
    29.             {
    30.                 for (int j = 0; j < linksPerNode; j++)
    31.                 {
    32.                     for (int k = 0; k < linksPerNode; k++)
    33.                     {
    34.                         //Creates the third level by finding all children from level 2 and assigning them as a parent for the creation of level 3
    35.                         CreateLevel(AllLvlArray[linksContainerArray[linksContainerArray[0, i + 1], j + 1]], k);
    36.                     }
    37.                 }
    38.             }
    39.         }
    40.  
    I hope that is helpful.
    I also thought of ways to optimise my algorithms, it is creating too many iterations of the while loop.
    There should be a way to have a more guided/biased random point generation that should rely on a "LookAway()" function that returns a vector that points in the opposite direction from a group of nearby existing nodes.
    I thought of this simple approach:
    https://i.imgur.com/dS3ZTc7.gif

    But it is only an idea now and for 2d and would not work if nodes 2, 3 and 6 are surrounding the parent node 1 from all sides.
    I also thought that another point where my algorithm is not optimised is that I discard forever the already generated potential child node positions if they fail the distance test, while they might pass a later relaxed distance test without the need to create new random positions.
    Thanks for any pointers :)
    Edit: Here is a new version of the while loop that does no lose "the closest to the threshold that did not pass" and every time when the threshold is relaxed checks that "best candidate" against the new relaxed threshold:
    Code (CSharp):
    1.  
    2. randomLoops = 0;
    3. extraDecrease = 0;
    4. float minDistScaled = minDistNode * scaleGrid;
    5. float lineLengthScaled = lineLength * scaleGrid;
    6. float maxSqrMagnitude = 0f;
    7. float minUnderThresholdSq=100f;
    8. int maxi = 0;
    9. while (foundNodePos == false)
    10. {
    11.     foundNodePos = true;
    12.     float minDist = minDistScaled - extraDecrease;
    13.     float minDistSq = minDist * minDist;
    14.     for (int i = 0; i < allLvlArrayNum; i++)
    15.     {
    16.         //Discards the new node position if it is too close to other existing nodes. This might make it difficult to put into a job, since there is a dependency
    17.         Vector3 newNodeToAllNodes = NodePositions[i] - newNodePos;
    18.         float curSqrMagnitude = Vector3.SqrMagnitude(newNodeToAllNodes);
    19.         float underThresholdSq = minDistSq - curSqrMagnitude;
    20.         if (curSqrMagnitude < minDistSq)
    21.         {
    22.             if (underThresholdSq < minUnderThresholdSq && i > maxi)
    23.             {
    24.                 minUnderThresholdSq = underThresholdSq;
    25.                 maxSqrMagnitude = curSqrMagnitude;
    26.                 furtherstNewNodePos = newNodePos;
    27.                 maxi = i;
    28.             }
    29.             foundNodePos = false;
    30.             newNodePos = NodePositions[parentNodeNum] + Random.onUnitSphere * lineLengthScaled;
    31.             //i = allLvlArrayNum;
    32.             break;
    33.         }
    34.     }
    35.     randomLoops++;
    36.     if (randomLoops > maxRandomLoops)
    37.     {
    38.         //foundNodePos = true;
    39.         extraDecrease += (extraDecreaseStep * scaleGrid);
    40.         randomLoops = 0;
    41.         minDist = minDistScaled - extraDecrease;
    42.         minDistSq = minDist * minDist;
    43.         if (maxSqrMagnitude> minDistSq) {
    44.             foundNodePos = true;
    45.             for (int i = 0; i < allLvlArrayNum; i++)
    46.             {
    47.                 //Checks the node that was closest to the threshold so far against the new threshold
    48.                 newNodePos= furtherstNewNodePos;
    49.                 Vector3 newNodeToAllNodes = NodePositions[i] - newNodePos;
    50.                 float curSqrMagnitude = Vector3.SqrMagnitude(newNodeToAllNodes);
    51.                 float underThresholdSq = minDistSq - curSqrMagnitude;
    52.                 if (curSqrMagnitude < minDistSq)
    53.                 {
    54.                     if (underThresholdSq < minUnderThresholdSq && i > maxi)
    55.                     {
    56.                         minUnderThresholdSq = underThresholdSq;
    57.                         maxSqrMagnitude = curSqrMagnitude;
    58.                         maxi = i;
    59.                     }
    60.                     foundNodePos = false;
    61.                     newNodePos = NodePositions[parentNodeNum] + Random.onUnitSphere * lineLengthScaled;
    62.                     break;
    63.                 }
    64.             }
    65.         }
    66.     }
    67.     //Debug.Log ("foundNodePos="+foundNodePos);
    68. }
    Edit2: I tested the above addition to the algorithm, which I call "second chance" :) The "second chance" bit generates around 15% of all nodes, provides around 10% speed-up and creates slightly more evenly distributed grids.
     
    Last edited: Apr 27, 2019
  10. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    4,271
    Alright! So some good news, this is totally achievable, and we probably won't even need IJobParallelFor to pull it off.

    But there's also some bad news. I was specifically asking for all the variables used in the algorithm (and by algorithm that includes both CreateGrid and CreateLevel because the first step is to get rid of all the reference types like GameObject.name. This does not include arrays 2D arrays, or lists, but does include arrays of lists and lists of lists.

    You are allowed to have patch-up steps before and after the main algorithm that can use all the reference types you want. But you have to make sure these patch-up steps take very little time as they will freeze your game. Though I think you should be able to get those steps down to under a frame.

    Unfortunately I can't help you with refactoring out those patch-up processes because your algorithm is using variables in class scope and I don't fully know those data types. But if you get stuck anywhere with this refactoring, feel free to ask.
     
  11. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    Oh, I see now what you wanted, sorry, did not get you the first time. You want a self-containing class with the functions and variables for the algorithm. Also, that passing the whole GameObject of the parent and then getting its number with int.Parse() was such a flop. I actually don't need it since all my positions, rotations and scales are in arrays, so I can pass an int with the parent's index instead.
    Hese is a link to the .cs file
    https://drive.google.com/file/d/1nt7GCn3Ea2szM0DA3QvAVnF6_S8kcqby/view?usp=sharing
    And here is the whole class:
    Code (CSharp):
    1.  
    2. using System.Collections;
    3. using System.Collections.Generic;
    4. using UnityEngine;
    5.  
    6. public class New : MonoBehaviour
    7. {
    8.  
    9.     int allLvlArrayNum = 0;
    10.     int allArrayLength = 2000;
    11.     Vector3 newNodePos;
    12.     float lineLength = 0.5f;
    13.     float scaleGrid = 1f;
    14.     int randomLoops;
    15.     float extraDecrease;
    16.     float minDistNode = 1f;
    17.     float extraDecreaseStep = 0.1f;
    18.     bool foundNodePos = false;
    19.     Vector3 furtherstNewNodePos;
    20.     int maxRandomLoops = 30;
    21.     Vector3 camPos = new Vector3(0, 0, 5);
    22.     Vector3[] NodePositions;
    23.     Quaternion[] NodeRotations;
    24.     Vector3[] NodeScales;
    25.     Vector3 camUp = new Vector3(0, 1, 0);
    26.     Vector3 fixedNormalScale = new Vector3(1f, 1f, 1f);
    27.     int[,] linksContainerArray;
    28.     int maxLinksPerLinkedNode = 9;
    29.     int branchingLevels = 6;
    30.     int linksPerNode = 3;
    31.  
    32.     void AssignValues()
    33.     {
    34.         NodePositions = new Vector3[allArrayLength];
    35.         NodeRotations = new Quaternion[allArrayLength];
    36.         NodeScales = new Vector3[allArrayLength];
    37.         linksContainerArray = new int[allArrayLength, maxLinksPerLinkedNode];
    38.     }
    39.  
    40.     void CreateLevel(int parentNodeNum, int childrenNum)
    41.     {
    42.         if (allLvlArrayNum < allArrayLength)
    43.         {
    44.             newNodePos = NodePositions[parentNodeNum] + Random.onUnitSphere * lineLength * scaleGrid;
    45.             randomLoops = 0;
    46.             extraDecrease = 0;
    47.             float minDistScaled = minDistNode * scaleGrid;
    48.             float lineLengthScaled = lineLength * scaleGrid;
    49.             float extraDecreaseStepScaled = extraDecreaseStep * scaleGrid;
    50.             float maxSqrMagnitude = 0f;
    51.             float minUnderThresholdSq = 100f;
    52.             int maxi = 0;
    53.             //This inner loop checks the distance between the new node and all other nodes and ensures that the new nodes are no too close to existing nodes, gradually lowering the threshold from 1 unit by 0.1 units every 30 attempts
    54.             while (foundNodePos == false)
    55.             {
    56.                 foundNodePos = true;
    57.                 float minDist = minDistScaled - extraDecrease;
    58.                 float minDistSq = minDist * minDist;
    59.                 for (int i = 0; i < allLvlArrayNum; i++)
    60.                 {
    61.                     //Discards the new node position if it is too close to other existing nodes. This might make it difficult to put into a job, since there is a dependency
    62.                     Vector3 newNodeToAllNodes = NodePositions[i] - newNodePos;
    63.                     float curSqrMagnitude = Vector3.SqrMagnitude(newNodeToAllNodes);
    64.                     float underThresholdSq = minDistSq - curSqrMagnitude;
    65.                     if (curSqrMagnitude < minDistSq)
    66.                     {
    67.                         //Debug.Log(underThresholdSq);
    68.                         //if (underThresholdSq < minUnderThresholdSq && i > maxi)
    69.                         if (underThresholdSq < minUnderThresholdSq)
    70.                         {
    71.                             minUnderThresholdSq = underThresholdSq;
    72.                             maxSqrMagnitude = curSqrMagnitude;
    73.                             furtherstNewNodePos = newNodePos;
    74.                             maxi = i;
    75.                         }
    76.                         foundNodePos = false;
    77.                         newNodePos = NodePositions[parentNodeNum] + Random.onUnitSphere * lineLengthScaled;
    78.                         break;
    79.                     }
    80.                 }
    81.                 randomLoops++;
    82.                 if (randomLoops > maxRandomLoops)
    83.                 {
    84.                     extraDecrease += extraDecreaseStepScaled;
    85.                     randomLoops = 0;
    86.                     minDist = minDistScaled - extraDecrease;
    87.                     minDistSq = minDist * minDist;
    88.                     if (maxSqrMagnitude > minDistSq)
    89.                     {
    90.                         foundNodePos = true;
    91.                         for (int i = 0; i < allLvlArrayNum; i++)
    92.                         {
    93.                             //Checks the node that was closest to the threshold so far against the new threshold
    94.                             newNodePos = furtherstNewNodePos;
    95.                             Vector3 newNodeToAllNodes = NodePositions[i] - newNodePos;
    96.                             float curSqrMagnitude = Vector3.SqrMagnitude(newNodeToAllNodes);
    97.                             float underThresholdSq = minDistSq - curSqrMagnitude;
    98.                             if (curSqrMagnitude < minDistSq)
    99.                             {
    100.                                 if (underThresholdSq < minUnderThresholdSq && i > maxi)
    101.                                 {
    102.                                     minUnderThresholdSq = underThresholdSq;
    103.                                     maxSqrMagnitude = curSqrMagnitude;
    104.                                     maxi = i;
    105.                                 }
    106.                                 foundNodePos = false;
    107.                                 newNodePos = NodePositions[parentNodeNum] + Random.onUnitSphere * lineLengthScaled;
    108.                                 break;
    109.                             }
    110.                         }
    111.                     }
    112.                 }
    113.             }
    114.             NodePositions[allLvlArrayNum] = newNodePos;
    115.             Vector3 nodeToCam = camPos - NodePositions[allLvlArrayNum];
    116.             NodeRotations[allLvlArrayNum] = Quaternion.LookRotation(nodeToCam, camUp);
    117.             NodeScales[allLvlArrayNum] = fixedNormalScale;
    118.             linksContainerArray[parentNodeNum, childrenNum + 1] = allLvlArrayNum;
    119.             linksContainerArray[allLvlArrayNum, 0] = parentNodeNum;
    120.             allLvlArrayNum++;
    121.             foundNodePos = false;
    122.         }
    123.     }
    124.  
    125.     void CreateCenterNode()
    126.     {
    127.         NodePositions[0] = Vector3.zero;
    128.         Vector3 nodeToCam = camPos - NodePositions[0];
    129.         NodeRotations[allLvlArrayNum] = Quaternion.LookRotation(nodeToCam, camUp);
    130.         NodeScales[allLvlArrayNum] = fixedNormalScale;
    131.         linksContainerArray[0, 0] = 0;
    132.         allLvlArrayNum++;
    133.     }
    134.  
    135.     void CreateGrid()
    136.     {
    137.         CreateCenterNode();
    138.         //We don't use fully nested for loops since this way we are building the tree gradually, which makes it more evenly spread
    139.         //Level 1 - Node # = linksPerNode
    140.         for (int i = 0; i < linksPerNode; i++)
    141.         {
    142.             CreateLevel(0, i);
    143.         }
    144.         //Level 2 - Node # = linksPerNode^2
    145.         if (branchingLevels >= 2 && allLvlArrayNum < allArrayLength)
    146.         {
    147.             for (int i = 0; i < linksPerNode; i++)
    148.             {
    149.                 for (int j = 0; j < linksPerNode; j++)
    150.                 {
    151.                     CreateLevel(linksContainerArray[0, i + 1], j);
    152.                 }
    153.             }
    154.         }
    155.         //Level 3 - Node # = linksPerNode^3
    156.         if (branchingLevels >= 3 && allLvlArrayNum < allArrayLength)
    157.         {
    158.             for (int i = 0; i < linksPerNode; i++)
    159.             {
    160.                 for (int j = 0; j < linksPerNode; j++)
    161.                 {
    162.                     for (int k = 0; k < linksPerNode; k++)
    163.                     {
    164.                         CreateLevel(linksContainerArray[linksContainerArray[0, i + 1], j + 1], k);
    165.                     }
    166.                 }
    167.             }
    168.         }
    169.  
    170.         //Level 4 - Node # = linksPerNode^4
    171.         if (branchingLevels >= 4 && allLvlArrayNum < allArrayLength)
    172.         {
    173.             for (int i = 0; i < linksPerNode; i++)
    174.             {
    175.                 for (int j = 0; j < linksPerNode; j++)
    176.                 {
    177.                     for (int k = 0; k < linksPerNode; k++)
    178.                     {
    179.                         for (int l = 0; l < linksPerNode; l++)
    180.                         {
    181.                             CreateLevel(linksContainerArray[linksContainerArray[linksContainerArray[0, i + 1], j + 1], k + 1], l);
    182.                         }
    183.                     }
    184.                 }
    185.             }
    186.         }
    187.         //Level 5 - Node # = linksPerNode^5
    188.         if (branchingLevels >= 5 && allLvlArrayNum < allArrayLength)
    189.         {
    190.             for (int i = 0; i < linksPerNode; i++)
    191.             {
    192.                 for (int j = 0; j < linksPerNode; j++)
    193.                 {
    194.                     for (int k = 0; k < linksPerNode; k++)
    195.                     {
    196.                         for (int l = 0; l < linksPerNode; l++)
    197.                         {
    198.                             for (int m = 0; m < linksPerNode; m++)
    199.                             {
    200.                                 CreateLevel(linksContainerArray[linksContainerArray[linksContainerArray[linksContainerArray[0, i + 1], j + 1], k + 1], l + 1], m);
    201.                             }
    202.                         }
    203.                     }
    204.                 }
    205.             }
    206.         }
    207.  
    208.         //Level 6 - Node # = linksPerNode^6
    209.         if (branchingLevels >= 6 && allLvlArrayNum < allArrayLength)
    210.         {
    211.             for (int i = 0; i < linksPerNode; i++)
    212.             {
    213.                 for (int j = 0; j < linksPerNode; j++)
    214.                 {
    215.                     for (int k = 0; k < linksPerNode; k++)
    216.                     {
    217.                         for (int l = 0; l < linksPerNode; l++)
    218.                         {
    219.                             for (int m = 0; m < linksPerNode; m++)
    220.                             {
    221.                                 for (int n = 0; n < linksPerNode; n++)
    222.                                 {
    223.                                     CreateLevel(linksContainerArray[linksContainerArray[linksContainerArray[linksContainerArray[linksContainerArray[0, i + 1], j + 1], k + 1], l + 1], m + 1], n);
    224.                                 }
    225.                             }
    226.                         }
    227.                     }
    228.                 }
    229.             }
    230.         }
    231.     }
    232. }
    I hope that is better.
    I should mention that my target is Android and I get the 7 sec loading time on Android.
     
    Last edited: Apr 27, 2019
  12. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    4,271
    Perfect!

    So our next step is we are going to turn this class you named "New" into a job struct. It no longer needs to be on a MonoBehaviour (you can have a MonoBehaviour create and populate the struct). The struct should implement the interface IJob, and the Execute method will just be a simple call to CreateGrid.

    We also need to get the remaining managed types to Job-safe types.

    For our random number generator, we will create a math.Random field in our struct. This is using Unity.Mathematics. Instead of the onUnitSphere property, we will use the NextFloat3Direction method from our math.Random. We can also change all usages of Vector3 to float3. There's implicit conversions between the two, so you shouldn't have any compatibility issues with the rest of the code, but the float3 will give you a good speedup later.

    Next, we need to convert our Arrays into NativeArrays. We'll allocate them using Allocator.TempJob. The 2D array is a little more involved, so I whipped up something for you quick.

    Code (CSharp):
    1. public struct NativeArray2D<T> : IDisposable where T : struct
    2.     {
    3.         private int xDim, yDim;
    4.         private NativeArray<T> data;
    5.  
    6.         public NativeArray2D(int x, int y, Allocator allocator)
    7.         {
    8.             xDim = x;
    9.             yDim = y;
    10.             data = new NativeArray<T>(x * y, allocator);
    11.         }
    12.  
    13.         public int CountX { get { return xDim; } }
    14.         public int CountY { get { return yDim; } }
    15.  
    16.         public T this[int x, int y]
    17.         {
    18.             get
    19.             {
    20.                 return data[yDim * x + y];
    21.             }
    22.             set
    23.             {
    24.                 data[yDim * x + y] = value;
    25.             }
    26.         }
    27.  
    28.         public void Dispose()
    29.         {
    30.             data.Dispose();
    31.         }
    32.     }
    By this point, we are pretty close to having something heavily optimizable, but we need to test some things to make sure they aren't broken. To ensure everything works, run the job and complete it immediately using .Schedule().Complete() and then copy the data out of the NativeArrays into the arrays that the rest of your game uses. First, make sure you get results on screen that you like. Second, make sure that the final copy step runs fast enough for your liking assuming that that little bit of code will freeze your game. If it takes too long, we'll have to do some additional gymnastics to speed that up.

    Assuming everything is checking out, the next step is to add [BurstCompile] on top of the job and see how much faster it runs. It might actually be fast enough that you don't need to do anything else, but if not, we can keep going.

    To get this thing running asynchronous, we can use a code snippet like this inside a coroutine:
    Code (CSharp):
    1. var jh = ourJobStructInstance.Schedule(this);
    2.             while (!jh.IsCompleted)
    3.             {
    4.                 yield return null;
    5.             }
    6.             jh.Complete();
    See how close that gets you to what you want. And if there's still some problems, we can work them out!
     
  13. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    Can I ask what your algorithm is trying to do, why do you need a group of random points with a minimum distance between them?
     
  14. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    Hi, Thanks for all the tips. This is quite new for me so it took some time to figure it out, but it was interesting.
    I decided to test everything in a new project to be sure that I am working on a clean basis with only 2 scripts:
    JobCaller.cs that is a MonoBehaviour and that starts the job upon pressing the space key:
    https://drive.google.com/file/d/1b_8PPa51ewFVwT1iK6jCMnBhX-cxzp2O/view?usp=sharing
    CreateGridStruct.cs that contains the IJob struct and the NativeArray2D struct:
    https://drive.google.com/file/d/1_gKGUTzuKKbALFMJ51c6fUglorwOVu6Z/view?usp=sharing
    I finally managed to compile the scripts, but I get this error message when I start the createGridJob:
    Code (CSharp):
    1. ArgumentException: Invalid state 0. Random object has not been properly initialized
    2. Unity.Mathematics.Random.CheckState () (at Library/PackageCache/com.unity.mathematics@1.0.1/Unity.Mathematics/random.cs:535)
    3. Unity.Mathematics.Random.NextState () (at Library/PackageCache/com.unity.mathematics@1.0.1/Unity.Mathematics/random.cs:513)
    4. Unity.Mathematics.Random.NextFloat2 () (at Library/PackageCache/com.unity.mathematics@1.0.1/Unity.Mathematics/random.cs:314)
    5. Unity.Mathematics.Random.NextFloat3Direction () (at Library/PackageCache/com.unity.mathematics@1.0.1/Unity.Mathematics/random.cs:469)
    6. CreateGridJob.CreateLevel (System.Int32 parentNodeNum, System.Int32 childrenNum) (at Assets/Scripts/CreateGridStruct.cs:91)
    7. CreateGridJob.CreateGrid () (at Assets/Scripts/CreateGridStruct.cs:232)
    8. CreateGridJob.Execute () (at Assets/Scripts/CreateGridStruct.cs:84)
    9. Unity.Jobs.IJobExtensions+JobStruct`1[T].Execute (T& data, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, Unity.Jobs.LowLevel.Unsafe.JobRanges& ranges, System.Int32 jobIndex) (at C:/buildslave/unity/build/Runtime/Jobs/Managed/IJob.cs:30)
    10. Unity.Jobs.JobHandle:ScheduleBatchedJobsAndComplete(JobHandle&)
    11. Unity.Jobs.JobHandle:Complete() (at C:/buildslave/unity/build/Runtime/Jobs/ScriptBindings/JobHandle.bindings.cs:20)
    12. JobCaller:CallCreateGridJob() (at Assets/Scripts/JobCaller.cs:156)
    13. JobCaller:Update() (at Assets/Scripts/JobCaller.cs:222)
    It says I have not properly initialized the Random object. For reference, this is what I did:
    Code (CSharp):
    1. using Rand = Unity.Mathematics.Random;
    2. ...
    3. public static Rand rand;
    4. ...
    5. public float3 randVector3;
    6. ...
    7. randVector3 = rand.NextFloat3Direction();
    It also spits the following message:
    Code (CSharp):
    1. Internal: JobTempAlloc has allocations that are more than 4 frames old - this is not allowed and likely a leak
    2. -----------------------------
    3. To Debug, enable the define: TLA_DEBUG_STACK_LEAK in ThreadsafeLinearAllocator.cpp. This will output the callstacks of the leaked allocations
    4.  
    In the job struct I also resorted to the Unity method for getting a Quaternion from a direction vector and up vector, because I did not know of any similar method in the new math library:
    Code (CSharp):
    1. longCubesRotNA[numberOfLinks] = Quaternion.LookRotation(nodeToNode, Vector3.up);
    I also don't know whether we should dispose of the NativeArrays...
    BTW, once I get this done, I plan to continue with converting to the job system the building of the secondary grid, but let's first get this done :)
    Here is the JobCaller.cs
    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4. using UnityEngine.UI;
    5. using Unity.Collections;
    6. using TMPro;
    7. using UnityEngine.SceneManagement;
    8. using UnityEngine.EventSystems;
    9. using Unity.Jobs;
    10. using UnityEngine.Jobs;
    11. using Unity.Mathematics;
    12.  
    13. public class JobCaller : MonoBehaviour
    14. {
    15.     public int allLvlArrayNum=0;
    16.     public int allArrayLength;
    17.     public Vector3 newNodePos;
    18.     public float lineLength;
    19.     public float scaleGrid;
    20.     public int randomLoops=0;
    21.     public float extraDecrease=0f;
    22.     public float minDistNode;
    23.     public float extraDecreaseStep;
    24.     public bool foundNodePos=false;
    25.     public Vector3 furtherstNewNodePos;
    26.     public int maxRandomLoops;
    27.     public Vector3 camPos;
    28.     public Vector3[] NodePositions;
    29.     public Quaternion[] NodeRotations;
    30.     public Vector3[] NodeScales;
    31.     public Vector3 camUp;
    32.     public Vector3 fixedNormalScale;
    33.     public int[,] linksContainerArray;
    34.     public int[] linksContainerArrayLength;
    35.     public int[,] lineToNodes;
    36.     public int numberOfLinks=0;
    37.     public Vector3[] longCubesPos;
    38.     public Quaternion[] longCubesRot;
    39.     public Vector3[] longCubesScale;
    40.     public bool[,] currentLinkedToTarget;
    41.     public int[,] instLongCubesNum;
    42.     public float[,] DistanceBetweenNodesArraySq;
    43.     public float linkWidth;
    44.     public int maxLinksPerLinkedNode;
    45.     public int branchingLevels;
    46.     public int linksPerNode;
    47.     public int secondChance;
    48.     public int maxLinks;
    49.  
    50.     void AssignValues()
    51.     {
    52.         allArrayLength = 2000;
    53.         maxLinksPerLinkedNode = 9;
    54.         lineLength = 0.5f;
    55.         scaleGrid = 1f;
    56.         minDistNode = 1f;
    57.         extraDecreaseStep = 0.1f;
    58.         maxRandomLoops = 30;
    59.         linkWidth = 0.01f;
    60.         branchingLevels = 6;
    61.         linksPerNode = 3;
    62.         NodePositions = new Vector3[allArrayLength];
    63.         NodeRotations = new Quaternion[allArrayLength];
    64.         NodeScales = new Vector3[allArrayLength];
    65.         maxLinks = allArrayLength * maxLinksPerLinkedNode;
    66.         instLongCubesNum = new int[allArrayLength, maxLinksPerLinkedNode];
    67.         linksContainerArray = new int[allArrayLength, maxLinksPerLinkedNode];
    68.         currentLinkedToTarget = new bool[allArrayLength, allArrayLength];
    69.         DistanceBetweenNodesArraySq = new float[allArrayLength, allArrayLength];
    70.         for (int i = 0; i < allArrayLength; i++)
    71.         {
    72.             for (int j = 0; j < allArrayLength; j++)
    73.             {
    74.                 currentLinkedToTarget[i, j] = false;
    75.             }
    76.         }
    77.         linksContainerArrayLength = new int[allArrayLength];
    78.         longCubesPos = new Vector3[maxLinks];
    79.         longCubesRot = new Quaternion[maxLinks];
    80.         longCubesScale = new Vector3[maxLinks];
    81.         lineToNodes = new int[maxLinks, 2];
    82.         fixedNormalScale = new Vector3(5f, 5f, 5f);
    83.         linkWidth = 0.01f;
    84.         camPos = new Vector3(0f, 0f, -3f);
    85.         camUp = new Vector3(0f, 1f, 0f);
    86.     }
    87.  
    88.  
    89.     void CallCreateGridJob()
    90.     {
    91.         var createGridJob = new CreateGridJob()
    92.         {
    93.             allLvlArrayNum = allLvlArrayNum,
    94.             allArrayLength = allArrayLength,
    95.             newNodePos = newNodePos,
    96.             lineLength = lineLength,
    97.             scaleGrid = scaleGrid,
    98.             randomLoops = randomLoops,
    99.             extraDecrease = extraDecrease,
    100.             minDistNode = minDistNode,
    101.             extraDecreaseStep = extraDecreaseStep,
    102.             foundNodePos = foundNodePos,
    103.             furtherstNewNodePos = furtherstNewNodePos,
    104.             maxRandomLoops = maxRandomLoops,
    105.             camPos = camPos,
    106.             NodePositionsNA = new NativeArray<float3>(allArrayLength, Allocator.TempJob),
    107.             NodeRotationsNA = new NativeArray<Quaternion>(NodeRotations, Allocator.TempJob),
    108.             NodeScalesNA = new NativeArray<float3>(allArrayLength, Allocator.TempJob),
    109.             camUp = camUp,
    110.             fixedNormalScale = fixedNormalScale,
    111.             linksContainerArrayNA = new NativeArray2D<int>(allArrayLength, maxLinksPerLinkedNode, Allocator.TempJob),
    112.             linksContainerArrayLengthNA = new NativeArray<int>(linksContainerArrayLength, Allocator.TempJob),
    113.             lineToNodesNA = new NativeArray2D<int>(maxLinks, 2, Allocator.TempJob),
    114.             numberOfLinks = numberOfLinks,
    115.             longCubesScaleNA = new NativeArray<float3>(maxLinks, Allocator.TempJob),
    116.             longCubesPosNA = new NativeArray<float3>(maxLinks, Allocator.TempJob),
    117.             longCubesRotNA = new NativeArray<Quaternion>(longCubesRot, Allocator.TempJob),
    118.             currentLinkedToTargetNA = new NativeArray2D<bool>(allArrayLength, allArrayLength, Allocator.TempJob),
    119.             instLongCubesNumNA = new NativeArray2D<int>(allArrayLength, maxLinksPerLinkedNode, Allocator.TempJob),
    120.             DistanceBetweenNodesArraySqNA = new NativeArray2D<float>(allArrayLength, allArrayLength, Allocator.TempJob),
    121.             linkWidth = linkWidth,
    122.             maxLinksPerLinkedNode = maxLinksPerLinkedNode,
    123.             branchingLevels = branchingLevels,
    124.             linksPerNode = linksPerNode,
    125.             secondChance = secondChance,
    126.         };
    127.  
    128.         for (int i = 0; i < allArrayLength; i++)
    129.         {
    130.             createGridJob.NodePositionsNA[i] = NodePositions[i];
    131.             createGridJob.NodeScalesNA[i] = NodeScales[i];
    132.             for (int j = 0; j < maxLinksPerLinkedNode; j++)
    133.             {
    134.                 createGridJob.linksContainerArrayNA[i, j] = linksContainerArray[i, j];
    135.                 createGridJob.instLongCubesNumNA[i, j] = instLongCubesNum[i, j];
    136.             }
    137.             for (int k = 0; k < allArrayLength; k++)
    138.             {
    139.                 createGridJob.currentLinkedToTargetNA[i, k] = currentLinkedToTarget[i, k];
    140.                 createGridJob.DistanceBetweenNodesArraySqNA[i, k] = DistanceBetweenNodesArraySq[i, k];
    141.             }
    142.         }
    143.  
    144.         for (int i = 0; i < maxLinks; i++)
    145.         {
    146.             createGridJob.lineToNodesNA[i, 0] = lineToNodes[i, 0];
    147.             createGridJob.lineToNodesNA[i, 1] = lineToNodes[i, 1];
    148.             createGridJob.longCubesScaleNA[i] = longCubesScale[i];
    149.             createGridJob.longCubesPosNA[i] = longCubesPos[i];
    150.             createGridJob.longCubesRotNA[i] = longCubesRot[i];
    151.         }
    152.  
    153.         JobHandle createGridJobHandle = createGridJob.Schedule();
    154.         createGridJobHandle.Complete();
    155.  
    156.         NodeRotations = createGridJob.NodeRotationsNA.ToArray();
    157.         linksContainerArrayLength = createGridJob.linksContainerArrayLengthNA.ToArray();
    158.         longCubesRot = createGridJob.longCubesRotNA.ToArray();
    159.         for (int i = 0; i < allArrayLength; i++)
    160.         {
    161.             NodePositions[i] = createGridJob.NodePositionsNA[i];
    162.             NodeScales[i] = createGridJob.NodeScalesNA[i];
    163.             for (int j = 0; j < maxLinksPerLinkedNode; j++)
    164.             {
    165.                 linksContainerArray[i, j] = createGridJob.linksContainerArrayNA[i, j];
    166.                 instLongCubesNum[i, j] = createGridJob.instLongCubesNumNA[i, j];
    167.             }
    168.             for (int k = 0; k < allArrayLength; k++)
    169.             {
    170.                 currentLinkedToTarget[i, k] = createGridJob.currentLinkedToTargetNA[i, k];
    171.                 DistanceBetweenNodesArraySq[i, k] = createGridJob.DistanceBetweenNodesArraySqNA[i, k];
    172.             }
    173.         }
    174.  
    175.         for (int i = 0; i < maxLinks; i++)
    176.         {
    177.             lineToNodes[i, 0] = createGridJob.lineToNodesNA[i, 0];
    178.             lineToNodes[i, 1] = createGridJob.lineToNodesNA[i, 1];
    179.             longCubesScale[i] = createGridJob.longCubesScaleNA[i];
    180.             longCubesPos[i] = createGridJob.longCubesPosNA[i];
    181.             longCubesRot[i] = createGridJob.longCubesRotNA[i];
    182.         }
    183.  
    184.         if (createGridJob.NodePositionsNA != null)
    185.         {
    186.             createGridJob.NodePositionsNA.Dispose();
    187.         }
    188.  
    189.         if (createGridJob.NodeRotationsNA != null)
    190.         {
    191.             createGridJob.NodeRotationsNA.Dispose();
    192.         }
    193.  
    194.         if (createGridJob.NodeScalesNA != null)
    195.         {
    196.             createGridJob.NodeScalesNA.Dispose();
    197.         }
    198.  
    199.         //if (createGridJob.linksContainerArrayNA != null)
    200.         //{
    201.         createGridJob.linksContainerArrayNA.Dispose();
    202.         //}
    203.  
    204.         createGridJob.allLvlArrayNum = allLvlArrayNum;
    205.         createGridJob.numberOfLinks = numberOfLinks;
    206.         createGridJob.secondChance = secondChance;
    207.     }
    208.  
    209.     // Start is called before the first frame update
    210.     void Start()
    211.     {
    212.         AssignValues();
    213.     }
    214.  
    215.     // Update is called once per frame
    216.     void Update()
    217.     {
    218.         if (Input.GetKeyDown("space"))
    219.         {
    220.             CallCreateGridJob();
    221.         }
    222.         //Debug.LogError(allArrayLength);
    223.     }
    224. }
    225.  
    and the CreateGridStruct.cs:
    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4. using Unity.Collections;
    5. using Unity.Jobs;
    6. using UnityEngine.Jobs;
    7. using Unity.Mathematics;
    8. using Rand = Unity.Mathematics.Random;
    9. using System;
    10.  
    11. public struct NativeArray2D<T> : IDisposable where T : struct
    12. {
    13.     private int xDim, yDim;
    14.     private NativeArray<T> data;
    15.  
    16.     public NativeArray2D(int x, int y, Allocator allocator)
    17.     {
    18.         xDim = x;
    19.         yDim = y;
    20.         data = new NativeArray<T>(x * y, allocator);
    21.     }
    22.  
    23.     public int CountX { get { return xDim; } }
    24.     public int CountY { get { return yDim; } }
    25.  
    26.     public T this[int x, int y]
    27.     {
    28.         get
    29.         {
    30.             return data[yDim * x + y];
    31.         }
    32.         set
    33.         {
    34.             data[yDim * x + y] = value;
    35.         }
    36.     }
    37.  
    38.     public void Dispose()
    39.     {
    40.         data.Dispose();
    41.     }
    42. }
    43.  
    44. public struct CreateGridJob : IJob
    45. {
    46.     public int allLvlArrayNum;
    47.     public int allArrayLength;
    48.     public float3 newNodePos;
    49.     public float lineLength;
    50.     public float scaleGrid;
    51.     public int randomLoops;
    52.     public float extraDecrease;
    53.     public float minDistNode;
    54.     public float extraDecreaseStep;
    55.     public bool foundNodePos;
    56.     public float3 furtherstNewNodePos;
    57.     public int maxRandomLoops;
    58.     public float3 camPos;
    59.     public NativeArray<float3> NodePositionsNA;
    60.     public NativeArray<Quaternion> NodeRotationsNA;
    61.     public NativeArray<float3> NodeScalesNA;
    62.     public float3 camUp;
    63.     public float3 fixedNormalScale;
    64.     public NativeArray2D<int> linksContainerArrayNA;
    65.     public NativeArray<int> linksContainerArrayLengthNA;
    66.     public NativeArray2D<int> lineToNodesNA;
    67.     public int numberOfLinks;
    68.     public NativeArray<float3> longCubesScaleNA;
    69.     public NativeArray<float3> longCubesPosNA;
    70.     public NativeArray<Quaternion> longCubesRotNA;
    71.     public NativeArray2D<bool> currentLinkedToTargetNA;
    72.     public NativeArray2D<int> instLongCubesNumNA;
    73.     public float linkWidth;
    74.     public int maxLinksPerLinkedNode;
    75.     public int branchingLevels;
    76.     public int linksPerNode;
    77.     public float3 randVector3;
    78.     public static Rand rand;
    79.     public int secondChance;
    80.     public NativeArray2D<float> DistanceBetweenNodesArraySqNA;
    81.  
    82.     public void Execute()
    83.     {
    84.         CreateGrid();
    85.     }
    86.  
    87.     void CreateLevel(int parentNodeNum, int childrenNum)
    88.     {
    89.         if (allLvlArrayNum < allArrayLength)
    90.         {
    91.             randVector3 = rand.NextFloat3Direction();
    92.             float lineLengthScaled = lineLength * scaleGrid;
    93.             newNodePos = NodePositionsNA[parentNodeNum] + randVector3 * lineLengthScaled;
    94.             randomLoops = 0;
    95.             extraDecrease = 0;
    96.             float minDistScaled = minDistNode * scaleGrid;
    97.             float extraDecreaseStepScaled = extraDecreaseStep * scaleGrid;
    98.             float maxSqrMagnitude = 0f;
    99.             float minUnderThresholdSq = 100f;
    100.             int maxi = 0;
    101.             while (foundNodePos == false)
    102.             {
    103.                 foundNodePos = true;
    104.                 float minDist = minDistScaled - extraDecrease;
    105.                 float minDistSq = minDist * minDist;
    106.                 for (int i = 0; i < allLvlArrayNum; i++)
    107.                 {
    108.                     //Discards the new node position if it is too close to other existing nodes.
    109.                     float3 newNodeToAllNodes = NodePositionsNA[i] - newNodePos;
    110.                     float curSqrMagnitude = newNodeToAllNodes.x* newNodeToAllNodes.x+ newNodeToAllNodes.y * newNodeToAllNodes.y+ newNodeToAllNodes.z * newNodeToAllNodes.z;
    111.                     float underThresholdSq = minDistSq - curSqrMagnitude;
    112.                     if (curSqrMagnitude < minDistSq)
    113.                     {
    114.                         if (underThresholdSq < minUnderThresholdSq && i > maxi)
    115.                         //if (underThresholdSq < minUnderThresholdSq)
    116.                         {
    117.                             minUnderThresholdSq = underThresholdSq;
    118.                             maxSqrMagnitude = curSqrMagnitude;
    119.                             furtherstNewNodePos = newNodePos;
    120.                             maxi = i;
    121.                         }
    122.                         foundNodePos = false;
    123.                         float3 randVector3 = rand.NextFloat3Direction();
    124.                         newNodePos = NodePositionsNA[parentNodeNum] + randVector3 * lineLengthScaled;
    125.                         break;
    126.                     }
    127.                 }
    128.                 randomLoops++;
    129.                 if (randomLoops > maxRandomLoops && foundNodePos==false)
    130.                 {
    131.                     extraDecrease += extraDecreaseStepScaled;
    132.                     randomLoops = 0;
    133.                     minDist = minDistScaled - extraDecrease;
    134.                     minDistSq = minDist * minDist;
    135.                     if (maxSqrMagnitude > minDistSq)
    136.                     {
    137.                         foundNodePos = true;
    138.                         for (int i = 0; i < allLvlArrayNum; i++)
    139.                         {
    140.                             //Checks the node that was closest to the threshold so far against the new threshold
    141.                             newNodePos = furtherstNewNodePos;
    142.                             float3 newNodeToAllNodes = NodePositionsNA[i] - newNodePos;
    143.                             float curSqrMagnitude = newNodeToAllNodes.x * newNodeToAllNodes.x + newNodeToAllNodes.y * newNodeToAllNodes.y + newNodeToAllNodes.z * newNodeToAllNodes.z;
    144.                             float underThresholdSq = minDistSq - curSqrMagnitude;
    145.                             if (curSqrMagnitude < minDistSq)
    146.                             {
    147.                                 if (underThresholdSq < minUnderThresholdSq && i > maxi)
    148.                                 {
    149.                                     minUnderThresholdSq = underThresholdSq;
    150.                                     maxSqrMagnitude = curSqrMagnitude;
    151.                                     maxi = i;
    152.                                 }
    153.                                 foundNodePos = false;
    154.                                 float3 randVector3 = rand.NextFloat3Direction();
    155.                                 newNodePos = NodePositionsNA[parentNodeNum] + randVector3 * lineLengthScaled;
    156.                                 break;
    157.                             }
    158.                         }
    159.                     }
    160.                     if (foundNodePos)
    161.                     {
    162.                         secondChance++;
    163.                     }
    164.                 }
    165.             }
    166.             NodePositionsNA[allLvlArrayNum] = newNodePos;
    167.             float3 nodeToCam = camPos - NodePositionsNA[allLvlArrayNum];
    168.             NodeRotationsNA[allLvlArrayNum] = Quaternion.LookRotation(nodeToCam, camUp);
    169.             NodeScalesNA[allLvlArrayNum] = fixedNormalScale;
    170.             float3 nodeToNode = NodePositionsNA[parentNodeNum] - NodePositionsNA[allLvlArrayNum];
    171.             float distBetweenNodes = math.sqrt(nodeToNode.x* nodeToNode.x+nodeToNode.y* nodeToNode.y+nodeToNode.z* nodeToNode.z);
    172.             float3 nodeMidPoint = NodePositionsNA[allLvlArrayNum] + nodeToNode * 0.5f;
    173.             longCubesScaleNA[numberOfLinks] = new float3(linkWidth, linkWidth, distBetweenNodes);
    174.             longCubesPosNA[numberOfLinks] = nodeMidPoint;
    175.             //How is this lookrotation implemented in the new math library?
    176.             longCubesRotNA[numberOfLinks] = Quaternion.LookRotation(nodeToNode, Vector3.up);
    177.             linksContainerArrayNA[parentNodeNum, childrenNum + 1] = allLvlArrayNum;
    178.             linksContainerArrayLengthNA[parentNodeNum]++;
    179.             currentLinkedToTargetNA[parentNodeNum, allLvlArrayNum] = true;
    180.             instLongCubesNumNA[parentNodeNum, childrenNum + 1] = numberOfLinks;
    181.             lineToNodesNA[numberOfLinks, 0] = parentNodeNum;
    182.             lineToNodesNA[numberOfLinks, 1] = allLvlArrayNum;
    183.             linksContainerArrayNA[allLvlArrayNum, 0] = parentNodeNum;
    184.             linksContainerArrayLengthNA[allLvlArrayNum]++;
    185.             currentLinkedToTargetNA[allLvlArrayNum, parentNodeNum] = true;
    186.             instLongCubesNumNA[allLvlArrayNum, 0] = numberOfLinks;
    187.             allLvlArrayNum++;
    188.             foundNodePos = false;
    189.             numberOfLinks++;
    190.         }
    191.     }
    192.  
    193.     void CreateCenterNode()
    194.     {
    195.         NodePositionsNA[0] = new float3(0f,0f,0f);
    196.         float3 nodeToCam = camPos - NodePositionsNA[0];
    197.         //can we use the math analogue to Quaternion.LookRotation?
    198.         NodeRotationsNA[allLvlArrayNum] = Quaternion.LookRotation(nodeToCam, camUp);
    199.         NodeScalesNA[allLvlArrayNum] = fixedNormalScale;
    200.         linksContainerArrayNA[0, 0] = 0;
    201.         linksContainerArrayLengthNA[allLvlArrayNum]=1;
    202.         currentLinkedToTargetNA[0, 0] = true;
    203.         longCubesScaleNA[0] = new float3(0f, 0f, 0f);
    204.         longCubesPosNA[0] = new float3(0f, 0f, 0f);
    205.         longCubesRotNA[0] = Quaternion.identity;
    206.         instLongCubesNumNA[0, 0] = 0;
    207.         lineToNodesNA[0, 0] = 0;
    208.         lineToNodesNA[0, 1] = 0;
    209.         numberOfLinks++;
    210.         allLvlArrayNum++;
    211.     }
    212.  
    213.     void FillDistanceArray()
    214.     {
    215.         for (int i = 0; i < allLvlArrayNum; i++)
    216.         {
    217.             for (int j = 0; j < allLvlArrayNum; j++)
    218.             {
    219.                 float3 NodeItoJ = NodePositionsNA[j] - NodePositionsNA[i];
    220.                 DistanceBetweenNodesArraySqNA[i, j] = NodeItoJ.x*NodeItoJ.x+ NodeItoJ.y * NodeItoJ.y+ NodeItoJ.z * NodeItoJ.z;
    221.             }
    222.         }
    223.     }
    224.  
    225.     void CreateGrid()
    226.     {
    227.         CreateCenterNode();
    228.         //We don't use fully nested for loops since this way we are building the tree gradually, which makes it more evenly spread
    229.         //Level 1 - Node # = linksPerNode
    230.         for (int i = 0; i < linksPerNode; i++)
    231.         {
    232.             CreateLevel(0, i);
    233.         }
    234.         //Level 2 - Node # = linksPerNode^2
    235.         if (branchingLevels >= 2 && allLvlArrayNum < allArrayLength)
    236.         {
    237.             for (int i = 0; i < linksPerNode; i++)
    238.             {
    239.                 for (int j = 0; j < linksPerNode; j++)
    240.                 {
    241.                     CreateLevel(linksContainerArrayNA[0, i + 1], j);
    242.                 }
    243.             }
    244.         }
    245.         //Level 3 - Node # = linksPerNode^3
    246.         if (branchingLevels >= 3 && allLvlArrayNum < allArrayLength)
    247.         {
    248.             for (int i = 0; i < linksPerNode; i++)
    249.             {
    250.                 for (int j = 0; j < linksPerNode; j++)
    251.                 {
    252.                     for (int k = 0; k < linksPerNode; k++)
    253.                     {
    254.                         CreateLevel(linksContainerArrayNA[linksContainerArrayNA[0, i + 1], j + 1], k);
    255.                     }
    256.                 }
    257.             }
    258.         }
    259.  
    260.         //Level 4 - Node # = linksPerNode^4
    261.         if (branchingLevels >= 4 && allLvlArrayNum < allArrayLength)
    262.         {
    263.             for (int i = 0; i < linksPerNode; i++)
    264.             {
    265.                 for (int j = 0; j < linksPerNode; j++)
    266.                 {
    267.                     for (int k = 0; k < linksPerNode; k++)
    268.                     {
    269.                         for (int l = 0; l < linksPerNode; l++)
    270.                         {
    271.                             CreateLevel(linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[0, i + 1], j + 1], k + 1], l);
    272.                         }
    273.                     }
    274.                 }
    275.             }
    276.         }
    277.         //Level 5 - Node # = linksPerNode^5
    278.         if (branchingLevels >= 5 && allLvlArrayNum < allArrayLength)
    279.         {
    280.             for (int i = 0; i < linksPerNode; i++)
    281.             {
    282.                 for (int j = 0; j < linksPerNode; j++)
    283.                 {
    284.                     for (int k = 0; k < linksPerNode; k++)
    285.                     {
    286.                         for (int l = 0; l < linksPerNode; l++)
    287.                         {
    288.                             for (int m = 0; m < linksPerNode; m++)
    289.                             {
    290.                                 CreateLevel(linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[0, i + 1], j + 1], k + 1], l + 1], m);
    291.                             }
    292.                         }
    293.                     }
    294.                 }
    295.             }
    296.         }
    297.  
    298.         //Level 6 - Node # = linksPerNode^6
    299.         if (branchingLevels >= 6 && allLvlArrayNum < allArrayLength)
    300.         {
    301.             for (int i = 0; i < linksPerNode; i++)
    302.             {
    303.                 for (int j = 0; j < linksPerNode; j++)
    304.                 {
    305.                     for (int k = 0; k < linksPerNode; k++)
    306.                     {
    307.                         for (int l = 0; l < linksPerNode; l++)
    308.                         {
    309.                             for (int m = 0; m < linksPerNode; m++)
    310.                             {
    311.                                 for (int n = 0; n < linksPerNode; n++)
    312.                                 {
    313.                                     CreateLevel(linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[0, i + 1], j + 1], k + 1], l + 1], m + 1], n);
    314.                                 }
    315.                             }
    316.                         }
    317.                     }
    318.                 }
    319.             }
    320.         }
    321.  
    322.         //Level 7 - Node # = linksPerNode^7
    323.         if (branchingLevels >= 7 && allLvlArrayNum < allArrayLength)
    324.         {
    325.             for (int i = 0; i < linksPerNode; i++)
    326.             {
    327.                 for (int j = 0; j < linksPerNode; j++)
    328.                 {
    329.                     for (int k = 0; k < linksPerNode; k++)
    330.                     {
    331.                         for (int l = 0; l < linksPerNode; l++)
    332.                         {
    333.                             for (int m = 0; m < linksPerNode; m++)
    334.                             {
    335.                                 for (int n = 0; n < linksPerNode; n++)
    336.                                 {
    337.                                     for (int o = 0; o < linksPerNode; o++)
    338.                                     {
    339.                                         CreateLevel(linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[0, i + 1], j + 1], k + 1], l + 1], m + 1], n + 1], o);
    340.                                     }
    341.                                 }
    342.                             }
    343.                         }
    344.                     }
    345.                 }
    346.             }
    347.         }
    348.  
    349.         //Level 8 - Node # = linksPerNode^8
    350.         if (branchingLevels >= 8 && allLvlArrayNum < allArrayLength)
    351.         {
    352.             for (int i = 0; i < linksPerNode; i++)
    353.             {
    354.                 for (int j = 0; j < linksPerNode; j++)
    355.                 {
    356.                     for (int k = 0; k < linksPerNode; k++)
    357.                     {
    358.                         for (int l = 0; l < linksPerNode; l++)
    359.                         {
    360.                             for (int m = 0; m < linksPerNode; m++)
    361.                             {
    362.                                 for (int n = 0; n < linksPerNode; n++)
    363.                                 {
    364.                                     for (int o = 0; o < linksPerNode; o++)
    365.                                     {
    366.                                         for (int p = 0; p < linksPerNode; p++)
    367.                                         {
    368.                                             CreateLevel(linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[0, i + 1], j + 1], k + 1], l + 1], m + 1], n + 1], o + 1], p);
    369.                                         }
    370.                                     }
    371.                                 }
    372.                             }
    373.                         }
    374.                     }
    375.                 }
    376.             }
    377.         }
    378.  
    379.         //Level 9 - Node # = linksPerNode^9
    380.         if (branchingLevels >= 9 && allLvlArrayNum < allArrayLength)
    381.         {
    382.             for (int i = 0; i < linksPerNode; i++)
    383.             {
    384.                 for (int j = 0; j < linksPerNode; j++)
    385.                 {
    386.                     for (int k = 0; k < linksPerNode; k++)
    387.                     {
    388.                         for (int l = 0; l < linksPerNode; l++)
    389.                         {
    390.                             for (int m = 0; m < linksPerNode; m++)
    391.                             {
    392.                                 for (int n = 0; n < linksPerNode; n++)
    393.                                 {
    394.                                     for (int o = 0; o < linksPerNode; o++)
    395.                                     {
    396.                                         for (int p = 0; p < linksPerNode; p++)
    397.                                         {
    398.                                             for (int q = 0; q < linksPerNode; q++)
    399.                                             {
    400.                                                 CreateLevel(linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[0, i + 1], j + 1], k + 1], l + 1], m + 1], n + 1], o + 1], p + 1], q);
    401.                                             }
    402.                                         }
    403.                                     }
    404.                                 }
    405.                             }
    406.                         }
    407.                     }
    408.                 }
    409.             }
    410.         }
    411.         FillDistanceArray();
    412.     }
    413. }
    414.  
    Thanks in advance.
     
  15. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    4,271
    Alright. First, let's address that Random issue.

    So in classical Unity, Unity provides a Random number generator singleton that generates a new value every time you call for a value. That works well in the single-threaded paradigm of classical Unity. However, it falls flat on its face when multithreading in job system enters the game. So instead of creating a static math.Random, we want our Random to be another variable in the CreateGridJob. And then we initialize it with a new math.Random(seed).

    Now there's this magic "seed" variable that we have to deal with. Every random number generator needs a seed, and since we are creating a new generator for our job, we need a new one. There are many ways to generate them. We can either use UnityEngine.Random or we can use the current time.
    Code (CSharp):
    1. uint seed  = (uint)(System.DateTime.Now.Ticks & 0x00000000FFFFFFFF);
    2. uint seed2 = math.asuint(Random.Range(float.MinValue, float.MaxValue));
    For the quaternion, you want math.quaternion.LookRotationSafe(nodeToNode, Vector3.up)

    And yes, you should call Dispose() when you are done with the NativeContainers at the end of your generation. Unfortunately there is a bug in Unity where if you encounter an exception after allocating a NativeContainer Unity will keep issuing that warning message until you restart Unity. I have learned to just ignore it until I get my algorithm stable, and then restart Unity when it is convenient.

    There's still a ton of headroom for optimization, but let's get this code running first!
     
  16. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    Ugh, that was underwhelming... I got it to work now, but it actually made the grid creation time 2-3 times slower (i.e. around 18 sec). I even used the burst compiler. I don't know why and I have to go to bed now. Here is how the Profiler looks like at 1000 nodes https://drive.google.com/file/d/1l3zF76rFVqiusuEzKwIKDsEjYIrYSmFj/view?usp=sharing
    I also had an idea how to use IJobParallelFor to create the nodes for all children per level at the same time: have the existing grid for the positions of all parents that should spawn nodes at this level and all their parents' positions in a read-only array. Then have another read/write array for all the child nodes that should be created this level pre-populated with very distant positions and have the IJobParallelFor iterate through all the positions in the read-only array and the read/write array.
    Here is how the working struct looks like
    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4. using Unity.Collections;
    5. using Unity.Jobs;
    6. using UnityEngine.Jobs;
    7. using Unity.Mathematics;
    8. using Rand = Unity.Mathematics.Random;
    9. using System;
    10. using Unity.Burst;
    11.  
    12. public struct NativeArray2D<T> : IDisposable where T : struct
    13. {
    14.     private int xDim, yDim;
    15.     private NativeArray<T> data;
    16.  
    17.     public NativeArray2D(int x, int y, Allocator allocator)
    18.     {
    19.         xDim = x;
    20.         yDim = y;
    21.         data = new NativeArray<T>(x * y, allocator);
    22.     }
    23.  
    24.     public int CountX { get { return xDim; } }
    25.     public int CountY { get { return yDim; } }
    26.  
    27.     public T this[int x, int y]
    28.     {
    29.         get
    30.         {
    31.             return data[yDim * x + y];
    32.         }
    33.         set
    34.         {
    35.             data[yDim * x + y] = value;
    36.         }
    37.     }
    38.  
    39.     public void Dispose()
    40.     {
    41.         data.Dispose();
    42.     }
    43. }
    44.  
    45. [BurstCompile]
    46. public struct CreateGridJob : IJob
    47. {
    48.     public NativeArray<int> allLvlArrayNumNA;
    49.     public int allArrayLength;
    50.     public float3 newNodePos;
    51.     public float lineLength;
    52.     public float scaleGrid;
    53.     public int randomLoops;
    54.     public float extraDecrease;
    55.     public float minDistNode;
    56.     public float extraDecreaseStep;
    57.     public bool foundNodePos;
    58.     public float3 furtherstNewNodePos;
    59.     public int maxRandomLoops;
    60.     public float3 camPos;
    61.     public NativeArray<float3> NodePositionsNA;
    62.     public NativeArray<Quaternion> NodeRotationsNA;
    63.     public NativeArray<float3> NodeScalesNA;
    64.     public float3 camUp;
    65.     public float3 fixedNormalScale;
    66.     public NativeArray2D<int> linksContainerArrayNA;
    67.     public NativeArray<int> linksContainerArrayLengthNA;
    68.     public NativeArray2D<int> lineToNodesNA;
    69.     public NativeArray<int> numberOfLinksNA;
    70.     public NativeArray<float3> longCubesScaleNA;
    71.     public NativeArray<float3> longCubesPosNA;
    72.     public NativeArray<Quaternion> longCubesRotNA;
    73.     public NativeArray2D<int> currentLinkedToTargetNA;
    74.     public NativeArray2D<int> instLongCubesNumNA;
    75.     public float linkWidth;
    76.     public int maxLinksPerLinkedNode;
    77.     public int branchingLevels;
    78.     public int linksPerNode;
    79.     public float3 randVector3;
    80.     //static Rand rand;
    81.     public Rand rand;
    82.     public NativeArray<int> secondChanceNA;
    83.     public NativeArray2D<float> DistanceBetweenNodesArraySqNA;
    84.  
    85.     public void Execute()
    86.     {
    87.         CreateGrid();
    88.     }
    89.  
    90.     void CreateLevel(int parentNodeNum, int childrenNum)
    91.     {
    92.         if (allLvlArrayNumNA[0] < allArrayLength)
    93.         {
    94.             uint seed = (uint)(System.DateTime.Now.Ticks & 0x00000000FFFFFFFF);
    95.             rand = new Rand(seed);
    96.             randVector3 = rand.NextFloat3Direction();
    97.             float lineLengthScaled = lineLength * scaleGrid;
    98.             newNodePos = NodePositionsNA[parentNodeNum] + randVector3 * lineLengthScaled;
    99.             randomLoops = 0;
    100.             extraDecrease = 0;
    101.             float minDistScaled = minDistNode * scaleGrid;
    102.             float extraDecreaseStepScaled = extraDecreaseStep * scaleGrid;
    103.             float maxSqrMagnitude = 0f;
    104.             float minUnderThresholdSq = 100f;
    105.             int maxi = 0;
    106.             while (foundNodePos == false)
    107.             {
    108.                 foundNodePos = true;
    109.                 float minDist = minDistScaled - extraDecrease;
    110.                 float minDistSq = minDist * minDist;
    111.                 for (int i = 0; i < allLvlArrayNumNA[0]; i++)
    112.                 {
    113.                     //Discards the new node position if it is too close to other existing nodes.
    114.                     float3 newNodeToAllNodes = NodePositionsNA[i] - newNodePos;
    115.                     float curSqrMagnitude = newNodeToAllNodes.x* newNodeToAllNodes.x+ newNodeToAllNodes.y * newNodeToAllNodes.y+ newNodeToAllNodes.z * newNodeToAllNodes.z;
    116.                     float underThresholdSq = minDistSq - curSqrMagnitude;
    117.                     if (curSqrMagnitude < minDistSq)
    118.                     {
    119.                         if (underThresholdSq < minUnderThresholdSq && i > maxi)
    120.                         //if (underThresholdSq < minUnderThresholdSq)
    121.                         {
    122.                             minUnderThresholdSq = underThresholdSq;
    123.                             maxSqrMagnitude = curSqrMagnitude;
    124.                             furtherstNewNodePos = newNodePos;
    125.                             maxi = i;
    126.                         }
    127.                         foundNodePos = false;
    128.                         uint seed1 = (uint)(System.DateTime.Now.Ticks & 0x00000000FFFFFFFF);
    129.                         rand = new Rand(seed1);
    130.                         randVector3 = rand.NextFloat3Direction();
    131.                         newNodePos = NodePositionsNA[parentNodeNum] + randVector3 * lineLengthScaled;
    132.                         break;
    133.                     }
    134.                 }
    135.                 randomLoops++;
    136.                 if (randomLoops > maxRandomLoops && foundNodePos==false)
    137.                 {
    138.                     extraDecrease += extraDecreaseStepScaled;
    139.                     randomLoops = 0;
    140.                     minDist = minDistScaled - extraDecrease;
    141.                     minDistSq = minDist * minDist;
    142.                     if (maxSqrMagnitude > minDistSq)
    143.                     {
    144.                         foundNodePos = true;
    145.                         for (int i = 0; i < allLvlArrayNumNA[0]; i++)
    146.                         {
    147.                             //Checks the node that was closest to the threshold so far against the new threshold
    148.                             newNodePos = furtherstNewNodePos;
    149.                             float3 newNodeToAllNodes = NodePositionsNA[i] - newNodePos;
    150.                             float curSqrMagnitude = newNodeToAllNodes.x * newNodeToAllNodes.x + newNodeToAllNodes.y * newNodeToAllNodes.y + newNodeToAllNodes.z * newNodeToAllNodes.z;
    151.                             float underThresholdSq = minDistSq - curSqrMagnitude;
    152.                             if (curSqrMagnitude < minDistSq)
    153.                             {
    154.                                 if (underThresholdSq < minUnderThresholdSq && i > maxi)
    155.                                 {
    156.                                     minUnderThresholdSq = underThresholdSq;
    157.                                     maxSqrMagnitude = curSqrMagnitude;
    158.                                     maxi = i;
    159.                                 }
    160.                                 foundNodePos = false;
    161.                                 uint seed2 = (uint)(System.DateTime.Now.Ticks & 0x00000000FFFFFFFF);
    162.                                 rand = new Rand(seed2);
    163.                                 randVector3 = rand.NextFloat3Direction();
    164.                                 newNodePos = NodePositionsNA[parentNodeNum] + randVector3 * lineLengthScaled;
    165.                                 break;
    166.                             }
    167.                         }
    168.                     }
    169.                     if (foundNodePos)
    170.                     {
    171.                         secondChanceNA[0]++;
    172.                     }
    173.                 }
    174.             }
    175.             NodePositionsNA[allLvlArrayNumNA[0]] = newNodePos;
    176.             float3 nodeToCam = camPos - NodePositionsNA[allLvlArrayNumNA[0]];
    177.             NodeRotationsNA[allLvlArrayNumNA[0]] = quaternion.LookRotationSafe(nodeToCam, camUp);
    178.             NodeScalesNA[allLvlArrayNumNA[0]] = fixedNormalScale;
    179.             float3 nodeToNode = NodePositionsNA[parentNodeNum] - NodePositionsNA[allLvlArrayNumNA[0]];
    180.             float distBetweenNodes = math.sqrt(nodeToNode.x* nodeToNode.x+nodeToNode.y* nodeToNode.y+nodeToNode.z* nodeToNode.z);
    181.             float3 nodeMidPoint = NodePositionsNA[allLvlArrayNumNA[0]] + nodeToNode * 0.5f;
    182.             longCubesScaleNA[numberOfLinksNA[0]] = new float3(linkWidth, linkWidth, distBetweenNodes);
    183.             longCubesPosNA[numberOfLinksNA[0]] = nodeMidPoint;
    184.             //How is this lookrotation implemented in the new math library?
    185.             longCubesRotNA[numberOfLinksNA[0]] = quaternion.LookRotationSafe(nodeToNode, Vector3.up);
    186.             linksContainerArrayNA[parentNodeNum, childrenNum + 1] = allLvlArrayNumNA[0];
    187.             linksContainerArrayLengthNA[parentNodeNum]++;
    188.             currentLinkedToTargetNA[parentNodeNum, allLvlArrayNumNA[0]] = 1;
    189.             instLongCubesNumNA[parentNodeNum, childrenNum + 1] = numberOfLinksNA[0];
    190.             lineToNodesNA[numberOfLinksNA[0], 0] = parentNodeNum;
    191.             lineToNodesNA[numberOfLinksNA[0], 1] = allLvlArrayNumNA[0];
    192.             linksContainerArrayNA[allLvlArrayNumNA[0], 0] = parentNodeNum;
    193.             linksContainerArrayLengthNA[allLvlArrayNumNA[0]]++;
    194.             currentLinkedToTargetNA[allLvlArrayNumNA[0], parentNodeNum] = 1;
    195.             instLongCubesNumNA[allLvlArrayNumNA[0], 0] = numberOfLinksNA[0];
    196.             allLvlArrayNumNA[0]++;
    197.             foundNodePos = false;
    198.             numberOfLinksNA[0]++;
    199.         }
    200.     }
    201.  
    202.     void CreateCenterNode()
    203.     {
    204.         NodePositionsNA[0] = new float3(0f,0f,0f);
    205.         float3 nodeToCam = camPos - NodePositionsNA[0];
    206.         //can we use the math analogue to Quaternion.LookRotation?
    207.         NodeRotationsNA[allLvlArrayNumNA[0]] = quaternion.LookRotationSafe(nodeToCam, camUp);
    208.         NodeScalesNA[allLvlArrayNumNA[0]] = fixedNormalScale;
    209.         linksContainerArrayNA[0, 0] = 0;
    210.         linksContainerArrayLengthNA[allLvlArrayNumNA[0]]=1;
    211.         currentLinkedToTargetNA[0, 0] = 1;
    212.         longCubesScaleNA[0] = new float3(0f, 0f, 0f);
    213.         longCubesPosNA[0] = new float3(0f, 0f, 0f);
    214.         longCubesRotNA[0] = Quaternion.identity;
    215.         instLongCubesNumNA[0, 0] = 0;
    216.         lineToNodesNA[0, 0] = 0;
    217.         lineToNodesNA[0, 1] = 0;
    218.         numberOfLinksNA[0]++;
    219.         allLvlArrayNumNA[0]++;
    220.     }
    221.  
    222.     void FillDistanceArray()
    223.     {
    224.         for (int i = 0; i < allLvlArrayNumNA[0]; i++)
    225.         {
    226.             for (int j = 0; j < allLvlArrayNumNA[0]; j++)
    227.             {
    228.                 float3 NodeItoJ = NodePositionsNA[j] - NodePositionsNA[i];
    229.                 DistanceBetweenNodesArraySqNA[i, j] = NodeItoJ.x*NodeItoJ.x+ NodeItoJ.y * NodeItoJ.y+ NodeItoJ.z * NodeItoJ.z;
    230.             }
    231.         }
    232.     }
    233.  
    234.     void CreateGrid()
    235.     {
    236.         CreateCenterNode();
    237.         //We don't use fully nested for loops since this way we are building the tree gradually, which makes it more evenly spread
    238.         //Level 1 - Node # = linksPerNode
    239.         for (int i = 0; i < linksPerNode; i++)
    240.         {
    241.             CreateLevel(0, i);
    242.         }
    243.         //Level 2 - Node # = linksPerNode^2
    244.         if (branchingLevels >= 2 && allLvlArrayNumNA[0] < allArrayLength)
    245.         {
    246.             for (int i = 0; i < linksPerNode; i++)
    247.             {
    248.                 for (int j = 0; j < linksPerNode; j++)
    249.                 {
    250.                     CreateLevel(linksContainerArrayNA[0, i + 1], j);
    251.                 }
    252.             }
    253.         }
    254.         //Level 3 - Node # = linksPerNode^3
    255.         if (branchingLevels >= 3 && allLvlArrayNumNA[0] < allArrayLength)
    256.         {
    257.             for (int i = 0; i < linksPerNode; i++)
    258.             {
    259.                 for (int j = 0; j < linksPerNode; j++)
    260.                 {
    261.                     for (int k = 0; k < linksPerNode; k++)
    262.                     {
    263.                         CreateLevel(linksContainerArrayNA[linksContainerArrayNA[0, i + 1], j + 1], k);
    264.                     }
    265.                 }
    266.             }
    267.         }
    268.  
    269.         //Level 4 - Node # = linksPerNode^4
    270.         if (branchingLevels >= 4 && allLvlArrayNumNA[0] < allArrayLength)
    271.         {
    272.             for (int i = 0; i < linksPerNode; i++)
    273.             {
    274.                 for (int j = 0; j < linksPerNode; j++)
    275.                 {
    276.                     for (int k = 0; k < linksPerNode; k++)
    277.                     {
    278.                         for (int l = 0; l < linksPerNode; l++)
    279.                         {
    280.                             CreateLevel(linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[0, i + 1], j + 1], k + 1], l);
    281.                         }
    282.                     }
    283.                 }
    284.             }
    285.         }
    286.         //Level 5 - Node # = linksPerNode^5
    287.         if (branchingLevels >= 5 && allLvlArrayNumNA[0] < allArrayLength)
    288.         {
    289.             for (int i = 0; i < linksPerNode; i++)
    290.             {
    291.                 for (int j = 0; j < linksPerNode; j++)
    292.                 {
    293.                     for (int k = 0; k < linksPerNode; k++)
    294.                     {
    295.                         for (int l = 0; l < linksPerNode; l++)
    296.                         {
    297.                             for (int m = 0; m < linksPerNode; m++)
    298.                             {
    299.                                 CreateLevel(linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[0, i + 1], j + 1], k + 1], l + 1], m);
    300.                             }
    301.                         }
    302.                     }
    303.                 }
    304.             }
    305.         }
    306.  
    307.         //Level 6 - Node # = linksPerNode^6
    308.         if (branchingLevels >= 6 && allLvlArrayNumNA[0] < allArrayLength)
    309.         {
    310.             for (int i = 0; i < linksPerNode; i++)
    311.             {
    312.                 for (int j = 0; j < linksPerNode; j++)
    313.                 {
    314.                     for (int k = 0; k < linksPerNode; k++)
    315.                     {
    316.                         for (int l = 0; l < linksPerNode; l++)
    317.                         {
    318.                             for (int m = 0; m < linksPerNode; m++)
    319.                             {
    320.                                 for (int n = 0; n < linksPerNode; n++)
    321.                                 {
    322.                                     CreateLevel(linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[0, i + 1], j + 1], k + 1], l + 1], m + 1], n);
    323.                                 }
    324.                             }
    325.                         }
    326.                     }
    327.                 }
    328.             }
    329.         }
    330.  
    331.         //Level 7 - Node # = linksPerNode^7
    332.         if (branchingLevels >= 7 && allLvlArrayNumNA[0] < allArrayLength)
    333.         {
    334.             for (int i = 0; i < linksPerNode; i++)
    335.             {
    336.                 for (int j = 0; j < linksPerNode; j++)
    337.                 {
    338.                     for (int k = 0; k < linksPerNode; k++)
    339.                     {
    340.                         for (int l = 0; l < linksPerNode; l++)
    341.                         {
    342.                             for (int m = 0; m < linksPerNode; m++)
    343.                             {
    344.                                 for (int n = 0; n < linksPerNode; n++)
    345.                                 {
    346.                                     for (int o = 0; o < linksPerNode; o++)
    347.                                     {
    348.                                         CreateLevel(linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[0, i + 1], j + 1], k + 1], l + 1], m + 1], n + 1], o);
    349.                                     }
    350.                                 }
    351.                             }
    352.                         }
    353.                     }
    354.                 }
    355.             }
    356.         }
    357.  
    358.         //Level 8 - Node # = linksPerNode^8
    359.         if (branchingLevels >= 8 && allLvlArrayNumNA[0] < allArrayLength)
    360.         {
    361.             for (int i = 0; i < linksPerNode; i++)
    362.             {
    363.                 for (int j = 0; j < linksPerNode; j++)
    364.                 {
    365.                     for (int k = 0; k < linksPerNode; k++)
    366.                     {
    367.                         for (int l = 0; l < linksPerNode; l++)
    368.                         {
    369.                             for (int m = 0; m < linksPerNode; m++)
    370.                             {
    371.                                 for (int n = 0; n < linksPerNode; n++)
    372.                                 {
    373.                                     for (int o = 0; o < linksPerNode; o++)
    374.                                     {
    375.                                         for (int p = 0; p < linksPerNode; p++)
    376.                                         {
    377.                                             CreateLevel(linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[0, i + 1], j + 1], k + 1], l + 1], m + 1], n + 1], o + 1], p);
    378.                                         }
    379.                                     }
    380.                                 }
    381.                             }
    382.                         }
    383.                     }
    384.                 }
    385.             }
    386.         }
    387.  
    388.         //Level 9 - Node # = linksPerNode^9
    389.         if (branchingLevels >= 9 && allLvlArrayNumNA[0] < allArrayLength)
    390.         {
    391.             for (int i = 0; i < linksPerNode; i++)
    392.             {
    393.                 for (int j = 0; j < linksPerNode; j++)
    394.                 {
    395.                     for (int k = 0; k < linksPerNode; k++)
    396.                     {
    397.                         for (int l = 0; l < linksPerNode; l++)
    398.                         {
    399.                             for (int m = 0; m < linksPerNode; m++)
    400.                             {
    401.                                 for (int n = 0; n < linksPerNode; n++)
    402.                                 {
    403.                                     for (int o = 0; o < linksPerNode; o++)
    404.                                     {
    405.                                         for (int p = 0; p < linksPerNode; p++)
    406.                                         {
    407.                                             for (int q = 0; q < linksPerNode; q++)
    408.                                             {
    409.                                                 CreateLevel(linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[linksContainerArrayNA[0, i + 1], j + 1], k + 1], l + 1], m + 1], n + 1], o + 1], p + 1], q);
    410.                                             }
    411.                                         }
    412.                                     }
    413.                                 }
    414.                             }
    415.                         }
    416.                     }
    417.                 }
    418.             }
    419.         }
    420.         FillDistanceArray();
    421.     }
    422. }
    423.  
    And here is how the function that calls it looks like:
    Code (CSharp):
    1.     void CallCreateGridJob()
    2.     {
    3.         var createGridJob = new CreateGridJob()
    4.         {
    5.             allArrayLength = allArrayLength,
    6.             newNodePos = newNodePos,
    7.             lineLength = lineLength,
    8.             scaleGrid = scaleGrid,
    9.             randomLoops = randomLoops,
    10.             extraDecrease = extraDecrease,
    11.             minDistNode = minDistNode,
    12.             extraDecreaseStep = extraDecreaseStep,
    13.             foundNodePos = foundNodePos,
    14.             furtherstNewNodePos = furtherstNewNodePos,
    15.             maxRandomLoops = maxRandomLoops,
    16.             camPos = camPos,
    17.             NodePositionsNA = new NativeArray<float3>(allArrayLength, Allocator.TempJob),
    18.             NodeRotationsNA = new NativeArray<Quaternion>(NodeRotations, Allocator.TempJob),
    19.             NodeScalesNA = new NativeArray<float3>(allArrayLength, Allocator.TempJob),
    20.             camUp = camUp,
    21.             fixedNormalScale = fixedNormalScale,
    22.             linksContainerArrayNA = new NativeArray2D<int>(allArrayLength, maxLinksPerLinkedNode, Allocator.TempJob),
    23.             linksContainerArrayLengthNA = new NativeArray<int>(linksContainerArrayLength, Allocator.TempJob),
    24.             lineToNodesNA = new NativeArray2D<int>(maxLinks, 2, Allocator.TempJob),
    25.             numberOfLinksNA = new NativeArray<int>(1, Allocator.TempJob),
    26.             longCubesScaleNA = new NativeArray<float3>(maxLinks, Allocator.TempJob),
    27.             longCubesPosNA = new NativeArray<float3>(maxLinks, Allocator.TempJob),
    28.             longCubesRotNA = new NativeArray<Quaternion>(longCubesRot, Allocator.TempJob),
    29.             currentLinkedToTargetNA = new NativeArray2D<int>(allArrayLength, allArrayLength, Allocator.TempJob),
    30.             instLongCubesNumNA = new NativeArray2D<int>(allArrayLength, maxLinksPerLinkedNode, Allocator.TempJob),
    31.             DistanceBetweenNodesArraySqNA = new NativeArray2D<float>(allArrayLength, allArrayLength, Allocator.TempJob),
    32.             allLvlArrayNumNA = new NativeArray<int>(1, Allocator.TempJob),
    33.             linkWidth = linkWidth,
    34.             maxLinksPerLinkedNode = maxLinksPerLinkedNode,
    35.             branchingLevels = branchingLevels,
    36.             linksPerNode = linksPerNode,
    37.             secondChanceNA = new NativeArray<int>(1, Allocator.TempJob),
    38.         };
    39.  
    40.         createGridJob.allLvlArrayNumNA[0] = allLvlArrayNum;
    41.  
    42.         for (int i = 0; i < allArrayLength; i++)
    43.         {
    44.             createGridJob.NodePositionsNA[i] = NodePositions[i];
    45.             createGridJob.NodeScalesNA[i] = NodeScales[i];
    46.             for (int j = 0; j < maxLinksPerLinkedNode; j++)
    47.             {
    48.                 createGridJob.linksContainerArrayNA[i, j] = linksContainerArray[i, j];
    49.                 createGridJob.instLongCubesNumNA[i, j] = instLongCubesNum[i, j];
    50.             }
    51.             for (int k = 0; k < allArrayLength; k++)
    52.             {
    53.                 createGridJob.currentLinkedToTargetNA[i, k] = currentLinkedToTarget[i, k] == false ? 0 : 1;
    54.                 createGridJob.DistanceBetweenNodesArraySqNA[i, k] = DistanceBetweenNodesArraySq[i, k];
    55.             }
    56.         }
    57.  
    58.         for (int i = 0; i < maxLinks; i++)
    59.         {
    60.             createGridJob.lineToNodesNA[i, 0] = lineToNodes[i, 0];
    61.             createGridJob.lineToNodesNA[i, 1] = lineToNodes[i, 1];
    62.             createGridJob.longCubesScaleNA[i] = longCubesScale[i];
    63.             createGridJob.longCubesPosNA[i] = longCubesPos[i];
    64.             createGridJob.longCubesRotNA[i] = longCubesRot[i];
    65.         }
    66.  
    67.         JobHandle createGridJobHandle = createGridJob.Schedule();
    68.  
    69.         createGridJobHandle.Complete();
    70.  
    71.         NodeRotations = createGridJob.NodeRotationsNA.ToArray();
    72.         linksContainerArrayLength = createGridJob.linksContainerArrayLengthNA.ToArray();
    73.         longCubesRot = createGridJob.longCubesRotNA.ToArray();
    74.         for (int i = 0; i < allArrayLength; i++)
    75.         {
    76.             NodePositions[i] = createGridJob.NodePositionsNA[i];
    77.             NodeScales[i] = createGridJob.NodeScalesNA[i];
    78.             for (int j = 0; j < maxLinksPerLinkedNode; j++)
    79.             {
    80.                 linksContainerArray[i, j] = createGridJob.linksContainerArrayNA[i, j];
    81.                 instLongCubesNum[i, j] = createGridJob.instLongCubesNumNA[i, j];
    82.             }
    83.             for (int k = 0; k < allArrayLength; k++)
    84.             {
    85.                 currentLinkedToTarget[i, k] = createGridJob.currentLinkedToTargetNA[i, k] == 0 ? false : true;
    86.                 DistanceBetweenNodesArraySq[i, k] = createGridJob.DistanceBetweenNodesArraySqNA[i, k];
    87.             }
    88.         }
    89.  
    90.         for (int i = 0; i < maxLinks; i++)
    91.         {
    92.             lineToNodes[i, 0] = createGridJob.lineToNodesNA[i, 0];
    93.             lineToNodes[i, 1] = createGridJob.lineToNodesNA[i, 1];
    94.             longCubesScale[i] = createGridJob.longCubesScaleNA[i];
    95.             longCubesPos[i] = createGridJob.longCubesPosNA[i];
    96.             longCubesRot[i] = createGridJob.longCubesRotNA[i];
    97.         }
    98.  
    99.         allLvlArrayNum = createGridJob.allLvlArrayNumNA[0];
    100.         numberOfLinks = createGridJob.numberOfLinksNA[0];
    101.         secondChance = createGridJob.secondChanceNA[0];
    102.  
    103.         createGridJob.NodePositionsNA.Dispose();
    104.         createGridJob.NodeRotationsNA.Dispose();
    105.         createGridJob.NodeScalesNA.Dispose();
    106.         createGridJob.linksContainerArrayNA.Dispose();
    107.         createGridJob.linksContainerArrayLengthNA.Dispose();
    108.         createGridJob.instLongCubesNumNA.Dispose();
    109.         createGridJob.currentLinkedToTargetNA.Dispose();
    110.         createGridJob.DistanceBetweenNodesArraySqNA.Dispose();
    111.         createGridJob.lineToNodesNA.Dispose();
    112.         createGridJob.longCubesPosNA.Dispose();
    113.         createGridJob.longCubesScaleNA.Dispose();
    114.         createGridJob.longCubesRotNA.Dispose();
    115.         createGridJob.allLvlArrayNumNA.Dispose();
    116.         createGridJob.numberOfLinksNA.Dispose();
    117.         createGridJob.secondChanceNA.Dispose();
    118.     }
    119.  
    As I said, I think what I will try next is IJobParallelFor for all children per level.
    Regardless, even if I did not manage to improve the code's performance, I would have learned something new :)
     
  17. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,203
    are you sure that the code you profiled is actually running in burst? (In the profiler it should say "CreateGridJob(Burst)" in the profiler marker in the timeline view.

    It is highly unlikely that converting code from normal C# to burst will not get you any speedups.

    It is however quite possible for burst compilation to fail (and give errors) and thus for the code to execute as a normal mono JIT'd code as a fallback.
     
  18. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    4,271
    I am also suspicious you are not actually running in Burst. The profiler shows no indication of Burst. Granted, I know you are running on Android and I don't remember if that issue for showing "(Burst)" in the profiler on Android was fixed yet. But given how the profiler is stating the actual job takes just under 8 seconds, which was nearly identical to the results you had on the main thread, I think it is more than likely Burst is not running. Also, there is a 66 ms JIT compiler invocation right before the job is run, which I wouldn't expect to see with Burst.

    You also have about a full second's worth of actual game freeze time due to your setup and patchup work. Those are also worth optimizing. I think you are copying a lot of data you don't need to copy in your setup. And the rest doesn't need for loops. NativeArrays have constructors that take managed arrays and do a fast memcpy.

    But still, you just got your game from being non-interactive for 8 seconds to being non-interactive for 1 second. That's already a huge improvement! And there's still a ton of headroom to optimize!

    Lastly, I think you are overlooking the case where two threads could spawn a Node too close together. I know you mentioned having a pre-generated array of distant positions, but I am not sure how you would generate such an array to work with randomly placed parents.
     
  19. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    How do I know that and how do I know the reason and how to fix it?
    The Profiler data is from the Editor. I develop for the Oculus Go, so I have to compile the .apk and send it over to the Oculus Go manually and then run it from "Unknown Sources".
    This is in the Editor, in Oculus Go it takes around 18 seconds, which is really too long.
    It's true that I use the constructors only for:
    Code (CSharp):
    1. NodeRotationsNA = new NativeArray<Quaternion>(NodeRotations, Allocator.TempJob),
    2. linksContainerArrayLengthNA = new NativeArray<int>(linksContainerArrayLength, Allocator.TempJob),
    3. longCubesRotNA = new NativeArray<Quaternion>(longCubesRot, Allocator.TempJob),
    I was not able to use that constructor for the NativeArrays<float3> since they have to accept Vector3 input. I also could not use that constructor with NativeArray2D. In addition, I read that NativeArrays cannot accept <bool>, so I had to pass the bools of the MonoBehaviour arrays to ints in the NativeArray. I could not think of any other way than a for loop...
    Technically not yet, since I still invoke createGridJobHandle.Complete(); right after the scheduling, so I get the same type of freeze (only now it's 18 seconds instead of 7). I do plan to put it in a coroutine, but I would want first to get the grid creation time down to at least 7 seconds.
    This is only an idea ATM, but here is how I see it:
    • As you have seen, the algorithm builds the grid in levels, i.e. first all the children of the center node, then all the children of those nodes, then all their children and so on.
    • I think I can run a IJobParallelFor for all the children per level
    • The array of the positions of all existing nodes before the currently built level will be read-only
    • The temporary array for the positions of all the children to be created in the current level will be a separate read/write array with length the exact number of children to be spawned in that level. It will be pre-populated with very distant positions.
    • Each parallel job (for each child) will be able to write only at its index in the temporary array. However, it will read all the contents of the array of the positions of all existing nodes and the temporary array of currently spawning children.
    • It will then determine whether the new potential position is too close to any existing nodes, including the child nodes that have just been spawned in parallel. For the children that have not spawned yet, the test will always be passed since the positions at their index will be very far.
    • If the potential new position for that child passes the test, it will re-write the position at its index in the temporary array. Any later nodes that are just in the process of spawning will be able to see the updated position in their own test.
    • Once the job completes, I will add the temporary array to the array of all existing nodes
    But that is really for after we get the algorithm back to at least 7 sec...
     
    Last edited: Apr 30, 2019
  20. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    How do I see those errors? I am profiling all this in the Editor, since I develop for Oculus Go, which does not let me "Build and Run".
     
  21. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,203
    You should be getting compile error in the console window if burst fails to compile.
     
  22. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    If you want to make a random grid then why not change the algorithm so that it grows a grid, e.g. you could set up a 3d array of booleans and pick a random one then grow into empty directions. If you keep a perimeter list of all available grid cells outside of the selected blocks you just randomly choose from this and update your perimeter list.

    If you want the grid to look random you could distort the points e.g. scale the grid by 2x then randomly shift the points using a InsideUnitSphere offset.

    Pros:
    • Does not suffer from becoming locked in.
    • Will never bump into itself.
    • Should be super fast.
    Cons:
    • Limited to 3d array size.
    • Will have a grid like appearance.
     
  23. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    That's what I don't want. I want it to have a fake organic look to it (the algorithm then continues on to draw the branches between the parents and the children and then also some random links between the children). Sort of like a molecule strucure.
     
  24. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    4,271
    Oh! Well then your editor run is definitely not running Burst! That's probably because in the editor, Burst recompiles all jobs after every domain reload (every time you enter playmode) and by default it does so asynchronously based on what jobs are actually used. So a job will always run once without Burst before Burst compiles it and swaps the optimized code in. However, you can change this behavior by going into the Jobs menu and under Burst checking "Synchronous Compilation". Profile that in the editor to get a real ballpark estimate as to what kind of speedups you should be seeing on Oculus Go.

    As for why Burst isn't running on your Oculus Go, I'm unfortunately not the best person to ask. I mostly target PC.

    You're right. I misread the code. There's some things we can do about that. But I think getting Burst to work is highest priority. Just out of curiosity, are GC allocs a concern with this algorithm?

    That smells like a nasty race condition.
     
  25. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    I have no idea what GC allocs are and why they should be a concern. I only recently started with game development as a hobby, but I like learning so much new stuff and tinkering with my code. Sorry if I sound too ignorant on that one...
    I am not experienced at all in writing multithreaded code, but from what I read a race condition is an undesired effect of one process changing the input for another process. But in my case the changing of the input would actually be desirable, i.e. a node that is currently being spawned would need to determine whether there is another child node that was already spawned by a parallel process (its position would have been re-written at its index) or not (the position at that other index would be the distant one, so the test will always be passed for that index). In addition, every node would write only to its own index in the temporary array. No two processes would be able to write to the same index in the temporary array, but will only read from the full temporary array. I seem to remember reading somewhere that the Unity Job safety system allows reading in parallel, but not writing in parallel. But I may be totally wrong on that one.
     
  26. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    4,271
    We'll address the GC allocs later. You have some. And depending on how often you want this algorithm to run and whether or not you can use the incremental garbage collector, they might be a real problem. But we can address that issue later.

    As for the race condition, consider this scenario:
    Node A randomly generates Position A on Thread A.
    At the same time, Node B randomly generates Position B on Thread B.

    Positions A and B are nearly overlapping.

    Next, Thread A reads the position at RwIndex B and sees a far away value w.r.t. Position A.
    At the same time, Thread B reads the position at RwIndex A and sees a far away value w.r.t. Position B.

    Both Positions A and B are written, despite being incompatible with each other.

    You can't random read and index write on the same array in parallel safely without some extremely clever atomic operations. And even then, it would probably be much slower on the CPU due to cache line conflicts between CPU cores. On GPU, I would probably solve this issue using group syncs and ballots which only works due to enforcing lockstep execution. Mimicking that behavior or using a spatial partitioning are the only two methods I can foresee actually working in parallel.
     
  27. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    I see now what you mean, thanks for the explanation. However, what is meant by "At the same time"? The time between the logical check for the test and the actual writing into memory? I have almost no idea how a milti-core system accesses the memory.
    In 2-3 seconds (the target time for the algorithm) for a grid of 6 levels and 3 links per node (overall 1092 nodes), the last level will have 729 nodes being created at the same time (in fact only 4 since the Oculus Go has 4 cores in the SoC, I don't know whether the batch count that you assign in the .Schedule() method has any impact on this).
    So I think the writes might not be as frequent in order for there to be an overlap. And even if there is, the simultaneous writes might not be of two nodes that are really close to each other. The result would be nodes that might be closer than allowed, but if there are 2-3 cases for the whole grid, it would not be an issue, if that provides a 3-4 times speed-up.
    I will try that out just to test, but that will be after we get the single-job performance back to at least 7 sec.
     
  28. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    You were right, I did get the error message that Burst cannot use System.DateTime time with regard to this line in the IJob struct:
    Code (CSharp):
    1. uint seed = (uint)(System.DateTime.Now.Ticks & 0x00000000FFFFFFFF);
    This is the error message:
    Code (CSharp):
    1. (0,0): error: The managed function `System.TimeZoneInfo.GetUtcOffsetFromUtc(System.DateTime time, System.TimeZoneInfo zone, ref bool isDaylightSavings, ref bool isAmbiguousLocalDst)` is not supported by burst
    2.  
    3. at System.TimeZoneInfo.GetDateTimeNowUtcOffsetFromUtc(System.DateTime time, ref bool isAmbiguousLocalDst)
    4.  
    5. at System.DateTime.get_Now()
    6.  
    7. at CreateGridJob.CreateLevel(CreateGridJob* this, int parentNodeNum, int childrenNum) (at C:\UnityProjects\3DMinesVReeper\Assets\Scripts\CreateGridStruct.cs:165)
    8. at CreateGridJob.CreateGrid(CreateGridJob* this) (at C:\UnityProjects\3DMinesVReeper\Assets\Scripts\CreateGridStruct.cs:413)
    9. at CreateGridJob.Execute(CreateGridJob* this) (at C:\UnityProjects\3DMinesVReeper\Assets\Scripts\CreateGridStruct.cs:88)
    10. at Unity.Jobs.IJobExtensions.JobStruct`1<CreateGridJob>.Execute(ref CreateGridJob data, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, ref Unity.Jobs.LowLevel.Unsafe.JobRanges ranges, int jobIndex) (at C:\buildslave\unity\build\Runtime\Jobs\Managed\IJob.cs:30)
    11. While compiling job: System.Void Unity.Jobs.IJobExtensions/JobStruct`1<CreateGridJob>::Execute(T&,System.IntPtr,System.IntPtr,Unity.Jobs.LowLevel.Unsafe.JobRanges&,System.Int32)
    If I try the other line that you suggested:
    Code (CSharp):
    1. uint seed = math.asuint(UnityEngine.Random.Range(0, 1));
    I get another message:
    Code (CSharp):
    1. UnityException: RandomRangeInt can only be called from the main thread.
    2. Constructors and field initializers will be executed from the loading thread when loading a scene.
    3. Don't use this function in the constructor or field initializers, instead move initialization code to the Awake or Start function.
    4. UnityEngine.Random.Range (System.Int32 min, System.Int32 max) (at C:/buildslave/unity/build/Runtime/Export/Random/Random.bindings.cs:48)
    5. CreateGridJob.CreateLevel (System.Int32 parentNodeNum, System.Int32 childrenNum) (at Assets/Scripts/CreateGridStruct.cs:95)
    6. CreateGridJob.CreateGrid () (at Assets/Scripts/CreateGridStruct.cs:245)
    7. CreateGridJob.Execute () (at Assets/Scripts/CreateGridStruct.cs:88)
    8. Unity.Jobs.IJobExtensions+JobStruct`1[T].Execute (T& data, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, Unity.Jobs.LowLevel.Unsafe.JobRanges& ranges, System.Int32 jobIndex) (at C:/buildslave/unity/build/Runtime/Jobs/Managed/IJob.cs:30)
    I tried to have a
    Code (CSharp):
    1. public uint seed;
    2. ...
    3. rand = new Rand(seed);
    4. randVector3 = rand.NextFloat3Direction();
    5.  
    in the IJob struct
    and in the MonoBehaviour:
    Code (CSharp):
    1. var createGridJob = new CreateGridJob()
    2.         {...
    3.          seed = math.asuint(UnityEngine.Random.Range(1, 2)),
    or
    Code (CSharp):
    1. var createGridJob = new CreateGridJob()
    2.         {...
    3.          seed = (uint)(System.DateTime.Now.Ticks & 0x00000000FFFFFFFF),
    Unity crashes when I hit play. I thought that last one should work, since that is how I have seen deltaTIme being passed from the main thread to a job. I am out of ideas.
    EDIT: I even tried
    In the IJob struct:
    Code (CSharp):
    1. public float randSeed;
    2. ...
    3. uint seed = math.asuint(randSeed);
    4. rand = new Rand(seed);
    5.  
    and in the MonoBehaviour:
    Code (CSharp):
    1. var createGridJob = new CreateGridJob()
    2. {
    3. ...
    4. randSeed = UnityEngine.Random.Range(1, 2),
    or
    Code (CSharp):
    1. var createGridJob = new CreateGridJob()
    2. {
    3. ...
    4. randSeed = System.DateTime.Now.Ticks & 0x00000000FFFFFFFF,
    and Unity crashes again
     
    Last edited: Apr 30, 2019
  29. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    Thanks, I did get this error message:
    Code (CSharp):
    1. (0,0): error: The managed function `System.TimeZoneInfo.get_Local()` is not supported by burst
    2.  
    3. at System.TimeZoneInfo.GetDateTimeNowUtcOffsetFromUtc(System.DateTime time, ref bool isAmbiguousLocalDst)
    4.  
    5. at System.DateTime.get_Now()
    6.  
    7. at CreateGridJob.CreateLevel(CreateGridJob* this, int parentNodeNum, int childrenNum) (at C:\UnityProjects\3DMinesVReeper\Assets\Scripts\CreateGridStruct.cs:165)
    8. at CreateGridJob.CreateGrid(CreateGridJob* this) (at C:\UnityProjects\3DMinesVReeper\Assets\Scripts\CreateGridStruct.cs:413)
    9. at CreateGridJob.Execute(CreateGridJob* this) (at C:\UnityProjects\3DMinesVReeper\Assets\Scripts\CreateGridStruct.cs:88)
    10. at Unity.Jobs.IJobExtensions.JobStruct`1<CreateGridJob>.Execute(ref CreateGridJob data, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, ref Unity.Jobs.LowLevel.Unsafe.JobRanges ranges, int jobIndex) (at C:\buildslave\unity\build\Runtime\Jobs\Managed\IJob.cs:30)
    11. While compiling job: System.Void Unity.Jobs.IJobExtensions/JobStruct`1<CreateGridJob>::Execute(T&,System.IntPtr,System.IntPtr,Unity.Jobs.LowLevel.Unsafe.JobRanges&,System.Int32)
     
  30. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    4,271
    Threads A and B are running at the same time on two different cores, so at the exact same time (real life time) that thread A is generating position A, thread B is generating position B. And due to the way memory latency and caching work, they don't have to have to be executing all instructions at exactly the same clock cycles for this issue to come up as there can be several clock cycles waiting on memory after the read check but before the write.

    Again, I wasn't looking close enough at the code yesterday and this morning to catch it, but you need to seed rand outside the job and assign it in CallCreateGridJob similar to what you are doing with the native containers.
     
  31. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    I think that's what I do:
    In the IJob struct:
    Code (CSharp):
    1. public long randSeed;
    2. ...
    3. void CreateLevel(int parentNodeNum, int childrenNum)
    4. {
    5.     if (allLvlArrayNumNA[0] < allArrayLength)
    6.     {
    7.         uint seed = (uint)(randSeed);
    8.         rand = new Rand(seed);
    9.         randVector3 = rand.NextFloat3Direction();
    And in the MonoBehaviour:
    Code (CSharp):
    1. var createGridJob = new CreateGridJob()
    2. {
    3.     randSeed= System.DateTime.Now.Ticks & 0x00000000FFFFFFFF,
    But for the resulting random vector I get always the same. I think it's because the seed does not update once I assign it to the job, so the job always uses the same seed. Here is the console log for the randVector3:
    https://i.imgur.com/8LmKgBQ.jpg
     
  32. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    4,271
    1. Is Burst working for you yet?
    2. Instead of a public long randSeed, just make a public Rand rand. And then when creating the job instance, use
    rand = new Rand(System.DateTime.Now.Ticks & 0x00000000FFFFFFFF),

    This will make each level use different random values since rand will be in scope during the entire duration of the job.
    3. Usually people create a new instance of a job struct every time they schedule one. If you do that, you would probably realize that you give the job a new seed every time you schedule it, which is what you need to do. You can give it a new seed by giving it a new Rand using the code above.
     
  33. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    Thanks I lot, I got it working now with Burst. Here is the Profiler data:
    https://drive.google.com/file/d/19rgO3Xr9JyadlsRq0d-F16JVm4ggmdCb/view?usp=sharing
    The grid creation frame took only 3 sec, out of which is JitCompilerService.CompileInternal(), which themselves consist of GC.Alloc - overall 2 sec. What is most impressive is the actual job thread takes 242ms!
    The bad news is that the burst compiler does not seem to work on the Oculus Go hardware, since I still get around 16 sec for the grid creation on the Oculus Go, whereas with MonoBehaviour I get around 7 sec. I will write above to the person from Unity that responded to ask for further details.
    You mentioned that we can further reduce the grid creation time by addressing GC allocs...
    EDIT: Got it working on Oculus Go! I had to untick and re-tick the Android SDK and NDK tick boxes in Preferences. Now the grid gets created in 3 seconds! Awesome :)
     
    Last edited: May 1, 2019
  34. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    I got it working with the burst compiler and got really good results in the Editor:
    https://drive.google.com/file/d/19rgO3Xr9JyadlsRq0d-F16JVm4ggmdCb/view?usp=sharing
    However, this does not seem to translate to my target hardware - Oculus Go. Is Oculus Go supported by the burst compiler at all? Thanks in advance for clarifying.
    EDIT: Got it working on Oculus Go! I had to untick and re-tick the Android SDK and NDK tick boxes in Preferences.
     
    Last edited: May 1, 2019
  35. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    4,271
    Hey! That's awesome!

    So I'll be honest, I have no idea what's going on with your JIT compilation in the editor. That's not something I've seen really on my machine.

    But as far as optimizations go moving forward, there's several different areas we can optimize now, but to do that, we're going to have to profile how long each step is taking so that we can figure out what to tackle first. To do that, you are either going to have to get Unity's profiler working with the Oculus Go or log time elapses of the different sections of code to a text file. Doesn't have to be super accurate, but getting a better idea of what parts are actually slow on the hardware is going to help you reach your goal of 2 interactive seconds of grid creation without freezes.

    Potential areas to analyze:
    • Job Generation
      • Allocating Native Containers
      • Copying data from existing arrays (you may need to explain to me where this data is coming from)
    • The Job
      • More Burst optimizations
      • This is probably not the issue anymore since it doesn't freeze the game once you put it in a coroutine, but it might be fun to make it faster anyways
    • Job Cleanup
      • Copying data back without type conversion
      • Copying data back with type conversion
      • GC Concerns (Most of these can be resolved with NativeArray's CopyTo method instead of ToArray)
    • Instantiation
      • I'm thinking you are going to want to use some sort of pooling mechanism for this
     
  36. Singtaa

    Singtaa

    Joined:
    Dec 14, 2010
    Posts:
    492
    Apologies that I didn't read all the posts. But from what I read I think your algorithm can be improved by utilizing Poisson Disc Sampling. It generates natural looking scatter and is O(n).
     
  37. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    Thanks, I had a look on Google and it seems quite interesting!
     
  38. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    Thanks for all the good advice so far! There is actually another important optimisation of the grid creation: I have a second algorithm that after building of the primary tree, then connects some of the closer nodes to each other so the whole structure looks more like a molecule than a tree branching in all directions. I will give that a shot myself, but I don't know if the burst compiler supports raycasting. I use raycasting in that second algorithm.
    On your questions below:
    As far as I tried and researched, it is not possible the "Build and Run" for the Oculus Go. On logging time elapses, I can try that, but can you actually get Time.time from within a job that is running with the burst compiler?
    I can profile that with Deep Profile, I guess, since it is in the MonoBehaviour. For most of the data coming from the existing arrays to the Native Containers I would not actually need to transfer the data, since it's basically zeros. I will remove the filling in of most of the Native Containers.
    The Job in burst takes only around 22ms, but I would be curious to learn new tips and tricks :)
    I do need to copy a lot of data back to MonoBehaviour Arrays and I do .ToArray() only for 3 out of 14 arrays since most of the others are NativeArray2D or float3 arrays. The currentLinkedToTargetNA[] and DistanceBetweenNodesArraySqNA[] are the largest ones, with maximum size 2000*2000 (allArrayLength=2000), but the nested filling for loops would need to be only as long as the actually created links allLvlArrayNumNA[0]*allLvlArrayNumNA[0] (which is 1093*1093 for 6 levels and 3 children per parent). So I will change all references to allArrayLength in the for loops to allLvlArrayNumNA[0].
    You are right, I do instantiate a lot of GameObjects. I do use batched instanced rendering for rendering the nodes, so I don't need GameObjects for that, but I need SphereColliders for the nodes. I read that the new version of ECS supports SphereColliders, so if I manage to implement the SphereColliders with ECS, the instantiation would be much faster.
    Besides the SphereColliders, one of the options also use a TextMesh Pro element in the prefab, so for that option I also have a TMP element in the prefab. I understood that TMP does not even support hybrid ECS. I do plan to try with quads instead of TMP elements to see if that provides good viewing quality. If so, I can have batched instanced rendering for the quads instead.
    About pooling of the nodes, that might be an idea, but the issue is that I don't know how many nodes would be spawned until the player chooses the number of branching levels and the number of children per branch and clicks Play.
    But this is definitely something that I will look into.
    Thanks again for all your help, I feel I have learned so much about the Unity's Job System already :)
     
  39. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    You could build the grid using a space filling polyhedra then distort it with random offsets this would give a non grid like structure but also allow for a very fast pre-generated node network.


    https://en.wikipedia.org/wiki/Gyrobifastigium


    Or what if you 'prefab' random grid sections that can be linked together and distorted allowing for a massive speed up in searching for a node as it would be pre-calculated?
     
    Last edited: May 2, 2019
  40. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    Thanks, looks interesting, will have a look.
     
  41. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    4,271
    Three options:
    1. Use RaycastCommands (only works with first hit despite the API suggesting otherwise
    2. Use Unity.Physics (requires quite a bit of plumbing to work in your use case)
    3. Build your own raycasting solution
    If your raycasts are primitive types like spheres and boxes, option 3 will probably be the fastest to get working. I've been building my own custom collision engine that handles some bizarre use cases using ECS and jobs, so I can help you out with the query algorithms if you need it.

    Wrapping your timing markers around the .Schedule and .Complete calls should be sufficient enough.

    Oculus Go may exhibit very different behavior. I was mostly asking how those arrays were initialized anyways.

    If CopyTo is significantly faster than the for loops on Oculus Go, then there's an easy solution. Working with a [,] array might be a bit of a pain, but we'll give it a shot anyways.

    Any Component on a GameObject can be processed in ECS using fluent queries. But if the text is predetermined, you could try grabbing the mesh and throwing it on a Render Mesh with the right material.

    Anyways, get some timing info on the Oculus Go and I am willing to help you address any other major performance issues.
     
  42. Arowx

    Arowx

    Joined:
    Nov 12, 2009
    Posts:
    8,194
    For those working on a Burst ECS solution could I suggest approaching the problem from the outside in...

    The current approach seems to be starting at a given point then expanding randomly outwards to produce a network of nodes in a random pattern. The underlying problem here is the probability of hitting a random overlap.

    What if you start with a random outline/surface then draw points on it at random range intervals then work inwards.*

    Potential advantages of in outside in approach is you can divide the outline/surface up into lots of smaller jobs/threads.

    Also internal inward points could be generated from face/line normals with a random offset (this also ensure each inward step has one less point). With a pass to merge any nodes too close together.

    In theory it should massively reduce the random/walk/collisions and allow for massive parallelism in proportion to the size of the point cloud.

    * The outline could be generated with a random walk with limited arc turns, or a more perfect shape that is distorted with noise or set of shapes combined with a fill algorithm used to find the outline.
     
  43. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    I agree that the current algorithm is heavy and discards too many random shots, while the gradual relaxation of the threshold has the risk of the grid collapsing on itself if it is too big, unless we drastically increase the random loops before each iterative relaxation of the threshold. It works quite well for smaller grids (e.g. around 1000 nodes), but it gets worse as the tree levels and children per level increase. But for my purpose, 1000 nodes is more than enough for the type of game I am thinking of. Also, learning the Unity's job system in order to speed it up is quite interesting.
    I received above a suggestion based on Poisson Disc Sampling. I will definitely investigate that. I think it's quite similar to what I want to achieve, but in the meantime I will continue optimising my original algorithm, since it gives me some challenge to learn how the job system and the burst compiler work.
     
  44. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    Thanks, That's what I will be doing in the coming days.
    In parallel, I thought about the issue of the raycasting for creating the secondary grid. I use it to avoid linking to a node that would intersect another node, like this:
    https://i.imgur.com/UCW9p7v.png

    In my algorithm, if node Y is within the maximum "reach" of a link from A, before accepting that link I do a raycast from node A to node Y and check if the first collider that the ray intersects with is the collider of Y. If it is not, I discard that link.
    I think it should not be that hard to work that out in the code based on the 2 vectors X and Y and the radius of the node.
    I put together that quickly:
    Code (CSharp):
    1. bool Y_Intersects_X(Vector3 Y, Vector3 X, float nodeRadius)
    2. {
    3.     Vector3 VectProjX=Vector3.Project(X,Y);
    4.     Vector3 OrthVectProjX= VectProjX-X;
    5.     if (Vector3.Dot(Vector3.Normalize(Y), Vector3.Normalize(VectProjX))==1 && Y.sqrMagnitude> X.sqrMagnitude) && OrthVectProjX.sqrMagnitude < nodeRadius.sqrMagnitude)
    6.     {
    7.         return true;
    8.     }
    9.     else
    10.     {
    11.         return false;
    12.     }
    13. }
    14.  
     
    Last edited: May 2, 2019
  45. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    5,203
    I am curious what the speed up from Original MonoBehaviour style code -> ECS / Burst / Jobified code was?
     
  46. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    4,271
    In your editor profile, your raycasts are only taking 1 ms. Unless you are trying to completely escape GameObjects altogether (tough with the TMP dependency), this doesn't seem like a problem worth optimizing yet.

    I don't think ECS is involved in this project. But both Jobs and Burst are showing their strength.

    This thread has caught a lot of attention, so I will summarize where everything is at since I have the time:

    This is a procedural generation algorithm which generates 1000 nodes using a guess-and-check approach that exhibited a desirable output. It also generates links between these nodes and then finally instantiates objects. This procedurally generated content is consumed by the rest of the game which uses MonoBehaviours and C# arrays.

    The full algorithm took about 7 seconds in both the editor and on Oculus Go (the target platform) using MonoBehaviours.

    With jobs and without Burst, the algorithm inflated to 9 seconds in the editor and 16 seconds on Oculus Go. In the editor, this happened because there is a large amount of data copied in and out of the NativeArrays before and after the job. On Oculus Go, my best guess was that neon was trying to run managed code in the job. Second best guess is that NativeContainers are just slow on ARM without Burst.

    With Burst, the node generation part of the algorithm went from about 8 seconds down to 242 ms!
    On Oculus Go, the full runtime of the algorithm dropped to 3 seconds!

    These numbers come from the profiler samples uploaded to this thread or @konstantin_lozev 's claims where profiler data is not available.

    There is still quite a bit of optimization left on the table, especially in regards to setting up and tearing down the job.
     
  47. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    Update from this evening: by eliminating the initial filling up of the NativeArrays and making them only as long as necessary (linksPerNode^branchingLevels+linksPerNode^(branchingLevels-1)+linksPerNode^(branchingLevels-2)...) now the full creation time is 2 sec for a 1000 grid!
     
    Last edited: May 3, 2019
  48. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    What I was most astonished was the difference that the Burst Compiler made to the running of the Job on the Oculus Go. It basically brought the grid generation from 7 seconds (no Job, only MonoBehaviour) down to 2 seconds (Job and Burst). I did not even have to go through trying to parallelise the algorithm into Jobs for each node (using IJobParallelFor)! So it's basically a 1:1 comparison of the same algorithm. I understood that parallelising the algorithm would have been a challenge since the generation of each node depends on the position of other nodes generated in parallel too. I am really happy with the result. Awesome job guys!
     
    FROS7 likes this.
  49. konstantin_lozev

    konstantin_lozev

    Joined:
    May 8, 2015
    Posts:
    99
    Here are the timings. I used System.Diagnostics.Stopwatch.
    With Jobs and Burst:
    Code (CSharp):
    1. CreateGrid_useJobs=True   00:00:01.4183260;  
    2. AssignMaterials()_PlaceNodes()   00:00:00.6560952;  
    3. CreateSecondaryGrid()_PlantMines()_AssignAdjMines()   00:00:00.0983083;    
    As you can see, the jobified grid creation takes less than 1.5 sec on the Oculus Go now. The next one is 0.65 sec that involves the instantiations of the spherecolliders. The third one at only 0.09 sec is the creation of the secondary grid.
    For comparison, here are the results for the same script in MonoBehaviour:
    Code (CSharp):
    1. CreateGrid_useJobs=False   00:00:11.3291720;  
    2. AssignMaterials()_PlaceNodes()   00:00:00.6201963;  
    3. CreateSecondaryGrid()_PlantMines()_AssignAdjMines()   00:00:00.0974781;
    Pretty impressive!
     
    DreamingImLatios likes this.
  50. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    4,271
    Really awesome!

    Looks like the memcpy after the job only takes 36 ms. It's up to you if you want to try to reach 100% interactive or just stuff everything in a coroutine with a 0.6 second faded black-out and call it a day.

    If you do decide to optimize further, besides slowly building a GameObject pool while your job is running, you could also try running another set of jobs to convert float3 into Vector3 and see if the implicit casts without Burst are wasting CPU cycles.
     
    konstantin_lozev likes this.