Search Unity

  1. Unity Asset Manager is now available in public beta. Try it out now and join the conversation here in the forums.
    Dismiss Notice
  2. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  3. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Official Introducing the new batch TransformPoint and Gizmo line drawing APIs arriving in 2023.1

Discussion in '2023.1 Beta' started by JohnW-Unity, Nov 23, 2022.

  1. JohnW-Unity

    JohnW-Unity

    Unity Technologies

    Joined:
    Jan 10, 2018
    Posts:
    1
    This is a heads-up to spread the word about a couple of new APIs that landed 2023.1.0a18 and provide the opportunity to significantly improve the performance of scripts that need to transform large numbers of points/lines or that draw many lines using the existing Gizmos.DrawLine() API.

    These new APIs deliver better performance by allowing arbitrarily sized batches of points/lines to be transformed or batches of lines to be drawn with a single API call drastically reducing the overhead involved in repeatedly marshalling data between and transitioning from managed user code (C#) to native Unity engine code (C++).

    The new APIs fall into two categories:

    Batch Transform APIs

    There are many situations when working with Unity where you have a requirement to convert points or vectors from one space to another - usually from the local space of a game object to world space or vice versa. The usual way to do this is with the family of TransformXXX and InverseTransformXXX functions found in the Transform class:
    Each of these functions takes a single Vector3, transforms it and returns the result which makes them very simple to use but as each API call made incurs an overhead marshalling data and transitioning from managed to native code they are also significantly inefficient when working on larger data sets as they have to be called repeatedly on each point/direction/vector in turn.

    From 2023.1 this inefficiency is largely eliminated with batch versions of each of these functions that allow you to transform as many points/directions/vectors as necessary with a single API call so the overhead is only incurred once:
    Each of these can take a single Span<Vector3> containing the source points/directions/vectors each of which will be overwritten with the transformed versions or they can take a ReadOnlySpan<Vector3> for the source data and a separate Span<Vector3> to receive the transformed versions leaving the source data intact (assuming the two spans don't overlap).

    Performance Improvement

    As part of the development of these new APIs, we did of course write performance tests and the results really speak for themselves: transforming one million points with the new batch APIs resulted in performance ranging from 16x the original speed to just over 150x!

    upload_2022-11-23_12-38-55.png

    Note though that you don't need to be transforming thousands of points to get a win here. While the existing APIs are fine for single instances, if you have even a few points to transform, the odds are it will be faster to use the batch APIs with the difference increasing as the size of your data set grows.

    Batch Gizmo Line Drawing APIs

    Also in 2023.1 are two new batch APIs for the Gizmos class extending its current line drawing abilities to allow multiple lines to be drawn in a single API call. Currently, for drawing simple lines the Gizmos class offers the DrawLine() API
    this API simply takes two Vector3 instances representing the beginning and end of the line to draw and does exactly that. However, in a manner identical to the Transform APIs described above, if you need to draw a large number of lines (maybe a curve made up of line segments for example or some other complex gizmo visualization), you need to repeatedly call it with each pair of endpoints incurring the data marshalling and managed/native transition overhead cost each time.

    To avoid this the new APIs in 2023.1 give you a better option in this situation:
    both of these APIs take a Span<Vector3> containing a sequence of line endpoints and draw multiple lines using them. The former DrawLineList() draws a single line between each pair of points allowing a sequence of disconnected lines to be drawn while the latter DrawLineStrip() draws a connected sequence of lines between each point in turn (so a line from point[0] to point[1], then another from point[1] to point[2] and so on) and also takes an optional flag to have an additional line drawn between the last point and the first to "close" the loop.

    Performance Improvement

    Again, as you would expect, we ran performance tests for the new APIs and the results are clear:

    upload_2022-11-23_12-39-6.png

    Here we can see it's about 14x faster to use the batch APIs purely due to the reduction in API call overhead, and again you don't need to have massive data sets to benefit. If you have even a few lines to draw the odds are it will be faster to use the new batch APIs than the existing single line one.

    Final Words

    This post was written to help highlight these new APIs that are available in 2023.1 as they provide such an enormous performance gain in the common situation where larger data sets are being worked on. Work is underway to backport them to the 2022 stream in time for LTS so they should be available there in due course to further expand their utility.
     

    Attached Files:

  2. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,225
    An API that speed things up without requiring us to completely change the way we program?

    *This* is the perfect model for high performance programming.
     
  3. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,294
    This is great!

    Any chance to look into batching for gizmo cubes or spheres? We have had some situations over the years where we had to draw a lot of those to debug eg. mesh generation or other things with a lot of data, and performance often tanked unless we did extra work to manually cull.
     
  4. IgorBoyko

    IgorBoyko

    Joined:
    Sep 28, 2020
    Posts:
    90
    I would also like to address something similar to suggested batch transform operations.

    Say, I want to get transform position, rotation, up, right, forward, so I just write:

    Code (CSharp):
    1. var position = transform.position;
    2. var rotation = transform.rotation;
    3. var up = transform.up;
    4. var right = transform.right;
    5. var forward = transform.forward;
    Seems kinda reasonable? But in fact, that's a lot of back-and-forth jumps between native and managed code, and most of it can be mitigated by doing manual calculations of up/right/forward vectors:

    Code (CSharp):
    1. var position = transform.position;
    2. var rotation = transform.rotation;
    3. var up = rotation * Vector3.up;
    4. var right = rotation * Vector3.right;
    5. var forward = rotation * Vector3.forward;
    So we get 3 less expensive operations. In larger count of such calculations, done every frame, it's a significant difference. Though it's not so obvious for most people.

    What I would like to suggest is to introduce some sort of "intermediate" structure, which gathers all important information from the transform in a single managed-to-native-back-to-managed call, and which has properties/extensions to calculate other useful variables. For example, such a struct might contain world and local matrices for the transform, and position + rotation + etc could be just calculated on demand without going back to managed code.

    This was just an example of solution. Main goal is "reduce overhead from marshaling and going native for repeated transform operations".
     
  5. Baste

    Baste

    Joined:
    Jan 24, 2013
    Posts:
    6,294
    I mean, we're almost there. We can get the Matrix4x4 localToWorld from a transform. It just needs a setter.
     
  6. TheOtherMonarch

    TheOtherMonarch

    Joined:
    Jul 28, 2012
    Posts:
    862
    A batched version of DrawSphere would be greatly appreciated.
     
    chadfranklin47 and DevDunk like this.
  7. aras-p

    aras-p

    Joined:
    Feb 17, 2022
    Posts:
    74
    Very cool!

    It's excellent that Span<T> as input is making an appearance. Someone(tm) should go over most existing APIs that work on arrays (and often have... or don't have, overloads for T[], List<T>, NativeArray<T>), and make them also take a Span.
     
  8. jbooth

    jbooth

    Joined:
    Jan 6, 2014
    Posts:
    5,461
    Nice Speedup! Kind of wondering about the naming convention though - since these are on transforms, and thus TransformVector is always going to be local to world, why not call them TransformLocalToWorldVector instead? I'd understand the generic naming if these were on Matrix4x4, since it could represent any space conversion, but these are explicit..
     
    laurentlavigne likes this.
  9. Peter77

    Peter77

    QA Jesus

    Joined:
    Jun 12, 2013
    Posts:
    6,591
    Is the engine itself making use of the new API too? Can we expect performance improvements on the engine side due to the new API?

    Being able to issue Gizmo calls outside of the OnGizmoSomething magic methods would be very useful also.
     
  10. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,225
    Yes, expanding Debug.Draw*
     
  11. Ofx360

    Ofx360

    Joined:
    Apr 30, 2013
    Posts:
    155
    This is great! Batch more things!

    Also having more Gizmos options would be great. Circles, discs, hemispheres, capsules, cones, arrows, text(?!!) etc would be nice to have out of the box within Gizmo.
     
  12. Slashbot64

    Slashbot64

    Joined:
    Jun 15, 2020
    Posts:
    313
    Does Gizmos.DrawLine() sill only work in editor and not game view?should be an option to change that
     
  13. John_Leorid

    John_Leorid

    Joined:
    Nov 5, 2012
    Posts:
    646
    Are Gizmos drawn the very moment we call the method (e.g.
    Gizmos.DrawLine()
    )? I thought the next step would be on the backend, not changing any of our code, just improving performance by silently building those lists.

    Gizmos is already a static class, somewhere inside Unity OnDrawGizmos and OnDrawGizmosSelected is called, I thought it would be like this:

    -> Call to OnDrawGizmos
    -> GameDev calls things like
    Gizmos.DrawLine()

    -> those calls end up filling lists in the Gizmos Static Class
    -> Once all OnDrawGizmos Methods were executed, the drawing beginns
    -> Draw internally with Batched Methods
    -> Clear lists
    -> Repeat

    Maybe I'm missing something here?
     
  14. aras-p

    aras-p

    Joined:
    Feb 17, 2022
    Posts:
    74
    Yes, the actual drawing of the gizmos is already similar to how you outlined: they are added into a bunch of "gizmo batches" etc.

    But! The explicit batched API avoids the pure overhead of doing a ton of tiny Gizmos.DrawLine functions. It has to take the data, marshal that into C++, append into these batch arrays, get back into C# land, etc. It's not a huge overhead, but if you are drawing a ton of lines, it all adds up.
     
  15. Per-Morten

    Per-Morten

    Joined:
    Aug 23, 2019
    Posts:
    119
    Always nice to get more batched API's so please keep them coming :)
     
    chadfranklin47 likes this.
  16. Kamyker

    Kamyker

    Joined:
    May 14, 2013
    Posts:
    1,085
    What about jobs and using them as dependency for https://docs.unity3d.com/2021.3/Documentation/ScriptReference/Jobs.IJobParallelForTransform.html? Feels like all batch Transform.Transform should be jobs and return JobHandle, no?
     
  17. Kamyker

    Kamyker

    Joined:
    May 14, 2013
    Posts:
    1,085
    I think what @Hannibal_Leo meant is that Gizmos.DrawLine could just fill up internal buffer in C# land and batch it automatically by Unity when needed.

    In other words code like:
    Code (CSharp):
    1. void OnDrawGizmosSelected()
    2. {
    3.     Gizmos.color = Color.yellow;
    4.     Gizmos.DrawSphere(transform.position, 1);
    5. }
    DrawSphere on Unity C# side:
    Code (CSharp):
    1. // old
    2. // public static void DrawSphere(Vector3 center, float radius) => Gizmos.DrawSphere_Injected(ref center, radius);
    3. static List<DrawSphereCommand> drawSphereCommands;
    4. public static void DrawSphere(Vector3 center, float radius)
    5. {
    6.   drawSphereCommands.Add(new () //structs
    7.   {
    8.     center = center,
    9.     radius = radius,
    10.     color = Gizmos.color,
    11.     matrix = Gizmos.matrix,
    12.     exposure = Gizmos.exposure
    13.   }
    14. }
    You can spot a problem that Gizmos.color etc call to native again but they also could be moved to C# side! This may look a bit heavy but it's really not. When using something like Gizmos.DrawLineList() you also have to add to your own list manually (or dozens of lists, one per OnDrawGizmos()).

    If copying "color, matrix, exposure" for every draw would make real difference in benchmarks then new DrawSphereList could be added with different List of Lists buffer.

    We end up with:
    - all old api being very fast
    - new batch apis even faster than 2023.1
     
    mariandev and laurentlavigne like this.
  18. John_Leorid

    John_Leorid

    Joined:
    Nov 5, 2012
    Posts:
    646
    Yes that's what I meant. ^^
     
  19. Kamyker

    Kamyker

    Joined:
    May 14, 2013
    Posts:
    1,085
    Was looking for a way to draw lines also in GameView and built game. Didn't manage to get CommandBuffers working but GL. It turns out there's sort of batch api that's internal:

    Could you add public api with spans?

    I got it working successfully using reflection:
    Code (CSharp):
    1. Material lineMaterial = new Material(Shader.Find("Unlit/Color"));
    2. lineMaterial.SetPass(0);
    3. GL.Begin(GL.LINES);
    4. {
    5.     GL.Color(Color.white);
    6.     fixed(Vector3* ptr = &points.AsReadOnly().AsReadOnlySpan().GetPinnableReference())
    7.         verticesDel.Invoke(ptr, (Vector3*)null, (Vector4*)null, points.Length);
    8. }
    9. GL.End();
    #Edit
    I have to say I prefer using GL batch api as it also supports vertex color. Here's a spiral traversal of points generated on NavMesh:
     
    Last edited: Jan 9, 2023
  20. jRocket

    jRocket

    Joined:
    Jul 12, 2012
    Posts:
    699
    The Gizmos API needs to be able to be called outside of on OnDrawGizmos. Either that or make Debug.DrawRay/Debug.DrawLine batched as well. I am already using a library that draws spheres and other shapes with Debug.DrawLine, but it is slow.
     
  21. Kamyker

    Kamyker

    Joined:
    May 14, 2013
    Posts:
    1,085
    Yep, I've searched a bit more for a better way and ended up using:
    Code (CSharp):
    1. Graphics.DrawMesh(lineMesh, Matrix4x4.identity, linesMaterial, 1);
    Seems the most flexible but not as easy to use as GL/Gizmos/Debug.
     
    Last edited: Jan 21, 2023
  22. John_Leorid

    John_Leorid

    Joined:
    Nov 5, 2012
    Posts:
    646
    Same, usually with custom shaders. But only in cases that are really complicated, like visualizing Trigger-Box-Intersections (or the decal-like variant "what is currently inside the trigger"). Tho, in 99% of all cases the DebugDrawingExtensions are good enough:
    https://assetstore.unity.com/packages/tools/debug-drawing-extension-11396
     
    mariandev likes this.
  23. TheCelt

    TheCelt

    Joined:
    Feb 27, 2013
    Posts:
    741
    Is this using jobs system in the background then returning the results to transform?
     
  24. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,225
    triiiiiiippy

    What's the benefit of spans in non jobs?
     
  25. Kamyker

    Kamyker

    Joined:
    May 14, 2013
    Posts:
    1,085
    Both managed array and NativeArray can be used as Span making copying less likely.

     
  26. laurentlavigne

    laurentlavigne

    Joined:
    Aug 16, 2012
    Posts:
    6,225
    You have an example of with and without span, in Unity?
     
  27. Gotmachine

    Gotmachine

    Joined:
    Sep 26, 2019
    Posts:
    34
    It would be really nice to also have APIs for setting (and retrieving) transforms position/rotation in batch, ie :
    Code (CSharp):
    1. // world pos/rot getters/setters
    2. Transform.GetPositions(Span<Transform> transforms, Span<Vector3> positions)
    3. Transform.GetRotations(Span<Transform> transforms, Span<Quaternion> rotations)
    4. Transform.GetPositionsAndRotations(Span<Transform> transforms, Span<Vector3> positions, Span<Quaternion> rotations)
    5. Transform.SetPositions(Span<Transform> transforms, Span<Vector3> positions)
    6. Transform.SetRotations(Span<Transform> transforms, Span<Quaternion> rotations)
    7. Transform.SetPositionsAndRotations(Span<Transform> transforms, Span<Vector3> positions, Span<Quaternion> rotations)
    8. // local pos/rot getters/setters
    9. Transform.GetLocalPositions(Span<Transform> transforms, Span<Vector3> positions)
    10. Transform.GetLocalRotations(Span<Transform> transforms, Span<Quaternion> rotations)
    11. Transform.GetLocalPositionsAndRotations(Span<Transform> transforms, Span<Vector3> positions, Span<Quaternion> rotations)
    12. Transform.SetLocalPositions(Span<Transform> transforms, Span<Vector3> positions)
    13. Transform.SetLocalRotations(Span<Transform> transforms, Span<Quaternion> rotations)
    14. Transform.SetLocalPositionsAndRotations(Span<Transform> transforms, Span<Vector3> positions, Span<Quaternion> rotations)
    15. // Couldn't hurt to have those too
    16. Transform.GetWorldToLocalMatrices(Span<Transform> transforms, Span<Matrix4x4> worldToLocalMatrices)
    17. Transform.GetLocalToWorldMatrices(Span<Transform> transforms, Span<Matrix4x4> localToWorldMatrices)
    There are many use cases for such batch operations (for ex, floating origin shifts, mass update of unit position, spatial queries...), and there as well, the native calls overhead quickly adds up to unsustainable levels, plus I would hope doing such operations in batch would open some optimization opportunities on the native side.
     
    Last edited: Sep 3, 2023
    chadfranklin47 and IgorBoyko like this.