Official Introducing the Vector API in Unity 2022.1

This will be a brief overview of the new UI Toolkit Vector API that recently landed in Unity 2022.1

First, a bit of (mesh generation) context:
UI Toolkit already provides the Mesh API which allows drawing custom content into a VisualElement. However, while everything is possible, it is hard to use. The code required to generate a single quad has to allocate vertices and indices, and provide coordinates in a clockwise direction. There is a few way it could not work as expected and it not the easiest thing to debug.

Code (CSharp):

1. void OnGenerateVisualContent(MeshGenerationContext mgc)
2.             {
3.         var mesh = mgc.Allocate(4, 6);
4.         mesh.SetNextVertex(new Vertex() { position = new Vector3(p0.x, p0.y, Vertex.nearZ), tint = wireColor});
5.         mesh.SetNextVertex(new Vertex() { position = new Vector3(p1.x, p1.y, Vertex.nearZ), tint = wireColor});
6.         mesh.SetNextVertex(new Vertex() { position = new Vector3(p2.x, p2.y, Vertex.nearZ), tint = wireColor});
7.         mesh.SetNextVertex(new Vertex() { position = new Vector3(p3.x, p3.y, Vertex.nearZ), tint = wireColor});
8.  
9.         mesh.SetNextIndex(0);
10.         mesh.SetNextIndex(1);
11.         mesh.SetNextIndex(2);
12.         mesh.SetNextIndex(0);
13.         mesh.SetNextIndex(2);
14.         mesh.SetNextIndex(3);
15.             }

Overall, this is a tool for power users, but most users only want to generate very simple geometry, so this is unfortunate.


Vector API
Inspired by HTML Canvas, the new API allow C# programmer to build at a higher level, expressing directly their intention. The API provide easy to use methods for

• Lines
• Arcs
• Beziers
• Fills
• Strokes
• and more!

​

This is the same example using the new Vector APIs.

Code (CSharp):

1. void OnGenerateVisualContent(MeshGenerationContext mgc)
2.             {
3.         var paint2D = mgc.painter2D;
4.  
5.         paint2D.fillColor = wireColor;
6.         paint2D.BeginPath();
7.         paint2D.MoveTo(p0);
8.         paint2D.LineTo(p1);
9.         paint2D.LineTo(p2);
10.         paint2D.LineTo(p3);
11.         paint2D.ClosePath();
12.         paint2D.Fill();
13.             }

To build paths, you use the painter2D object in the context and issue commands that moves a kind of virtual pen. So here, we move the pen to the first position, and chain lines together to build a quad.
Once the path is done, you can either Fill it or Stroke it.

This remove the need to do many manual tasks as the allocation and the overall management of the data is abstracted, and the process is much more scalable for more complex shapes.

When doing some visual, doing a quad is the basic, but this API allows doing munch more than that using the different methods (LineTo, Arc, ArcTo, BezierCurveTo) and their arguments.

Another Example Setting the Color, Caps and Joins
Setting the color, the way segments are connected, the width of the line is easy with a one liner addition to an existing path. You can use the following options for the connections :

Code (CSharp):

1.  painter.strokeColor = Color.white;
2.         painter.lineJoin = LineJoin.Round;
3.         painter.lineCap = LineCap.Round;
4.         painter.lineWidth = 10.0f;
5.         painter.BeginPath();
6.         painter.MoveTo(new Vector2(100, 100));
7.        painter.LineTo(new Vector2(120, 120));
8.        painter.LineTo(new Vector2(140, 100));
9.        painter.LineTo(new Vector2(160, 120));
10.        painter.LineTo(new Vector2(180, 100));
11.        painter.LineTo(new Vector2(200, 120));
12.        painter.LineTo(new Vector2(220, 100));
13.         painter.Stroke();

With these tool combined, complex path can be achieved:





Fill the shapes:
The path previously defined can also be filled.

Code (CSharp):

1. paint2D.fillColor = Color.Blue;
2. paint2D.BeginPath();
3. paint2D.Arc(new Vector2(100,100) 50.0f, 0.0f, 360.0f);
4. paint2D.Fill();

Holes:
Holes can be created by using the an argument with

paint2D.Fill

We support two fill rules: odd-even and non-zero.

• When using odd-even, new inner segments will toggle if the shape is filled or not.
• When using non-zero, the fill is toggled when crossing a shape with a different winding order.

You can read about those fill rules on the web, this is well documented.




Technical:
The systems generate all the geometry, but it also allows the rendering to use arcs with an infinite precision without having an infinitely complex geometry.


Here is a zoom on the result. We can see some color compression artifact when the gif was generated, but no vertices.



The anti-aliasing used on these curves is the same as what UI toolkit uses for the rounded corners of the visual elements.


Not implemented:
We don't have any option for line patterns, gradients, texturing yet.
The anti-aliasing does not work on self intersecting path.

Final Words
We hope this will enable new uses cases for some projects, and we aim at collecting even more feedback to improve the tool in the future. Lets us know your thought and questions!

 

If your geometry is static, I would recommend an external SVG editor where you would be more comfortable to edit the shapes. You can then import the same in unity with the SVG importer package and assign the result in the UI Builder. The svg importer does not yet support the "infinite resolution curves" but let's hope that it won't take much time now that we have the backend in place

This programming API is more interesting if you need dynamic elements in c#. These elements could then be use in the builder like any other elements. One example is the connectors in the VFX/Shader graph that are generated dynamically depending on the position of the elements.

 

The shape-with-hole has to happen within the same path (that is, you define the shape and the hole between BeginPath() and Fill()). So, you would have to regenerate the hole shape at every frame.

 

No idea, but certainly looks like a bug. We'll get that fixed pronto.

 

There's no way to draw an SVG shape at this time, but we will work on the SVG importer to make it compatible with the Vector API. I can't provide an ETA at this time, however.

 

Correct, this is 2022.1+. This won't be backported.

 

Performance is good for generating simple geometry as far I as I could test. You should not see any overhead compared to building a mesh "by hand". It is probably not ready to support a custom particle system with thousands of elements updated at every frame on mobile, but I don't think it would be doable with any generic framework (you may need a more manual approach or this) If you have any use case that you think would push the limit, don't hesitate to share with us!

 

This API is meant to create geometry that leverage the rendering features of UI toolkit, such as anti-aliasing, arc rendering and vertex data. Because of this, the implementation is closely tied to the rendering implementation, so they are developed together. The Vector Graphics Package will use that API to make even better scalable graphics when importing for UI toolkit if everything goes according to plan

You are probably looking for the new (and experimental) Spline Package. As the UI toolkit Vector API is meant to create the vertex data that the shader used by UI Toolkit requires, it is close to useless in a 3D environment as there is no depth information, no occlusion against opaque geometry etc. possible with that shader... Shapes is a great library, better suited for some usages as it is lighter, but it could not scale to the complexity of the UI that we are targeting.

 

It works with both

 

A performance fix is coming soon for 2022.1 which may give a 10x-20x performance boost, depending what you are drawing.

That said, if you can share your code that generates these drawings we could investigate to see if this isn't hiding some other problem.

In 2022.2 you will be able to instantiate a new Painter2D any where an start accumulating vector primitives in it. This is not yet landed in the 2022.2 branch, but it is coming soon. This can't be done in 2022.1 unfortunately.

 

Looking deeper into this, this particular use-case is problematic. We unfortunately cannot append new sub-paths to another path that is already rendered with Stroke(). So the alternative is what you are doing: rebuild the whole path from scratch.

This works for simple geometries, but for shapes that may grow indefinitely, the cost of generating an increasingly complex strokes will get slower overtime, and consume more and more memory. (Your examples grows to a few MBs of vertex data after a few seconds).

We will look to see if we can find a way to keep the path data alive to allow for more efficient appends, but in the meantime it would be preferable to avoid this situation.

I can think of valid use-cases where this situation may arise (a painting program, for example), but these would probably benefit from being baked into a texture instead of being regenerated from scratch. We have plans to provide code samples that does just that.

 

We'll have a look at ways to improve the performance for this use case. As it is right now, paths cannot be modified once Stroke()/Fill() are called, meaning that we cannot go back a modify already existing geometry.

This is exactly it. With a MoveTo() before each segments, there is no need for joins between sub-paths. This will be particularly efficient with a "Butt" line ending.

Thanks again for sharing.

 

Thanks for your suggestion, we will definitely consider them.

I think the color must be set before the

BeginPath

as we cannnot change the color along the path:

ex

Code (CSharp):

1.         var painter = ctx.painter2D;
2.         painter.strokeColor = Color.white;
3.         painter.lineWidth = propertyTools.lineWidth;
4.         painter.lineCap = propertyTools.lineCap;      
5.         painter.lineWidth = 2.0f;
6.  
7.         painter.BeginPath();
8.         painter.MoveTo(vertices[0] + offset);
9.         foreach (var v in vertices)
10.             painter.LineTo(v + offset);
11.         painter.LineTo(vertices[0] + offset);
12.         painter.Stroke();

 

Not yet, we are definitely looking into enabling that but for now the built-in borders+backgrounds+masking is active on all elements.

 

Would it be possible to share more details about this? I'd like to know if there's a bug in that use-case.

 

Yes we are because we intend on using the Vector API to render the imported SVG on the long term and that the SVG do support gradient. We are still under planning/consideration for this, so there is no ETA yet available.

Unlike SVG and HTML`s canvas, the team was tempted to add a "gradient along path" for the stroke option instead of just being able to do uniform gradient. The handling of the corner cases is probably manageable, but that feature may well be impossible with the current tessellation algorithm we developed. Would that divergence with the standard be something useful? (the current alternative that we need to implement for SVG support anyway is to do multiples small segment with their own gradient/color, which would require more vertices)

 

We are aiming to provide the SVG features on the long term, (look for the svg support item in the roadmap) and that include texturing. No ETA available for now. I think it will be broken down in smaller features and gradients will probably be prioritized before texturing.

 

There is a lot of performance improvements in 2022.2 compared to 2022.1, most of them are 10x.

The vector API essentially generates the geometry for the UI Toolkit mesh API, so unless you are doing really simple mesh, the performance should be better with the vector API because we already optimized the work.

I have no comparison with the regular/3d mesh object

 

There is no plan to do so right now, you can possibly suggest your idea to the https://portal.productboard.com/xvo8kmgtx7tkv7desh8cyaj7/tabs/49-ui-design for consideration by the product team.

That being said, I don't think it will be prioritized because we already have means to achieve that currently.

• You can use an SVG editor to create the shape wanted, and then use the shape in UI toolkit
• You can programmatically create your geometry using the vector API

External SVG editor will provide a much better workflow for creating most shapes compared to the build that concentrate on organizing the hierarchy.

For circle, you could set the rounded corners on a square visual element to 50% and it will appear as a circle.

 

To use a SVG image as a background for a visual element or in the Image component (from c# only), you must install the package. To do so, in the Package Manager window, install the package com.unity.vectorgraphics from git URL.

With the package installed, I think the importer option were self-explanatory, and you should be good to go. The package does not currently use the Vector API from UITK, so there is no AA on the generated geometry, but we want to make this happen on the long term.

 

It would be 2023.1 at minimum.

 

Yes, I think that should be as simple as changing a dropdown or a checkbox and applying the setting.