Feedback Wanted: Mesh scripting API improvements

Do you plan to add utility function like SharedToUnique Vertices, UniqueToShared, Extrude, Quad based Mesh construction, Constructing meshes from parametric curves, stitching meshes together or similar things? Over time I wrote lots of these things myself but having a foundation of a bit more high-level tools to create procedural meshes would be appreciated by a lot of users I think.

 

ya you are right, would make sense to have that as separate package, I think probuilder actually already has some of these things but the documentation there is rather thin.

 

Hi, Do you have a plan to improve Mesh.CombineMeshes() method? In my case, I manually combined meshes into single mesh for UV adjusting, bone index modification and so on. This is commonly need for Texture Atlasing and combining skinned mesh.

It will be very helpful if CombineMeshes have UV offset (+scaling) and bone index remapping features.​

 

The proposed changes all look good. But I can't help returning to an old bugbear of mine with mesh drawing in Unity. Please add the ability to have meshes with no index buffer and a property for specifying the topology for the draw command. It's frustrating that you can do everything you want with DrawProcedural, but that it requires binding ComputeBuffers which aren't available on all targets.

 

This looks like a major step forward.

One thing I'd wonder about is having multiple streams as a replacement for additionalVertexStreams. I used that in my vertex painter, and while it works it has a number of annoying bugs and limitations (always need to set position even if your not overriding it, breaks batching, needs to be reset in some cases). Something that can be serialized in the scene would be ideal, as right now you have to have a component which stores it and copies it to the object at runtime (which prevents the mesh from being read only, breaking batching, and then requiring a separate baking pass). Ideally the data gets intertwined at some point automatically and everything 'just works'.

There's also the question of if it makes sense to get away from additional UV sets as a way to say "additional user data". While every engine seems to use this terminology, it seems a little silly.

 

It’s should be reposted to DOTS thread

 

We also need a way to specify one submesh to render instead of entire mesh with all submeshes.
I thinks it must be part of MeshFilter component in old system and all draw api must support to draw exact submesh instead of entire mesh.

Unity already have this inside static batching but it is not exposed to us.

This is like mesh atlasing that allow to have one mesh and effectively draw different parts of it.

 

Am I right that I can declare position as byte2 (not 2 bytes per component but 2 bytes for entire position ) with this api?

 

Yay, custom vertex layouts would be great! Right now I have to awkwardly pack everything to UVs.
jbooth also made a good point about additionalVertexStreams.

 

Another tangential thing is physics; a project I’m working on now uses jobs to generate lots of mesh data, copying the native arrays to arrays via unsafe code. It works well, but the fundamental bottleneck is by far updating physics representations. I can process dozens of times more meshes than I can update physics on, which causes a lot of complexity in amortizing that work.

 

I'd be happy with properties that accept NativeArray instead of managed array. We're making procedural meshes using ECS but the problem right now is only managed arrays are allowed. We need to do an extra step of copying the contents of the NativeArray into a managed array.

Code (CSharp):

1. mesh.nativeVertices = ...;
2. mesh.nativeUvs = ...;
3. mesh.nativeColors = ...;

 

I am personally interested with Mesh Buffers usability over computer shaders. I'd love to use it. I just have a question. Does this changes will require us to use Entities ?

 

>VertexAttributeDescriptor[] GetVertexAttributes();
Can we get a non-allocating alternative for this? Like a "VertexAttributesCount" and "GetVertexAttribute(int index)", or something filling a List<>

>void SetVertexBufferParams(int vertexCount, params VertexAttributeDescriptor[] attributes);
Can this have the slice-like overloads as well?

These may be planned already, but they're not mentioned in the doc.

P.S: Current use case time: I'm creating mesh data on a dedicated .net thread, writing into a static allochglobal buffer. Then when the size is known, I Marshal.AllocHGlobal() a new buffer that's used to temp hold the data while sending it to the main thread. There I copy the data to a static List<> as that's the only one I can reasonably use with the current API. This copying is fun in itself, as I'm pinning the internal array of the List<> gotten with reflection, getting the pointer, and doing a memcpy from said malloc'd buffer into the List<>'s backing array. Then I got to set the List<>'s count as well with reflection. This turned out to be noticably quicker then simply copying data over using List<>.Clear() and List<>Add(). And I pack 2 int16 values into the color channel.

So I may be a happy guy when I can scratch all that for a burst job generating a few NativeArray<>'s and calling this new API

 

Please add a replacement or fix for Unwrapping.GenerateSecondaryUVSet since this is broken for 32bit indexing. And I may be mistaken but I believe we need serialization of the mesh object to allow for undo/redo functionality. I would also suggest moving towards a format that contains adjacency information like a winged edge data structure or similar.

 

It does work in 2019.1.0f2 but I get an IndexOutOfRangeException when using Unwrapping.GenerateSecondaryUVSet in the LTS version 2017.4.25f1. This is caused by the triangle indices into the vertex array having been changed to random values like 1048591 for a mesh with only 60 vertices for example.

 

"It primarily has issues with performance, too much extra memory copying, memory allocation"​

I'm running into the exact same problems (copying data for no reason, because the API demands it ... and having to jump through hoops to prevent GC allocations that should never have happened in the first place) with the 2D Mesh manipulation APIs (i.e. VertexHelper - which is (correct me if I'm wrong) currently required to create meshes for UnityUI?).

Will this update also fix/replace the VertexHelper class with the same type of improvements? (VH is almost a great class, but the fact it forces you to copy data to/from low-performance List's on every frame is painful).

 

On the 10th of May, my timing is either very good or very bad. Thanks, glad to hear it is now rectified.

 

I like the proposed changes so far. Especially overloads for NativeArrays is very much welcome at the moment.

Something which isn't made very clear in the document however, is whether this will allow us to reference the Mesh object directly from within jobs? From peeking in the source code with a decompiler, the mesh class appears to be a thin wrapper around the native object. Will the Mesh class be turned into a blittable struct such that it can be used in jobs, or do we still need to post-process our calculations in the main thread in order to apply them to the mesh?

My current approach to this, due to the lacking API is as follows:
Create a MeshData struct that contains NativeLists for vertices, indices, uvs, normals etc.
Create jobs that generate the mesh data. The Job has a field of type MeshData and modifies its NativeLists.
On the mainthread, check and wait for the job to complete (polling IsCompleted and/or calling Complete() on the last job in the dependency chain). After job completion, copy the NativeList contents into a managed array and passing that to the Mesh API.

With the changes described in the document, the last step (copy contents from NativeList to managed array) won't be necessary anymore. But what about the other steps? For me, especially having to synchronize with the mainthread by calling Complete() is a bottleneck I would be glad to see removed.

 

A core problem with the mesh API is that modifying a mesh at run time is a blocking operation. Vertex data can be built on either seperate threads or the job system, but ultimately you are bottlenecked by the part where you have to assign the data to the mesh to update it since this has to be done on the main thread. The workaround I use is time slicing where I allocate time per frame to process a queue of meshes that need updating.

What I have always wanted is for the entire process of updating a mesh to be completely non-blocking. Either allow us to do it all on the job system or alternatively introduce a thread safe variant of the Mesh class where you can lock/unlock it for writing.

 

from the document

"Direct NativeArray access to mesh vertex/index buffers, without a memcpy incurred by SetVertexBufferData/SetIndexBufferData. That would probably work similar to how NativeArray Texture2D.GetRawTextureData works."

You can with this pre create the mesh (if you know the size) and manipulate the content memory direct in a job.
But I guess you still need to flag the mesh to be uploaded to the GPU again on the main thread.

 

Yep, I meant to say "A core problem with the current mesh API". NativeArray access to the mesh buffers would allow us to take more of the task off the main thread which is great. I am hoping for the entire task of updating a mesh to be completely non-blocking so that theres no requirement to do any sort of main thread operation (such as flagging the mesh for upload).

 

It would be nice if we could directly manipulate the mesh data (including adding and removing vertices) in separate thread/job, and then mark the mesh to be updated on the main thread. Right now we have to make copy of the vertex buffer and assign it back to the mesh in order to manipulate the mesh on a different thread.

 

Is GPU submission in Unity still all on the main thread? Wasn't there graphics jobs already?

 

Please add a method for extracting a submesh, that works properly with skinned meshes!

 

Can we get an option to specify mesh colours(https://docs.unity3d.com/ScriptReference/Mesh-colors.html) without normalization? For... nefarious data encoding purposes.

Currently it clamps input if it exceeds 1. I work around it by using Color32 instead, but that's just because fortunately I can get by with the 0-255 range.

Never? Maybe?

 

You can use UVs instead, and there are currently 8 sets of UVs which are up to float4, so quite a bit of data to play with.

 

That doesn't seem to be the case. What I do in C#:

Code (CSharp):

1. Mesh msh = new Mesh();
2. Color[] clr_test = new Color[vertexCount];
3. clr_test[0].a = 2f;
4. msh.colors = clr_test;

What I do in the shader:

Code (CSharp):

1. float instanceID = v.color.w; // clamped
2. //float instanceID = v.color.w * 255.0; // what I use for color32

Based on visual shader debugging I can see that with regular colors any value above 1 arrives to the shader clamped. I'm not sure why that happens, I just assumed Unity does that somewhere along the way and I just rolled with it. But I would be greateful if you managed to pull an explanation of how to get above 1.

 

In the google doc Jonas Bötel mentions adding NativeSlice overloads. How about just replacing all NativeArrays in parameters with NativeSlices? They are basically the same and NativeArray implicitely converts to NativeSlice.

Span<T> is even more universal type. If it were supported, it could replace both native and managed arrays and also their slices.

 

This is great! Thanks for tackling this.
I don't see instancing support and consider this important.
I want to make sure all multiple stream things can be done like the hardware can do it.
The API should not try to abstract too much, I rather have different layouts for different platforms than a system that tries to guess a good layout for me.

I wonder if this all happens on the main thread and the thread talking to the API (render thread) makes it needed to add a copy. I hope this can be avoided.

As mentioned by someone else - it would be best to consider this use case: one large indexbuffer, one large vertex buffer, all game objects that was to render have a pointer/reference to those and access ranges in there. CPU updates to the buffers don't require a wait until API GPU is finished reading it (the client app has to ensure it does not cause those race conditions). If this is difficult we can consider a intermediate buffer that allows to upload to and then do a GPU copy from that to the large buffer. Imagine we want to update vertex buffers each frame and we want to bound the performance impact, doing it async and distributing over multiple frames is acceptable - in many cases you can have existing data being used until the new data arrives. I would be happy to stay in contact to try an easy version of the new API.

 

Please adds functionality to generate SDF from mesh and mesh from SDF.

 

Definitely agreed on this. Currently the list of overloads seems to be:

Code (CSharp):

1. SetFoo(T[])
2. SetFoo(T[], int start, int length)
3. SetFoo(NativeArray<T>, int start, int length)
4. SetFoo(List<T>, int start, int length)

Although ideally we'd only need the following:

Code (CSharp):

1. SetFoo(List<T>, int start, int length)
2. SetFoo(ReadOnlySpan<T>)

With NativeSlice<T> and NativeArray<T> given implicit conversions to Span<T> and ReadOnlySpan<T> once .NET Standard 2.1 drops.
On that note, would there be value in an ArraySegment<T> overload over the T[] with start and length version? Probably not, but I figured I'd throw it out there.

 

@Aras hello! There are two main things I'd actually like to suggest - one of which may not be suited here.

Firstly: Unity seems to be going forwards with using proper Microsoft guideline naming conventions (PascalCase everywhere, vs Unity's old camelCase naming) - this would be really nice. Having to switch between casing can be a little jarring and annoying for those who are somewhat OCD like myself. If you'd like sources I'm happy to quote

Secondly(might not belong here): Better compatibility with ECS. Now, I see you've got some general ideas layed out, but having a MeshComponent that we could use directly in the Job System would be... amazing. I've been developing a Voxel based video game lately, and of course, world generation needed to be multithreaded. However, passing in a NativeArray<Vector3> NativeArray<int> etc was very jarring. However, an IJobParallelFor<MeshComponent> would be a whole ballpark nicer.

Thanks for asking for feedback here

 

Nice.
A couple of requests:
- A set of APIs that enqueue mesh data being uploaded and work in a non-blocking way using internal jobs/manager and allow us to control CPU budget per frame if necessary including any NavMesh/Physics update tasks. Commands like Flush(), SetCPUBudget(Max ms), etc.
- Ability to obtain original mesh when batching applies.
Thanks

 

Am i correct in assuming that the mesh recalculations immediately apply to the updated part of the collider? No mention of this in the document. If so, would be also nice to see these optimizations ported into the Terrain collider.

Though with the:

Code (CSharp):

1. SetFoo(List<T>, int start, int length)

linear methods it wouldn't be enough. If possible, it'd be nice if the methods were extented to follow SetHeights(int xBase, int yBase, float[,] heights) functionality where only the changed patch of it would be updated.

 

Amazing to hear that.

I've been doing some procedural animated meshes and the need of copying/setting the entire mesh data every frame kills the performance. From tests with my case scenario, it's completely memory access(copy) bounded. Having such "low-level" access would be a "game changing" pun intended.

 

Not sure what the use case for generating a mesh from an SDF on the scripting side would be, as an SDF is usually something generated from geometry you already had. Generally you simply render the SDF in shader in a way that makes it look like a geometric object, without having to process any significant geometry. The only thing I could see it being useful for on the scripting side is if you wanted to generate a collider from it at runtime, but that's a complicated feature to develop for very niche utility. What would be cool is if PhysX or Unity's DOTS based physics they're working on could utilize SDFs as collider data itself to do physics operations on...

 

I want it to be easy to modify meshes on the GPU with Compute Shaders without unnecessary memory transfer to the CPU. It's very critical that this is possible.
I haven't looked into it but I think this might be already possible with Graphics.DrawProcedural. If not that definitely should be in there.

 

from 2019.3.0a7 release notes:

• Graphics: Ability to specify custom Mesh vertex data formats, and set vertex buffer data from NativeArrays. See VertexAttributeFormat, VertexAttributeDescriptor, Mesh.SetVertexBufferParams, Mesh.SetVertexBufferData.
  
  
• Graphics: Ability to specify Mesh submesh information directly, and set index buffer data from NativeArrays. See SubMeshDescriptor, Mesh.SetIndexBufferParams, Mesh.SetIndexBufferData.

 

Not sure if its possible.

• I would like to have a static custom data chunk in a mesh. Then able to read/write from a vertex function of a shader. These data chunks should independent from the current vertex or uv index as a global info.

 

1) Dumping MeshFilter.
2) Creating a Skeleton file when importing a SkinnedMeshRenderer to prevent bones ordering F***up.
3) Kill the LOD system and redo it. It's rotten, not multithreaded and scale very poorly.

 

This sounds great!

• The most important thing for me is performance. I am not interested in naming conventions or high-level apis with hidden costs. Just give us low-level stuff to let us write better games.
• Ideally, we can write directly to vertex streams in uncached/write combined/video memory/whatever using an appropriate platform format from another thread (I don't care if we can or have to use DOTS or whatever).
• Same for indexes
• Any existing API's that could be made thread safe would be nice.
• Bonus points if there were a way to blit vertex/index streams to readable memory asynchronously on write-only meshes for those rare cases in which you might need access to vertex information from one mesh (but not the entire FBX) and you can afford to wait for it. Like when your artists have 300 meshes within an FBX and there are rare cases that you want vertex information from one of them for some UI stuff but you don't know which ahead of time and your target platform doesn't always support compute shaders. We could do the translation from platform formats like F16 to CPU friendly formats and deinterleave data to pick what we want.

 

This is possible with DrawProceduralIndirect, which is how I'm handling it currently, but you have to use a custom vertex shader to read the data back out, which means you can't combine it with the HDRP shaders or shader graph. You used to be able to with custom master nodes, but you can't any more. The ability to assign a buffer to a mesh or otherwise tell the standard shaders "this is where your vertex data is" without sending the data back to the GPU would be a godsend.

 

If you can do this easily.. especially with HDRP it would epic. Efficient GPU mesh modification with compute shaders is where it's at.