Graphics [Removed from Asset Store]Cluster Mesh Rendering (rendering acceleration technique)

Asset has been removed from Unity Asset Store.
When we will have a better management of URP, we will probably add it back.
Sorry for the inconvenience.

 

We added a short video to show you how this tool is working at this moment inside Unity:

 

Hi all,

Here is our latest addition to the Mesh Cluster tool – Frustum Culling!

It seems that there are some confusions about this tool. This is NOT a mesh combiner that will create one giant mesh and render that combined mesh with one draw call. Even if this technique is working in some situations, it is not very efficient, as you are sending a lot of triangles to the GPU that are not actually visible on screen. Even if the GPU is fast in detecting polygons that are out of screen, it will still cost time (everything depends on the number of triangles that are out of the view frustum).

Here is a small video showing a single mesh rendered using this system. On the right side of the screen you will see what we actually send to the GPU for rendering. On the right side you see what the game camera is rendering.

The second video is showing the Frustum Culling on the entire map.
On the left, you see what we’re sending to the GPU, and on the right is what the player camera is seeing.

 

looks amazing, can this also work on WebGL 2.0? are there any limitation for mobile?

 

Hi, so I'm afraid this tool can only work on targets that allow ComputeBuffers as input to the Vertex Shaders. I didn't try WebGL 2.0 yet so I cannot tell you this now.
The tool is not ready yet as I need to add a lot more features to it and then I'll sort out the compatibility list for you guys. For sure this code will work on PC (DX11 and above), new mobiles (probably IPhone 8 and above, and the equivalent on Android using Vulkan), and on all consoles (including Switch using Vulkan).
Thanks for the appreciation : )

 

Hi all,

Here’s a new demo and some more explanations about the Mesh Clustering.
In the video you will see the current system that we intend to simplify (there are still too many steps before you can use the system and we know we can do better to make your life easier : )

So, in this demo we start by creating a standard Unity sphere and assign a pre-created red material (it is using Universal Render Pipeline\Lit).

Then, we collect all materials in the scene and assign the pre-created material for the Cluster Mesh Rendering. The creation of the material is simple, just duplicate the original material and replace the Universal Render Pipeline\Lit with CRO\URP_Lit.

Collect the geometry and poof!, everything is done! You can now play the game : )

As you will see, we send to the GPU only the parts of the sphere that are visible, not the entire object. This will make the rendering a lot more efficient. Of course, this is still use one single draw call. As a bonus, in case you are wondering, this will work also for instanced geometry. For every instance we will send only what is visible from that instance.

Hope you like it and let me know what will make it more useful for your particular case, or what demo video should we add.

 

We just wanted to test the current performances of the system. Please note that the system is not yet fully optimized, we need to work a little more to get better performances, but it's not so bad even now. Take a look and leave a comment.
Enjoy!

 

Hi all,

We added a new feature to the Mesh Cluster: Screen size percentage culling.

In general, when an object is too small it doesn’t impact the visible scene anymore and it’s best to not render it. For this reason, we added a new option for you where you can control the threshold that establishes when an object is too small and it can be removed from screen.
Be aware that an object for us is a Cluster Mesh, not the entire object as it is exported from your favorite 3D modeling tool. We will explain this in detail in a future video demonstration.

Take a look at the demo and let us know what other features will be useful to add for your project.

 

Am curious to know if this can work for/on VR oculus quest ?

 

Hi, Mark! In theory it should work, I don't see why it wouldn't work. The biggest problem would be what rendering pipeline you want to use. For the moment I tested it with Universal Render Pipeline and Standard Unity (deferred rendering). As there are too many variations I'll concentrate only on one pipeline (Universal Render Pipeline) and later I'll try to add support for others.

 

Thanks so much for your reply. Once I get the cable for the quest in the mail and start
playing around with making my own levels for vr quest , I will come back here
I think for vr you use the Universal Render Pipeline.

I think this could be a very good, well liked asset once you have it on the store.
AFAIK there is not to many assets that does this well. ;-)

 

Thank you for the kind words. Check back here from time to time for updates!

 

Hello all,

We added a free checkup tool on GitHub – MeshClusterCompatibility (https://github.com/CodeRoadOne/MeshClusterCompatibility) for Unity 2019.4.1f1. This tool will allow you to know in advance if Mesh Cluster is compatible with the minimum device that you want to support. Just download the code, do a build for that platform and run it on that machine. As soon as the application is starting you will see a screen telling you if the current implementation of the Mesh Cluster is compatible with that platform.

If all the current fields are green you have maximum compatibility. If some fields are yellow, the tool will work, but there will be a speed penalty.

Let us know if your targeted device is not compatible and where it is failing. We will decide if we try to increase the compatibility to include your target device.

The Compatibility Test Scene can be found in: CodeRoadOne/MeshCluster/DemoScene/CompatibilityTest/CRO_CompatibilityTest

Send us your feedback!

 

Hi guys,
Is this still in active development or have you stopped developing? Looks very interesting.

 

Still working on it, but with a different strategy. URP has too many constrains so we are creating a new rendering pipeline that will have the indirect rendering. Also the usage was not so user friendly so we are trying to approach it a little differently. Also we are trying to add support for dynamic objects not only static ones.

 

Is there any plans for HDRP?

 

We are preparing right now a release for Unity URP and we will investigate HDRP shortly after. We need to see if the HDRP have the hooks that we need to make this rendered as smoothly as possible. Even for URP we are missing hooks for shadows (when rendering every cascade) so we need to send to the GPU a lot more data that is actually needed.

 

Hello @CodeRoadOne , this is really cool stuff! I'm really looking forward to the URP release
In the meantime, can I ask you what algorithm have you used for generating the mesh triangle clusters?

Thank you and good luck with the project!

 

Hi @gmodarelli, for the first release we will use the closest triangle. It is not giving perfect results, but as a first release we think is good enough. If enough people are interested on the project we will use a different algorithm in a later release that will take into account also the orientation of the triangle and it will try to keep the triangles closer together. We will try to get closer to what Nanites from Unreal is doing so we will be able to render one mesh with different lods. A road map will be presented when we release the tool on the Asset Store.

 

hi @CodeRoadOne
Tool looks amazing thanks

Can you say approximate eta for first release of tool?

 

Hi @Jes28,
The tool is working but we are trying to find out the possible points of failure to assert or message so we know what is going on. Also we are looking at performances, we will post here a new set of videos showing what is possible and what are the restrictions (there are a lot of them at this moment). On the last tests we find out that the tool works very well if there are a lot of low poly models but if you go to high poly the GPU is not going to keep up, so everything is dependent on the scene and we want to make it clear from the start so you know if the tool would help or not. Of course we have a lot of plans on how to improve it but that will depend only on the interest and the direction people will want us to go on.
We are planning to release it in about 1 months.

 

Hi all,

We have a few more videos of the current features:

Stand alone version for windows:

Stand alone with generation of the depth texture:

In Unity Editor demo:

Hope that you like it, and don't forget to leave us a comment of what you would like to see in our next demo.

 

Thank you for answering. I agree on the "closest triangle" approach to start with. No need to overcomplicate things at the beginning

The new demo videos look really promising, awesome job! Can wait to try the tool.

I don't know how easy it is to get the info from Unity, but it would be cool if you could display the time spent in the various rendering passes (frustum culling, occlusion culling, etc..).

 

It is a good point, we will investigate and try to add the requested info. We will add some more info also (number of triangles rendered, number of clusters/Lod rendered), this way you will know where to look if you want to get more speed.
Thank you for the feedback!

 

@gmodarelli As requested we added debug timing view so you can see how much time is spent in different passes:
I hope you like it and find it useful.

 

That's perfect, thank you

 

Out of curiosity, does this work with skinned meshes? And what about particles?

 

Hi @FiftyTifty,
The list of restrictions will be available soon.
Skinned mesh and particles are not supported, but it will be rendered by the default unity renderer.
Our system is just an addition, if a feature is not supported by us is still going to be displayed in game, but using Unity renderer. I hope this answer your question.
Just to let you know we will investigate the support for skinned meshes. For the particles, URP has a very good system so changing that we don't think it will give a big boost in performance.

 

Amazing job, any clue about when this is gonna be available on the Asset store?

 

Hi @Kanaka1047, we are preparing right now the package for submission, so we are hopping that in about 2 weeks to be able to submit it and then it will depend only on the approval time.
We will post here when we will do the submission (it will be soon).

 

Hi!
Congrats on your progress! Waiting for the HDRP version

 

Looks good. I'm working on a similar project for vegetation in HDRP - Did you ever manage to fix the culling issue with cascade shadow maps?

 

Hi @SimonFearn, yes we managed to fix all the issues with the cascade shadow maps.

We now support frustum culling for all cameras and occlusion culling only for the main camera (if requested we can add occlusion culling for all cameras).

We are having some issues with the VR in this moment as for some reason our code is executed over 8 times comparing to only 3 time when is using Unity standard rendering, so we are doing a mistake somewhere and we are trying to find a fix for it. Even with this issue we are in pair with Unity rendering (we have same FPS, but as soon as we will render data only 3 times we will beat Unity rendering by a lot).
.
We are also working on a demo with a lot of different objects to showcase the power of the system.
Stay tune for new updates and videos (we made it simpler to setup everything)!

 

cant wait, amigo ! i recall that u said it would be on the asset store this week?

 

Hi all,

We updated the first post with the current know issues and limitations. We are not sure that the current version is a good release candidate so that's the reason we didn't submit it yet to Unity Asset Store.

We need some feedback from you all to know if there is some interest in this tool even with all the mentioned restrictions or we need to work more on it before the release.

Also we want to test it more and give you more info about performances so you know exactly what you buy and not to be disappointed.

We are waiting for your feedback.

 

Yup interested, just release it
Can also all it (beta), like several packages do in the store, and can use big font to notify that must read limitations disclaimer first.

 

Is there a package to test and give feedback though?

 

The feedback is actually on the post, after you read it, are you interested in the package or not. We don't have a test package. Initially we wanted to have two packages, a simple version where you could test the package without source code and a different one with full source code. Eventually we said we will release the tool only with full source code, so no there is no test package.

 

Based on the post, the tool looks in interesting in general, and would be particularly useful for me if it supported HDRP and had less restrictions.

 

Yes, this kind of feedback is interesting for us, what restrictions you want to see gone?
We didn't check HDRP, but we assume we have the same kind of restrictions, maybe less if HDRP is using Deferred Rendering like URP 12.
For shadows from different source light we wrote to Unity, but we didn't receive any answer/feedback, so we don't know in what direction we should go. Actually we know in what direction, but it will be a pain (create our own rendering pipeline, a copy of the URP or HDRP, and we insert our code directly there, but for a few events that Unity needs to expose we think this is an extreme measure, but if it is not possible to have it in a different way, we will do that also).

 

Having a separate rendering pipeline would be almost a total deal-breaker/no-go. HDRP support would be great, to be honest. Plus, if there are things that are not exposed, you can send a pull request to the graphics repo, they accept outsiders' pull request as long as you explain why you need it and what's the benefit (as long as the pull request is not changing too much stuff)

 

So, I'm talking about my project, so this will be subjective, but it's a real use case, though.

First af all, I'm on HDRP and going to URP is not an option.

Baked lighting is a must have for me. I rely very heavily on it. More than that, I use Bakery and its a very popular asset for baking in general. So if lightmaps are supported, some kind of Bakery support would be really useful as well. Or at least it would be good to make sure that the assets don't break each other.
Also shadowmask support.

All light types (punctual and area as well) support with all features (like high-quality pcss filtering) in deferred.

And ShaderGraph support. There was a discussion in some thread not so long ago about how you can't ship a big game made only with the default Unity shaders like "Lit". I totally agree with that. I have all shaders made in shadergrpah for my project and since its the only official way of creating custom shaders for SRPs right now, there's no way around it. But I'm okay to, say, edit generated shadergraph code if it takes that to work. Maybe it would be also good to support Better Shaders asset as well.

Virtual Texture support. It's HDRP only and it relies on ShaderGraph too. If we are talking about Virtual Geometry, it makes sense to look in the future and support VT as well. Maybe it will work out of the box and doesn't need some special support on your side, I'm not sure.

Also, have you considered shipping your asset along with a modified HDRP/URP package? This is what the author of the NGSS asset is planning to do. Seems like a valid approach, if you're prepared to support non-LTS version as well (all stable releases).

 

With the lighting and shadow limitations, how detrimental are they? You mention that only one dynamic light can be used. Which is unusable for most games and art directions; say, Skyrim with a grand total of only one dynamic light? That is useless. Even DOS games such as TES: Arena supported multiple dynamic lights.

Or is that simply a miscommunication?

 

it seems like it do support Builtin RP right?

 

Hi @passeridaepc,

Thank you for your feedback this is very useful to know the needs of the users. When we said we would like to have our own SRP is just a copy of UPR with our code injected, this will allow us to do a lot more.
We didn't investigate a lot the HDRP support but we can do that. The main issue with the system is that everything that is supported by HDRP will not work with our system.
What our system is doing is something like that:
- We take all different meshes and combine them as one single mesh.
- When we render a material we emit a single draw call for all instances and meshes.
- Unity has no idea about the meshes, materials or object position.

Because we render all meshes with a single draw call we pay a high cost in the vertex processing, so having meshes with millions of triangles and we don't yet support multiple level of details per mesh, it will make it run very slow at runtime.

Also the baked UV's are difficult to manage at this moment as we need a different set of data per instance (we didn't add that yet, but this will make the shader porting a little more difficult).

Virtual texturing is counting on active meshes and trying to see what to put in a texture for rendering, because we bypass Unity completely streaming and virtual texturing cannot work by default.

Shader graph is something that we would like to add for sure as support, but we don't have an expert in this so we will need to learn on how to implement it correctly.

For lights at this moment Unity is using an internal list for visible light that has no connection the actual lights in the scene (or we didn't find it yet). This makes everything not so easy to manage. We are investigating how to add support for 64K lights in scene with max 256 lights active and max 4 lights per cluster. Adding this will make everything more complex and we cannot remove that cost from Unity pipeline as it is not exposed (so the lighting will be done twice)

Thank you for the honest feedback and we will try to add all the requested features in time of course.

 

Hi @FiftyTifty,

Indeed having only one light is not ideal, like I reply in the previous post we are investigating the support of 64K lights per scene and a maximum of 256 visible lights.

Shadows are having the same issue as before, we will need to re-write a lot of Unity code just to be able to render correctly the shadow, but we will investigate how we can copy the entire URP and inject our code only where is needed. We are not sure how to do that and still be approve by Unity to distribute most of their code in our tool.

Thank you for your feedback.

 

Hi @JacksparrowPhillip,
At this moment we don't support Builtin RP. We had supported it one year ago, but we will investigate how we can add it back, and how difficult it will be.

Thank you for your feedback.

 

That's good news. Have compromises been considered? Such as having the scene be split up into separate tiles. Rather than one clustered mesh, two. Or four. Or eight. Etc.

Would that help with handling lights at all?

 

Hi @FiftyTifty,
Yes we considered a lot of things, the main issue are the moving lights with moving objects.
The static lights with static objects can be computed offline and feed to every cluster easily.
Moving lights need to modify the exported list and check if a moving light is closer to a cluster than the list it already has.
The most complex case is the moving light with a moving object. We need to identify for every moving object the closest lights and we need to do this at high speed so the fps will not suffer.
Thank you for trying to find solutions for us!

 

Hi all,

We added a new video with the current demo scene that the project will be delivered with.

Some info about the scene:

• Over 150 different meshes
• Over 55.000 objects in the scene
• 1 Material used

Some info about the video and the machine that was used to capture the video.

• CPU: Intel I7- 9700K @ 3.60GHz
  • 8 Cores with 8 Logical Cores
• GPU: Nvidia RTX 2080
• FPS
  • Mesh Cluster Enabled: 250-350 (with video recording)
    • 290 - 420 (without video recording)
  • Mesh Cluster Disabled: 45-75

We tested the same scene on a different CPU/GPU:

• CPU: Intel Xenon E5-2630 @ 2.30GHz
  • 12 Cores with 24 Logical Cores
• GPU: Nvidia GeForce GTX 1050 Ti
• FPS
  • Mesh Cluster Enabled: 70-80
  • Mesh Cluster Disabled: 25-40