Discussion [iOS] Best way to display millions vertex point cloud on mobile

Hello,

What is the best way to display millions vertex point cloud on mobile ?

We try to scan with Lidar on iOS and we generate a point cloud (https://github.com/cdmvision/arfoundation-densepointcloud).
So this point cloud is updated every frame.
We've been able to display 1 millions vertex at 60 FPS, afterwards it drops in FPS (https://github.com/cdmvision/arfoundation-densepointcloud/issues/3).

We are searching for an technical alternative (like: compute buffer ? DOTS ? Mesh API ?) to get same performance like Zappcha scan application.
Does anyone have any insights on this ?

 

You are probably best off using a compute shader and doing some point based rasterization there. I would first try with a very simple compute shader that just outputs a 4x4 (or whatever size point) into a texture. Basically just conveet point to view space then write pixels.

There are lots of great papers on efficient point cloud rendering though. All seem to be using compute shaders last I looked.

 

We try to implement ComputeBuffer to display dense point cloud but we didn't see any performance improve compare to mesh (https://github.com/bloc-in-bloc/arfoundation-densepointcloud/tree/compute-buffer).

We go FPS drop at 5 million vertex.

 

Well 5m is not an insignificant amount of points, but also I would expect the latest iPhone to handle at least a bit more that that if that's all it is doing. You are also running lidar at the same time so that will contribute to performance loss / thermal throttling. Have you actually profiles this? Or are you just looking at FPS. You really need to profile this to get any kind of usable metrics.

I would suggest profiling in Xcode GPU profiler and see exactly which parts are slowing down your application.

At 5m points, your biggest constraint will be memory reads/writes. You can do some to improve this though. Like storing points in halfs instead of floats, and writing to a 16 bit instead of a 32 bit color buffer. Also be sure you aren't running this at full resolution. That is gonna hurt performance a lot.

Also why do you need so many points on screen to begin with? 5m points even at only 1 pixel wide will cover a 1920x1080 screen res 2.5 times over. Maybe you can get away with that amount or less? Or you can do some kind of culling if they aren't all on screen at once.

 

Hi !

Been profiling but need to learn the Xcode GPU profiler. We indeed have high read and writes metrics on our frames.




GPU write is indeed close to maximum level.

We indeed try half float and uint encode/decode in both shader and c# without any noticeable performance improvements. Pushed the modifications in the branch "optimization-buffer-stride" https://github.com/cdmvision/arfoundation-densepointcloud.

What do you mean by "full resolution" ? We can't actually set the Screen.SetResolution to half, but if there is anyting to do shader side, we'll be glad to learn !

We show all the points on screen so users know where they already scanned, and maybe improve some badly covered area. This will be use on pretty large infrastructures (from 2 to 50m long). So after scanning we should see all point cloud.

And we think that this is feasible as https://zappcha.com/ is capable of showing 90+ millions points cloud easily.

 

Show your whole frame. Looks like you have some vertex work that is overlapping some fragment work, which is a bit odd to me.

Also I don't see you using a compute shader (I think you misunderstood as compute buffer?). That will be your best bet. With small points, a regular vertex/fragment shader setup is going to be suboptimal due to poor quad utilization. Small triangles = poor quad utilization. Read up a bit on that here : http://filmicworlds.com/blog/visibility-buffer-rendering-with-material-graphs/ (half way through the page).

And why can't you use a lower screen res? It is pretty common to do so in most demanding realtime mobile apps. That is going to improve your performance a lot.

I imagine with the above changes you could at least double your performance. Maybe quadruple.

This is a somewhat advanced topic, which means you will have to learn a decent amount to really get into it. I would recommend really studying the GPU counters in the profiler and things will start to make sense.

Also there are some other things you can do like render less and less particles the further away they are, as they contribute less to the overall image. Also if you can separate your particles into sections you can frustum cull those sections.