[Released] Point Cloud Viewer Tools

Roughly how many points there are in single tree? (and is that single mesh, or each tree is split to smaller meshes)

Or would it be possible to have those files for testing?

 

Hi Mgear,

First, awesome work. Is it possible to detect collision with the point cloud? Is the asset updated to the latest version of unity? Does it work with URP? Is it possible to change the color of each point according to the proximity to some point (like a heat map)?

Thanks,

Victorino

 

Kind of, there is example scene that checks for if bounds intersects cloud. (but not suitable for huge clouds and not 100% accurate either).. but let me know your use cases so i can test.


Personally if i'd need accurate better collisions, i'd try creating regular mesh from point cloud using external tools (mesh reconstruction), then use those as low poly collision meshes.

Have tested 2020, should work on newer versions too.

Some features do work, the main viewer (not all shaders are tested though, but if you encounter issues submit here)
https://forum.unity.com/threads/released-point-cloud-viewer-tools.240536/page-12#post-6293071

HDRP requires some modifications (not included in the store version yet)
https://github.com/unitycoder/UnityPointCloudViewer/issues/105

Can modify the existing shaders, i can help with that.
(should be easy if its single Vector3 point to compare distance with, and then gradient texture to pick color from) or such.


*sorry missed notification for this question earlier.[/QUOTE]

 

Can you make a script (runtime) that allows me to send in the vertex and color data of the point cloud... and then the Utility allows me to create the data as a physical mesh/virtual cloud or tile/non-tiled point cloud?

Basically asking for you to make a runtime utility script the allows people to bring in their own data in Vector3[] and Color[] or Color32[] data (essentially skipping the parsing functionality of your scripts and moving on to the other processing).

I created runtime scripts that work well for parsing the data and is faster, and I am able to create point meshes and see it, however, I want to do some more testing for performance and see if it is an issue with VR or the shader/pointmesh method. Testing with 55million points in VR is useable, but im pretty sure it can be improved significantly using yours.

Also, I havent imported your asset into this current project yet, is it URP compatible? Is the Video of 10billion points Tiled as Meshes or Uploaded to the GPU separately?

 

check example scene "PointCloud-YourOwnData.unity", it has small script to show how to feed your own point data.
issue there is that if you feed a huge array, there is noticeable lag to send it on gpu, since that part of unity runs in main thread.. but let me know how it goes.

10b demo: its using the V3 tiled viewer (no meshes, i tried using mesh tiles briefly but its slower and still missing color data), but there is no 10b visible points, dataset is that size, but regular limitations apply for how many points can be actually visible. (tiles can decrease visible point count by drawing less points when further away)

urp: not recently tested, most features should work though (since URP supports OnRenderObject and meshes with basic unlit shaders).. but report any missing features.

 

Ill update this post more later... first issue though...

PointCloudViewerDX11.CS pointColors = type Vector3? instead of Color32? why is this? You could do a byte.tryparse from the float and get the correct r,g,b values. For instance, if it fails, you can use the other method.

** Note after looking through the shader... Assuming this is because shaders like <float4>?

 

yes, shader uses float3, and binary data is saved to that format so would have to change that also for faster parsing,
i think there were some benefits from using this originally. (of course better if i eventually start using bytes)

 

What is the method that is used to upload the data to graphics card? If I were to isolate that out? I don’t see it in the code so I assume it’s part of a compute shader, but how do you LOD points that are uploaded to the GPU directly vs the mesh lod stuff?

I messed with it for a little bit but couldn’t get it to work. Also the example your own data scene does not work in VR. I can see the points on the monitor but not in the headset. I checked to make sure the Camera is main camera also.

 

in v3 it adjusts the point counts based on distance, and loads new data if not enough points available, in code:

Code (CSharp):

1. tiles[e.index].visiblePoints = newpointcount

vr issue: if possible, can you try that it works without urp? (sounds like it could be due to urp+vr, i havent tested urp or hdrp in vr)

 

Here is a link. You can see when I rotate the headset it rotates the camera and everything is setup for VR.

In my headset, I can see everything in the scene, controller line, skybox etc... however, in the little window you can see the points as being shown... but in reality there are no points in the VR.

Edit** This is without URP in standard

Here is my point cloud data imported into a mesh, however, when all points come into range, it starts to get very choppy. Otherwise it is fine. I would like to basically just copy the data that is in the mesh, into a buffer, delete the mesh etc.

 

sorry havent had time to check into that URP yet.

other info that someone asked:
added example script for runtime las/laz conversion, using the commandline tool:
https://github.com/unitycoder/UnityPointCloudViewer/issues/119

 

Hello,

I've checked the changelogs but I've not seen a feature that was commented time ago.

I had limit of 800M points because in version I used it was neccesary to merge the point cloud into big cloud before converting. Is it already possible to feed the converter with loose tiles so we can have an "infinite" dataset not only limited by storate rather than RAM because of converter requirement,?

 

commandline version can convert multiple files into one v3 cloud (thats how i converted that 10 billion points dataset)

v2 viewer can read only about 178000000 points max, since its single array.

 

Awesome, I saw the video of the lidar today, I was guessing that was the way but may be you had super hardwhare . Awesome to know. Can´t wait to find a moment to test it out.

 

I've prepared a new test point cloud and I'm testing the converter, an small thing is that you may whant to update the converter sample comand line for V3 since syntax has changed a bit:

https://github.com/unitycoder/UnityPointCloudViewer/wiki/Commandline-Tools

some params instead of "=" use now ":" aswell as " -exportformat=pcroot" is now needed, also name of .exe has changed,

PointCloudConverter.exe -input="G:\LIDAR\2021\00_InputConverter" -randomize=true -gridsize=500 -minpoints=5000 -version:3 -offset=true -pack=true  -exportformat=pcroot  -output=G:\LIDAR\2021\00_Output\PuertoRico.pcroot

 

output path does not like spaces:

 

having the path inside ".." should work?

 

*heads-up*
this plugin will be on a discount at the Black Friday Sale (November 14 - December 5, 2021)

 

Hello, we are creating a demo for HoloLens 2, in the editor it works just fine, but when we build it for HoloLens the point cloud is not visible. We want to use the Point Cloud Viewer DX11 and we have tried different shaders but with no results so far. Could you lend us a hand here?

Thanks!

 

Hololens support is bit limited, but you could try the Mesh version first.

the DX11 version would require manual edits,
to remove that unsupported Threading/Tasks for UWP (at least that was one issue previously)
https://github.com/unitycoder/UnityPointCloudViewer/issues/25

If possible, check your device log for what could be the error message?
Also does hololens 2 support DX11?

 

Thanks mgear,
We already tried the mesh version but we rather use DX11 version because of the performance.
We also tried disabling threading but in this case the application never opened (it seemed stuck).
Hololens2 supports DX11, in Visual Studio you can create a DX11 App for Hololens, we are using the MRKT in Unity with the Direct3D11 option for the Graphics API activated... so we guess it is using it.

 

ok, so the PointCloud-BinaryLoaderV1.scene still works if disable threading toggle, (v2 or v3 doesnt)
but previously the threading parts in code caused UWP issues and errors. (so check device log if thats the case)

 

Hi again mgear, These are the only references in the log we can find: unitycodercom_PointCloudBinaryViewer.PointCloudViewerDX11:CallReadPointCloudThreaded(String) unitycodercom_PointCloudBinaryViewer.PointCloudViewerDX11:Start() unitycodercom_PointCloudBinaryViewer.PointCloudViewerDX11:ReadPointCloudThreaded(Object) unitycodercom_PointCloudBinaryViewer.PointCloudViewerDX11:ReadPointCloudThreaded(Object) unitycodercom_PointCloudBinaryViewer.PointCloudViewerDX11:OnLoadingCompleteCallBack(Object)

There are no more references after that, we could not find ones that appear in Unity like "(Viewer) Reading threaded pointcloud file: " or "Loading old format: 3042525 points.."

 

if there is CallReadPointCloudThreaded(), that one is still using threading,
make sure the filename (converted format) is .bin
.bin = V1
.ucpc = V2
.pcroot = V3

 

Hello again mgear,
After some tests, we decided to render into a RenderTexture and this way we can check that the HoloLens are really rendering the PointCloud though we cannot see it directly from the glasses.
I attached a screenshot from the emulator but the same thing happens in the real one.

Something we noticed is that if the color format we select in the RenderTexture has an alpha channel, the PointCloud render area will be rendered transparent, as you can see in the second screenshot.

Does something of this ring a bell for you?

 

interesting, haven't seen anything like that..
what you could try, is to use different shaders for that cloud material,
this one (in screenshot) is the most basic one so try that first, then maybe "PointCloudColorAlphaDX11", and maybe "PointCloudFixedColorOpaqueDX11-VR"

 

What is the setup for the 10 Billion Points - cloud scene?
(Scripts that are in use)

I have written scripts that read in the data and tile them, (using this because my files usually fail with yours for one reason or another). Now I would like to write a script that takes each tile and writes it to a format that can be read into your your system. The tiles are in mesh data. I've written scripts that export the mesh data and can import really fast, but per usual the gameobject and drawcalls are a problem.

Code (CSharp):

1. public struct V3_Header
2. {
3.     public int version;
4.     public int gridSize;
5.     public int totalPointCount;
6.     public float minX;
7.     public float miny;
8.     public float minz;
9.  
10.     public float maxX;
11.     public float maxY;
12.     public float maxZ;
13.  
14.     public float auto_offset_X;
15.     public float auto_offset_Y;
16.  
17.     public int auto_offset_Z;//??
18.     public int packMagicValue;//??
19. }

Are these data types correct?


Oh Also, i know you had mentioned possibly using Color32 Data in the future. I found that a super easy conversion was converting your float data like this (since the float is really a percent of a range).

r = (byte)RValue * 255;

Additionally, saving it as a byte saves more data and is faster to read, since
r = BR.ReadByte();
g = BR.ReadByte();
b = BR.ReadByte();
Not sure how Unity handles the Color vs Color32 data, but I know a while back I was reading that they said Color32 has a lot more performance.

 

When I upload the data to the PCVDX11 script, I have an issue when looking at it directly in VR. The VR shader/mat is the only one that works for me. Here is my setup as well... and a video.

 

Thanks, mgear

The VR shader does finally render, however I have been trying two approaches in order to add colors and modify the point size:

• 1) Modify the "PointCloudFixedColorOpaqueDX11-VR" shader to support both features: I was able to support colors but I have been struggling to change the size of the points.
• 2) Modify the "PointCloud-DX11-Uber" shader to support XR, I noticed the "PointCloudFixedColorOpaqueDX11-VR" has UNITY_VERTEX_OUTPUT_STEREO, and I also found that other shaders for XR uses that directive, so I tried doing the same with the Uber:

Code (csharp):

1. Shader "UnityCoder/PointCloud/DX11/UberV2"
2. {
3.     Properties
4.     {
5.         _Size ("Size", Float) = 1.0
6.         [KeywordEnum(Off, On)] _Circle ("Circular Point",float) = 0
7.         [KeywordEnum(Off, On)] _EnableColor ("Enable Tint",float) = 0
8.         _Tint ("Color Tint", Color) = (0,1,0,1)
9.         [KeywordEnum(Off, On)] _EnableScaling ("Enable Distance Scaling",float) = 0
10.         //_Origin ("Origin", vector) = (0,0,0)
11.         _MinDist ("Min Distance", float) = 0
12.         _MaxDist ("Max Distance", float) = 100
13.         _MinSize ("Min Size", float) = 0.1
14.         _MaxSize ("Max Size", float) = 1.0
15.     }
16.  
17.     SubShader
18.     {
19.         Pass
20.         {
21.             Tags { "Queue" = "Geometry" "RenderType"="Opaque" }
22.             ZWrite On
23.             LOD 200
24.             Cull Off
25.      
26.             CGPROGRAM
27.             #pragma target 4.0
28.             #pragma vertex VS_Main
29.             #pragma fragment FS_Main
30.             #pragma geometry GS_Main
31.             #pragma multi_compile _CIRCLE_ON _CIRCLE_OFF
32.             #pragma multi_compile _ENABLECOLOR_ON _ENABLECOLOR_OFF
33.             #pragma multi_compile _ENABLESCALING_ON _ENABLESCALING_OFF
34.             #include "UnityCG.cginc"
35.             #pragma fragmentoption ARB_precision_hint_fastest
36.  
37.             StructuredBuffer<half3> buf_Points;
38.             StructuredBuffer<fixed3> buf_Colors;
39.  
40.             struct appdata
41.             {
42.                 fixed3 color : COLOR;              
43.             };
44.      
45.             struct GS_INPUT
46.             {
47.                 float4    pos        : POSITION;
48.                 fixed3 color     : COLOR;
49.                 #ifdef _ENABLESCALING_ON
50.                 float dist      : TEXCOORD1;
51.                 #endif
52.                 UNITY_VERTEX_INPUT_INSTANCE_ID
53.             };
54.  
55.             struct FS_INPUT
56.             {
57.                 float4    pos        : POSITION;
58.                 fixed3 color     : COLOR;
59.                 #ifdef _CIRCLE_ON
60.                 float2 uv : TEXCOORD0;
61.                 #endif
62.                 UNITY_VERTEX_OUTPUT_STEREO
63.             };
64.  
65.             float _Size;
66.  
67.             #ifdef _ENABLESCALING_ON
68.             //float3 _Origin;
69.             float _MinDist, _MaxDist;
70.             float _MinSize, _MaxSize;
71.             #endif
72.  
73.             #ifdef _ENABLECOLOR_ON
74.             fixed4 _Tint;
75.             #endif
76.  
77.             GS_INPUT VS_Main(uint id : SV_VertexID, uint inst : SV_InstanceID)
78.             {
79.                 GS_INPUT o = (GS_INPUT)0;
80.                 UNITY_SETUP_INSTANCE_ID (id);
81.                 UNITY_INITIALIZE_OUTPUT(GS_INPUT, o);
82.                 UNITY_INITIALIZE_VERTEX_OUTPUT_STEREO(o);
83.                 UNITY_TRANSFER_INSTANCE_ID(v, o);
84.                 float3 pos = buf_Points[id];
85.                 #ifdef _ENABLESCALING_ON
86.                 //pos+=_Origin; // TODO check if issues when transform is moved, since transform.pos gets added to points?
87.                 float dist = distance(buf_Points[id],_WorldSpaceCameraPos);
88.                 o.dist = dist;
89.                 #endif
90.  
91.                 o.pos = UnityObjectToClipPos(buf_Points[id]);
92.  
93.                 fixed3 col = buf_Colors[id];
94.                 #ifdef _ENABLECOLOR_ON
95.                 col *= _Tint;
96.                 #endif
97.  
98.                 #if !UNITY_COLORSPACE_GAMMA
99.                 col = col*col; // linear
100.                 #endif
101.                 o.color = col;
102.                 return o;
103.             }
104.  
105.             [maxvertexcount(4)]
106.             void GS_Main(point GS_INPUT p[1], inout TriangleStream<FS_INPUT> triStream)
107.             {
108.                 #ifdef _ENABLESCALING_ON
109.                 _Size = _MinSize + (p[0].dist-_MinDist)*(_MaxSize-_MinSize)/(_MaxDist-_MinDist);
110.                 #endif
111.  
112.                 float width = _Size*(_ScreenParams.z-1);
113.                 float height = _Size*(_ScreenParams.w-1);
114.                 float4 vertPos = p[0].pos;
115.              
116.                 DEFAULT_UNITY_SETUP_STEREO_EYE_INDEX_POST_VERTEX(p[0]);
117.              
118.                 FS_INPUT newVert;
119.              
120.                 newVert.pos = vertPos + float4(-width,-height,0,0);
121.                 newVert.color = p[0].color;
122.                 #ifdef _CIRCLE_ON
123.                 newVert.uv = float2(0,0);
124.                 #endif
125.                 UNITY_TRANSFER_VERTEX_OUTPUT_STEREO(p[0], newVert);
126.                 triStream.Append(newVert);
127.  
128.                 newVert.pos = vertPos + float4(-width,height,0,0);
129.                 #ifdef _CIRCLE_ON
130.                 newVert.uv = float2(1,0);
131.                 #endif
132.                 UNITY_TRANSFER_VERTEX_OUTPUT_STEREO(p[0], newVert);
133.                 triStream.Append(newVert);
134.  
135.                 newVert.pos = vertPos + float4(width,-height,0,0);
136.                 #ifdef _CIRCLE_ON
137.                 newVert.uv = float2(0,1);
138.                 #endif
139.                 UNITY_TRANSFER_VERTEX_OUTPUT_STEREO(p[0], newVert);
140.                 triStream.Append(newVert);
141.  
142.                 newVert.pos = vertPos + float4(width,height,0,0);
143.                 #ifdef _CIRCLE_ON
144.                 newVert.uv = float2(1,1);
145.                 #endif
146.                 UNITY_TRANSFER_VERTEX_OUTPUT_STEREO(p[0], newVert);
147.                 triStream.Append(newVert);
148.             }
149.  
150.             #ifdef _CIRCLE_ON
151.             // source https://thebookofshaders.com/07/
152.             float circle(float2 _st, float _radius)
153.             {
154.                 float2 dist = _st-float2(0.5,0.5);
155.                 return 1.-smoothstep(_radius-(_radius*0.01),_radius+(_radius*0.01),dot(dist,dist)*4.0);
156.             }
157.             #endif
158.  
159.             float4 FS_Main(FS_INPUT input) : SV_Target
160.             {
161.                 #ifdef _CIRCLE_ON
162.                 clip(circle(input.uv,0.9) - 0.999);
163.                 #endif
164.                 return float4(input.color,1);
165.             }
166.             ENDCG
167.         }
168.     }
169. }

I haven't achieved changing the point size or create a triangle approach, any help would be much appreciated!

Thanks!

 

ill check into that tomorrow, im guessing this line take now p[0], which is same for all points, so
UNITY_TRANSFER_VERTEX_OUTPUT_STEREO(p[0], newVert);
probably should be
UNITY_TRANSFER_VERTEX_OUTPUT_STEREO(newVert, newVert);

but will test tomorrow.

 

That shader code looks like a lot of processing. Is there a simple shader for Stereo that simply gets the vertex position and color and renders it? Similar to the Unity URP Particles which is what you would use for URP MeshTopology.Points? You cant adjust size though.

 

that shader has the billboard geometry stuff, so the simplest one is that PointCloudColorDX11 (no size adjust) so that could be tested or modified into vr.

 

so for desktop vr, the original uber, and that modified one works.. (with single and multipass, at least with Quest + link)

 

Thanks, mgear
Then it seems to be a problem with HoloLens, we'll continue to investigate.

 

Hello mgear,

I started using your plugin recently, and I would like to ask you a few things.

I am interested in a use case where I want to position a point cloud on a simplified 3D model of the area which was scanned.
For example, if I have modelised some room, which I have also scanned to get the corresponding point cloud.
I would like to easily position my cloud to match the model of the room, which implies repositionning, rotation and scaling at runtime.
Is something like this possible? What would be the best solution to do something like this using your tool?
And then, would it be possible to instantiate separatly multiple clouds in the same manner?

Thank you in advance

 

Easier option:
Use the PointCloud2Mesh converter, then you get regular gameobjects (that can be moved, rotated, scaled, instantiated).
But if you have really large cloud, then meshes can be too slow.

Complicated Option:
Using the DX11 viewer, need to modify shader to have offset/rotation/scale parameters (well this part is not super complicated, but still requires few modifications, i can help with that, and also setting those properties to shader from script)
For extra clouds, can add extra/separate viewers for each cloud.


How many points you might roughly have on your case, and is this for desktop usage?

 

Thanks for the reply!

In my case, I have to figure out the limitations on the point count, while maintaining an acceptable performance and visual quality. Hence, I have no real data on the number of points I have to display. I would say that the worst case scenario would be, all clouds combined, around 200 million points, but it really is a guess.
And yes, it is for desktop usage.

I will test out the easier option, to figure out where the limits are.

For the shader modification, it would really be great to have access to this kind of parameters. I have tweaked your shader a bit to get a position offset property, but rotation and scale seemed a bit more complicated to deal with. Also, I ran into a visualization issue when applying the offset, so there must be some implications I did'nt have time to dig into (using the tiles viewer, so I suppose the issue was that I didn't apply the offset to the code choosing which tile to display based on distance). So, your support would be really welcome on this, especially if I can get better performance with this method.

 

200m is certainly too much for the mesh version, but try to start with that until hit the wall. (also can just limit the view distance+fog, if its over really large area, to reduce drawing).

ok yeah, v3 tiles viewer will be more complicated to modify,
since its requires rebuilding the visibility bounds for each tile.

so might want to try v2 format after mesh,
its already more performant compared to mesh, and has relatively simple shader.
Also using color packing, its even more performant, so 200m might be possible. (of course filtering the cloud in advance is still a good idea)

*ive sent one V2 test manipulation shader in private message, if you want to test it:

 

Hello mgear,

Thank you for your support with this!
I have tried to work a bit with the mesh version, but I ran into some issues with the converter:
- I for some clouds, it seems that the converter doesn't work properly. Visually, I get 5 points in the scene, it is quite weird. For others, it works well. I always use pts format for input.
- When the conversion works, it seems that the generated prefabs are broken. The GameObjects added directly to the hierarchy are fine, however the mesh in the meshFilter seems to be impossible to serialize, and thus I can't generate a proper prefab from it (either it is broken, or the mesh filter do not retain the corresponding meshes).

Anyway, it seems that performance-wise, it is less performant than V2 with your shader, which works well!
Do you think that combining it with other shaders might be acceptable performance-wise?

Also, I noticed that when ticking the color packing option, it suggests using the V4-packed material, which is broken (the associed shader file is missing from the plugin's assets).

In my case, all input clouds I will receive as input will already be filtered, so it isn't something I will have to worry about.

 

ok that should be v2 packed material. (ill update the text)

which unity version?
*Ok i see the same issue now, ill check. For now then, need to create prefabs from the objects created into scene.

Maybe can send screenshot in private message? and or share small cloud file where it happens.
Check also that your material pointsize is not too small?

For mesh performance,
can test [x] Sort points in X axis with [x] Split to Grid, to have better culling of the meshes (when they are outside view), and [x] create lods can be tested also (less points to draw based on distance)
https://github.com/unitycoder/UnityPointCloudViewer/wiki/PointCloud2Mesh#advanced-settings

PointCloudColorMesh.mat is the simplest material/shader for meshes, doesn't have size option, but can work in some cases.

 

I tried doing that too, it didn't work correctly, the mesh not being serialized. I am working on Unity 2021.1.25.

I'll do that. But I don't think point size is the issue, it happens mostly with big clouds (60M points), and when "Sort points in X axis" is not ticked.

Sorry, I did'nt express my thought well enough. I was talking about the PointCloudDX11ColorSizeV2-manipulate shader. Do you think that combining the matricial computations with other operations performed in the other shaders like packed or sizeByDistance would be fine performace-wise, espacially in a multi-instanciation scenario? For small shaders like the Tint or the Alpha, it would seem to not have much impact, but how about the more complex ones?
*Edit : I already combined it with Tint and Alpha, works quite well.

Thanks again for your great support!

 

I modified the VR shader and was able to get color working in it, but there is still the same problem where when looking at the points it flies away.

When the camera pulls reference data for current.camera or vice versa in stereo the camera data is null. Would that have something to do with it? How does the shader calculate the camera position in general (which line)?

 

ill check this today, did you add just the color or also the geometry billboards?

 

Just the color.

 

ive sent one test version vr shader in PM.

 

Does anyone else that follows this forum have issues with VR? Such that, when a user interface enters into the scene the point cloud moves towards it at a different scale? (specifically world space canvas). Also using URP

When I look at a canvas or it is visible to the camera, the point cloud changes/scale location and I assume it is related to Unity Engine Subsystems regarding trying to render the points behind/in front of the camera.

I've formatted my computer, reinstalled Unity, created new projects etc and still have this issue with SinglePass enabled.

I did try some of Mgears suggestions but still have this problem and was hoping someone may have either figured it out or maybe share their settings? Anyone using URP or are most using Standard Built in which will eventually be depracated?

 

Hey, is there a way to have the pointcloud show up in the scene instead of only when the game is running? I'm working with very small pointclouds that don't even need to be tiled.

 

Convert them to meshes and load them into the scene. (Game Running = Runtime). This method is done in the editor and therefore if you save the point clouds they will be present at both Editor/SceneView and Runtime.

Meshes are more than adequate for small point clouds.

 

Yes, using meshes is the easy way. ^

If you want to try, you can also enable executeineditmode, from the PointCloudViewerDX11.cs

but first disable threading,


And might have to click play once, then stop.
But it might have unknown side-effects..

 

if you have issues with URP + WorldSpaceCanvas, need to do this:
https://github.com/unitycoder/UnityPointCloudViewer/issues/126

(for some reason, worldspace canvas "eats" the pointcloud when UI element visible or in front)