Question Render TrailRenderer to texture

Hi, I'm trying to implement a VR finger painting board where a TrailRenderer following the user's hand gets rendered to a RenderTexture on a plane.

It basically works, but the rendered trail is oddly distorted compared to the original.

Code here:

Code (CSharp):

1. private void RenderMesh(TrailRenderer trail, Material material)
2. {
3.     var mesh = new Mesh();
4.     trail.BakeMesh(mesh);
5.    
6.     var ttr = trail.transform;
7.     var lookMatrix = Matrix4x4.LookAt(mesh.bounds.center, mesh.bounds.center + ttr.forward, -ttr.right);
8.     var unRotate = Quaternion.Euler(0, 0, 0);
9.     var scaleMatrix = Matrix4x4.TRS(Vector3.back, unRotate, Vector3.one);
10.     var viewMatrix = scaleMatrix * lookMatrix.inverse;
11.     var projMatrix = Matrix4x4.Perspective(120f, 0.5f, 1f, 10f);
12.     var transMatrix = Matrix4x4.TRS(mesh.bounds.center - this.transform.position, Quaternion.identity, Vector3.one);
13.     _com.SetRenderTarget(canvasTexture);
14.     _com.SetViewProjectionMatrices(viewMatrix, projMatrix);
15.     _com.DrawMesh(mesh, transMatrix, material, 0, 0);
16. }
17.  
18. public void OnWillRenderObject()
19. {
20.     if (wantClear)
21.     {
22.         wantClear = false;
23.         _com.SetRenderTarget(canvasTexture);
24.         _com.ClearRenderTarget(true, true, Color.clear);
25.     }
26.     Graphics.ExecuteCommandBuffer(_com);
27.     _com.Clear();
28. }
29.  

Trails are drawn in local space, parented to the Plane which displays the RenderTexture.

Attached image is the result. The large line is the user input, and the small copy is the render.

What I've come up with so far has been mostly trial and error. I started with the Unity documentation, but I haven't been able to find a clear explanation of what each of these matrices are supposed to represent. Any thoughts are much appreciated!

 

TrailRenderers (and LineRenderers) don't make very good geometry when the adjacent verts are very close together. The mitering gets all bollixed up because of the tessellation appears to only consider the current point and next, without regard to width or other adjacent possibly degenerate edge verts.

You may have better luck just rendering a circular sprite from one finger sample to the next, basically drawing the line chunks yourself by repeated sprite instantiations each frame, then destroying the sprites after they have been rendered to the RenderTexture.

 

Yeah, immediately after I started this project I realized that the Line and Trail renderers are not at all built for this kind of thing. It's been a massive headache, honestly.

Rendering sprites is definitely a good option. Assuming that rendering them to the texture is fast enough to not drop frames on a Quest 2, it's probably what we'll go with.

I do still want to understand how the transform and projection matrices are supposed to work in this instance. To my eye, it looks like I'm in the ballpark, but the LineRenderer itself just doesn't work well. From what I understand so far, the view matrix is supposed to represent a camera in some local space, and we have to use the projection matrix to translate it back to world space? In particular, I haven't been able to find any real info on the translation matrix on the drawMesh command.

 

Did you ever figure out the matrices? I need to do something similar now and haven't managed to really figure it out yet...