Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

Question Rendering mesh on a texture

Discussion in 'General Graphics' started by Mozmarto, Feb 8, 2021.

  1. Mozmarto

    Mozmarto

    Joined:
    Dec 27, 2014
    Posts:
    4
    I am wondering whether there is a way to render a mesh from MeshRenderer directly to a RenderTexture.

    The reason I need this is because I am using Spine 2d for character animations and in need of making a custom outline shader. Spine 2d uses MeshRenderer to render animations onto a screen so I need to take the rendered image and apply shader to it (otherwise the outline will be applied to a spritesheet and not the whole animation). Unfortunately rendering an image from a separate camera is not a solution in my case since there will be quite a lot of characters with unique outlines applied to them.

    I guess the closest thing I've found is Graphics.SetRenderTarget function along with Graphics.DrawMeshNow but I couldn't figure out how to correctly use them in my case.

    The closest thing I got to is this code:

    Code (CSharp):
    1. public class animation_test : MonoBehaviour
    2. {
    3.     //buffers that camera render to by default
    4.  
    5.     //a render texture that mesh should render to (potentially created in runtime)
    6.     public RenderTexture rt;
    7.     //mesh renderer that should render itself into a texture
    8.     public MeshRenderer mr;
    9.     public MeshFilter mf;
    10.     public Texture2D tex;
    11.  
    12.     public SkeletonAnimation anim;
    13.     // Start is called before the first frame update
    14.  
    15.     private void Awake()
    16.     {
    17.     }
    18.     void Start()
    19.     {
    20.         var animation_state = anim.AnimationState;
    21.         animation_state.SetAnimation(0, "Walk", true); //start spine animation
    22.  
    23.         RenderPipelineManager.endCameraRendering += OnPR;
    24.     }
    25.  
    26.     // Update is called once per frame
    27.     void Update()
    28.     {
    29.     }
    30.  
    31.     void OnPR(ScriptableRenderContext c, Camera cc)
    32.     {
    33.         if (!Application.isPlaying) return; //don't render when in editor
    34.         if (mr == null) return;
    35.  
    36.         RenderTexture.active = rt;
    37.  
    38.         Graphics.DrawMeshNow(mf.mesh,Vector3.one,Quaternion.identity); //render a mesh into a render texture immediately
    39.         //Graphics.DrawTexture(new Rect(0, 0, 20, 20), tex);
    40.        
    41.         //RenderTexture.active = null;
    42.  
    43.     }
    44. }
    It works okay when drawing texture (although I still haven't figured out how to properly set position and scale). But whenever I try to draw a mesh it renders the whole image (including image from a camera) on small square in the middle of the screen.
     
    Eugen78 likes this.
  2. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,396
    Look into command buffers. Specifically
    DrawRenderer
    .
     
    Mozmarto likes this.
  3. Mozmarto

    Mozmarto

    Joined:
    Dec 27, 2014
    Posts:
    4
    Using command buffers definitely helped to make some progress with this.
    Here's how I used it, I don't know whether it is 100% correct:

    Code (CSharp):
    1.     private void OnWillRenderObject()
    2.     {
    3.         rt.Release(); //clear the texture
    4.  
    5.         mr.transform.localScale = new Vector3(100f, -100f, 1f); //switch scale to match texture pixel coordinates
    6.  
    7.         buffer.SetRenderTarget(rt); //setrender texture [rt]
    8.  
    9.         buffer.DrawRenderer(mr, mr.material); //draw MeshRenderer [mr]
    10.  
    11.         Graphics.ExecuteCommandBuffer(buffer);
    12.  
    13.         mr.transform.localScale = new Vector3(1f, 1f, 1f); //reset scale
    14.     }
    As far as I understand DrawRenderer transforms position and scale units directly into pixels? Is there a way to affect this without changing the transform of the object?
     
  4. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,396
    You want to also set the view and projection matrices to align a virtual camera to the mesh renderer you're trying to capture.
    https://docs.unity3d.com/ScriptReference/Rendering.CommandBuffer.SetViewProjectionMatrices.html
    https://docs.unity3d.com/ScriptReference/Matrix4x4.TRS.html
    https://docs.unity3d.com/ScriptReference/Matrix4x4.Ortho.html

    Position the view at the center of the mesh renderer's world bounds, or at its pivot if it's in a useful location, rotated to aim towards your mesh, and apply the scale matrix as shown in the first link. Then create an orthographic projection that's the size you need to cover the entire renderer (which you can use the bounds of the mesh renderer to figure out precisely if you want, though for an outline you'll presumably want it a bit bigger than that).

    That way your mesh doesn't have to move at all.

    Alternatively, you could use
    CommandBuffer.DrawMesh
    or your original
    Graphics.DrawMeshNow
    with a transform matrix that scales up the mesh to cover the screen. Use that same
    Matrix.TRS
    function mentioned above. Some confusion you might be having is even though you're not assigning one, the rendering does use a view and projection matrix, probably some default orthographic one.

    You might also find this article useful.
    https://bgolus.medium.com/the-quest-for-very-wide-outlines-ba82ed442cd9
    For that I don't bother trying to limit the render texture to just the object in question, but instead to the entire screen (with some experimental, but ultimately unused screen space limiting techniques) and still manage to draw outlines very quickly.
     
    nath1339 and Mozmarto like this.
  5. techmage

    techmage

    Joined:
    Oct 31, 2009
    Posts:
    2,133
    I am trying to get this to work but have the issue that my object is still rendering to my main framebuffer through my main camera when I call Graphics.ExecuteCommandBuffer(m_CommandBuffer).

    Is there anyway to prevent that? I just want to render it straight into the rendertexture and be entirely non-visible through the main camera.
     
  6. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    12,396
    Then you don’t have a being render target set by the command buffer.

    Make sure there’s a
    SetRenderTarget
    , and that the render target is still valid when you’re executing the command buffer.
     
  7. techmage

    techmage

    Joined:
    Oct 31, 2009
    Posts:
    2,133

    It is also rendering into the rendertexture as expected, so I believe that is set properly? But it still renders to the main camera.
     
  8. techmage

    techmage

    Joined:
    Oct 31, 2009
    Posts:
    2,133
    Actually, got it working. Have to make sure you clear the command buffer after executing, otherwise I guess it somehow stays in some command queue somewhere and renders to the main camera? I don't really understand that though if I am making a new CommandBuffer.

    Also seems I have to do that oddity with m_ExecuteBuffer = true in Update then checking that in OnRenderObject because OnRenderObject seems to get called multiple times per frame?


    Code (CSharp):
    1.     void Start()
    2.     {
    3.         m_RenderTexture = new RenderTexture(m_Resolution.x, m_Resolution.y, 24);
    4.         m_TestDisplay.material.mainTexture = m_RenderTexture;
    5.         m_CommandBuffer = new CommandBuffer();
    6.     }
    7.  
    8.     void Update()
    9.     {
    10.         m_ExecuteBuffer = true;
    11.     }
    12.  
    13.     void OnRenderObject()
    14.     {
    15.         if (!m_ExecuteBuffer)
    16.             return;
    17.  
    18.         m_ExecuteBuffer = false;
    19.        
    20.         Debug.Log(Time.time);
    21.        
    22.         m_CommandBuffer.Clear();
    23.         m_CommandBuffer.SetRenderTarget(m_RenderTexture);
    24.         m_CommandBuffer.ClearRenderTarget(true, true, Color.clear);
    25.        
    26.         Matrix4x4 matrix = Matrix4x4.TRS(transform.position, transform.rotation, transform.lossyScale);
    27.         m_CommandBuffer.DrawMesh(m_Renderer.sharedMesh, matrix, m_SplineBakerMaterial, 0);
    28.  
    29.         Graphics.ExecuteCommandBuffer(m_CommandBuffer);
    30.         m_CommandBuffer.Clear();
    31.     }
     
  9. techmage

    techmage

    Joined:
    Oct 31, 2009
    Posts:
    2,133
    Ok forget above post. It was all whack.

    If your rendering straight to rendertexture you can do it all in Update and then when you take it out of OnRenderObject you don't have to clear the command buffer.

    Code (CSharp):
    1.  
    2. void Start()
    3. {
    4.     m_RenderTexture = new RenderTexture(m_Resolution.x, m_Resolution.y, 24);
    5.     m_TestDisplay.material.mainTexture = m_RenderTexture;
    6.     m_CommandBuffer = new CommandBuffer();
    7.     m_CommandBuffer.name = "SplineBaker";
    8.     m_CommandBuffer.SetRenderTarget(m_RenderTexture);
    9.    
    10.     var lookMatrix = Matrix4x4.TRS(new Vector3(0, 0, -1), Quaternion.identity, Vector3.one * 100);
    11.     var orthoMatrix = Matrix4x4.Ortho(-1, 1, -1, 1, 0.01f, 2);
    12.     m_CommandBuffer.SetViewProjectionMatrices(lookMatrix, orthoMatrix);
    13. }
    14.  
    15. void Update()
    16. {
    17.     m_CommandBuffer.ClearRenderTarget(true, true, Color.clear);
    18.     m_CommandBuffer.DrawMesh(m_Renderer.sharedMesh, m_IdentityTRS, m_SplineBakerMaterial, 0);
    19.     Graphics.ExecuteCommandBuffer(m_CommandBuffer);
    20. }
    21.  
    My issue now is I am trying to get worldPosition rendered as color output from the fragment through this rendertexture and mesh, but seems that doesn't come through? I can see worldPosition color if I just look at the mesh through a plain ortho cam, but through my rendertexture and command buffer its just solid.
     
    unity_uM_sDw6vPRNRBw likes this.
  10. techmage

    techmage

    Joined:
    Oct 31, 2009
    Posts:
    2,133
    Ok forget prior two posts above, serious issues. You do need to keep calling CommandBuffer.Clear before every submission otherwise it just keeps stacking DrawMesh commands in it, duh

    This is good. Also figured my issue with worldPosition colors was my mesh was scaled too small:
    Code (CSharp):
    1.     readonly Matrix4x4 m_LookMatrix = Matrix4x4.TRS(new Vector3(0, 0, -1), Quaternion.identity, Vector3.one);
    2.     readonly Matrix4x4 m_OrthoMatrix = Matrix4x4.Ortho(-1, 1, -1, 1, 0.01f, 2);
    3.     Matrix4x4 m_TransformTRS;
    4.     Texture2D m_Texture2D;
    5.     RenderTexture m_RenderTexture;
    6.     CommandBuffer m_CommandBuffer;
    7.  
    8.     void Start()
    9.     {
    10.         m_RenderTexture = new RenderTexture(m_Resolution.x, m_Resolution.y, 24);
    11.         m_TestDisplay.material.mainTexture = m_RenderTexture;
    12.         m_CommandBuffer = new CommandBuffer();
    13.         m_CommandBuffer.name = "SplineBaker";
    14.         m_TransformTRS = Matrix4x4.TRS(Vector3.zero, Quaternion.identity, Vector3.one * m_MeshScale);
    15.     }
    16.  
    17.     void Update()
    18.     {
    19.         m_CommandBuffer.Clear();
    20.         m_CommandBuffer.SetRenderTarget(m_RenderTexture);
    21.         m_CommandBuffer.ClearRenderTarget(true, true, Color.clear);
    22.         m_CommandBuffer.SetViewProjectionMatrices(m_LookMatrix, m_OrthoMatrix);
    23.         m_CommandBuffer.DrawMesh(m_Renderer.sharedMesh, m_TransformTRS, m_SplineBakerMaterial, 0);
    24.         Graphics.ExecuteCommandBuffer(m_CommandBuffer);
    25.     }