Search Unity

Question Live voxelized resrsentaion of a shader

Discussion in 'Visual Effect Graph' started by Afarshezif, Feb 18, 2022.

  1. Afarshezif

    Afarshezif

    Joined:
    Dec 6, 2020
    Posts:
    6
    Hello all.

    I am seeking a working solution for feeding live texture to my VFX graph.
    I have a VFX graph based on the Voxelized Terrain example which creates a voxelized representation of a texture2D. Aside to it i have a C# script (or a shader graph) that creates an animated heightMap texture.
    I like to run a process to dynamically set the graph with the new data.

    I was trying to use Graphics.Blit to sample the shader and set the texture property on each frame, but the result is quite glitchy.


    Code (CSharp):
    1.         // Create a temporary RenderTexture of the same size as the texture
    2.             RenderTexture tmp = RenderTexture.GetTemporary(
    3.                                 size,
    4.                                 size,
    5.                                 0,
    6.                                 RenderTextureFormat.Default,
    7.                                 RenderTextureReadWrite.Linear);
    8.  
    9.             // Blit the pixels on texture to the RenderTexture
    10.             Graphics.Blit(null, tmp, material);
    11.  
    12.             // Backup the currently set RenderTexture
    13.             RenderTexture previous = RenderTexture.active;
    14.  
    15.             // Set the current RenderTexture to the temporary one we created
    16.             RenderTexture.active = tmp;
    17.  
    18.             // Create a new readable Texture2D to copy the pixels to it
    19.             Texture2D myTexture2D = new Texture2D(size, size);
    20.  
    21.             // Copy the pixels from the RenderTexture to the new Texture
    22.             myTexture2D.ReadPixels(new Rect(0, 0, tmp.width, tmp.height), 0, 0, false);
    23.             myTexture2D.Apply();
    24.  
    25.             // Reset the active RenderTexture
    26.             RenderTexture.active = previous;
    27.  
    28.             // Release the temporary RenderTexture
    29.             RenderTexture.ReleaseTemporary(tmp);
    30.  
    31.             myTexture2D.wrapMode = TextureWrapMode.Clamp;
    32.             visualEffect.SetTexture(propartyName, myTexture2D);


    Inspired by some repos demonstrating live input of Realsense and Kinect cameras, i believe i can achieve a working solution.

    Thanks for your help!
     
    Afarshezif, Feb 18, 2022
    #1

  2. VladVNeykov

    VladVNeykov

    Unity Technologies

    Joined:
    Sep 16, 2016
    Posts:
    550
    Hi @Afarshezif ,

    If you have already created the render texture in your project folder, you just need to assign it within your graph, no need for any additional updating.
    upload_2022-2-18_20-6-37.png

    If the render texture is procedurally created, then you can expose a Texture2D in blackboard and assign it with
    Code (CSharp):
    1. yourVisualEffectComponent.SetTexture("ExposedTextureName", yourRenderTexture);
    upload_2022-2-18_20-6-28.png

    You'll just need to set it once, and can then keep modifying it at will in your code and its changes should just appear in the VFX Graph.

    Hope this helps!
     
    VladVNeykov, Feb 18, 2022
    #2

  3. Afarshezif

    Afarshezif

    Joined:
    Dec 6, 2020
    Posts:
    6
    Dear @VladVNeykov , thank you for your reply!

    I was trying to work with the "set position from map" component but it seems it does not correlate with the current heightMap texture. I'm attaching screenshots here of my graph structure (all the texture position calculations are taken from the voxelized sample project).

    I am trying to achieve a live procedural texture generation. I am aware I can set the texture to a VisualEffect instance, but I fail to convert a running shader material to a texture2D without glitching errors. To be clear - the shader is animating while the game is running, I like to sample the shader on each frame and pass it to the VFX graph. (For example to result can be similar to passing a live webcam feed).

    Thank you again for your help!
     

    Attached Files:

    Afarshezif, Feb 18, 2022
    #3

  4. VladVNeykov

    VladVNeykov

    Unity Technologies

    Joined:
    Sep 16, 2016
    Posts:
    550
    Hi,
    I looked through your graph, but I'm not sure what to look for. What is the issue you are having?

    In what way does it not correlate?

    Same here, how is it glitchy? You are getting unstable results, reliable results, but offset somehow, or just noise?

    Trying to understand if you are having a VFX Graph issue per se, or something with how you are deriving your texture / the shader you blit with, etc.

    I'm afraid I won't be of much help, so will just throw a bunch of things at the wall to see if any will stick :D

    If it's an issue with the data, and not the VFX Graph, I'd recommend exposing and assigning to all your textures (whatever you are getting from your webcam feed, the temp and previous render textures, your myTexture, etc.) and double-clicking them in the inspector to preview and debug at what stage your data gets messy.

    Potentially check also if you are doing any color space conversion on the textures and if you have any mipmaps enabled, which could skew the results you are getting. Also see if your wrap mode is what you want it to (i.e. clamp if you don't want any repetition on the edges).

    On the VFX Graph side the blend scale logic from what I could see looks good, maybe clamp your Remap (Vector2) operator to ensure your UVs are in the 0-1 range.

    And you could also plug the result from your HeightMap SampleTexture2D to a SetColor block to debug what values you are getting in the graph itself.

    Hopefully some of this might hint towards what exactly is happening :)
     
    VladVNeykov, Feb 19, 2022
    #4