Search Unity

  1. The 2022.1 beta is now available for testing. To find out what's new, have a look at our 2022.1 beta blog post.
    Dismiss Notice

How to create point cache (of texture2D) from code?

Discussion in 'VFX Toolbox' started by cyz2727327, May 6, 2021.

  1. cyz2727327

    cyz2727327

    Joined:
    Nov 4, 2016
    Posts:
    1
    Hi everyone,

    I want to simply visualize the RGBD camera image (texture2D) using the VFX graph as 3D point cloud. I would like to update the image frame-by-frame to show the live captured 3D image, or a recorded video as point cloud.

    Based on my current understanding, VFX graph only takes Attribute Map pixel position and color, and these Attribute Maps are properties of Point Cache, which is generated from texture2D data by a tool called Point Cache Bake Tool. However, I cannot access this tool via code, and therefore cannot automate the entire process frame-by-frame.

    This seems to be a very easy thing to do, but I have been struggling to find a solution. Can anyone please tell me what I've been missing?
     
  2. AndrewStyan

    AndrewStyan

    Joined:
    Apr 6, 2020
    Posts:
    13
    This is probably not what you need as it is not point cloud data and not live, but may give a clue as I did tackle how to get position and colour into Textures that could then be used by VFX Graph.

    First create 2 Textures. My data came from a CSV file that held ~5000 x, y, z values and another value that I converted to a colour. The position data went into texPos texture and the colour went into the valPos texture. Here is the method (there was a separate csv load method).
    texSize * texSize needs to be large enough to hold all the values.

    You will see from the code that values are first stored 4 at a time as floats into an array with R=x, G=y, B=z as it is an RGBA float32 format. Even though I filled the A value it wasn't used as I recall that the A value is not readily accessible in VFX graph (but I think I now know how to get it by treating is as an actual alpha value).

    Each array is then loaded into the texture using SetPixelData.

    There may be a way of doing this on a frame by frame basis if your needs are for a live data feed. Don't know.

    Code (CSharp):
    1. void Load2DTexture () {
    2.  
    3.     float minalt = 1;
    4.     float maxalt = 0;
    5.     // resize the texture
    6.     texPos.Resize (texSize, texSize, TextureFormat.RGBAFloat, false); // 4 data per vertex
    7.     texVal.Resize (texSize, texSize, TextureFormat.RGBAFloat, false); // 4 data per vertex
    8.  
    9.     // split lines into records
    10.     string[] records = fileContents.Split ('\n');
    11.     if (debugging) print ("...................................................   Processed file : " + filename + " at folder " + Application.persistentDataPath + "  [records: " + records.Length + "]");
    12.     if (debugging) Debug.Log ("File records: " + records.Length);
    13.     // + "  format supported " + SystemInfo.SupportsTextureFormat(TextureFormat.RGBAFloat) + " max records : "+texSize*texSize);
    14.  
    15.     int indPos = 0;
    16.     int indVal = 0;
    17.     float[] dataPos = new float[texSize * texSize * 4];
    18.     float[] dataVal = new float[texSize * texSize * 4];
    19.     // read file and assign array values
    20.     for (int i = 1; i < records.Length - 1; ++i) // why is records number over-reported by 1 ???
    21.     {
    22.       // if(indPos > 27830) Debug.Log("indPos is " + indPos + "  i is " + i);
    23.       string[] fields = records[i].Split (',');
    24.       // create position data ( must be 4 per record)
    25.       dataPos[indPos] = float.Parse (fields[2]); // x
    26.       indPos++;
    27.       dataPos[indPos] = float.Parse (fields[3]); // y
    28.       indPos++;
    29.       dataPos[indPos] = float.Parse (fields[4]); // z
    30.       indPos++;
    31.       float altRatio = float.Parse (fields[5]) / altMax; // altitude ratio
    32.       dataPos[indPos] = altRatio; // [0f] height as a ratio (accessible in VFX using a Sample Texture2DArray block)
    33.       indPos++;
    34.  
    35.       // create values data (must be 4 per record) - mostly color variables rgb
    36.       float age = (float) i / vertexCount; // age (0...1)
    37.       //  Color col = Color.Lerp(Color.yellow, Color.blue, altRatio);
    38.       //  float _hue = Mathf.Lerp(0.07f, 0.83f, altRatio);
    39.       minalt = Mathf.Min (minalt, altRatio);
    40.       maxalt = Mathf.Max (maxalt, altRatio);
    41.       Color col = gradient.Evaluate (altRatio);
    42.       // Color col = Color.HSVToRGB(_hue, 1, 1, true);
    43.       dataVal[indVal] = col.r;
    44.       indVal++;
    45.       dataVal[indVal] = col.g;
    46.       indVal++;
    47.       dataVal[indVal] = col.b;
    48.       indVal++;
    49.       dataVal[indVal] = age; // [1.0f] could be age or id;  does this have to be alpha (=1.0f) if it is to be used in a 'Set Color from Map' block?
    50.       indVal++;
    51.     }
    52.     if (debugging) Debug.Log ("min/max alt : " + minalt + "  " + maxalt);
    53.     texPos.SetPixelData (dataPos, 0, 0); // mip level 0
    54.     texPos.Apply ();
    55.     texVal.SetPixelData (dataVal, 0, 0); // mip level 0
    56.     texVal.Apply ();
    57.   }
    In VFX graph here are the relevant nodes and parameters.

    Number of particles to spawn = number of points in the data
    position map = texPos
    values map = texVal

    I was outputting to a Particle line so needed Set Target Position from Map, you may only need Set Position from Map. This function indexes over the texPos texture and uses the RGB values as x,y,z. Likewise for Set Color from Map. The index comes from Get Attribute: particleId.

    Hope this helps, not sure I can help more but ask away.

    I recall I had to piece this together from an example in https://github.com/keijiro/VfxGraphTestbed

    upload_2021-8-16_16-22-41.png
    upload_2021-8-16_16-43-8.png


    The entire graph is this and was used to produce the following 3D model .
    upload_2021-8-16_17-9-6.png upload_2021-8-16_17-10-17.png
     
    cyz2727327 likes this.
unityunity