Question Most computationally friendly method of spawning and controlling a large number of objects

I am building a VR (and possibly AR) application where I have a large dataset of particles with positions (x,y,z) at different time steps. The particles update their position at each timestep (30 fps), and there are loads of particles (approx. 10 000). This makes for a very large JSON dataset.

I want to display as many of these particles as possible. What is a VR feasible and computational friendly way of doing so? If I try to spawn all particles and separate sphere GameObjects, I very quickly get performance issues.

For reference, I am using a Meta Quest 2 VR headset.

 

Quest2 won't do 10k render calls, period. 1 separate unique object = 1 render call in most circumstances. A particle system should be able to do them all in one draw call though -- so if they're just particles just render them as particles. 10k is a lot for the cpu based particles system but may be workable, it's nothing for the VFX graph though.

If you want to update, or at all interact with the particles' positions every frame with code, using jobs with CPU particles will be the only workable way at your type of scale : https://docs.unity3d.com/es/2019.4/Manual/particle-system-job-system-integration.html. You can write to the particles' native data directly that way, otherwise you'll have like a 3-5ms delay every frame it updated the positions as that info gets copied to the gpu for displaying. Note you'll have to code all of the particles interactions manually if you're doing it this way, with all of the limitations of the job system (e.g. no strings (JSON)/objects/if using Burst). Even then this may be too much for mobile, I would not attempt this without Burst.

For the JSON input data, you'll have to copy it to native data structure (e.g. NativeArrays) before using it in the jobs.

Alternatively you can also use jobs to write the positions to a float3 texture and then read them if you would rather use VFX graph, (added later: you can use a time scrolling texture to write out all of the positions over time. The texture would need to be at least like 10k pixels on a side... the absolute max number of particles you could uniquely address is 16384 as that's the texture size limit on one side in unity... unless you start using multiple textures... however I would assume that much compute would be hard to do with VFX graph on modern mobile VR headsets. I've made a few PC VR based data visualizers this way and can confirm it works very well when using like 1-2k particles, and I can run a dozen of them at a time only using a small amount of a 3080-TI's power. I haven't tested higher particle counts though.)

 

Also you could look into using DOTS

 

Look into ECS and the Entities Graphics package. I haven't tried Entities Graphics on Quest for a while, but XR is now officially supported (so file a bug report if something breaks). It will do all the heavy lifting for you in terms of instancing and updating the rendering data. It is not entirely feature complete yet (mesh animation/skinning are still not great), but should be fine for simple particle rendering.

With this you will want to use the ECS Baking system to bake your JSON data into a blob asset. No good messing with JSON at runtime.

Sounds like your positions are all pre-calculated in a dataset so I would be tempted to go through and find the maximum extents for all the points and, if the range isn't too great, encode them all as normalized fixed-point 16-bit values to save memory. Bandwidth is always tight on mobile VR.

 

how many frames it is?
if it can fit all in memory, then that would be fast to playback..
(either using particle system or single mesh with point rendering)

and do convert json into even raw binary first, so that loading is faster.

other option to test could be converting/encoding those positions into textures,
then make mp4 video out of it
and play that on a high resolution plane
where shader moves the vertices based on texture RGB colors.. (but would be lossy)

 

Is VFX more computationally friendly compared to using the particle system?

The number of particles is changing throughout the simulation so I need something that can account for that. Also, I need to set the exact position of each particle at each frame.

I am not familiar with particle system job system integration or using float3 texture with VFX, so I would appreciate your recommendation and tips on getting started.

How do I need to format the data properly in to NativeArrays? My JSON data is on the following format (I have allready interpolated the data to a fps of 30 so all I am doing is setting the application framerate to 30 and iterate through the keys):

Code (CSharp):

1. {
2.     '0.033 (timestamp)': [
3.         {
4.             'particle ID': 1,
5.             'x': 1,
6.             'y': 2,
7.             'z': 2,
8.        },
9.        {
10.             'particle ID': 2,
11.             'x': 1,
12.             'y': 2,
13.             'z': 2,
14.        }
15.     ],
16.     '0.066': [
17.         {
18.             'particle ID': 1,
19.             'x': 2,
20.             'y': 3,
21.             'z': 4,
22.        },
23.        {
24.             'particle ID': 2,
25.             'x': 5,
26.             'y': 2,
27.             'z': 2,
28.        }
29.     ]
30. }

 

I created a script to control Unity's Particle System. The position updates looks fine, and I can run with a lot of particles on the Quest 2, however, I have problems with the rate of emission. The particle system is constantly spawning new particles, while I only want the particles contained in the data set for the current timestep to be displayed. Any idea how I can improve the script to do so?

Also, I am loading the JSON file here which works, but is very slow. How can I make it faster? I can transform the data on beforehand to whatever format if that helps performance.

Extra: I try to cap the framerate to 30 as that is what the data is interpolated to, however, the meta quest 2 seems to ignore Application.targetFrameRate = 30; and runs at its own FPS (I believe 72). Is there a way to cap it to 30, or do I have to interpolate to 72 FPS?

Code (CSharp):

1. using System.Collections;
2. using System.Collections.Generic;
3. using Newtonsoft.Json;
4. using UnityEngine;
5.  
6. public class ParticleController : MonoBehaviour
7. {
8.     public TextAsset jsonFile;
9.  
10.     private Dictionary<float, List<DataItem>> jsonData;
11.     private List<KeyValuePair<float, List<DataItem>>> keyValuePairs;
12.     private int jsonIndex = 0;
13.     ParticleSystem ps;
14.  
15.     void Start()
16.     {
17.         // Set frame rate
18.         Application.targetFrameRate = 30;
19.         ps = GetComponent<ParticleSystem>();
20.  
21.         // Load JSON from Resources
22.         string jsonString = jsonFile.text;
23.  
24.         // Deserialize JSON
25.         jsonData = JsonConvert.DeserializeObject<Dictionary<float, List<DataItem>>>(jsonString);
26.  
27.         // Convert dictionary to a list of key-value pairs for iteration
28.         keyValuePairs = new List<KeyValuePair<float, List<DataItem>>>(jsonData);
29.     }
30.  
31.     private void LateUpdate()
32.     {
33.         // RESET SCRIPT IF END OF JSON IS REACHED
34.         if (jsonIndex >= keyValuePairs.Count)
35.         {
36.             Debug.Log("Restarting script.");
37.             jsonIndex = 0;
38.         }
39.  
40.         // Get the current key-value pair
41.         KeyValuePair<float, List<DataItem>> currentKeyValuePair = keyValuePairs[jsonIndex];
42.  
43.         // Get the length of the list of data items
44.         int dataItemsLength = currentKeyValuePair.Value.Count;
45.         List<DataItem> currentRows = currentKeyValuePair.Value;
46.  
47.         ParticleSystem.Particle[] particles = new ParticleSystem.Particle[dataItemsLength];
48.         ps.GetParticles(particles);
49.  
50.         for (int i = 0; i < particles.Length; i++)
51.         {
52.             DataItem item = currentRows[i];
53.             particles[i].position = new Vector3(item.X, item.Y, -item.Z);
54.         }
55.  
56.         ps.SetParticles(particles, particles.Length);
57.  
58.         jsonIndex++;
59.     }
60. }
61.  

 

1) You'll probably want to make the particles have a infinite or like 10,000 sec lifetime, and spawn the exact number you need, or manually spawn them as needed and work out the proclivities for yourself. I would use a burst compiled job to write the positions to a texture's native data using: https://docs.unity3d.com/ScriptReference/Texture.GetNativeTexturePtr.html

2) How you handle converting your input data into native data a job could use I don't know. You just need to copy the data to a native data structure to make it work with burst, you could do it in C# if you need to work with a lot of JSON data (classes / strings), do so on a thread if performance is important, because you're doing this in VR and it is. I don't know your setup to get in more detail.

3) You could also just write to the particles positions directly to the particles native data, if you use burst you can't get faster than doing this, and you have prefect control over any aspects of the particles you want: https://docs.unity3d.com/es/2019.4/Manual/particle-system-job-system-integration.html".

4) Or use the texture route and make your own shader to draw the data if you want to make a custom shader graph like this: (See the Voxalized Terrain example here for how to remap a 1d or 2d, or 3d surface into a map (graph): https://github.com/Unity-Technologies/VisualEffectGraph-Samples)

 

Ask GPT4 for help with the row switching if it's confusing. It just copying data stored in one table row configuration into a different order (time -> id -> cord), could map to a pixel location at [x: id for a particle on a row, y: time stamp for the pixel] -> (pixel to store data in for that timestep), holding (cord) [position to place the particle in 1, 2 or 3D space].