Smoke particles suitable for VR

Hi!

Is it possible to make the unity particle system suitable for VR (e.g: volumetric particles)?
In VR, you can see a strong rotation effect when moving the head.

Thanks!

 

In Unity 2017.1, you can change the smoke particle "render alignment" in renderer module to "facing" which to my knowledge is for removing that unwanted rotation.

 

That helps remove one of the unwanted rotations, but rolling your head side to side it'll still feel attached to your face. It also makes the rotation from head translation worse.

The usual solution for these kinds of effects is to use meshes of some kind. Either sphere-like shapes or intersecting fans that fade out planes when viewed from edge on. The biggest thing to do regardless of what technique you use is to fade out of the effects when the camera gets close.

 

If you still didn't fix that, you can change from Billboard to Mesh. It will make your particle into a cube that will look pretty good.

 

I've been trying to resolve this issue on a gpu particle system I wrote a couple months back, the issue came to my attention after importing it into VR. The particle system in question creates the billboard effect using the following line, where q are the uv's for the current vertex on the quad being rendered:

o.position = mul(UNITY_MATRIX_VP, o.position + float4(q, 0.0, 0.0) * s.starSize * 0.12f);

starsize and the 0.12f are just for scaling the quad as you probably can figure out. I've been trying to figure out how to eliminate the roll associated with rotating the camera to no avail. One of your other posts said to use a separate vector rather than the camera forward, but I'm not entirely sure I understand how that would work or how it would prevent the quads from not just rotating away from the camera at a certain angle, thus invalidating the idea of them being billboarded. I would appreciate if you could shed some light on this issue, discussing the aforementioned method or any others you have in mind, thank you very much for your time.

 

That code seems incomplete. That alone would cause the particles to always be world axis aligned, so the camera would have no impact on their orientation. Usually I see something more like:

float4 viewPos = mul(UNITY_MATRIX_V, worldPos);
o.pos = mul(UNITY_MATRIX_P, viewPos + float4(vertex.xy, 0.0, 0.0) * scale);

That would produce the usual view aligned particles.

The default Unity particles use "View" alignment, and that comment probably comes from an era before they added the "Facing" alignment option, but you'd have to show me the post for me to know for sure that's what I was referring to.

View aligned particles are such that their forward vector is the inverse of the camera's forward vector, and their up vector is the same as the camera's up vector. This means when you turn your head, all of the particles turn too.

Facing aligned particles are such that their forward vector is the direction from that particle's position to the camera's position, and their up vector is derived from the camera's up vector. This means they rotate to face the camera as they move around, but not when just looking around*, which usually looks a little better than View aligned particles. However they're still using the camera's up vector so they rotate when you roll your head.

The real key for VR billboards is to use the world up, or some other world space vector (like their velocity), to derive the particle's up. Using Unity's particles in 2018.2 and earlier, this had to be done with some kind of ugly Get/SetParticle routines. I posted some example code of how to handle this in the Oculus forums some years ago:
https://forums.oculusvr.com/developer/discussion/comment/201733/#Comment_201733

Since 2018.3 though, there's an option to do this in the particle's renderer settings.


Obviously none of that helps if you're writing your own particle system though. The key bit of knowledge you need is how to calculate a rotation matrix from two vectors, mainly the camera to particle direction, and the world up.

Here's an example of how I do it:

Code (csharp):

1. // _WorldSpaceCameraPos is the eye position in VR, not the gameObject camera's position.
2. // Use an average of the two eye positions to get a good middle position so the facing is
3. // the same for both eyes.
4. #ifdef UNITY_SINGLE_PASS_STEREO
5. float3 camWorldPos = (unity_StereoWorldSpaceCameraPos[0] + unity_StereoWorldSpaceCameraPos[1]) * 0.5;
6. #else
7. float3 camWorldPos = _WorldSpaceCameraPos;
8. #endif
9. float3 cameraToParticleOffset = particleWorldPos - camPos;
10. float3 worldUp = float3(0.0, 1.0, 0.0); // world space up
11.  
12. // This assumes you're using the default Unity quad with the surface normal facing -Z
13. float3 forward = normalize(cameraToParticleOffset);
14. float3 right = normalize(cross(up, forward));
15. float3 up = -normalize(cross(right, forward));
16. float3x3 transposedLocalToWorldRotation = float3x3(right, up, forward);
17.  
18. float3 particleLocalVertex = v.vertex.xyz * particleScale;
19. float3 vertexWorldPos = particleWorldPos + mul(particleLocalVertex, transposedLocalToWorldRotation);
20.  
21. o.pos = mul(UNITY_MATRIX_VP, float4(vertexWorldPos, 1.0));

 

You were right, the segment I posted was incomplete, I mistakenly thought I could multiply the vertex position by the VP matrix and achieve the same result as first multiplying the vertex by the View matrix and multiplying that by the Projection matrix. I ran it and it seemed to achieve the same thing but that was only because I have a high concentration of particles and I didn't bother to look around the scene, had I done so, I would've realized some of them are being culled.

I tried out your solution and it fixes the issue of camera roll, but it gives rise to a new issue: When the particles are exceptionally close to the camera's clipping plane and are nearly out of view, they kind of roll as they pass by. This gives the effect of there being an invisible wedge in front of the camera that reorients the particles as they get close. I'm guessing this has to do with how we calculate the forward vector. I think to eliminate this, I will be using a combination of the conventional billboard method, as well as the new method using the world up vector and particle to eye vector, toggling between the two based on distance to particle. Is this something you've noticed as well, or is it maybe a fault in my implementation? Thanks again.

 

Correction, this has nothing to do with the clipping plane, but rather the position of the camera.

 

Yep, that's the exact reason.

Using the camera forward vector is an easy solution if you don't mind the existing rotations that happen when you turn your head, but find the rotation from translations do bother you. Just replace the forward vector in that calculation with the camera's forward vector.

float3 forward = -UNITY_MATRIX_V._m20_m21_m22;

The other kind of hacky solution is to move the position the particles are aiming at behind the player's view by some offset.

float3 cameraToParticleOffset = particleWorldPos - camPos + UNITY_MATRIX_V._m20_m21_m22 * _BillboardAimOffset;

 

I think to get the particle position a little bit behind the camera it'd be

float3 cameraToParticleOffset = particleWorldPos - camPos - UNITY_MATRIX_V._m20_m21_m22 * _BillboardAimOffset;

Subtracting the camera forward vector instead of adding it. I think that's what you meant to write, or perhaps I misunderstood what you meant. I didn't know the camera forward was represented by

UNITY_MATRIX_V._m20_m21_m22

, very helpful, doesn't seem to be enclosed on the helper function page for shaderlab by unity.

Your proposed solution (or at least as I understand it) of setting the forward vector to offshoot the camera by a factor worked pretty well for me.

 

Depends on how you want to set your _BillboardAimOffset value.

There's a lot of stuff that's not enclosed in any kind of helper function. The most egregious to me is the lack of object to world space wrapper, so people have something like this line all over the place:

float3 worldPos = mul(unity_ObjectToWorld, pos).xyz;

Especially annoying since depending on if pos is a float3 or a float4 it can get people in trouble (it needs to be a float4), and in Unity's own code they swap between using using "pos" straight and "float4(pos.xyz, 1.0)" to ensure it gets translation applied. Ah well.

But yeah, there's a ton of useful data that can be extracted from the various matrices. It's the kind of thing experienced shader devs already know since, well, it's a matrix, it holds that kind of stuff, but isn't obvious to people just starting out that don't have the knowledge on what matrices are or how they work.

Other good ones are the object's world position:
unity_ObjectToWorld._m03_m13_m23

Or extracting the object's scale:

Code (csharp):

1. float3 scale = float3(
2.     length(unity_ObjectToWorld._m00_m10_m20),
3.     length(unity_ObjectToWorld._m01_m11_m21),
4.     length(unity_ObjectToWorld._m02_m12_m22)
5.     );

Or just the rotation:

Code (csharp):

1. float3 c0 = normalize(unity_ObjectToWorld._m00_m10_m20);
2. float3 c1 = normalize(unity_ObjectToWorld._m01_m11_m21);
3. float3 c2 = normalize(unity_ObjectToWorld._m02_m12_m22);
4. float3x3 objectToWorldRotation = float3x3(
5.     c0.x, c1.x, c2.x,
6.     c0.y, c1.y, c2.y,
7.     c0.z, c1.z, c2.z
8.     );

You can also get the object's forward vector the same way as you get the camera's forward vector, though you'll want to normalize that one since it might be scaled and the view matrix never is.

 

It is unreasonable to expect that all would be in documentation, I've just got to acquire the experience.
I am quite unlearned on the topic of shaders, but I'm getting there little by little. In no small part thanks to people like you. Thank you very much for your guidance through this issue and all the other information you've provided.