Bounding box around the shadow of an object

Hello,

Is it possible to draw an outline/bounding box around the object's shadow only?

I found this thread, which explains how to draw a bounding box around an object. http://answers.unity3d.com/questions/292031/how-to-display-a-rectangle-around-a-player.html.

However, is it possible to draw one around the shadow of an object?

This is what I'd like to achieve:
https://drive.google.com/open?id=0B0gogWMq7zHXOXFVQWdiSUZjYW8

I need this to make the a screenshot of only the shadow. And the screenshot's width and height has to be as minimum as possible. The shadow must fit in perfectly.

Thanks

 

It's always possible, but it might be very tricky.

Is the surface the shadow casts on always flat? In that case it's very easy, just make a copy of the model and project it onto the plane. Then draw a bounding box around that object.

In the case the surface is not always flat, it becomes a very different exercise.

 

Yes, it is always flat. However, I quite don't understand the solution you propose. You suggest I make a copy of the object that casts the shadow (a box for example) and somehow project it onto the plane. How does one project it onto the plane? (and yes, the shadow is always cast on a plane)

Thank you!

PS. Another, a more difficult solution, which I thought of was to create a setup of only one directional light. I will know the angle of the light. I also know the position of the object that casts the shadow. With this information, I should be able to calculate the length (size) of the shadow: its final starting and end position.

 

Well, just a little vector math can do the projection on a plane. Can be done for any plane normal, but in this case I think it's save to assume the plane normal is just positive y.

Vector3 sunDir; // Sun direction
Vector3 vertex; // Vertex xyz
float planeY; // Y value of plane
vertex += sunDir * ((vertex.y - planeY) / sunDir.y); // Projection

This also assumes a directional light of course.

 

I tried this solution and came up with this code:

Code (CSharp):

1.  
2. [SerializeField] Transform sunTransform;
3. [SerializeField] Transform planeTransform;
4. [SerializeField] MeshFilter mf;
5.    
6. void Update () {
7.         Vector3 sunDir =sunTransform.localRotation.eulerAngles;
8.         for (int i=0; i< mf.mesh.vertexCount; i++)
9.         {
10.             Vector3 vertex = mf.mesh.vertices[i];
11.             float planeY = planeTransform.position.y; // Y value of plane
12.             vertex += sunDir * ((vertex.y - planeY) / sunDir.y); // Projection
13.         }
14.     }

This however, froze Unity due to the high amount of vertices. And secondly, when I tried it with a cube (to prevent freezing), nothing really happened.

 

No, nothing will really happen that way. You need to actually maintain the list of vertices and set it back on the mesh. Things are pass by copy in C#, not pass by reference.

 

For what you want I would suggest projecting only the bounding box of your object onto the plane. This kind of mass calculation is what GPUs eat for breakfast, far more complex math is being done to render it on screen every frame. CPUs aren't quite as good at this, so even this simple projection is going to be way too slow to do, as you experienced.

Basic steps:
1. Get local mesh bounds.
2. Transform corners into world space.
3. Project corners onto plane.
4. Transform from world space into screen space.
5. Get min & max bounds of those transformed corners in screen space.
6. Draw bounds.
7. ...
8. Profit!

I suggest starting with the local mesh bounds rather than the world space renderer bounds as it should result in a tighter on screen bounds than using the renderer bounds. The 8 additional local to world transforms shouldn't be a problem for performance.

 

Also, because it'll bite you of you don't, you need to use https://docs.unity3d.com/ScriptReference/Renderer-localToWorldMatrix.html to transform the local mesh bounds into world space, not the Transform local to world function.

 

@bgolus
Hey, so I tried putting the steps you wrote into code and came up with this:

Spoiler: ProjectScript

Code (CSharp):

1. using UnityEngine;
2.  
3. public class ModelProjector : MonoBehaviour {
4.  
5.     [SerializeField] Camera cam;
6.     [SerializeField] Transform sunTransform;
7.     [SerializeField] Transform planeTransform;
8.     [SerializeField] MeshFilter mf;
9.     [SerializeField] MeshRenderer mr;
10.  
11.     private Vector3[] pts = new Vector3[8];
12.  
13.     void OnGUI() {
14.  
15.         // 1. Get local mesh bounds
16.         // Bounds mfBounds = mf.mesh.bounds;
17.         Bounds mfBounds = mr.bounds;
18.  
19.         // 2. Transform corners into world space
20.         // All 8 vertices of the bounds
21.         pts[0] = new Vector3(mfBounds.center.x + mfBounds.extents.x, mfBounds.center.y + mfBounds.extents.y, mfBounds.center.z + mfBounds.extents.z);
22.         pts[1] = new Vector3(mfBounds.center.x + mfBounds.extents.x, mfBounds.center.y + mfBounds.extents.y, mfBounds.center.z - mfBounds.extents.z);
23.         pts[2] = new Vector3(mfBounds.center.x + mfBounds.extents.x, mfBounds.center.y - mfBounds.extents.y, mfBounds.center.z + mfBounds.extents.z);
24.         pts[3] = new Vector3(mfBounds.center.x + mfBounds.extents.x, mfBounds.center.y - mfBounds.extents.y, mfBounds.center.z - mfBounds.extents.z);
25.         pts[4] = new Vector3(mfBounds.center.x - mfBounds.extents.x, mfBounds.center.y + mfBounds.extents.y, mfBounds.center.z + mfBounds.extents.z);
26.         pts[5] = new Vector3(mfBounds.center.x - mfBounds.extents.x, mfBounds.center.y + mfBounds.extents.y, mfBounds.center.z - mfBounds.extents.z);
27.         pts[6] = new Vector3(mfBounds.center.x - mfBounds.extents.x, mfBounds.center.y - mfBounds.extents.y, mfBounds.center.z + mfBounds.extents.z);
28.         pts[7] = new Vector3(mfBounds.center.x - mfBounds.extents.x, mfBounds.center.y - mfBounds.extents.y, mfBounds.center.z - mfBounds.extents.z);
29.  
30.  
31.         // 3. Project corners onto plane
32.         Vector3 sunDir = sunTransform.localRotation.eulerAngles;
33.         for (int i=0; i < pts.Length; i++)
34.         {
35.        
36.             Vector3 cornerPoint = pts[i];
37.             float planeY = planeTransform.position.y; // Y value of plane
38.             pts[i] += sunDir * ((cornerPoint.y - planeY) / sunDir.y); // Projection
39.        
40.             // Vector3.ProjectOnPlane(pts[i], planeTransform.position.normalized);
41.         }
42.  
43.         // 4. Transform from world space into screen space
44.         for (int i = 0; i < pts.Length; i++)
45.         {
46.             Vector3 cornerPoint = pts[i];
47.             pts[i] = cam.WorldToScreenPoint(cornerPoint);
48.         }
49.  
50.         //Get them in GUI space
51.         for (int i = 0; i < pts.Length; i++) pts[i].y = Screen.height - pts[i].y;
52.  
53.         // 5. Get min & max bounds of those transformed corners in screen space.
54.         Vector3 min = pts[0];
55.         Vector3 max = pts[0];
56.         for (int i = 1; i < pts.Length; i++)
57.         {
58.             min = Vector3.Min(min, pts[i]);
59.             max = Vector3.Max(max, pts[i]);
60.         }
61.  
62.         // 6. Draw bounds
63.         Rect r = Rect.MinMaxRect(min.x, min.y, max.x, max.y);
64.         GUI.Box(r, "Shadow");
65.  
66.         // 7. ...
67.  
68.         // 8. Where are you, Mr. Profit?
69.     }
70. }
71.  

However, I am getting faulty results:
https://drive.google.com/open?id=0B0gogWMq7zHXMGxWZU14RFBOem8

Any idea what I might be doing wrong? Perhaps, it's the reason I am not using Renderer.localToWorldMatrix. But how do I use it? Even if I do get the 4x4 matrix from the bounds, how do I proceed with it?

It also does not seem to matter if I use MeshFilter.mesh.bounds or MeshRenderer.bounds.
I think the error is at step 3. Projecting. I tried to use the solution proposed above and the built in Vector3.ProjectOnPlane(). Neither seemed to do the trick. @jvo3dc mentioned I need to set them back on the mesh. Is that the problem?

Cheers

 

The renderer bounds are already in world space, so those are fine to use. At least for now to make sure the rest of the code is working properly. To use the localToWorldMatrix you just need to transform the corners using:

point = Renderer.localToWorldMatrix(point);
https://docs.unity3d.com/ScriptReference/Matrix4x4.MultiplyPoint.html

But only do this if you're using the local mesh bounds, not when using the renderer bounds. Leave it as is for now.

Definitely don't use this. It's doing something very different than what you want. What you want is to find a position projected onto a plane at an arbitrary projection angle. That function only handles the case where the projection angle is orthogonal to the plane. Another way to describe that is it's finding the nearest point on a plane. In your case where the plane is the flat ground it would act as if the light was directly above aimed straight down. In that case point.y = groundHeight would be just as accurate, and faster! But that's not what you need.

Here's the real problem:

Both @jvo3dc and I missed this in your previous post. The sunDir value should be the sun transform's forward vector, not the Euler angles. And by forward vector I mean literally sunTransform.forward. I'm honestly surprised the result you're getting using the Euler angles is even appearing on screen, let alone aligned with the box.

edit: One minor comment on the code above. You have float planeY = planeTransform.position.y; inside your loop. Do that outside the loop, as that value is calculated every time it's accessed. Technically it gets cached after the first access now, but it'll still be faster to do that just before the loop instead of reaccessing it every time.

 

I tried to do this. And the result which I got was this:

First of all, it seems the drawn box is flipped. And secondly, it seems to be too big in most of the cases.

 

Yep. Something isn't going right. It's not obvious to me where or why.

Maybe "/ -sunDir.y"

I would suggest logging the projected corners, or have them as a public variable so you can see then in the inspector, and make sure their y value is properly equal to your plane's height.

 

Yes, thank you! You are correct. I have a working solution for directional light now.
Any ideas how to make it work with point lights?

 

Sure. You just need to do the same math, but replace the "sunDir" with (cornerPoint - worldLightPos).normalized for each light. Same should work for spotlights, but that won't handle the spotlight's cone. That's a lot more work.

 

Hey, I tried this, and it does work for one pointlight. But having more than one makes the box be rendered incorrectly:

To the code, I added the functionality, which finds each point light in the scene at "Start()". And then changed the projection code:

Spoiler: Code

Code (CSharp):

1. List<Transform> pointLightTransforms = new List<Transform>();
2.  
3.     private void Start()
4.     {
5.         foreach (Light l in FindObjectsOfType<Light>())
6.         {
7.             if (l.type == LightType.Point)
8.             {
9.                 pointLightTransforms.Add(l.transform);
10.             }
11.         }
12.     }
13.  
14.     void OnGUI() {
15.  
16.         //    .
17.         //    . Commented this out for forum post
18.         //    .
19.  
20.         // 3. Project corners onto plane
21.      
22.         float planeY = 0;
23.         for (int i=0; i < pts.Length; i++)
24.         {
25.             Vector3 cornerPoint = pts[i];
26.             // Vector3 sunDir = sunTransform.forward;
27.             foreach (Transform pointLightTransform in pointLightTransforms)
28.             {
29.                 Vector3 pointLightDir = (cornerPoint - pointLightTransform.position).normalized;
30.                 planeY = planeTransform.position.y; // Y value of plane
31.                 pts[i] += -pointLightDir * ((cornerPoint.y - planeY) / pointLightDir.y); // Projection
32.             }
33.         }

Do I somehow need to add/combine all the pointLightTransform.position vectors before adding them to the corners? Or is there something else missing? Or somehow check, which point light creates the furthest shadows in each of the 4 directions? (top, bottom, left and right).

 

You can't add the projections together. You need to calculate the on-screen points for each light's shadow separately. If you have 3 lights instead of 8 points to track you now have 24 points to track and find the on-screen position of.

I think the most efficient way would be to do the full loop of corners to on screen min-max for each light and keep track of the largest min and max on screen bounds.

Code (CSharp):

1.  
2. Vector3 minBounds = new Vector3(Mathf.Infinity, Mathf.Infinity, Mathf.Infinity);
3. Vector3 maxBounds = new Vector3(Mathf.NegativeInfinity, Mathf.NegativeInfinity, Mathf.NegativeInfinity);
4. foreach light
5.   foreach point
6.     Vector3 tempPos = // project to plane w/ if for directional vs other
7.     tempPos = // transform to screen pos
8.     tempPos = // transform to GUI pos
9.     minBounds = Vector3.min(minBounds, tempPos);
10.     maxBounds = Vector3.max(maxBounds, tempPos);
11.  
12. // draw bounds
13. // profit!
14.  

 

I tried using this solution, however it seems the projection is done incorrectly for me. It draws the bounding box around the object correctly, but not the shadow:
https://drive.google.com/open?id=0B0gogWMq7zHXU2RYWFNWUlliS1U

The code now looks like this:

Code (CSharp):

1.  
2.         float planeY = 0;
3.  
4.         Vector3 minBounds = new Vector3(Mathf.Infinity, Mathf.Infinity, Mathf.Infinity);
5.         Vector3 maxBounds = new Vector3(Mathf.NegativeInfinity, Mathf.NegativeInfinity, Mathf.NegativeInfinity);
6.         Vector3 tempPos = Vector3.zero;
7.         Vector3 cornerPoint = Vector3.zero;
8.  
9.         foreach (Transform pointLightTransform in pointLightTransforms)
10.         {
11.             for (int i=0; i < pts.Length; i++)
12.             {
13.                 cornerPoint = pts[i];
14.                 Vector3 pointLightDir = (cornerPoint - pointLightTransform.position).normalized;
15.                 planeY = planeTransform.position.y;
16.                 tempPos += -pointLightDir * ((cornerPoint.y - planeY) / pointLightDir.y); // Project to plane
17.                 tempPos = cam.WorldToScreenPoint(cornerPoint); // Transform to screen pos
18.                 tempPos.y = Screen.height - tempPos.y; // Transform to GUI pos
19.                 minBounds = Vector3.Min(minBounds, tempPos);
20.                 maxBounds = Vector3.Max(maxBounds, tempPos);
21.             }
22.         }
23.  
24.         Rect r = Rect.MinMaxRect(minBounds.x, minBounds.y, maxBounds.x, maxBounds.y);
25.         GUI.Box(r, "Shadow");

For some reason I feel as if this is an easy fix.

 

Look carefully at how you're using tempPos and cornerPoint...

(Hint: remove one entirely and replace it with the other )

 

But of course...My bad.

Code (CSharp):

1. tempPos = cam.WorldToScreenPoint(cornerPoint); // Transform to screen pos

should be changed to

Code (CSharp):

1. tempPos = cam.WorldToScreenPoint(tempPos); // Transform to screen pos

Now it works perfectly! Thank you! We are almost there...It works not so perfectly on objects with multiple meshrenderers. Let's pretend I have a bed and it consists of multiple mesh renderers. For each mesh renderer I find the 8 points of the bounds. And then try to find min and max. It seems to be working kind of right. Except that there is a lot of free gap between the edges of the bounding box and the actual mesh.

Another video, which first demonstrates that the solution you've helped me achieve works with multiple point and directional lights (even combined together), on objects with a single mesh renderer. Then it demonstrates that things get messy as soon as an object consists of more than 1 mesh renderer:

The changed code looks like this:

Spoiler: Code

Code (CSharp):

1. [SerializeField] Camera cam;
2.     [SerializeField] Transform planeTransform;
3.  
4.     Vector3[] pts = new Vector3[8];
5.     List<Light> lights = new List<Light>();
6.  
7.     public GameObject _target;
8.    
9.    
10.  
11.     private void Start()
12.     {
13.         foreach (Light l in FindObjectsOfType<Light>())
14.         {
15.             if (l.type == LightType.Point || l.type == LightType.Directional)
16.             {
17.                 lights.Add(l);
18.             }
19.         }
20.     }
21.  
22.     void OnGUI() {
23.         List<MeshRenderer> meshRenderers = _target.GetComponentsInChildren<MeshRenderer>().ToList();
24.         if (meshRenderers.Count >= 1) {
25.             pts = new Vector3[8 * meshRenderers.Count];
26.             int counter = 0;
27.             foreach (MeshRenderer mr in meshRenderers)
28.             {
29.                 Bounds mrBounds = mr.bounds;
30.  
31.                 if (cam.WorldToScreenPoint(mrBounds.center).z < 0) { Debug.Log("returning"); }
32.  
33.                 pts[counter + 0] = new Vector3(mrBounds.center.x + mrBounds.extents.x, mrBounds.center.y + mrBounds.extents.y, mrBounds.center.z + mrBounds.extents.z);
34.                 pts[counter + 1] = new Vector3(mrBounds.center.x + mrBounds.extents.x, mrBounds.center.y + mrBounds.extents.y, mrBounds.center.z - mrBounds.extents.z);
35.                 pts[counter + 2] = new Vector3(mrBounds.center.x + mrBounds.extents.x, mrBounds.center.y - mrBounds.extents.y, mrBounds.center.z + mrBounds.extents.z);
36.                 pts[counter + 3] = new Vector3(mrBounds.center.x + mrBounds.extents.x, mrBounds.center.y - mrBounds.extents.y, mrBounds.center.z - mrBounds.extents.z);
37.                 pts[counter + 4] = new Vector3(mrBounds.center.x - mrBounds.extents.x, mrBounds.center.y + mrBounds.extents.y, mrBounds.center.z + mrBounds.extents.z);
38.                 pts[counter + 5] = new Vector3(mrBounds.center.x - mrBounds.extents.x, mrBounds.center.y + mrBounds.extents.y, mrBounds.center.z - mrBounds.extents.z);
39.                 pts[counter + 6] = new Vector3(mrBounds.center.x - mrBounds.extents.x, mrBounds.center.y - mrBounds.extents.y, mrBounds.center.z + mrBounds.extents.z);
40.                 pts[counter + 7] = new Vector3(mrBounds.center.x - mrBounds.extents.x, mrBounds.center.y - mrBounds.extents.y, mrBounds.center.z - mrBounds.extents.z);
41.                 counter += 8;
42.             }
43.         }
44.  
45.         // 3. Project corners onto plane
46.        
47.         float planeY = 0;
48.  
49.         Vector3 minBounds = new Vector3(Mathf.Infinity, Mathf.Infinity, Mathf.Infinity);
50.         Vector3 maxBounds = new Vector3(Mathf.NegativeInfinity, Mathf.NegativeInfinity, Mathf.NegativeInfinity);
51.         Vector3 tempPos = Vector3.zero;
52.         Vector3 cornerPoint = Vector3.zero;
53.         Vector3 pointLightDir = Vector3.zero;
54.  
55.         foreach (Light l in lights)
56.         {
57.             for (int i=0; i < pts.Length; i++)
58.             {
59.                 cornerPoint = pts[i];
60.                 pointLightDir = l.type == LightType.Point ? (cornerPoint - l.transform.position).normalized : l.transform.forward;
61.                 planeY = planeTransform.position.y;
62.                 tempPos = cornerPoint + (- pointLightDir * ((cornerPoint.y - planeY) / pointLightDir.y)); // Project to plane
63.                 tempPos = cam.WorldToScreenPoint(tempPos); // Transform to screen pos
64.                 tempPos.y = Screen.height - tempPos.y; // Transform to GUI pos
65.                 minBounds = Vector3.Min(minBounds, tempPos);
66.                 maxBounds = Vector3.Max(maxBounds, tempPos);
67.             }
68.         }
69.  
70.         // 6. Draw bounds
71.         Rect r = Rect.MinMaxRect(minBounds.x, minBounds.y, maxBounds.x, maxBounds.y);
72.         GUI.Box(r, "Shadow");

Any ideas how to remove the empty space?

 

That's in part a product of using the renderer bounds rather than the mesh's local bounds. Any object that's slightly rotated is going to get a significantly larger renderer bounding box because the renderer bounds is world space axis aligned. Try placing a box that covers the entire object with out rotating it, that's what you're actually projecting the corners of. The mesh's local bounds will be much smaller, though something like a bed will always have a smaller shadow than calculated. Again, you're just projecting a box, so the foot of the bed is being treated just as tall as the headboard.

In some ways this is the same problem as doing physics collisions on complex objects. You want a simplified representation of the object that still matches the visual bounds of the object. For complex objects like this you may need to resort to manually placing a small number of proxy boxes that conform to the shape of the object you want to know the shadow bounds of. Then iterate over each proxy & light combination and find that min-max. If your mesh is multiple sub meshes they should have their own local bounds you could use.

However the other problem is the code makes no allowances for how visible a shadow is. In that first example from that video the bounds are "correct", but the shadow from the second light isn't visible either due to the relative brightness, or the light's range, etc. To solve this you will basically have to calculate all of the lighting the GPU is doing on the CPU by hand, figuring out falloff and perceptual contribution of the shadows, etc.

 

Yes, I can see how it becomes a problem quickly with multiple meshes. I will try to use the local mesh bounds and see what results I am able to achieve.

Thank you for your patience with me, and for your invaluable help in getting this to work!

Cheers,
Frosty