Trying to get color of a pixel on texture with raycasting.

Hey all.
I've made research about this but solutions didn't worked for me , so i decided to create new post about this.
I am working with 3D mode.
I have a quad in my scene and a texture attached to it.
I'm trying to cast ray on quad and get the color of pixel that ray collided with.
Currently it isn't giving me the color that i expected , it's always give me same color.
Here is the code that i'm working on.

Code (CSharp):

1. private void FireRay() {
2.             if (Input.GetButtonDown(controllerInput.FireButtonName)) {
3.                 Vector3 direction = requiredComponents.RaycastPosition.position - transform.position;
4.  
5.                 RaycastHit raycastHit;
6.  
7.                 if (Physics.Raycast(requiredComponents.RaycastPosition.position , direction , out raycastHit)) {
8.                     if (raycastHit.collider.tag != "Billboard") {
9.                         return;
10.                     }
11.  
12.                     Texture2D tMap = (Texture2D)raycastHit.collider.GetComponent<Renderer>().material.mainTexture;
13.                     /*Vector2 pCoord = raycastHit.textureCoord;
14.                     pCoord.x *= tMap.width;
15.                     pCoord.y *= tMap.height;*/
16.  
17.                     int x = Mathf.FloorToInt(raycastHit.point.x);
18.                     int y = Mathf.FloorToInt(raycastHit.point.y);
19.  
20.                     Color pColor = tMap.GetPixel(x , y);
21.                     requiredComponents.CollisionEvents.PixelColor = pColor;
22.  
23.                     Debug.Log("The picked color is :" + pColor);
24.  
25.                     if (requiredComponents.CollisionEvents.OnRayHitBillboard != null) {
26.                         requiredComponents.CollisionEvents.OnRayHitBillboard.Invoke();
27.                     }
28.                 }
29.             }
30.         }

Note : The texture that i'm using is already set to readable/writeable.
Thanks for any help

 

You need to use a UV texture coordinate, not a point in space.

https://docs.unity3d.com/ScriptReference/RaycastHit-textureCoord.html

 

@GroZZleR Hey , thanks for answer.
I updated my code as follows.

Code (CSharp):

1.  private void FireRay() {
2.             if (Input.GetButtonDown(controllerInput.FireButtonName)) {
3.                 Vector3 direction = requiredComponents.RaycastPosition.position - transform.position;
4.  
5.                 RaycastHit raycastHit;
6.  
7.                 if (Physics.Raycast(requiredComponents.RaycastPosition.position , direction , out raycastHit)) {
8.                     if (raycastHit.collider.tag != "Billboard") {
9.                         return;
10.                     }
11.  
12.                     Renderer renderer = raycastHit.collider.GetComponent<MeshRenderer>();
13.                     Texture2D texture2D = renderer.material.mainTexture as Texture2D;
14.                     Vector2 pCoord = raycastHit.textureCoord;
15.                     pCoord.x *= texture2D.width;
16.                     pCoord.y *= texture2D.height;
17.  
18.                     Vector2 tiling = renderer.material.mainTextureScale;
19.                     Color color = texture2D.GetPixel(Mathf.FloorToInt(pCoord.x * tiling.x) , Mathf.FloorToInt(pCoord.y * tiling.y));
20.  
21.                     Debug.Log("Picked color : " + color);
22.                     requiredComponents.CollisionEvents.PixelColor = color;
23.  
24.                     if (requiredComponents.CollisionEvents.OnRayHitBillboard != null) {
25.                         requiredComponents.CollisionEvents.OnRayHitBillboard.Invoke();
26.                     }
27.                 }
28.             }
29.         }

Now it's giving me better results , but i'm still not sure if it's correct or not.

 

Hopefully someone else can chime in, you're further than I've ever gone with it.

 

I understand , thanks for the answers

 

Are you sure you have to multiply the texture coordinates using renderer.material.mainTextureScale? I would think that the coordinates in raycastHit.textureCoord would already be including that. The example in the Unity documentation for RaycastHit.textureCoord looks similar to your code, but does not multiply the UVs by mainTextureScale:

Code (csharp):

1. // Write black pixels onto the GameObject that is located
2. // by the script. The script is attached to the camera.
3. // Determine where the collider hits and modify the texture at that point.
4. //
5. // Note that the MeshCollider on the GameObject must have Convex turned off. This allows
6. // concave GameObjects to be included in collision in this example.
7. //
8. // Also to allow the texture to be updated by mouse button clicks it must have the Read/Write
9. // Enabled option set to true in its Advanced import settings.
10.  
11. using UnityEngine;
12. using System.Collections;
13.  
14. public class ExampleClass : MonoBehaviour
15. {
16.    public Camera cam;
17.  
18.     void Start()
19.    {
20.        cam = GetComponent<Camera>();
21.    }
22.  
23.     void Update()
24.    {
25.        if (!Input.GetMouseButton(0))
26.            return;
27.  
28.         RaycastHit hit;
29.        if (!Physics.Raycast(cam.ScreenPointToRay(Input.mousePosition), out hit))
30.            return;
31.  
32.         Renderer rend = hit.transform.GetComponent<Renderer>();
33.        MeshCollider meshCollider = hit.collider as MeshCollider;
34.  
35.         if (rend == null || rend.sharedMaterial == null || rend.sharedMaterial.mainTexture == null || meshCollider == null)
36.            return;
37.  
38.         Texture2D tex = rend.material.mainTexture as Texture2D;
39.        Vector2 pixelUV = hit.textureCoord;
40.        pixelUV.x *= tex.width;
41.        pixelUV.y *= tex.height;
42.  
43.         tex.SetPixel((int)pixelUV.x, (int)pixelUV.y, Color.black);
44.        tex.Apply();
45.    }
46. }

Note that I just copied this code from the documentation, I don't know if it works or is correct

 

@T5Shared Hey , i watched a video about this topic , the guy in the video was doing that and saying it's needed for tiling textures. So i am not sure.
Well , it gives me the results that i expected now , but i am not sure if the results are correct.

 

@agneng_dev Could you post a link to that video? Also, you should be able to test this relatively easy. Use a material with a black and white checkerboard texture. Set tiling to 1, 1. Check if the colour is correctly black or white depending on where you are clicking on the object. Change the tiling to 3, 3. Check again.

 

@T5Shared Hey here is the link (You can start watching from 6:40 if you want)

Thanks for the solution by the way.

 

Yup, he's doing that and demonstrating that it works when he changes the texture tiling on the material. Maybe he is right and the Unity example is wrong (it works until you start changing the tiling). The Unity documentation isn't very precise ('RaycastHit._textureCoord is a texture coordinate when a hit occurs'). Perhaps the returned UV coordinate is just the interpolated value of the three vertex UVs of the triangle. Since it doesn't say, you will have to test this for yourself.

If the UV is not already corrected by Unity, it means that the video and your code are correct.

 

@T5Shared Hey , thanks for the answers. I will try your solution and some other solutions.
I will update the post again after i get results.

 

@GroZZleR @T5Shared
Alright , i've done some testings and it seems like my code is working and i am getting the results that i expected.
Here is the code that works well for me. I hope it can help someone at some point.

Code (CSharp):

1. private void FireRay() {
2.             if (Input.GetButtonDown(controllerInput.FireButtonName)) {
3.                 Vector3 direction = requiredComponents.RaycastPosition.position - transform.position;
4.  
5.                 RaycastHit raycastHit;
6.  
7.                 if (Physics.Raycast(requiredComponents.RaycastPosition.position , direction , out raycastHit)) {
8.                     if (raycastHit.collider.tag != "Billboard") {
9.                         return;
10.                     }
11.  
12.                     Renderer renderer = raycastHit.collider.GetComponent<MeshRenderer>();
13.                     Texture2D texture2D = renderer.material.mainTexture as Texture2D;
14.                     Vector2 pCoord = raycastHit.textureCoord;
15.                     pCoord.x *= texture2D.width;
16.                     pCoord.y *= texture2D.height;
17.  
18.                     Vector2 tiling = renderer.material.mainTextureScale;
19.                     Color color = texture2D.GetPixel(Mathf.FloorToInt(pCoord.x * tiling.x) , Mathf.FloorToInt(pCoord.y * tiling.y));
20.  
21.                     // Debug.Log(raycastHit.point.x + " " + raycastHit.point.y);
22.                     requiredComponents.CollisionEvents.PixelColor = color;
23.  
24.                     if (requiredComponents.CollisionEvents.OnRayHitBillboard != null) {
25.                         requiredComponents.CollisionEvents.OnRayHitBillboard.Invoke();
26.                     }
27.                 }
28.             }
29.         }

Thank you everyone for helping me out with this process.

 

Thanks for posting your working code! One possible improvement could be to use renderer.sharedMaterial, since accessing renderer.material (yes, even without changing anything) will result in Unity creating a copy of the material and assigning that to this renderer instead of the original material. This might not be a problem in this case, but it is something to be aware of.

 

Do you mean, when i say "renderer.material" it's creating new material in the background ?
Just like passing struct to function without "ref" keyword right ?

 

Not really like a struct, no. Check the Unity docs: https://docs.unity3d.com/ScriptReference/Renderer-material.html

You can actually see if a material has been cloned, because its name will be different in the Inspector of the renderer

 

if (sprite != null)
{
texture2D = sprite.texture;
}
else
{
texture2D = renderer.material.mainTexture as Texture2D;
}

Raycast hit;
if(Physics.Raycast(myRay, out hit, 500f, myMask))
{

int x = Mathf.FloorToInt(_th.hit[0].textureCoord.x * texture2D.width);
int y = Mathf.FloorToInt(_th.hit[0].textureCoord.y * texture2D.height);

Debug.Log(x);
Debug.Log(y);
Debug.Log(texture2D.GetPixel(x, y));

if (texture2D.GetPixel(x, y).a != 0f) {
Debug.Log("the pixel is transparent!");
}

}