Question Raycasting for interactables

First off, newbie disclaimer: Started self-teaching myself Unity and C# in February, so still learning my way around things, so please be gentle, lol! I'm using 2021.3.20f1, if that makes any difference.

I'm trying to use raycasting instead of OnMouseOver/Exit for my interactable objects, because from what I understand raycasting is better for VR, which is something I would like to explore down the line.

For the most part, I've got it working pretty well, but if there are two interactable objects near each other (eg: a light switch next to a door) or nested one inside the other (eg: a collectable object hidden within a drawer), the OnLoseFocus and OnInteract functions start to get glitchy.

For example, in the case of the light switch next to the door: say the light switch is to the left of the door, if the player is focusing on the light switch and then looks right (toward the door), the light switch will get stuck in "OnFocus" even after the player is not looking at either object. If the player looks right (away from the door, and therefore not at anything interactable), the light switch behaves as it should and transfers from "OnFocus" to "OnLoseFocus."

Similar glitch occurs with the second case I mentioned (interactable object nested within another interactable object), except that it doesn't matter which way you look, it just stays stuck in "OnFocus."

How this manifests itself in the individual scripts varies from case-to-case. Sometimes if the object is set to be highlighted when in "OnFocus" the object will stay highlighted, other times if there is a UI message prompt reminding the player which key to press to interact the prompt will remain on screen until another prompt cancels it out, and then in other instances if it's a collectable object and there's a UI prompt letting the player know that they collected the object after "OnInteract," that prompt will not delete itself after the 3 seconds I've set it to.

There doesn't seem to be any consistency between these various glitches, but I've narrowed it down to the one constant for all of them being related to an interactable object being too close to another one, and the OnFocus function getting stuck - what I don't know is how to go about fixing that!

I'm currently working on a compromise of using a mixture of raycasting and OnMouseOver, but would like to have it be more uniform, if I can (and I'm sure there must be a way). Here are the portions of my FPS controller that pertains to interaction (the whole FPS controller is roughly 900 lines of code, so I figured cut-and-paste would be more efficient in this instance):

Code (CSharp):

1. public class FirstPersonController : MonoBehaviour
2. {
3. [Header("Interaction")]
4.     [SerializeField] private Vector3 interactionRayPoint = default;
5.     [SerializeField] private float interactionDistance = default;
6.     [SerializeField] private LayerMask interactionLayer = default;
7.     private Interactable currentInteractable;
8.  
9. void Update()
10.     {
11.             if (canInteract)
12.             {
13.                 HandleInteractionCheck();
14.                 HandleInteractionInput();
15.             }
16.             ApplyFinalMovements();
17.         }
18.     }
19. private void HandleInteractionCheck()
20.         {
21.             if (Physics.Raycast(playerCamera.ViewportPointToRay(interactionRayPoint), out RaycastHit hit, interactionDistance))
22.             {
23.                 if (hit.collider.gameObject.layer == 3 && (currentInteractable == null || hit.collider.gameObject.GetInstanceID() != currentInteractable.GetInstanceID()))
24.                 {
25.                     hit.collider.TryGetComponent(out currentInteractable);
26.  
27.                     if (currentInteractable)
28.                         currentInteractable.OnFocus();
29.                 }
30.             }
31.             else if (currentInteractable)
32.             {
33.                 currentInteractable.OnLoseFocus();
34.                 currentInteractable = null;
35.             }
36.         }
37.  
38.         private void HandleInteractionInput()
39.         {
40.             if (Input.GetKeyDown(interactKey) && currentInteractable != null && Physics.Raycast(playerCamera.ViewportPointToRay(interactionRayPoint), out RaycastHit hit, interactionDistance, interactionLayer))
41.             {
42.                 currentInteractable.OnInteract();
43.             }
44.         }
45. }
46.  

 

You probably want to move to a

Physics.RaycastAll

so you can catch all interactable objects the player is looking at, so you don't lose track of one if it happens to be close to another one.

You probably want to boil down the ray cast into its own method(s), as well.

 

Mmkay, so I tried altering Physics.Raycast to Physics.RaycastAll, but that gave me the following compiler errors:
Argument 1: cannot convert from 'UnityEngine.Ray' to 'UnityEngine.Vector3'
Argument 2 may not be passed with the 'out' keyword

 

Yeah its not a drop in replacement mate. You're going to have to rework your logic with the change.

Refer to the documention.

 

Gotcha, cheers.

 

I don't think you really need RaycastAll. That'd be useful if there was multiple overlapping things and you wanted to sort them by some priority (imagine if you had a lock for the door and it was on the door itself, and once you unlocked the door you wanted to ignore the lock and allow activating the door without actually disabling the lock itself).

...

From reading your post along with your code what I bet is happening is when you look towards the door there is no single frame (update) where the raycast is on nothing. So it just goes:

switch
switch
switch
door

As opposed to when you look the other way, it goes to nothing.

switch
switch
switch
nothing

Your logic here:

Code (csharp):

1.     private void HandleInteractionCheck()
2.     {
3.         if (Physics.Raycast(playerCamera.ViewportPointToRay(interactionRayPoint), out RaycastHit hit, interactionDistance))
4.         {
5.             if (hit.collider.gameObject.layer == 3 && (currentInteractable == null || hit.collider.gameObject.GetInstanceID() != currentInteractable.GetInstanceID()))
6.             {
7.                 hit.collider.TryGetComponent(out currentInteractable);
8.  
9.                 if (currentInteractable)
10.                     currentInteractable.OnFocus();
11.             }
12.         }
13.         else if (currentInteractable)
14.         {
15.             currentInteractable.OnLoseFocus();
16.             currentInteractable = null;
17.         }
18.     }

This logic on calls OnLoseFocus if NOTHING was hit.

But if you go from switch to door, there is no moment where NOTHING could be hit.

Try something like this instead:

Code (csharp):

1.     private void HandleInteractionCheck()
2.     {
3.         //I put the 'interactionLayer' in here because I assume that was just missed. You use it in HandleInteractionInput, why not here? Aren't these related?
4.         if (Physics.Raycast(playerCamera.ViewportPointToRay(interactionRayPoint), out RaycastHit hit, interactionDistance, interactionLayer) && hit.collider.gameObject.layer == 3)
5.         {
6.             var next = hit.collider.GetComponent<Interactable>();
7.             if (next != currentInteractable)
8.             {
9.                 if (currentInteractable) currentInteractable.OnLoseFocus();
10.                 currentInteractable = next;
11.                 if (currentInteractable) currentInteractable.OnFocus();
12.             }
13.         }
14.         else if (currentInteractable)
15.         {
16.             currentInteractable.OnLoseFocus();
17.             currentInteractable = null;
18.         }
19.     }

Mind you this is completely untested and written here in the browser. Furthermore it's 1:30am and I'm on my nightcap right now... so the booze may have sprinkled some typos in. But you should get the gist from it.

Also, your update might want to turn into something like:

Code (csharp):

1.     void Update()
2.     {
3.         if (canInteract)
4.         {
5.             HandleInteractionCheck();
6.             HandleInteractionInput();
7.         }
8.         else if (currentInteractable)
9.         {
10.             currentInteractable.OnLoseFocus();
11.             currentInteractable = null;
12.         }
13.         ApplyFinalMovements();
14.     }

I don't know how 'canInteract' is intended to work (it's not even defined in your code, I presume you trimmed out all the stuff not related to this logic). But if you toggle that bad boy false the same thing will happen where the object will stay stuck in focused.

Oh, and finally, if you need to define a out argument that you won't actually use like in your HandleInteractionInput. You can use _ to do so:

Code (csharp):

1. Physics.Raycast(playerCamera.ViewportPointToRay(interactionRayPoint), out _, interactionDistance, interactionLayer)

Although, there's an overload of Raycast that doesn't require the out param:

Code (csharp):

1. Physics.Raycast(playerCamera.ViewportPointToRay(interactionRayPoint), interactionDistance, interactionLayer)

https://docs.unity3d.com/ScriptReference/Physics.Raycast.html

And finally... why not use it? You could use it as a sanity check...

Code (csharp):

1.     private void HandleInteractionInput()
2.     {
3.         if (Input.GetKeyDown(interactKey) &&
4.             currentInteractable != null &&
5.             Physics.Raycast(playerCamera.ViewportPointToRay(interactionRayPoint), out RaycastHit hit, interactionDistance, interactionLayer)
6.             && hit.collider.gameObject == currentInteractable.gameObject)
7.         {
8.             currentInteractable.OnInteract();
9.         }
10.     }

Cause I mean if you don't want to sanity check. What's the point of the raycast anyways? The raycast was already done, currentInteractable wouldn't be set to anything if it didn't hit something. You could have just done:

Code (csharp):

1.     private void HandleInteractionInput()
2.     {
3.         if (currentInteractable && Input.GetKeyDown(interactKey))
4.         {
5.             currentInteractable.OnInteract();
6.         }
7.     }

And last last last last... why you using GetKeyDown? Use the input system the way it was intended so you can swap the actual key for the end user (say they use a gamepad, or they just don't like that you have it mapped to space and prefer enter).

And we get:

Code (csharp):

1. public class FirstPersonController : MonoBehaviour
2. {
3.     [Header("Interaction")]
4.     [SerializeField] private Vector3 interactionRayPoint = default;
5.     [SerializeField] private float interactionDistance = default;
6.     [SerializeField] private LayerMask interactionLayer = default;
7.     private Interactable currentInteractable;
8.  
9.     void Update()
10.     {
11.         if (canInteract)
12.         {
13.             HandleInteractionCheck();
14.             HandleInteractionInput();
15.         }
16.         else if (currentInteractable)
17.         {
18.             currentInteractable.OnLoseFocus();
19.             currentInteractable = null;
20.         }
21.         ApplyFinalMovements();
22.     }
23.    
24.     private void HandleInteractionCheck()
25.     {
26.         if (Physics.Raycast(playerCamera.ViewportPointToRay(interactionRayPoint), out RaycastHit hit, interactionDistance, interactionLayer) && hit.collider.gameObject.layer == 3)
27.         {
28.             var next = hit.collider.GetComponent<Interactable>();
29.             if (next != currentInteractable)
30.             {
31.                 if (currentInteractable) currentInteractable.OnLoseFocus();
32.                 currentInteractable = next;
33.                 if (currentInteractable) currentInteractable.OnFocus();
34.             }
35.         }
36.         else if (currentInteractable)
37.         {
38.             currentInteractable.OnLoseFocus();
39.             currentInteractable = null;
40.         }
41.     }
42.  
43.     private void HandleInteractionInput()
44.     {
45.         if (currentInteractable && Input.GetButtonDown("Interact")) //obviously you'll need to define the input "Interact" in the input system, or change this for another input system you may prefer
46.         {
47.             currentInteractable.OnInteract();
48.         }
49.     }
50. }

 

First of all - thank you so much for that incredibly helpful and insightful reply. As an utter noob, I really appreciate the clarity and attention to detail you provided. I refuse to believe that you wrote this at 1:30 am after a nightcap, lmao! I did find one minor type-o (and extra closing parenthesis) as I was adding your suggestions to my script, but for 1:30 am after a nightcap, that's effing impressive my man!

Alas, after implementing everything accordingly (and making sure there weren't any compiler errors), it sadly still is doing the same thing as before, so I guess it's back to the drawing board. But I've taken your suggestions for the various stages and applied them to other areas of my coding for more conciseness, so thank you for those tips!

 

Did you copy it correctly?

I just copied it into my own test project and I didn't have the parens issue you spoke of, which clues to me that maybe you didn't copy it into your code correctly.

After doing that, I added a little extra to fill in the gaps that your OP doesn't have, and then setup a scene:

Code (csharp):

1.     public class zTest02 : MonoBehaviour
2.     {
3.  
4.         public bool canInteract = true;
5.         private Camera playerCamera;
6.         private Vector2 lastMousePos;
7.  
8.         [Header("Interaction")]
9.         [SerializeField] private Vector3 interactionRayPoint = default;
10.         [SerializeField] private float interactionDistance = default;
11.         [SerializeField] private LayerMask interactionLayer = default;
12.         private Interactable currentInteractable;
13.  
14.         private void OnEnable()
15.         {
16.             lastMousePos = Input.mousePosition;
17.         }
18.  
19.         void Update()
20.         {
21.             playerCamera = Camera.main;
22.             var delta = (Vector2)Input.mousePosition - lastMousePos;
23.             lastMousePos = Input.mousePosition;
24.             if (Input.GetMouseButton(1))
25.             {
26.                 playerCamera.transform.rotation *= Quaternion.Euler(delta.y * Time.deltaTime * -5f, delta.x * Time.deltaTime * -5f, 0f);
27.             }
28.  
29.             if (canInteract)
30.             {
31.                 HandleInteractionCheck();
32.                 HandleInteractionInput();
33.             }
34.             else if (currentInteractable)
35.             {
36.                 currentInteractable.OnLoseFocus();
37.                 currentInteractable = null;
38.             }
39.             //ApplyFinalMovements();
40.         }
41.  
42.         private void HandleInteractionCheck()
43.         {
44.             if (Physics.Raycast(playerCamera.ViewportPointToRay(interactionRayPoint), out RaycastHit hit, interactionDistance, interactionLayer) && hit.collider.gameObject.layer == 3)
45.             {
46.                 Debug.Log("HIT: " + hit.collider.name, hit.collider);
47.                 var next = hit.collider.GetComponent<Interactable>();
48.                 if (next != currentInteractable)
49.                 {
50.                     if (currentInteractable) currentInteractable.OnLoseFocus();
51.                     currentInteractable = next;
52.                     if (currentInteractable) currentInteractable.OnFocus();
53.                 }
54.             }
55.             else if (currentInteractable)
56.             {
57.                 currentInteractable.OnLoseFocus();
58.                 currentInteractable = null;
59.             }
60.         }
61.  
62.         private void HandleInteractionInput()
63.         {
64.             if (currentInteractable && Input.GetKeyDown(KeyCode.Space))
65.             {
66.                 currentInteractable.OnInteract();
67.             }
68.         }
69.  
70.     }

Code (csharp):

1. public class Interactable : MonoBehaviour
2. {
3.  
4.     public SPEvent OnFocusEvent = new SPEvent();
5.     public SPEvent OnLoseFocusEvent = new SPEvent();
6.  
7.     public void OnLoseFocus()
8.     {
9.         this.OnLoseFocusEvent.ActivateTrigger(this, null);
10.     }
11.  
12.     public void OnFocus()
13.     {
14.         this.OnFocusEvent.ActivateTrigger(this, null);
15.     }
16.  
17.     public void OnInteract()
18.     {
19.         Debug.Log("Interact: " + this.name, this);
20.     }
21.  
22. }

And I got this:

 

I was just sitting here and realized... maybe I should have put a retical in so it reads clearer. So I did:

 

OP I've liked your post for being able to articulate your design and your ideas like a human being. Glad for the high quality of the responses you've got.

 

Agreed, I often tell people with poor posts how I'm willing to put in as much effort as they do. OP put in great effort for their question, so I'll try to figure out what their problem is.

 

Last time I did type everything in, but this time I copy/pasted your tested script and it works! Thank you so much - I'd spent at least a month (more likely two, lol) narrowing this down, so this is a HUGE weight off my mind now!

While we're on a roll - I took your initial suggestion to use "GetButtonDown" instead of "GetKeyDown," and applied that to my other actions - which worked great except for my sprinting function. Can't see why that one didn't work like the others did. Here's the sprint code (As with the previous issue, I selectively omitted my full movement script to just the portions pertaining to sprinting, which is why some functions/bools/etc. are quoted but not used - if the full script would be better to diagnose, I can share that, but it's about 900 lines, so I figured this would be easier):

Code (CSharp):

1. public class FirstPersonController : MonoBehaviour
2. {
3.  
4. public bool IsSprinting => canSprint && Input.GeButtonDown("Sprint");
5.  
6. [SerializeField] private bool canSprint = true;
7. [SerializeField] private float sprintSpeed = 6.0f;
8. private FMOD.Studio.EventInstance sprintingSound;
9. [SerializeField] private float sprintBobSpeed = 18f;
10. [SerializeField] private float sprintBobAmount = 0.10f;
11. [SerializeField] private float sprintStepMultiplier = 0.6f;
12.  
13. private float GetCurrentOffset => isCrouching ? baseStepSpeed * crouchStepMultiplier : IsSprinting ? baseStepSpeed * sprintStepMultiplier : baseStepSpeed;
14.  
15. private void HandleMovementInput()
16.     {
17.         currentInput = new Vector2((isCrouching ? crouchSpeed : IsSprinting ? sprintSpeed : walkSpeed) * Input.GetAxis("Vertical"),(isCrouching ? crouchSpeed : IsSprinting ? sprintSpeed : walkSpeed) * Input.GetAxis("Horizontal"));
18.  
19.         float moveDirectionY = moveDirection.y;
20.         moveDirection = (transform.TransformDirection(Vector3.forward) * currentInput.x) + (transform.TransformDirection(Vector3.right) * currentInput.y);
21.         moveDirection.y = moveDirectionY;
22.     }
23.  
24. private void HandleStamina()
25.         {
26.             if(IsSprinting && currentInput != Vector2.zero)
27.             {
28.                 if (regeneratingStamina != null)
29.                 {
30.                     StopCoroutine(regeneratingStamina);
31.                     regeneratingStamina = null;
32.                     sprintingSound = FMODUnity.RuntimeManager.CreateInstance("event:/Player/Footsteps/Run/RunningLoop");
33.                     sprintingSound.start();
34.                 }
35.  
36.                 currentStamina -= staminaUseMultiplier * Time.deltaTime;
37.  
38.                 if (currentStamina < 0)
39.                     currentStamina = 0;
40.  
41.                 OnStaminaChange?.Invoke(currentStamina);
42.  
43.                 if (currentStamina <= 0)
44.                 {
45.                     canSprint = false;
46.                     FMODUnity.RuntimeManager.PlayOneShot("event:/Player/Footsteps/Run/SprintRecover");
47.                     sprintingSound.stop(FMOD.Studio.STOP_MODE.ALLOWFADEOUT);
48.                 }
49.             }
50.  
51.             if(!IsSprinting && currentStamina < maxStamina && regeneratingStamina == null)
52.             {
53.                 regeneratingStamina = StartCoroutine(RegenerateStamina());
54.             }
55.  
56.             if(!IsSprinting)
57.             {
58.                 sprintingSound.stop(FMOD.Studio.STOP_MODE.ALLOWFADEOUT);
59.             }
60.         }

Also attaching a screenshot of my input settings within Unity, in case that's where the problem lies.

 

Cheers! I'm doing my best to not fall in over my head as I teach myself all of this. It's something I've always wanted to do, but just never got around to until now - but better late than never, lol.

 

Never mind this one - I eventually realized that using "GetButton" instead of "GetButtonDown" fixed that!