Filter which objects I can select in the scene object via editor tool - how to do custom selection

I would like to implement a tool (probably a simple editor window), which adds selection modes to the Unity scene. In my case, I would like to only make specific object selectable while in my special mode, because I'm working in a large scene with many objects and clicking on an object by accident may scroll the hierarchy down to a place far away from where I was working.

I know that I can set the current selection via Selection.activeGameObject and that I can block clicks in the scene view by using the current event with Event.Use().

Now, when a click happens (I can check in the current event), I would need to see if the mouse has selected one of my special objects and here comes the tricky part. I can't use raycasting because the objects are only visualized via editor handles (more specifically lines and boxes). So I would want to achieve the default selection behavior of these selectable controls, such as Handles.Button(), but I don't want to actually implement those controls in my additional selection mode tool. Instead, I want to keep all of the individual handles and their custom editor code (Gizmos etc) in their respective MonoBehaviors and custom editors and then somehow filter the selection to exclude all other objects via an additional editor window.

Any ideas how this could be done?

 

Thank you for the suggestion! I've got the basic callback working. At least, when I put GameObjects into the ignore and filter arrays, the picking behavior in the scene changes.

Code (CSharp):

1. using System.Reflection;
2.  
3. public class SceneSelectionFilter : EditorWindow
4. {
5.     [MenuItem("Window/Scene Selection Filter")]
6.     public static void ShowWindow()
7.     {
8.         EditorWindow.GetWindow<SceneSelectionFilter>("Selection Filter").Show();
9.     }
10.  
11.     private MethodInfo Internal_PickClosestGO;
12.  
13.     private void OnEnable()
14.     {
15.         Assembly editorAssembly = typeof(Editor).Assembly;
16.         System.Type handleUtilityType = editorAssembly.GetType("UnityEditor.HandleUtility");
17.  
18.         FieldInfo pickClosestDelegateInfo = handleUtilityType.GetField("pickClosestGameObjectDelegate", BindingFlags.Static | BindingFlags.NonPublic);
19.         Delegate pickHandler = Delegate.CreateDelegate(pickClosestDelegateInfo.FieldType, this, "OnPick");
20.         pickClosestDelegateInfo.SetValue(null, pickHandler);
21.  
22.         Internal_PickClosestGO = handleUtilityType.GetMethod("Internal_PickClosestGO", BindingFlags.Static | BindingFlags.NonPublic);
23.     }
24.  
25.     private GameObject OnPick(Camera cam, int layers, Vector2 position, GameObject[] ignore, GameObject[] filter, out int materialIndex)
26.     {
27.         materialIndex = -1;
28.  
29.         ignore = new GameObject[] { /* ? */ };
30.         filter = new GameObject[] { /* ? */ };
31.  
32.         return (GameObject)Internal_PickClosestGO.Invoke(null, new object[] { cam, layers, position, ignore, filter, materialIndex });
33.     }
34. }

However, I don't understand how those arrays need to be filled to achieve my desired behavior:

• GameObjects should not be pickable by default (ground tiles, models, etc)
• A set of specific objects, which I can define, should be pickable

I've tried a few configurations but couldn't manage to completely exclude objects from being selected. It does look like the right approach to handle the problem, hopefully, I just missed something.

 

@shawn: Thank you for explaining. The filtering by Internal_PickClosestGO returns the desired result: either one of the pickable objects or null.

However, I noticed, that null is not a valid value for the internal method PickGameObjectDelegated, which will call the internal method again without the filtering applied to get a non-null pickable object. I managed to force the selection to stay at the current correct object and not select anything else with the following code. The behavior is how I want it, but Unity logs an error.

I don't expect any sort of technical support for these internal features, so I don't want to be asking anymore. I'm still posting my current code, with the small error-logging problem for reference, if anyone else has a similar issue.

Code (CSharp):

1. private GameObject OnPick(Camera cam, int layers, Vector2 position, GameObject[] ignore, GameObject[] filter, out int materialIndex)
2. {
3.     materialIndex = -1;
4.  
5.     filter = GetPickableObjects();
6.  
7.     Assembly editorAssembly = typeof(Editor).Assembly;
8.     Type handleUtilityType = editorAssembly.GetType("UnityEditor.HandleUtility");
9.     MethodInfo Internal_PickClosestGO = handleUtilityType.GetMethod("Internal_PickClosestGO", BindingFlags.Static | BindingFlags.NonPublic);
10.     GameObject go = (GameObject)Internal_PickClosestGO.Invoke(null, new object[] { cam, layers, position, ignore, filter, materialIndex });
11.  
12.     // The returned object is either one of our desired ones, or null.
13.     // If we return null, the internal Unity method 'PickGameObjectDelegated' will ignore our custom filter
14.     // and find a different pick.
15.     if (go == null)
16.     {
17.         // Now, if we continue here, everything works as I want it to,
18.         // however, Unity will throw the following error, when we have a pickable object ("MyCube") already selected
19.         // and attempt to click on something, which shouldn't be selected:
20.         // "GetAllOverlapping failed, could not ignore game object 'MyCube' when picking".
21.         // I'm not explicitly ignoring this object, I actually want it to remain selection. I tried to:
22.         //    add the current selection ("MyCube") to ignore
23.         //    add the current selection to filter and everything else to ignore
24.         // The error still gets logged, but the behavior is what I want, the old correct selection remains.
25.  
26.         // Instead of letting Unity pick something else, I want to stay at the current selection.
27.         return Selection.activeGameObject;
28.     }
29.  
30.     // This is one the pickable desired objects, all good if we reach this.
31.     return go;
32. }

I've tried to find the issue in the source, but it looks like the error is logged, but then the null is still handled correctly, so I'm not sure how to change the ignore and filter arrays in this special case to keep the selection.

For now, I can also use the following workaround:

Code (CSharp):

1. public class SceneSelectionFilter : EditorWindow
2. {
3.     [MenuItem("Window/Scene Selection Filter")]
4.     public static void ShowWindow()
5.     {
6.         EditorWindow.GetWindow<SceneSelectionFilter>("Selection Filter").Show();
7.     }
8.  
9.     private void OnEnable()
10.     {
11.         SceneView.onSceneGUIDelegate += OnSceneGUI;
12.     }
13.  
14.     private void OnDisable()
15.     {
16.         SceneView.onSceneGUIDelegate -= OnSceneGUI;
17.     }
18.  
19.     private void OnSceneGUI(SceneView sceneView)
20.     {
21.         // Block all clicks from propagating to the scene view.
22.         HandleUtility.AddDefaultControl(-1);
23.  
24.         Event e = Event.current;
25.         if (e.type == EventType.MouseUp)
26.         {
27.             // This returns a picked object as it normally does, but does not select it yet.
28.             GameObject picked = HandleUtility.PickGameObject(e.mousePosition, false);
29.  
30.             if (ShouldBeSelected(picked))
31.             {
32.                 // We select it, if valid for our needs.
33.                 Selection.activeObject = picked;
34.                 e.Use();
35.             }
36.         }
37.     }
38.  
39.     private bool ShouldBeSelected(GameObject go)
40.     {
41.         // TODO: go contained in my set of selectable objects.
42.         return go == GameObject.Find("MyCube");
43.     }
44. }

This is more like the original idea I had: Block all clicks from the scene view, then pick an object close to the mouse, but only select it if within our defined set. Made possible by the supported PickGameObject method. Not sure if there are any other side-effects, but if we only want to select an object, this works fine.

One drawback, however, is that since we are blocking the scene view completely, our relevant components do not receive a click event themselves. My original use-case was: The selection filter tool blocks selection of unwanted objects, but lets the desired click through to my objects, which have a custom editor and gizmo /handle code with buttons, etc. This would work flawlessly in the first approach with the internal delegate if I could get rid of the logged error, but in the second approach, I need to forward scene events captured by the selection filter to those selected objects, which is a little ugly (because now this code needs to change if any of the other tools requires a different click or layout message).

 

Right now there is one more way to do it. You can use SceneVisibilityManager.
It is easy way to disable picking and view in editor only. It has some drawbacks because it changes visibility in hierarchy but still it can be useful.

 

Not quite working, for that the scene view clicking behavior actually looping through objects that the raycast hits to become selection, including those set to be "unpickable".

More details: the actual behavior is that:
1. you have a prefab, the prefab has 1 child object.
2. disable picking for the child object.
3. click the prefab mulitple times.

Voila, you can select the unpickable object now.

EDIT:

my bad. this does not happen.

The actual situation is that, when you set the child object to be unpickable, the root of the prefab becomes unpickable as well, because:
the selection comes from the "selection base", which indicate that the root of the prefab is the actual selection when you click any child object. Now that the child object can't be selected, you can never select the root now.

Sad story: what I want to have is that by selecting the child object, give me the root object, and stop the iterating to any child object anymore.