Move UI above TouchKeyboard for Mobile

Hi,

In our game, we have a chat area and would like to move the UI above the keyboard so that the last sent message is at the bottom of the visible space (like Facebook chat or WhatsApp etc does)

I've tried a few different ways to do this, but I'm having a hard time converting the TouchKeyboard height into something a RectTransform relates to. Is there an easy way to handle this that I'm missing? seems like it should be something fairly easy to do...

Thanks

 

I did something similar a while back. I will see if I can find the code, but I think I basically created a container within the canvas that contained everything so it could be shifted up easily.

IIRC I converted touch keyboard height into a percentage of the screen height and then used that percentage against the RectTransform anchored position.

 

Similar method to what I was using, I just can't make the anchored positions make any sense with the percentages... its either too high and leaving a big huge gap, or on other resolutions its not moved enough

 

Ok, I've dug out the code.

I had two scripts - one which is on the input field and just checks when it becomes active and sets the transform (this app has about 500+ different input fields)

Then I have a second script which is attached to the containing game object (first item below the Canvas).
I used a bit of code I found elsewhere to get a corrected screen corners position: http://answers.unity3d.com/questions/781643/unity-46-beta-rect-transform-position-new-ui-syste.html for the GetScreenRect function, although I inverted the Y output from it. It could be this function is all you need

Ok my code below, it's not pretty or tidy but seems to work smoothly for me across iPhone 4, iPhone 5, iPad portrait/horiz both retina and non-retina. This is all about making sure the input field is always visible above the keyboard.

This is currently for iOS - not tested on android as yet, hence the #if statements surrounding it so it falls back to not hiding the mobile input on android.

No idea how to format code properly here, sorry!

Apply this to the input field: (change MainContainer to whatever your top item is called)

using UnityEngine;
using System.Collections;
using UnityEngine.UI;

public class InputfieldFocused : MonoBehaviour {

InputfieldSlideScreen slideScreen;
InputField inputField;


void Start () {
slideScreen = GameObject.Find ("MainContainer").GetComponent<InputfieldSlideScreen>();
inputField = transform.GetComponent<InputField>();
#if UNITY_IOS
inputField.shouldHideMobileInput=true;
#endif
}

#if UNITY_IOS

void Update () {
if (inputField.isFocused)
{
// Input field focused, let the slide screen script know about it.
slideScreen.InputFieldActive = true;
slideScreen.childRectTransform = transform.GetComponent<RectTransform>();
}
}
#endif
}


Now, apply this to the MainContainer (or whatever you call it)


using UnityEngine;
using System.Collections;

public class InputfieldSlideScreen : MonoBehaviour {

// Assign canvas here in editor
public Canvas canvas;


// Used internally - set by InputfieldFocused.cs
public bool InputFieldActive = false;
public RectTransform childRectTransform;

#if UNITY_IOS


void LateUpdate () {
if ((InputFieldActive)&&((TouchScreenKeyboard.visible)))
{
transform.GetComponent<RectTransform>().anchoredPosition = Vector2.zero;

Vector3[] corners = {Vector3.zero,Vector3.zero,Vector3.zero,Vector3.zero};
Rect rect = GetScreenRect(corners,childRectTransform);
float keyboardHeight = TouchScreenKeyboard.area.height;

float heightPercentOfKeyboard = keyboardHeight/Screen.height*100f;
float heightPercentOfInput = (Screen.height-(rect.y+rect.height))/Screen.height*100f;


if (heightPercentOfKeyboard>heightPercentOfInput)
{
// keyboard covers input field so move screen up to show keyboard
float differenceHeightPercent = heightPercentOfKeyboard - heightPercentOfInput;
float newYPos = transform.GetComponent<RectTransform>().rect.height /100f*differenceHeightPercent;

Vector2 newAnchorPosition = Vector2.zero;
newAnchorPosition.y = newYPos;
transform.GetComponent<RectTransform>().anchoredPosition = newAnchorPosition;
} else {
// Keyboard top is below the position of the input field, so leave screen anchored at zero
transform.GetComponent<RectTransform>().anchoredPosition = Vector2.zero;
}
} else {
// No focus or touchscreen invisible, set screen anchor to zero
transform.GetComponent<RectTransform>().anchoredPosition = Vector2.zero;
}
InputFieldActive = false;
}


public Rect GetScreenRect(Vector3[] corners,RectTransform rectTransform) {
rectTransform.GetWorldCorners(corners);

float xMin = float.PositiveInfinity, xMax = float.NegativeInfinity, yMin = float.PositiveInfinity, yMax = float.NegativeInfinity;
for (int i = 0; i < 4; ++i) {
// For Canvas mode Screen Space - Overlay there is no Camera; best solution I've found
// is to use RectTransformUtility.WorldToScreenPoint) with a null camera.
Vector3 screenCoord = RectTransformUtility.WorldToScreenPoint(null, corners);
if (screenCoord.x < xMin) xMin = screenCoord.x;
if (screenCoord.x > xMax) xMax = screenCoord.x;
if (screenCoord.y < yMin) yMin = screenCoord.y;
if (screenCoord.y > yMax) yMax = screenCoord.y;
corners = screenCoord;
}
Rect result = new Rect(xMin, Screen.height-yMin - (yMax - yMin), xMax - xMin, yMax - yMin);
return result;
}
#endif
}

 

Thanks for the code, I'll have a look at integrating it into my game.

 

Hi,

I've just been working on this code, but found an issue in the GetScreenRect code. WorldToScreenPoint doesn't take an array of Vector3's, at least not in Unity 4.6... what version are you using or is this a mistake?

should this be corners and when assigning back, coners = screenCoord?

 

Hi there @kujo

I'm working on something similar and running into the same issue. Did you happen to figure out the error?

 

I've updated the second script so that it works with Unity 5.X. and fixes the error in the above script:

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3.  
4. public class InputfieldSlideScreen : MonoBehaviour {
5.  
6.     // Assign canvas here in editor
7.     public Canvas canvas;
8.  
9.  
10.     // Used internally - set by InputfieldFocused.cs
11.     public bool InputFieldActive = false;
12.     public RectTransform childRectTransform;
13.  
14.     #if UNITY_IOS
15.  
16.  
17.     void LateUpdate () {
18.         if ((InputFieldActive)&&((TouchScreenKeyboard.visible)))
19.         {
20.             transform.GetComponent<RectTransform>().anchoredPosition = Vector2.zero;
21.  
22.             Vector3[] corners = {Vector3.zero,Vector3.zero,Vector3.zero,Vector3.zero};
23.             Rect rect = RectTransformExtension.GetScreenRect(childRectTransform, canvas);
24.             float keyboardHeight = TouchScreenKeyboard.area.height;
25.  
26.             float heightPercentOfKeyboard = keyboardHeight/Screen.height*100f;
27.             float heightPercentOfInput = (Screen.height-(rect.y+rect.height))/Screen.height*100f;
28.  
29.  
30.             if (heightPercentOfKeyboard>heightPercentOfInput)
31.             {
32.                 // keyboard covers input field so move screen up to show keyboard
33.                 float differenceHeightPercent = heightPercentOfKeyboard - heightPercentOfInput;
34.                 float newYPos = transform.GetComponent<RectTransform>().rect.height /100f*differenceHeightPercent;
35.  
36.                 Vector2 newAnchorPosition = Vector2.zero;
37.                 newAnchorPosition.y = newYPos;
38.                 transform.GetComponent<RectTransform>().anchoredPosition = newAnchorPosition;
39.             } else {
40.                 // Keyboard top is below the position of the input field, so leave screen anchored at zero
41.                 transform.GetComponent<RectTransform>().anchoredPosition = Vector2.zero;
42.             }
43.         } else {
44.             // No focus or touchscreen invisible, set screen anchor to zero
45.             transform.GetComponent<RectTransform>().anchoredPosition = Vector2.zero;
46.         }
47.         InputFieldActive = false;
48.     }
49.  
50. }
51.    
52. public static class RectTransformExtension {
53.  
54.     public static Rect GetScreenRect(this RectTransform rectTransform, Canvas canvas) {
55.  
56.         Vector3[] corners = new Vector3[4];
57.         Vector3[] screenCorners = new Vector3[2];
58.  
59.         rectTransform.GetWorldCorners(corners);
60.  
61.         if (canvas.renderMode == RenderMode.ScreenSpaceCamera || canvas.renderMode == RenderMode.WorldSpace)
62.         {
63.             screenCorners[0] = RectTransformUtility.WorldToScreenPoint(canvas.worldCamera, corners[1]);
64.             screenCorners[1] = RectTransformUtility.WorldToScreenPoint(canvas.worldCamera, corners[3]);
65.         }
66.         else
67.         {
68.             screenCorners[0] = RectTransformUtility.WorldToScreenPoint(null, corners[1]);
69.             screenCorners[1] = RectTransformUtility.WorldToScreenPoint(null, corners[3]);
70.         }
71.  
72.         screenCorners[0].y = Screen.height - screenCorners[0].y;
73.         screenCorners[1].y = Screen.height - screenCorners[1].y;
74.  
75.         return new Rect(screenCorners[0], screenCorners[1] - screenCorners[0]);
76.     }
77.  
78. }    #endif
79.  

 

Also, rather than doing a GameObject.Find("MainContainer") and being hardcoded to that name, i changed the assignment line from

slideScreen = GameObject.Find ("MainContainer").GetComponent<InputfieldSlideScreen>();
|
to
|
slideScreen = gameObject.GetComponentInParent<InputfieldSlideScreen>();

This will pick walk up from the input field that the script is attached to, and find the first "InputfieldSlideScreen" it can find, which is a must for me, as i want to be able to have different "shifters" throughout my app, with different naming/structures.

 

Tested with Android, not working.

Any suggestion ?

 

Unity does not have much Android native code. Im currently looking for some plugin which would disable it. Still nothing.

 

Try to use it:

Code (CSharp):

1. public static int GetKeyboardHeight(bool includeInput)
2.     {
3. #if UNITY_ANDROID
4.         using (AndroidJavaClass UnityClass = new AndroidJavaClass("com.unity3d.player.UnityPlayer"))
5.         {
6.             AndroidJavaObject View = UnityClass.GetStatic<AndroidJavaObject>("currentActivity").Get<AndroidJavaObject>("mUnityPlayer").Call<AndroidJavaObject>("getView");
7.  
8.             using (AndroidJavaObject Rct = new AndroidJavaObject("android.graphics.Rect"))
9.             {
10.                 View.Call("getWindowVisibleDisplayFrame", Rct);
11.  
12.                 int height = Rct.Call<int>("height");
13.                 int width = Rct.Call<int>("width");
14.  
15.                 int systemHeight = Display.main.systemHeight;
16.                 int systemWidth = Display.main.systemWidth;
17.  
18.                 return Screen.height - Rct.Call<int>("height");
19.             }
20.         }
21. #elif UNITY_IOS
22.         return (int)TouchScreenKeyboard.area.height;
23. #else
24.         return 0;
25. #endif
26.     }

 

I have an issue with currentActivity field - it simply cannot find it in unity editor. Web says it's fine - on build everything should work but it doesn't. I don't even know how to check if it returns actual keyboard height during Android build
By the way what is the best way to debug such Android builds? Keyboard doesn't want to display properly in emulators

 

You can debug by build a Development Build and check Script Debugging (you can use the build and run option to speed up the device deploy):
https://bertt.wordpress.com/2018/06...unity3d-android-application-in-visual-studio/

this is the only way I know for proper debugging on Android.

 

Thnx a lot! I've finally done it!

Moreover, this solution only works if your screen is the same as canvas. Otherwise, for example, if you set UI Scale mode to "Scale with screen size" you have to find relative to canvas keyboard height. To do this simply try this:

Code (CSharp):

1.  float KBrelativeTOscreen = MobileUtilities.GetKeyboardHeight(true);
2.             float screenToCanvasRatio = Screen.height / canvas.rect.height;
3.  
4.             float KBrelativeToCanvas = KBrelativeTOscreen / screenToCanvasRatio;
5.  
6.              fullChatOverlay.anchoredPosition = new Vector2(0f, KBrelativeToCanvas);

it's a little off-top, but it solves the next problem - to resize properly chat-box(or whatever) using this value.

p.s. Do not forget to use float instead of int - or, more likely, it will break everything.

 

I got this to work relative to the canvas size, here is the code... All you need is to call it with the canvas RectTransform reference.

Code (CSharp):

1.  
2.     public static int GetRelativeKeyboardHeight(RectTransform rectTransform, bool includeInput)
3.         {
4.             int keyboardHeight = GetKeyboardHeight(includeInput);
5.             float screenToRectRatio = Screen.height / rectTransform.rect.height;
6.             float keyboardHeightRelativeToRect = keyboardHeight / screenToRectRatio;
7.    
8.             return (int) keyboardHeightRelativeToRect;
9.         }
10.    
11.         private static int GetKeyboardHeight(bool includeInput)
12.         {
13.     #if UNITY_EDITOR
14.             return 0;
15.     #elif UNITY_ANDROID
16.             using (AndroidJavaClass unityClass = new AndroidJavaClass("com.unity3d.player.UnityPlayer"))
17.             {
18.                 AndroidJavaObject unityPlayer = unityClass.GetStatic<AndroidJavaObject>("currentActivity").Get<AndroidJavaObject>("mUnityPlayer");
19.                 AndroidJavaObject view = unityPlayer.Call<AndroidJavaObject>("getView");
20.                 AndroidJavaObject dialog = unityPlayer.Get<AndroidJavaObject>("mSoftInputDialog");
21.                 if (view == null || dialog == null)
22.                     return 0;
23.                 var decorHeight = 0;
24.                 if (includeInput)
25.                 {
26.                     AndroidJavaObject decorView = dialog.Call<AndroidJavaObject>("getWindow").Call<AndroidJavaObject>("getDecorView");
27.                     if (decorView != null)
28.                         decorHeight = decorView.Call<int>("getHeight");
29.                 }
30.                 using (AndroidJavaObject rect = new AndroidJavaObject("android.graphics.Rect"))
31.                 {
32.                     view.Call("getWindowVisibleDisplayFrame", rect);
33.                     return Screen.height - rect.Call<int>("height") + decorHeight;
34.                 }
35.             }
36.     #elif UNITY_IOS
37.             return (int)TouchScreenKeyboard.area.height;
38.     #endif
39.         }
40.  

 

Code (CSharp):

1. TouchScreenKeyboard.area.height

I got a full screen height with this code on iOS. What am I doing wrong?
Unity 2019.4.12f, iPhone 6

 

Oh, gosh It was long ago. As far as I remember it should work on IOS - there isn't any problem with it

 

I have the same problem on IOS: when i don't make Open for TouchScreenKeyboard (it was opened when was touch in Webview form) and try to get TouchScreenKeyboard.area.height - it returns value that equals to Screen.height

 

Unfortunately, I have tested it only for android, and you have to probably do some research for your own, if it is not working as expected(Probably it involves some native code manipulation, but I am not completely sure). btw you can pin some code to have a look here.