Search Unity

Understanding Canvas Scaler Screen Match Mode and Reference Resolution

Discussion in 'UGUI & TextMesh Pro' started by CodeRonnie, Mar 29, 2018.

  1. CodeRonnie

    CodeRonnie

    Joined:
    Oct 2, 2015
    Posts:
    29
    EDIT: TL;DR These are my original, personal notes from trying to learn about the canvas scaler which are in depth, but sometimes nearly unreadable. I added a post further down that does a better job of explaining my general thoughts on how you should configure the scaler.

    I've been doing a lot of work with UI lately and for my own sake I spent a lot of time trying to get my head fully around the CanvasScaler, specifically the Screen Match Mode and Reference Resolution. It's important to fully understand this component and its properties at the outset of your UI design. Otherwise you may not fully grok how to specify the size of your UI elements in their rect transform without just anchoring their corners. You may not have a complete visualization of how your game or app will look on many different screen sizes and orientations. If you have to come back later and change the values of your main CanvasScaler it can really effect your whole UI design. I've spent a lot of time digging around in the UI source code (thanks to Unity for exposing this!) to understand this and many other aspects of Unity UI. Hopefully these notes I made for myself can be useful to others trying to understand what to think of CanvasScaler.


    Canvas
    RenderMode: Screen Space - Overlay vs. Camera
    Camera mode applies an aditional scalar to the scale factor to shrink the UI to the size of the frustum.

    Canvas Scaler
    One important thing to visualize is that while the canvas scaler changes the scale of the canvas it simultaneously changes the size of the canvas. The end result is an effect similar to the way a dolly zoom works. The size of the canvas in world space does not change because of the canvas scaler. It always matches the pixel resolution of the screen. The canvas is sized and scaled to fit that space based on the chosen canvas scaler properties.

    UI Scale Mode: (Scale With Screen Size) - You almost never want to have constant pixel size or a constant physical size.

    Reference Resolution: (1024x768) - The basis on which our other numbers for sizing UI are framed. If the reference resolution is 1920 x 1080 then sizing an image's rect transform to 512 x 512 means 512 pixels in the context of a 1920 x 1080 screen, then scale accordingly based on screen match mode and the actual resolution of the current device. The common denominator for aspect ratio on all platforms is 4:3 / 3:4. There are some desktop monitors that are 5:4, but it's 1/16 more narrow than 4:3 and not very common. We'll get to aspect ratios and orientations in the next section, but 1024x768 is by far the most common pixel resolution for 4:3 ratios.

    Screen Match Mode: The scale factor for the entire canvas will either be (screenSize.x / m_ReferenceResolution.x) or (screenSize.y / m_ReferenceResolution.y) with match mode expand or shrink and somewhere between those two values with match width or height. (The screenSize variable accounts for the actual size of the viewport rectangle for that camera display on screen if the camera display does not occupy the full screen resolution.) Match width or height is useful for forcing the exclusive use of one of those calculations because the expand and shrink match modes will automatically select between them depending on the screen resolution. The expand match mode chooses the smallest value from those two calculations to use as a scalar and shrink chooses the largest. The size of the canvas will be the pixel resolution of the screen divided by the chosen scale factor, but it wall have that scale factor applied so in world space in the scene view it will be the size of the screen resolution. This is the dolly zoom like effect mentioned before. So if the screen resolution is not the same aspect ratio as the reference resolution, one of the canvas dimensions will match the reference resolution, and the other will stretch by a number of pixels to meet the screen aspect ratio. Which axis is chosen to match the reference resolution (horizontal or vertical) depends not only on the screen match mode, but the relationship between the screen resolution and reference resolution. For expand, the axis that needs to be scaled down the most, or scaled up the least will match the reference resolution. For shrink, the axis that needs to be scaled down the least, or scaled up the most will match the reference resolution. When considering landscape aspect ratios, the only reason a anyone would prefer a wide screen monitor is is if the height of elements on screen stay the same as they would on say a 4:3 screen. If so, you simply have more space horizontally on the wider the screen. If the relative width of elements on screen remain the same, then a wider screen effectively cuts out more and more vertical space as it gets wider. Therefore, on landscape it likely makes the most sense to only respect the height of the screen when determining the scale factor. However, the opposite is true for portrait orientation. This means that for landscape you should design for the most narrow aspect ratio you expect to support, like 4:3. Then, have the elements retain their relative height on screen while spacing UI elements out on wider aspect ratios. Designing for the widest supported aspect ratio would mean that elements would come together and overlap on more narrow screens. Similarly for portrait orientation (phone & tablet), you should design on the shortest supported aspect ratio. If your application supports both landscape and portrait orientation, you may need to create a different canvas for each orientation simply because it makes more sense to have a completely different UI layout for portrait than you do for landscape. So just set the canvas scaler settings appropriately for each (match width or height and set height for landscape and width for portrat). If the aspect ratio of the screen matches the aspect ratio of the reference resolution, then the screen match mode setting does not matter. The calculated scale factor value will be the same no matter the match mode setting. It's only when the aspect ratio is different from the original design that the match mode becomes important. Also, this may not even change the size of elements in your UI if you are using anchors for everything rather than pixel size values. Remember, the pixel size values are in the context of your reference resolution, not the actual screen size!

    ScreenMatchMode.Expand: Expand always scales the rectangle of the reference resolution to fit completely inside of the actual screen resolution. That's not to say that your actual UI will be formatted that way, but the parent scale factor for all elements in the canvas will be calculated according to that concept. So if the screen aspect ratio is more narrow than the reference, it will choose a scale factor based on width and make sure all of the scaled elements fit the width of the screen properly, while creating additonal empty space vertically between elements. If the screen is wider than the reference it will scale based on height and fit every element's height to the proper ratio on screen, providing extra horizontal space between elements. This is a good all around choice for handling both landscape and portrait with the same canvas, but should probably be designed with and use a reference resolution that most resembles a square like 4:3 1024x768, or literally using a square reference resolution.
    scaleFactor = Mathf.Min(screenSize.x / m_ReferenceResolution.x, screenSize.y / m_ReferenceResolution.y);

    ScreenMatchMode.Shrink: Shrink makes the screen fit over the smallest part of the reference aspect ratio, so some part of that reference rectangle will always hang outside one axis of the screen rectangle. That's not to say that your actual UI will appear that way, but the parent scale factor for all elements in the canvas will be calculate according to that concept. This almost never seems like what you would want unless you were desiging for portrait and had your reference set to 9:16, but then you could just match by width and that would probably be better. I'm imagining a full screen mobile app where items in a list get consumed into a scrollable rect on shorter screens. So you've designed the items in the scroll rect to be the correct relative height on the tallest screen, and the relative width of elements is matched. Therefore as screens get smaller the screen "shrinks" the vertical space available over the design in reference space. Effectively on a tall screen you can see more elements in the list. I don't see as much use for this concept horizontally in landscape.
    scaleFactor = Mathf.Max(screenSize.x / m_ReferenceResolution.x, screenSize.y / m_ReferenceResolution.y);

    ScreenMatchMode.MatchWidthOrHeight:
    // If one axis has twice resolution and the other has half, it should even out if widthOrHeight value is at 0.5.
    // In normal space the average would be (0.5 + 2) / 2 = 1.25
    // In logarithmic space the average is (-1 + 1) / 2 = 0
    float logWidth = Mathf.Log(screenSize.x / m_ReferenceResolution.x, kLogBase);
    float logHeight = Mathf.Log(screenSize.y / m_ReferenceResolution.y, kLogBase);
    float logWeightedAverage = Mathf.Lerp(logWidth, logHeight, m_MatchWidthOrHeight);
    scaleFactor = Mathf.Pow(kLogBase, logWeightedAverage);

    I prefer to set this all the way to height for landscape designs and width for portrat designs because it has the most predictable results. Let's say you have a 1024x768(4:3) reference resolution for a typical landscape oriented video game and you're creating the UI. So, instead of using anchors for every bit of your layout, you anchor an image to the corner of the screen and set the size of the rect transform to be 128x128. That size specification is in the context of the reference resolution when using a canvas scaler with scale mode set to scale with screen size. So 128/1024 is 1/8 screen width and 128/768 is 1/6 screen height. Now you can see that those pixel size values on your rect transforms are actually relative screen size values when considered in context of your reference resolution. So if the end user is actually playing on a 1920x1080(16:9) screen and you are set to match height, the scale factor will be 1080/768=1.40625. Every rect in your canvas will be scaled by 1.40625. If that rect is anchored at its corners it doesn't usually matter. So the size of your image will change to 180 pixels on that screen. So, 180/1080 is 1/6 screen height, just like on our reference design. However, since this is a wider monitor and we designed on our most square supported resolution, 180/1920 is a little more than 1/10 of the screen width. So now there is more horizontal empty space between our UI elements on the wider monitor. Since our element is anchored at its center to the corner of the layout that empty space opens up in the center of the screen for more of the game world.

    Reference Pixels Per Unit: Specifically affects images. Similar to reference resolution. If the pixels per unit of the image asset is set to 100 (default) then when divided by reference pixels per unit of 100 (default) the scaling on the image will be 1. This only matters when clicking the "Set Native Size" button and in calculations for tiled and sliced images.
     
    Last edited: Apr 17, 2020
  2. RobWu

    RobWu

    Joined:
    Jan 16, 2018
    Posts:
    65
    Great post!
    But it could use some more formatting.. ;)

    Found your post after running into some canvas related issues with WebGL as well.
    cheers for the elaborate explanations!

    rob
     
  3. joxtheman

    joxtheman

    Joined:
    Feb 18, 2016
    Posts:
    5
    Thank you. That helps a lot! :)
     
  4. justtime

    justtime

    Joined:
    Oct 6, 2013
    Posts:
    392
    @CodeRonnie Hi there! Could you help me a little bit with a CanvasScaler?
    I'am making an android/ios app, which is looks like native material design. But i can't handle how to reach my goal: make layout adaptive for tablets.
    My CanvasScaler settings - Scale With Screen Size, Reference:360*640, Match Mode:Expand, Pixels Per Unit:100.
    I made app photos on real devices, i think it much better for understanding what i want.
    Here is nexus 5 ( 4.95", 1920x1080) and Lenovo TAB 2 A7-20F (7", 1024x600) with default settings(above)

    And here is my goal, i changed Reference to 544*640 to reach this effect


    How i got 544*640 reference? Nexus dpi is 445, lenovo is 170. I got 445/170 = 2.617. And width difference is 1080-600 = 480. So 480/2.6 = 184. 360(initial width reference) + 184 = 544.
    My question: Is this formula correct? Or may be there is another formula without using dpi(cause i can't get device dpi in editor). Thanks!
     
  5. drewdough

    drewdough

    Joined:
    Oct 5, 2016
    Posts:
    38
    This thread has been insanely helpful for me.

    Why do you recommend 4:3 though as the most narrow resolutions?
    Isn't there still a lot of Android devices that are 5:4 and so slightly narrower?
    I think if I'm developing a locked portrait game, using 5:4, such as 720x576 as the reference resolution, matching 100% on height, is the best solution?

    Please let me know your thoughts!
     
  6. CodeRonnie

    CodeRonnie

    Joined:
    Oct 2, 2015
    Posts:
    29
    @drewdough Spot on! If that's the kind of platform you're targeting those sound like good settings for your game!
     
  7. CodeRonnie

    CodeRonnie

    Joined:
    Oct 2, 2015
    Posts:
    29
    Upon further review, my thoughts are actually that you should match width on portrait mode. As I reviewed my old post and wrote this reply, it ended up rather long because I wanted to clarify some things that may not have been obvious in my original post. Those were really just notes I accumulated for myself while learning, then polished up a little for grammar. I figured it would be better to post them than delete them. But, I didn't spend much time to guarantee they made sense or were easy to read. So, this isn't all directed at you @drewdough. Much of it is just further thoughts on canvas scaler.

    Before I contradict you on matching height for portrait mode, I think you do understand the canvas scaler. Firstly, maybe those are the right settings for you, how would I know? Secondly, I agree with you about everything else. Choosing a common 4:5 (portrait) resolution you want to target is a good idea indeed if you intend to support it. I just thought that some people might not be worried about 5:4 (specifically landscape), but yes you are right that everyone should probably just do it anyway. Otherwise you risk your UI elements overlapping on those slightly smaller ratios.

    To anyone confused by why you would start by developing for the most square, uncommon ratio & resolution, it doesn't mean that you don't develop and test on all other ratios & resolutions you intend to support. It's just the reference resolution, which is actually more about aspect ratio than resolution, and it combines with the screen match mode to establish a shared axis for relative scale on all displays. So, the UI elements will always take up the same relative amount of screen space on one axis, regardless of display, ratio, resolution, even regardless of how you use anchors on that axis. On landscape everything matches height, and is always scaled appropriately on the vertical axis. On wider screens the UI elements will space out and take up less screen space. That's why you start with the most square aspect ratio. It's better than starting with the widest, and then having all of your UI run together on screens that are not as wide. If you don't want the elements to space out, if you want them to also stretch, that's what anchors are for! You can achieve any blend of both effects with ease. Anchors should be to help you control that scaling horizontally as things space out on wider screens. You should design for the smallest display, but test in the editor for every ratio and resolution that you plan to support to make sure everything scales the way you expect.

    Anyway, if I were targeting a locked portrait game I would actually want to match the reference resolution 100% to the target device's width, not height. Now, if you meant to say landscape instead of portrait, and it was just a typo, then yes I would match height on landscape and everything you said is right. Otherwise, on portrait I would match width and here's why:

    I match height on landscape layouts like a PC/console game or landscape mobile orientation, and for the same reasons I match width on portrait layouts. When you choose to match width or height, you're always committing to which axis of the display should scale predictably, regardless of anchors or pixel values in the transform rect. The UI elements will always appear to take up the same percentage of the screen along that axis. For instance, TVs and monitors are almost always used in landscape. People generally think of wide screen TVs and monitors like a 16:9 as having more real estate than a 4:3 display. So, more rectangular aspect ratios are thought of as showing more, not less. That logic also translates to mobile in portrait mode. After all it's just a "wide screen" phone rotated to be held in one hand right? It's still the same device and people still think of it as having more screen space. This is all based on aspect ratio (not resolution) and based on the way humans think about these things. So, when we're in portrait we should match the "width" because that's essentially the same physical axis on the real device as "height" in landscape if you were to just rotate the phone. The device has just been turned on it's side, so the operating system reports it as width in portrait and height in landscape. It's the shortest axis of the rectangle we want to match on all devices.

    In addition, the reference resolution should always be set to the most square aspect ratio for that orientation. (Unless of course you have a special case and are doing something different.) As the aspect ratio of an end user's display gets wider (landscape) or taller (portrait) than the designed reference ratio, the pixel values we specified in our UI elements' rect transforms should scale to take up the same relative height (landscape) or width (portrait) on ALL displays regardless of the change in aspect ratio or resolution. And that's exactly what the canvas scaler does under these configurations of [landscape/match height/5:4 reference] and [portrait/match width/4:5 reference].

    It's a good reason why we should make a different UI for landscape and portrait, even if the game or app supports both orientations. One match mode setting will not generally work for the other screen orientation, and you'll get less predictable scaling when the ratio inverts. From what I have seen, most apps really do have a completely different UI layout depending on the orientation in which the app is viewed. I'm pretty sure I saw that recommended in the Android Studio developer documentation (nothing to do with Unity, just mobile apps.) Create separate UI views for each orientation. And since this is a game engine after all, actual games don't usually support two orientations. Perhaps someone could run your game on their vertical monitor just to see what would happen, all the more reason to have a good grasp on this important component and its properties. Even in that rare situation I imagine the game would also benefit from a completely different UI depending on orientation (or a forced minimum aspect ratio). So you may as well have different canvases, scalers, and match modes for your release target. If your app rotates, just use a script to calculate orientation from the display resolution and show the correct canvas. If anyone really wants to use one UI canvas to handle this type of scaling in both landscape and portrait orientations, the best option is actually to use the expand match mode, see below.

    Interestingly, the reference resolution along that axis is just preference for you as the designer, to understand what the number values in your rect transforms mean. Every value is basically just a relative value to the reference resolution. If you are set to match reference height of 768 and you type 100 into the height value of a rect transform, you can rest assured that means you want the height of that object on all devices to be 100/768ths of the screen height. That's a percentage. It's about 13% of the screen, vertically. What about horizontally? Well, that depends on what the target ratio is. It will take up the same percentage of screen width for the same aspect ratio, regardless of pixel resolution, thanks to canvas scaler. But, the percentage will change based on the width of the ratio. However, understanding those values becomes a lot simpler with the configuration recommended here.

    The actual pixel values chosen for the reference resolution should be a common resolution for that aspect ratio, or one that we prefer to think of as our design resolution, your preference. It only affects which devices will have to apply what amount of scaling. Is it better design at low resolution and scale up, or design at high resolution and scale down? That's just a behind the scenes calculation. The visual results for a user should be about the same across all devices. Anything more in depth is up to you to decide. It's the aspect ratio that matters. The specific resolution is just what the Unity designer needs in mind when manually entering UI rect size values. And as pointed out, the aspect ratio of that most square resolution on pretty much any platform is technically 5:4(landscape) / 4:5(portrait) depending on the display orientation, although 5:4's are not that common for gaming in landscape. I don't know many gamers who play on a 5:4 monitor. But, if you want to design defensively you may as well pick a 5:4 reference resolution, even for a PC game. Then you know for a fact that your UI can never run together and overlap. Less bug reports! For portrait it probably makes a lot more sense to just say, always set your reference resolution to something that has a 4:5 ratio. I don't know the market share of mobile aspect ratios off the top of my head, but yes you are right about that!

    Expand - This match mode is actually useful in the case of using one canvas to handle both screen orientations. In this case you actually want the reference resolution to have equal numbers in both fields, a 1:1 aspect ratio, an actual square. Then, your UI can stretch to fit anything, but the layout of your UI will never change. Anchors now become important on both axes. That's the best and shortest way to explain Expand.

    Can I just pick expand, and not have to think about matching width or height if I'm just making a simple landscape game or whatever? Sure, the end result will likely be the same, but accidentally, not explicitly. Technically speaking, it requires the UI to run more conditional code in the background to check your display orientation than it actually needs to. What a waste!!! :) Also, if you don't understand explicitly which axis the UI is going to match based on your intended target platforms, then you also won't understand what those numbers are that you're punching into your rect transforms. They'll just be arbitrary numbers with no frame of reference to you. You may as well understand explicitly how your UI is intended to scale and specify it.

    Shrink - What is shrink for? I don't know. It's basically the opposite of everything I've described here. As the screen gets "smaller" (in it's aspect ratio) your UI elements will space out more and more based on their non anchored pixel values. Fun fact, I have helped release a game where the camera worked this way. It was a mobile game, and 4:3 iPads were granted more view of the world than 16:9 devices. Which reminds me, setting up the canvas scaler as I've described has the benefit of working conceptually the same way cameras work in relation to user aspect ratios. Perhaps shrink is specifically for the case that you are making a game where the camera has been scripted to have opposite behavior and the UI should follow suit. It seems more likely on mobile to me for some reason to think that more square displays would see more of the game or app.

    Should match width or height ever be set to something in between width and height? I don't know, you tell me, but I wouldn't recommend it for most games.

    Unity provides many options for this component, but many of them are unhelpful most of the time (IMHO). I think it's more a matter of the UI developers wanting to expose as many options as possible to us, and then we as the game developers get to decide what's best for ourselves. For me that's [landscape/height/5:4], [portrait/width/4:5], and sometimes expand.
     
    Last edited: May 5, 2020
    Atrixx and SlimeProphet like this.
  8. CodeRonnie

    CodeRonnie

    Joined:
    Oct 2, 2015
    Posts:
    29
    @justtime

    Based on these settings:

    Canvas Scaler:
    360 x 640
    9:16
    Expand

    Nexus 5:
    1080 x 1920
    9:16

    Tab 2
    600 x 1024
    9.375:16

    These devices have *almost* the exact same aspect ratio. So, based on everything I've described above, the first image is what I would expect to see. If the aspect ratio changed I might expect you to see more or less items in the list, but it's basically the same. What you're describing is a change in constant physical size, real world dimensions. The larger device in the real world should display more, and the buttons and UI elements should be the same size in the real world. That is indeed something you base on DPI calculations. Input on mobile often should conform to some minimum physical size to guarantee you aren't expecting user to tap on button that are too small to actually hit. Maybe on a GIANT screen you still want the button to be in the bottom right, and not bigger than your head. These are physical real world dimensions and require you to know the resolution and DPI of your display device. Unity cannot always determine these numbers reliably. There is in fact a canvas scaler mode called Constant Physical Size. However, pretty much everything I've written about in both posts is based on the scale mode Scale With Screen Size. If you select Constant Physical Size it's doing similar calculations on the UI back end. You can specify a fallback DPI in the case that Unity can't properly detect it. If you need only some parts of your UI to have a constant physical size, then you will need to either perform those calculations yourself in scripts that manually control your UI settings, or have sub-canvases with their own canvas scalers. I have a lot less experience with this kind of thing and your mileage may vary.
     
  9. YoungRichi

    YoungRichi

    Joined:
    Apr 19, 2018
    Posts:
    14
    Hey i really appreciate the explanation! Cheers :)
     
    CodeRonnie likes this.
  10. SlimeProphet

    SlimeProphet

    Joined:
    Sep 30, 2019
    Posts:
    50
    This was a very helpful post and saved me time. Thanks!
     
    JQV and CodeRonnie like this.
unityunity