Search Unity

  1. Calling all beginners! Join the FPS Beginners Mods Challenge until December 13.
    Dismiss Notice
  2. It's Cyber Week at the Asset Store!
    Dismiss Notice

Working Multi-Resolution Sprite Sheets

Discussion in '2D' started by TheFuntastic, Oct 18, 2014.

  1. TheFuntastic

    TheFuntastic

    Joined:
    Feb 11, 2013
    Posts:
    17
    Does anybody have strategies for how to use different sprite sheets for different resolutions? This is a very common workflow in just about every other app authoring environment I've worked in, and I find it very surprising there aren't more people talking about this?

    Before you say just use Reference Resolution, that is only half the solution. Inherently this means you are either upscaling low-res assets which looks poor on hi-res devices, or you are downscaling hi-res assets which looks great but incurs a huge memory hit on older devices. The official word is that we should roll our own solution:
    Okay so I get that we're expected to roll our own solution, but that is very non-trivial:

    The sprite versions you're not using should never be loaded into memory. This means:
    1. You can't reference multiple versions of sprites in the inspector, as they will consume application memory when the object is created on level load or instantiated (correct me if I'm wrong. I think you will at least avoid texture memory until it is set as the active sprite)
    2. You instead need some kind of scheme involving Resources.Load. This gets tricky because the APIs for working with sprites sheets aren't great. E.g. it's difficult to enumerate over the sprites in a sprite sheet or to say which spritesheet asset a sprite came from.
    3. You will still incur the memory hit of whatever sprite is used to set up your scene. Eg if you set up your scene using 1x assets then you will still pay the memory cost of those when switching to a 2x environment

      Furthermore, you need to ensure sprites are correctly referenced at any point in it's life cycle, such as:
    1. Sprite present in the scene on Load
    2. Sprites dynamically instantiated from prefabs
    3. Sprites referenced in animations.
    Quite frankly this sounds like a world of pain. If anybody has strategies for dealing with this I'd love to hear it. Are most people just using 2x assets, scaling down and saying to hell with older devices?

    And to the devs, ideally for unity to be a competent 2D app development platform this should be automagical. There are numerous ways to go about this, but the most straightforward I can think of is specifying different scaling versions of sprites in the property inspector or asset importer.

    I'm not sure how memory management would work, but I imagine there would be a global flag you could set to choose your desired resolution. This means loading the correct assets could be built into the scene loading scheme. As a dev that would be wonderful. ;)
     
  2. runevision

    runevision

    Unity Technologies

    Joined:
    Nov 28, 2007
    Posts:
    1,666
    Hi!

    The "official word" is a bit different now since beta 20 introduced some differences. Can you let me know where the post is you quoted so I can update it?

    Basically, it doesn't matter now if you use SD or HD as reference, text will be rendered according to the current resolution regardless. Also, sprite resolution can now be controlled by their pixelsPerUnit settings in the importer rather than having to set it from script on the Image components. There is also proper mip-map support now without having to go to the advanced settings of the texture importer.

    As for how to switching between different versions of sprites, that's still not well solved. I believe this talk by Veli from Unity has suggestions for best practices (it's about the 2D feature but that part should apply to UI as well):
     
    sluice likes this.
  3. TheFuntastic

    TheFuntastic

    Joined:
    Feb 11, 2013
    Posts:
    17
    Okay sure, was quoted by Tim C in this thread:
    http://forum.unity3d.com/threads/ui-scaling.263381/

    And wow, I really wish I'd seen that a few months ago - great resource. Anyway, I've arrived at pretty much the same solution as the one described in the talk:
    1. An Editor script to pre-process all sprites/images/etc and replace sprite references with string references to assets in the Resources folder
    2. A SpriteResolver component which fulfils these string references and loads in the correct sprite using Resources.Load() when the object is created.
    This means there are no inspector references to any sprites (and thus the scene looks empty). But crucially because textures are loaded on demand from the Resources folder, it means we always have the most efficient memory profile depending on the resolution of the device.

    Some points that help make this workable: by sticking to a strict naming convention and folder structure it’s actually quite feasible to find the source path of a sprite and determine if it has a matching 2x version.

    Secondly, in order to layout the scene properly Sprite Resolver uses [ExecuteInMode] to load the textures in the editor. This works but is far from perfect as it creates those crucial-to-avoid inspector references which may get saved with the scene. It’s possible to customise the build process to guarantee sprites are never referenced, but I’ll leave that for a more advanced version.

    I'll post my scripts as soon as I have something stable.
     
  4. TheFuntastic

    TheFuntastic

    Joined:
    Feb 11, 2013
    Posts:
    17
    For the sake of completeness I should say that another strategy seems to be mipmap level management. By generating mipmaps on your highest res textures and using QualitySettings.masterTextureLimit you can define which resolution level is drawn on device.

    This has caveats: Mipmaps can’t be manually defined, so you’re relying on scaling algorithms to downscale your assets. Secondly when you’re using the base texture level (ie your highest resolution) all the mipmaps in the chain are loaded. So this means at 2x you are paying the memory price of 1x as well (plus a little bit of interest for the lower mipmap levels).

    I couldn't actually get this working right (testing on Android) but at least know the option exists.

    To the devs: From an API point of view this is quite an elegant system to emulate. All of my textures are defined as a single asset and using a single API call I can change the resolution. Memory management would have to be quite magical behind the scenes, but I see no reason why it isn't feasible. In fact a less manual version of the asset bundle approach (as highlighted in the video) could work quite well in that regard.

    It's true that older devices with SD resolutions are gradually being phased out, But phones are already coming out in ultra HD and regular pc displays are going Retina, so I don't exactly see the problem as going away anytime soon.
     
  5. andsee

    andsee

    Joined:
    Mar 20, 2012
    Posts:
    88
    I use the mipmap approach and did some performance tests to confirm that on iOS at least setting this speeds up loading and reduces the memory footprint as expected.

    Note that the Unity Profiler does not report the amount of memory used by the textures correctly it still reports the full size image even though it's never loaded. Confirmed by using large textures and watching the memory via xCode instruments.

    I'd be interested if anyone has done similar tests on Android.

    The main downside to this solution is that you're wasting memory with the below the minimum size you want since as far as I'm aware all of the mipmaps down to 1x1 or some small size have to be loaded. Although on the flip side it allows you to scale your assets in transitions and get better looking results.

    On the plus, these days with so many device resolutions I've given up on pixel perfect and expect some scaling across devices (We design so that our main device has no scaling), mip maps and trilinear should give a visual improvement when you're between two sizes.
     
  6. TheFuntastic

    TheFuntastic

    Joined:
    Feb 11, 2013
    Posts:
    17
    Ah, so if the profiler reports the wrong figure that would explain things. Does anybody know if there is an equivalent of xCode instruments for Android?

    Also it's worth mentioning the extra memory for mipmaps will always be bound to 33%.
    Mip level base: 2048x2048 tex is ~16mb + 33% mip maps = ~ 22mb total
    Mip level 1 : 1024x1024 tex is ~4mb + 33% mip maps = ~5.3mb total

    So from a pure memory point of view this can be very effective.
     
  7. TheFuntastic

    TheFuntastic

    Joined:
    Feb 11, 2013
    Posts:
    17
    As promised the scripts I'm using. They aren't prefect but it seems to work. Biggest point to remember is that this should always be run before you execute a build to make sure there are no lingering inspector references to sprites.

    Also I'm not doing anything about animations or prefabs that aren't present in your scene:

    SpriteTools.cs:
    Code (CSharp):
    1. using UnityEditor;
    2. using UnityEngine;
    3. using UnityEngine.UI;
    4.  
    5. public class SpriteTools : Editor
    6. {
    7.     public const string SUFFIX_SD = "";
    8.     public const string SUFFIX_HD = "@2x";
    9.  
    10.  
    11.     /// <summary>
    12.     /// Can be used as part of a build tool chain to make sure all sprites are mem friendly string references
    13.     /// to textures in the Resources folder.
    14.     /// </summary>
    15.     [MenuItem("Sprites/Preprocess All Scenes")]
    16.     public static void PreprocessAllScenes()
    17.     {
    18.         if (!EditorApplication.SaveCurrentSceneIfUserWantsTo())
    19.         {
    20.             return;
    21.         }
    22.  
    23.         var currentScene = EditorApplication.currentScene;
    24.  
    25.         foreach (var scene in EditorBuildSettings.scenes)
    26.         {
    27.             EditorApplication.OpenScene(scene.path);
    28.             PreprocessSprites();
    29.         }
    30.  
    31.         EditorApplication.OpenScene(currentScene);
    32.     }
    33.  
    34.     /// <summary>
    35.     /// This removes all scene references and make sure they are being loaded from Resources.
    36.     /// </summary>
    37.     [MenuItem("Sprites/Preprocess Current Scene")]
    38.     public static void PreprocessSprites()
    39.     {
    40.         if (!EditorApplication.SaveCurrentSceneIfUserWantsTo())
    41.         {
    42.             return;
    43.         }
    44.        
    45.         //Resources.FindObjectsOfTypeAll is the same as FindObjectsOfType, except if finds disabled objects as well
    46.         var spriteRenderers = Resources.FindObjectsOfTypeAll<SpriteRenderer>();
    47.         for (int i = 0; i < spriteRenderers.Length; i++)
    48.         {
    49.             var renderer = spriteRenderers[i];
    50.             if(MapSprite(renderer.sprite, renderer))
    51.             {
    52.                 //A valid mapping has been created, safe to remove scene reference
    53.                 renderer.sprite = null;
    54.             }
    55.         }
    56.  
    57.         var imageRenderers = Resources.FindObjectsOfTypeAll<Image>();
    58.         for (int i = 0; i < imageRenderers.Length; i++)
    59.         {
    60.             var image = imageRenderers[i];
    61.             if(MapSprite(image.sprite, image))
    62.             {
    63.                 //A valid mapping has been created, safe to remove scene reference
    64.                 image.sprite = null;
    65.             }
    66.         }
    67.  
    68.         EditorApplication.SaveScene();
    69.     }
    70.  
    71.     private static bool MapSprite(Sprite sprite, Component rendererComponent)
    72.     {
    73.         if(sprite == null) return false;
    74.  
    75.         var path = AssetDatabase.GetAssetPath(sprite.texture);
    76.  
    77.         if(path.IndexOf("Resources/unity_builtin_extra") > -1)
    78.         {
    79.             //Ignore default unity assets
    80.             return false;
    81.         }
    82.  
    83.         if(path.IndexOf("Assets/Resources") < 0)
    84.         {
    85.             Debug.LogError(string.Format("The sprite {0} doesn't exist in the resources folder and will be ignored. Scene: {1} Current path: {2}",
    86.                                     sprite.name,
    87.                                     EditorApplication.currentScene,
    88.                                     path)
    89.                                 );
    90.             return false;
    91.         }
    92.  
    93.         var resolver = rendererComponent.GetComponent<SpriteResolver>();
    94.         if (resolver == null)
    95.         {
    96.             resolver = rendererComponent.gameObject.AddComponent<SpriteResolver>();
    97.             resolver.Preinitialize();
    98.         }
    99.  
    100.         BuildDefinitions(resolver, sprite);
    101.  
    102.         return true;
    103.     }
    104.  
    105.     private static void BuildDefinitions(SpriteResolver resolver, Sprite sprite)
    106.     {
    107.  
    108.         resolver.spriteName = StripName(sprite.name);
    109.         resolver.assetPath = StripName(StripPath(AssetDatabase.GetAssetPath(sprite.texture)));
    110.  
    111.         BuildDefinitonAtScale(resolver, 1, SUFFIX_SD);
    112.         BuildDefinitonAtScale(resolver, 2, SUFFIX_HD);
    113.     }
    114.  
    115.     private static void BuildDefinitonAtScale(SpriteResolver resolver, float scale, string suffix)
    116.     {
    117.         var resolution = resolver.GetResolution(scale, false);
    118.        
    119.         if(resolution == null)
    120.         {
    121.             resolution = new SpriteResolution() {suffix = suffix, scale = scale};
    122.             resolver.resolutions.Add(resolution);
    123.         }
    124.  
    125.         if (!CheckResourceExists(resolver.spriteName, suffix, resolver.assetPath))
    126.         {
    127.             resolver.resolutions.Remove(resolution);
    128.             Debug.LogError(string.Format("{0} has a missing resolution at scale {1}", resolver.spriteName, scale));
    129.         }
    130.     }
    131.  
    132.     private static bool CheckResourceExists(string spriteName, string suffix, string assetPath)
    133.     {
    134.         //Sadly it seems to only way to verify the existence of sprites is to load them.
    135.         //Though this only happens in the editor, so should have no gameplay memory consequences
    136.  
    137.         var sprites = Resources.LoadAll<Sprite>(assetPath + suffix);
    138.  
    139.         if (sprites == null || sprites.Length < 1) return false;
    140.  
    141.         for (int i = 0; i < sprites.Length; i++)
    142.         {
    143.             if (spriteName + suffix == sprites[i].name) return true;
    144.         }
    145.         return false;
    146.     }
    147.  
    148.     [MenuItem("Sprites/Test SD")]
    149.     private static void TestSD()
    150.     {
    151.         var spriteResolvers = FindObjectsOfType<SpriteResolver>();
    152.         foreach (var sr in spriteResolvers)
    153.         {
    154.             sr.ForceSD();
    155.         }
    156.     }
    157.  
    158.     [MenuItem("Sprites/Test HD")]
    159.     private static void TestHD()
    160.     {
    161.         var spriteResolvers = FindObjectsOfType<SpriteResolver>();
    162.         foreach (var sr in spriteResolvers)
    163.         {
    164.             sr.ForceHD();
    165.         }
    166.     }
    167.  
    168.  
    169.     #region Utils
    170.     private static bool IsHD(string name)
    171.     {
    172.         if(name.IndexOf(SUFFIX_HD) > -1)
    173.         {
    174.             return true;
    175.         }
    176.         return false;
    177.     }
    178.  
    179.     private static string StripName(string name)
    180.     {
    181.         var stripped = name;
    182.         if(SUFFIX_SD.Length> 0)
    183.         {
    184.             stripped = stripped.Replace(SUFFIX_SD, string.Empty);
    185.         }
    186.         if(SUFFIX_HD.Length > 0)
    187.         {
    188.             stripped = stripped.Replace(SUFFIX_HD, string.Empty);
    189.         }
    190.         return stripped;
    191.     }
    192.  
    193.     private static string StripPath(string path)
    194.     {
    195.         var stripped = path.Replace("Assets/Resources/", string.Empty);
    196.         var ind = stripped.LastIndexOf(".");
    197.         stripped = stripped.Substring(0, ind);
    198.         return stripped;
    199.     }
    200.  
    201.     private static string GetExtension(string filepath)
    202.     {
    203.         var result = filepath.Split(".".ToCharArray());
    204.         return "." + result[result.Length - 1];
    205.     }
    206.     #endregion
    207.  
    208. }
    209.  
    SpriteResolver.cs:
    Code (CSharp):
    1. using System.Collections.Generic;
    2. using UnityEngine;
    3. using System.Collections;
    4. using UnityEngine.UI;
    5.  
    6. [ExecuteInEditMode]
    7. public class SpriteResolver : MonoBehaviour
    8. {
    9.     //Set this on Applicaiton start up to determine which sprites get loaded.
    10.     public static float scaleFactor = 1;
    11.  
    12.     public string spriteName;
    13.     public string assetPath;
    14.  
    15.     public List<SpriteResolution> resolutions;
    16.  
    17.     private SpriteRenderer spriteRenderer;
    18.     private Image imageRenderer;
    19.  
    20.     //Called from editor script
    21.     public void Preinitialize()
    22.     {
    23.         if(resolutions == null)
    24.         {
    25.             resolutions = new List<SpriteResolution>();
    26.         }
    27.     }
    28.  
    29.     void Awake()
    30.     {
    31.         spriteRenderer = GetComponent<SpriteRenderer>();
    32.         imageRenderer = GetComponent<Image>();
    33.  
    34.         //No point loading if the component has just been created by the editor
    35.         if (resolutions != null && resolutions.Count > 0)
    36.         {
    37.             Load(GetResolution());
    38.         }
    39.     }
    40.  
    41.     private void Load(SpriteResolution resolution)
    42.     {
    43.         var sprite = GetSprite(resolution);
    44.  
    45.         if(sprite == null)
    46.         {
    47.             Debug.LogError(string.Format("The sprite {0} doesn't exist", spriteName));
    48.         }
    49.  
    50.         if (spriteRenderer != null)
    51.         {
    52.             spriteRenderer.sprite = sprite;
    53.         }
    54.         if (imageRenderer != null)
    55.         {
    56.             imageRenderer.sprite = sprite;
    57.         }
    58.     }
    59.  
    60.     private SpriteResolution GetResolution()
    61.     {
    62.         //No point seeing high res on pc //Can override for testing
    63.         if (Application.isEditor || !Application.isMobilePlatform)
    64.         {
    65.             return GetResolution(1);
    66.         }
    67.  
    68.         //Normal behavior
    69.         return GetResolution(scaleFactor);
    70.     }
    71.  
    72.     public SpriteResolution GetResolution(float scale, bool fallback = true)
    73.     {
    74.         for (int i = 0; i < resolutions.Count; i++)
    75.         {
    76.             if(resolutions[i].scale == scale)
    77.             {
    78.                 return resolutions[i];
    79.             }
    80.         }
    81.  
    82.         //Try accomodate missing assets
    83.         if(fallback && resolutions.Count > 0)
    84.         {  
    85.             Debug.LogError(string.Format("The sprite {0} doesn't exist at scale {1}, falling back to scale {2}", spriteName,scale, resolutions[0].scale));
    86.             return resolutions[0];
    87.         }
    88.  
    89.         return null;
    90.     }
    91.  
    92.     private Sprite GetSprite(SpriteResolution resolution)
    93.     {
    94.         if(resolution == null)
    95.         {
    96.             return null;
    97.         }
    98.  
    99.         var fullAssetPath = assetPath + resolution.suffix;
    100.         var sprites = Resources.LoadAll<Sprite>(fullAssetPath);
    101.  
    102.         if (sprites == null || sprites.Length < 1)
    103.         {
    104.             Debug.LogError("No sprite assets can be found at the path " + fullAssetPath + ". Please be sure to reprocess sprites");
    105.             return null;
    106.         }
    107.  
    108.         Sprite sprite = null;
    109.  
    110.         for (int i = 0; i < sprites.Length; i++)
    111.         {
    112.             string fullname = spriteName + resolution.suffix;
    113.             if (fullname == sprites[i].name)
    114.             {
    115.                 sprite = sprites[i];
    116.                 break;
    117.             }
    118.         }
    119.  
    120.         return sprite;
    121.  
    122.     }
    123.  
    124.     //Use this for testing form the editor to check which sprites are present at each resolution
    125.     public void ForceHD()
    126.     {
    127.         Load(GetResolution(2, false));
    128.     }
    129.  
    130.     //Use this for testing form the editor to check which sprites are present at each resolution
    131.     public void ForceSD()
    132.     {
    133.         Load(GetResolution(1, false));
    134.     }
    135. }
    136.  
    137.  
    138. [System.Serializable]
    139. public class SpriteResolution
    140. {
    141.     public float scale = 1;
    142.     public string suffix;
    143. }
    144.  
     
  8. kujo

    kujo

    Joined:
    Aug 19, 2013
    Posts:
    106
    Great work, looks good. Been wanting a solution like this in Unity since we started playing with the new UI. 2D Toolkit had a really nice way of handling it, but as this isn't available in Unity, we have used 4x assets and hoping for the best. Its not been tested on low end devices yet, but this will definitely ensure we don't blow memory.
     
  9. dchau_hh

    dchau_hh

    Joined:
    Jan 22, 2014
    Posts:
    24
  10. andsee

    andsee

    Joined:
    Mar 20, 2012
    Posts:
    88
    Yep you're correct, take a look at http://forum.unity3d.com/threads/un...t-pack-images-inside-resources-folder.248349/ It appears to be the current thinking of 'correct' functionality but I think it's really an oversight as packing them would not introduce double storage and would instead save memory and provide the functionality we are after.
     
  11. CalaveraX

    CalaveraX

    Joined:
    Jul 11, 2013
    Posts:
    131
    I'm in research also for get a solution like this. but i'm doing it for my interfaces with unity 4.6

    All the "tileable" elemets are not problems, they are not resized thanks to the sliced sprites, but for some backgrounds and avatar images that i have, its a mess.

    And i need all of these images in the view on editor for the GUI guy (The ones who makes the interfaces visually) needs to see them :(
     
  12. TheFuntastic

    TheFuntastic

    Joined:
    Feb 11, 2013
    Posts:
    17
    Interesting, I was actually mostly unaware of Unity's default packing options.

    I can happily report though that sprite sheets imported from TexturePacker work as expected in the Resources folder.
     
  13. Kiori

    Kiori

    Joined:
    Jun 25, 2014
    Posts:
    160
    I was looking for a solution to this, also possibly 2dtk based like the kujo mentioned above. I'll give the script a run later on, if I make any improvements i'll report back.
     
  14. luispedrofonseca

    luispedrofonseca

    Joined:
    Aug 29, 2012
    Posts:
    800
    I'm surprised there's no solution to this yet. 2D Toolkit handles it perfectly with the sprite collections, but would be great to see a "native" solution.
     
  15. mdrotar

    mdrotar

    Joined:
    Aug 26, 2013
    Posts:
    377
    I'm disappointed this is still an issue in Unity 5. Could we get an update from Unity on this? Can we expect a solution soon?
     
  16. Kiori

    Kiori

    Joined:
    Jun 25, 2014
    Posts:
    160
    The funny thing is unity UI has the canvas scaler, so really its clear that it would take a engine dev a few minutes to whipup a built-in, editor based(no coding on our side), all encompassing, multi device/resolutin solution... yet it doesnt happen.
     
  17. Adam-Buckner

    Adam-Buckner

    Joined:
    Jun 27, 2007
    Posts:
    5,670
    That point of view is naïve, Kiori.

    Due to the huge number of devices, aspect ratios and resolutions, it is not possible to automate UI design for every possible situation - and more importantly - every possible game.

    These decisions need to be made by the team developing the project.

    Each situation, each device and each game or app has it's own requirements.

    Control schemes can vary immensely between devices, and even more between platforms.

    When developing your project in Unity, these are decisions you need to make.

    There are many tools, including anchors, the canvas scaler and more, at the disposal of the project developers such as yourself... but whether a UI is scaled, anchored or completely re-created depending on the target device must be a decision by the people building the project based on their project and the targeted devices.

    We have a session on Resolution Independence in the Live Training Archive.

    If, for some reason, I don't understand your point of view, and you do find it easy to do: please create this package and release it on the Asset Store! I'm sure people will find it a useful resource. Just be aware that this will probably not be trivial.

    Now, these people claim to have a solution: https://www.assetstore.unity3d.com/en/#!/content/2116 They've been working hard at developing this solution. I've not tested it.
     
  18. mdrotar

    mdrotar

    Joined:
    Aug 26, 2013
    Posts:
    377
    @Adam Buckner

    It sounds like you've misunderstood the problem.

    This isn't an issue about anchors, placement, sizes or aspect ratios. It's about loading low or high resolution images depending on the pixel density (or perhaps other criteria) of the running device. It's a concept that is built right into Android and iOS because it's essential for mobile development. Unity doesn't have built-in support for this like native iOS/Android development does and it doesn't seem like there is a good way to work around that.

    Please take the time to read through this thread. So far it seems impossible to do things "the Unity way" without wasting memory or cpu/graphics power. This issue does come up during the question period of your Resolution & Device Independence video at around this point www.youtube.com/watch?v=ezeoYnLBpnE&t=35m30s. The issue is eventually left off with "put the images in your resources folder and load from there" which is what TheFuntastic's script above does. But then read along further in this thread and it's noted that sprites in the Resources folder don't get packed by the sprite packer.

    If I'm wrong about it not being possible, then please do explain how to accomplish this.

    Also, FYI, it rubs customers the wrong way when a Unity employee's response to a basic problem in Unity is to spend an extra $150 on the asset store. And judging by the reviews, it doesn't even support Unity 5. I'd say there's a good chance it's just another abandoned asset.

    So could @Adam Buckner , @runevision or any other Unity employee let us know if anything is in the works to address this and if so, what priority is it? If there is no plan to address this, please let us know in that case as well so we can make decisions accordingly.
     
  19. runevision

    runevision

    Unity Technologies

    Joined:
    Nov 28, 2007
    Posts:
    1,666
    Sprite swap (using different sprites depending on resolution or other criteria) affects all usage of sprites and is under the domain of 2D Team. UI Team is not actively involved. I've notified 2D Team about this thread.
     
    charmi212121, psyydack, Kiori and 2 others like this.
  20. Raimis

    Raimis

    Joined:
    Aug 27, 2014
    Posts:
    160
    + 1 on this subject. It would really be nice if unity would provide a built in solution for displaying assets of different resolutions. We are rebuilding one of our UIs really soon. For now we might be using a solution involving asset bundles and some scripting for ui itself. Basically we will try to achieve result where we can see all assets in stage (so that we can work in editor with all assets visible) but save stage with no assets assigned and reassign them during runtime after selecting proper assetbundle or folder in resources. Assetbundle sounds little better due to async loading.
     
  21. Mike-Geig

    Mike-Geig

    Unity Technologies

    Joined:
    Aug 16, 2013
    Posts:
    170
    Howdy, I currently have simulated the SD/HD behavior quite nicely with Unity 5's new asset bundle variants. I have created two project folders with the exact same content (one folder with SD images and the other HD). Then you just need to build the two variants and choose which to load at run time. This solution works great if you are planning on using assetbundles or don't mind a little code overhead.
     
  22. StarManta

    StarManta

    Joined:
    Oct 23, 2006
    Posts:
    6,877
    .....we're expected to maintain two project files to make this solution work? We have a very different idea of what the phrase "works great" means.....

    For Unity, this really is a one-guy, one-day sort of problem. After the Sprite Packer runs, it should generate one at normal resolution and one at half resolution. At runtime, it looks at the device's resolution, and picks which one to use. This is incredibly easy to solve on the engine side, and we shouldn't pretend that hacky, complicated, labor-intensive workarounds are a solution.
     
    Kiori likes this.
  23. Raimis

    Raimis

    Joined:
    Aug 27, 2014
    Posts:
    160
    Two project folders, not files. Though some kind of solution that would skip the labor involved in maintaining asset in more than 1 resolution would work better (something more like @StarManta described), but if I could get my hands on your solution @Mike, that would be a great start for us (we can't afford to wait until new solution comes out, as we start UI redesign in few days from now).
     
  24. Mike-Geig

    Mike-Geig

    Unity Technologies

    Joined:
    Aug 16, 2013
    Posts:
    170
    You are correct. We do appear to have a different definition of what "works". I know that I would prefer to actually inspect my assets before building instead of allow some algorithm to determine how my final product works. Your solution, while easy, is not exactly robust. What if you wanted 2 different aspect ratios? Is the engine just supposed to skew or distort the image? Is that a working solution. Perhaps I want different graphics entirely for different resolutions or target devices. Again, my solution handles that. Yours doesn't. I'm not suggesting that a one click solution wouldn't be cool (or won't even happen eventually). I am saying that the way I handled it does, in fact, work great and is incredibly robust.

    I need to clean it up a bit. I will try and get something reasonably well thought out put together for you.
     
    MarimoAli and luispedrofonseca like this.
  25. mdrotar

    mdrotar

    Joined:
    Aug 26, 2013
    Posts:
    377
    When exactly is a good time to do that?

    The original sprite still needs to be referenced in the scene while in editing mode so a person can see the UI. Is there a point in time during scene saving we can intercept and remove that reference so that the original sprite is not tagged as "in use" and packaged in the exported project? And then during scene loading (asynchronously or not), when is a good time to load the appropriate asset bundle (asynchronously or not), and fill the sprite slot with a reference to the asset bundled sprite so that all other objects/scripts referencing the Image/SpriteRenderer work seamlessly?

    To be honest, I'm not thrilled about the asset bundle solution. iOS/Android/Desktop asset bundles are not compatible with each other according to this, which means I'd need HD and SD asset bundles for each platform... which means I'd need to write even more code to manage the asset bundles. Seems like a deep rabbit hole.

    Edit: also keeping in mind that the sprite will likely have a packing tag set and expect to be drawn using an atlas to save draw calls.
     
    Last edited: Mar 17, 2015
    Kiori likes this.
  26. StarManta

    StarManta

    Joined:
    Oct 23, 2006
    Posts:
    6,877
    I'm not seeing how the resolution of the sprite sheet is related to the aspect ratio of the screen. So..easy answer to that question, no, it doesn't do anything at all with the aspect ratio. It just imports the exact same atlas image at 2048x2048 (the @2x version) and 1024x1024. (More accurately, one at the standard resolution, and one at half that, using a quarter of the memory.)
     
  27. Mike-Geig

    Mike-Geig

    Unity Technologies

    Joined:
    Aug 16, 2013
    Posts:
    170
    My apologies, I had a specific use case in mind and forgot to mention that first. When thinking about UI, the hardest issues in regards to image size are things like background images or full screen sprites. Often, buttons can be 9-sliced. If they can't then their size makes it less of an issue if they are slightly too big or too small. Floating images can also fall under the same premise. Background images however, are the largest images you will have on your screen at once. They generally solid quality because any degradation will be very noticeable. Furthermore, aspect ratio is a big deal in regards to full screen images. So generally, when tackling a UI solution I start with the heaviest pieces.

    Now, if that isn't an issue for you and you have very simple needs, then my solution is obviously a bit heavy. My point is that not all use cases require a simple SD / HD conversion with no inbetween or variance. If you do need variance, then your solution is a bit light. All said in done, there is no perfect solution at the moment. I don't know what the status of any solution akin to your suggestion might be. I do know that my solution does currently work. I also agree that there is a bit of overhead in getting it started. Hopefully with the continued use and development of our asset bundles and 2D tools, a system will be released that will satisfy all use cases.
     
  28. AlexVK79

    AlexVK79

    Joined:
    Mar 7, 2015
    Posts:
    2
    Hi Mike, the main point in the need of loading lower-res 2d sprites is to reduce memory requirenments and icrease frame-rate on old or entry-level devices (old iPhones, low-RAM Android or WP8 devices)
    or easily make our OLD games look better (by simply adding 2x/4x art) on new retina/retina-hd devices.

    For 3D there is a QualitySettings.masterTextureLimit which does similar thing, but it requires mipmaps,
    which means extra 33% RAM and bigger .ipa/.apk size.

    What we need is something like QualitySettings.master2DSpriteLimit which works very similar to masterTextureLimit, but:
    - only works with 2D textures (Sprite, uGUI) - i.e. Textures without mipmaps, so that it can work together with masterTextureLimit;
    - applies downscaling at loading in runtime (so that ipa/apk size is smaller - no extra mipmap levels)
    - the system works fine with automatic Sprite Packer (so that it adds proper padding considering that the atlas can be downscaled to 2x or to 1x (assuming that working set is 4x) - or just to 1x (assuming that working set is 2x) - i.e need some project-wide setting for that).

    Alternatively this system could create 1/2 and 1/4 downscaled sprites and atlases at build time and put them into ipa/apk for faster game loading but bigger ipa/apk size. (This could also be a project-wide setting).

    If developer has chosen that he designs a game in 2x resolution, the system could warn him if any sprites are not multiple of 2 (so that they can downscale to 1x without problems).
    If developer has chosen that he designs a game in 4x resolution, the system could warn him if any sprites are not multiple of 4 (so that they can downscale to 2x and 1x without problems).

    There could be some more things here and there, so it is probably much more than a single day to add such system to Unity, but overall such system can be helpful for many devs right now and it can later work without conflicts with any more advanced systems which Unity team maybe implements (or not) in Unity 6 or 7 :)
    ----------------------------------------------------------------------------------------------------------
    An alternative solution could be to allow to _override_ resource loading somewhere in MonoBehaviour.
    So that when scene is loading the script is notified each time some Sprite or Texture is about to load
    and we could choose to load another version (2x, 4x, or even different aspect ratio sprite).

    This must work not only with assets put in Resources folder, so that SpritePacker still works on alternative asset versions.

    But if they are not in Resources, Unity should have a way to detect that these asstes still need to be included during build. This could be something like a system of "Linked Asset versions". So when you link one sprite (2x, or 4x) to another (1x) in the editor, Unity would add 2x/4x assets to the build if 1x sprite is referenced in scene.

    And this same "Linked Asset versions" property would allow MonoBehaviour script to _discover_ alternative asset versions (no need for Resource.Load) and decide which one to load (main one or some alternative).
    This solution could be more robust becasue it would allow to load alternative Sprite versions depending not only on device resolution (1x,2x,4x) but also on aspect ratio (normal/wide, etc) or anything else such as target OS, current language selected, etc.

    Even more, you could add an option to allow downscaling the main Sprite/Texture to 1/2 or 1/4 of its size
    instead of selecting alternative asset - thus such solution could have all benefits of the first one.
    ----------------------------------------------------------------------------------------------------------
     
    Stef_Morojna and Kiori like this.
  29. Raimis

    Raimis

    Joined:
    Aug 27, 2014
    Posts:
    160
    Well out of the box solution might work, but when you dig deeper into things you find it very limiting. After giving it a better thought I don't agree that unity should handle asset resizing automatically - there are many problems related to that like image settings, image might be used as metadata for the game (we use bitmaps in one of our games as guidance for AI, such image cannot be modified), etc. Or this could be optional with some settings. What would really be nice is to have an ability to load different asset sets easily + maintaining preview in scene view (but scene shouldn't have assets assigned when launching the project). Anyway, nice thing that you can build this easily with unity! If i'd have to choose I'd rather choose that unity focuses on patching current bugs in engine rather than develop new features :)
     
  30. luispedrofonseca

    luispedrofonseca

    Joined:
    Aug 29, 2012
    Posts:
    800
  31. mdrotar

    mdrotar

    Joined:
    Aug 26, 2013
    Posts:
    377
    @Mike Geig Any chance we could still get that example using asset bundles? If it's taking a while to put together, that's fine. I just wanted to make sure you didn't forget about it.
     
  32. Kiori

    Kiori

    Joined:
    Jun 25, 2014
    Posts:
    160
    Everything written in this thread further states what i said. Unity needs better ways to handle differently size/shaped sprites/atlasses/whatever, which will have to include a camera assistant or whatnot.
    I feel that the UI in particular has a lot of tools for it, but the overall 2d handling could be more editor bound.
    The best tutorial i've seen so far is this one:
    and yet its not what i'd hope for.

    Like @luispedrofonseca said, 2dtoolkit has great solutions for the all the issues you may encounter, not just our personal use cases.(including resolution handling, etc, you name it)

    In the end 2dtk is a better engine than unity3d. 2d remains a second citizen in the unityverse.
     
    rakkarage likes this.
  33. Raimis

    Raimis

    Joined:
    Aug 27, 2014
    Posts:
    160
    @Mike Geig how is your solution coming together? Can you please publish it in whatever state it is? We are starting the new UI this week and I'd really love to see that beforehand.
     
    JohnTube likes this.
  34. a.sami.sdd

    a.sami.sdd

    Joined:
    Mar 20, 2014
    Posts:
    2
    Bump!! This is a Serious Issue. Should be first in Priority List to actually get the 2D Fully Built in.

    Its a humble request from any staff from unity to atleast let us know when this thing is planned or we shouldn't expect it. I really would like to Ask Veli about it. As he said way before that it will be better.
     
  35. Mike-Geig

    Mike-Geig

    Unity Technologies

    Joined:
    Aug 16, 2013
    Posts:
    170
    Hey sorry, this thread totally fell off my radar. I am working on a complete, full system demo of asset bundles and their use for variant graphics. It isn't finished yet, but hopefully I can wrap it up soon (It's not my primary work responsibility so unfortunately I only get to work on it in my free time). I will definitely be posting it and doing a live session on it as soon as it's ready.

    I can't really share it right now (it's a giant piece of non-working crap at the moment) as I am trying out a lot of edge cases and the whole thing is a mess.
     
    rakkarage and luispedrofonseca like this.
  36. ColossalPaul

    ColossalPaul

    Unity Technologies

    Joined:
    May 1, 2013
    Posts:
    169
    Hi,

    First of all, I apologise for not getting on here sooner. No excuse. Will get better at this.

    The solution: AssetBundle Variants.

    How does it work? (See attached demo project)
    1. Create 2 folders with identical structure. Tag them as asset bundle with different variants (hd and sd)
    2. Put your scene in an asset bundle all by itself.
    3. In another 'bootstrap' scene, use WWW to load the variant bundle that you want (hd or sd) then load the scene asset bundle.
    4. Then call Application.LoadLevel.
    Is this only for sprites?
    • No. In the demo you will see that I have done the variant remapping for a material too.
    That's it?
    • That represents the simplest case and of highest clarity for demonstration purposes
    • With some work, you could take into account for scenes that has assets coming from multiple bundles. And only some of the bundles have variants.
    • That's when you need to inspect the asset bundle manifest
    Manifest?
    Where is the 'one click' solution?
    • We are thinking hard about it. The usage from project to project is just too varied to easily be captured by a simplistic design. For maximum flexibility, we leave it scriptable for now until we evolve the asset bundle system more.
    Why not do it like tk2d, @2x etc?
    • We want to avoid building specialized systems that solves only a specific case. As much as we could, we'd like to have a system that could do more than handle sprites. This reduces the sheer amount of code that we have to maintain thus improving quality.
    • In other words, we thought that it could be useful to be able to swap out stuff other than just sprites.
    • Asset bundles has a big role to play in the future so we want to evolve that system.
    Hope this helps.
     

    Attached Files:

    rakkarage, luispedrofonseca and Kiori like this.
  37. mdrotar

    mdrotar

    Joined:
    Aug 26, 2013
    Posts:
    377
    Thanks for putting this together, @ColossalPaul. It was helpful. And I'll look forward to @Mike Geig's live session on asset bundles.
     
  38. Tim-Wiese

    Tim-Wiese

    Joined:
    Jul 7, 2012
    Posts:
    54
    I'm curious why variants only work with Streamed Scene AssetBundles? When I read the description of how any hd or sd (variant) bundle could be swapped out at runtime, I assumed this would work with bundled prefabs and other assets as well.
    Ideally I would like to be able to bundle a prefab in a AssetBundle, then make an hd and sd bundle for the texture of the prefab, at runtime I load either the hd or sd bundle and the prefab uses the right texture from that bundle. This way I could do the same for UI as well. Bundle the prefab for a UI menu, the sprites on the prefab reference an atlas, then I bundle an hd and sd version of that atlas. At runtime if I load the hd or sd bundle the prefab will use the atlas from which ever bundle is loaded.
    Having to do this within a streamed scene seems pretty limiting.
     
  39. ColossalPaul

    ColossalPaul

    Unity Technologies

    Joined:
    May 1, 2013
    Posts:
    169
    @Tim Wiese , what you described will be possible in the very near future. I'll post another demo when that feature is done. Stay tuned. :)
     
    swipeware, scott-havird and Tim-Wiese like this.
  40. mdrotar

    mdrotar

    Joined:
    Aug 26, 2013
    Posts:
    377
    @ColossalPaul, The example you put together does seem to avoid the issues of loading both SD and HD assets which is great. It may have some other trade-offs though.

    I don't know exactly how asset bundles behave but it seems like I might end up with 2 copies of the assets on disk. If I include the asset bundle files in the build output, does the application have to unpack the asset bundles into a cache to be usable? LoadFromCacheOrDownload seems to work something like that. So then I would have the asset bundle file plus the unpacked assets stored on the device.

    In addition to that, it must take extra loading time to unpack the asset bundle first, before loading the actual assets. Not great, but with LoadFromCacheOrDownload it should only happen on first run, correct?
     
  41. ColossalPaul

    ColossalPaul

    Unity Technologies

    Joined:
    May 1, 2013
    Posts:
    169
    Hi @mdrotar

    Currently there are 2 ways to distribute asset bundle.
    1. Put them on a server
    2. As part of the game build
    There are 3 ways to read an asset bundle
    1. LoadFromCacheOrDownload
    2. WWW(url)
    3. AssetBundle.CreateFromFile
    Details:
    • LoadFromCacheOrDownload will decompress the bundle to the cache folder, and then read from there. Only stores the decompressed bundle, so 1x device storage. This only happens once.
    • WWW(url), keeps everything in memory, and does it (download and decompress) every time. Therefore more memory intensive than previous.
    • CreateFromFile only works if you distribute the uncompressed bundle with the app in the 'StreamingAssets' folder. App gets bigger, but no double space and no double memory and no decompression.
      See http://docs.unity3d.com/Manual/StreamingAssets.html
    In the near future, we will be able to read direct from compressed bundles. Stay tuned.

    Hope this helps.
     
    Last edited: Apr 2, 2015
    rakkarage likes this.
  42. mdrotar

    mdrotar

    Joined:
    Aug 26, 2013
    Posts:
    377
  43. YuriyVotintsev

    YuriyVotintsev

    Joined:
    Jun 11, 2013
    Posts:
    31
  44. Kiori

    Kiori

    Joined:
    Jun 25, 2014
    Posts:
    160
    @ColossalPaul thanks for the feedback, please make a blog post once these features are truly finished.
    All in all i wish you guys luck with improving sprite atlas/bundles support. It could be better indeed.

    Also, on a side not as i mentioned in another post, you guys need more tutorials on how to handle multi res situations.
    Right now as far as i know, for 2d at least, the best way is to handle the ortho size resizing, etc.(2d game)
    2dtk has camera bult-in overrides for it, we have the canvas-scaler for uGUI, but nothing on the camera front.

    Being that for the engineering team with source access its not that difficult of a feature to implement, you guys should look into it. Cocos2d- has this built-int and the engine isnt even feature comparable with unity, but you can easily, resize to fit screen, stretch here like this, there like that, nearly all the options you can think of.

    To me currently, the multi res sprite/asset handling and screen size/res/view/whatever handling are 2 of the biggest reasons i think unity2d is soso, while 2dtk is great. I tell ppl about 2dtk not unity.

    I've mentioned this before and suggested at least more tutorials, for the uninitiated, on the matter(handling various ratios/sizes/cameras/etc), since this -is- a recurring issue.
    But i was called unpolite and naive by a unity representative, so maybe you'll see this differently.
     
  45. swipeware

    swipeware

    Joined:
    Mar 24, 2015
    Posts:
    3
    @ColossalPaul Why does the scene itself need to be put in an asset bundle of its own (Step 2 in your post above)? I would have thought that the textures/sprites/etc would have been enough.

    I'm quite new to Unity (Coming from Corona SDK) and I'm looking for an SD/HD approach that will work well for 2D game dev, and asset bundle variants seem to be the way to go...
     
    Last edited: Apr 4, 2015
  46. ColossalPaul

    ColossalPaul

    Unity Technologies

    Joined:
    May 1, 2013
    Posts:
    169
    @YuriyVotintsev, BuildPipeline.BuildAssetBundles is the new API. Here: http://docs.unity3d.com/ScriptReference/BuildPipeline.BuildAssetBundles.html. This new system will scan the project for all asset bundles and does an incremental build and generate dependency manifests, instead of blindly building everything everytime.

    @Kiori, your suggestions are valid and we would definitely look into handling multi-res in all relevant areas. It is one of our top priorities. The learn team is planning a big expose on these features. Please stay tuned.

    @swipeware, in the very near future release, you can just load your textures variant bundle and then load your prefab bundle and instantiate the prefab from the bundle (the engine will relink the prefab to the variant textures). The scene can remain in the main app.

    Hope this helps...
     
  47. YuriyVotintsev

    YuriyVotintsev

    Joined:
    Jun 11, 2013
    Posts:
    31
    The question was about duplicating of sprite texture. One texture is asset itself, and second texture is sprite atlas. I don't need the asset Texture2D. It would be sufficient to have only sprite atlas in AssetBundle. But currently (according to my tests) there are both textures in AssetBundle. How can I add only sprite atlas to bundle? Or am I wrong with my calculations of bundle size?
     
  48. ColossalPaul

    ColossalPaul

    Unity Technologies

    Joined:
    May 1, 2013
    Posts:
    169
    It is currently not possible to do so, however it is high on our list of improvements. Please stay tuned for more info when this gets in.

    Hope this helps.
     
  49. Mike-Geig

    Mike-Geig

    Unity Technologies

    Joined:
    Aug 16, 2013
    Posts:
    170
    For anyone interested, I scheduled a live session on Asset Bundles for the end of the month. I will cover variant bundles for multi-resolution during that. http://unity3d.com/learn/live-training
     
    rakkarage, scott-havird and Kiori like this.
  50. Kiori

    Kiori

    Joined:
    Jun 25, 2014
    Posts:
    160
    @ColossalPaul

    I tried the example you posted, it seems to work fine so far, i like the way you handled it with a 'bootstrap' scene.
    There are things that can be improved with the system...

    OK first, Currently the bundles don't auto build atlases for images, maybe that should be looked into, we could tag the images we wanted to be packed or whatnot, not sure.
    So right now, basically if you wanna save in draw calls you need sprite atlasses with high res and low res, with same name/position I guess.

    Another annoyance of the system, currently, is not having a multi-platform solution, which i hope will come in the future.

    Other things i noticed that were not addressed in your example: the editor has asset bundle "name" and "tag" right below the myTexture box, that part was completely ignored, i guess its not feature ready?

    Another thing i noticed is both textures are the same size, in a real world scenario they would have 1x, 2x the size, so we would need to change the pixel to units? to adjust and have everything fit?
    You example should have used actual high and low res textures for us to know.

    Also, for whatever reason myTexture low res is marked as sprite mode: multiple, i guess its accidental.

    Thanks either way, this was nice to see.

    @Mike Geig

    As usual you are the hero around here.
    Please be as thorough as you can with the tutorial, looking forward to it.
    Also, perhaps you could mention unity's 'new' atlas creation system, this is a feature that few ppl know about, most ppl use TexturePacker or whatnot. Even though unity has this since 4.5?
    This is quite a useful feature to have built into the engine, it would be good to hear more about it.
    Ideally it should be integrated into everything, generating all sorts of atlases for all sorts of use cases.