Search Unity

  1. Improved Prefab workflow (includes Nested Prefabs!), 2D isometric Tilemap and more! Get the 2018.3 Beta now.
    Dismiss Notice
  2. The Unity Pro & Visual Studio Professional Bundle gives you the tools you need to develop faster & collaborate more efficiently. Learn more.
    Dismiss Notice
  3. Let us know a bit about your interests, and if you'd like to become more directly involved. Take our survey!
    Dismiss Notice
  4. Improve your Unity skills with a certified instructor in a private, interactive classroom. Watch the overview now.
    Dismiss Notice
  5. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice

[RELEASED] UI Accessibility Plugin - UAP V1.0

Discussion in 'Assets and Asset Store' started by Holy-Manfred, May 3, 2017.

  1. Holy-Manfred

    Holy-Manfred

    Joined:
    Nov 30, 2013
    Posts:
    6

    Hello everyone!

    We have just released our new UI Accessibility Plugin today (UAP). The plugin allows you to make your Unity game UI accessible to blind or visually impaired players with just a few clicks and make your UI ‘speak’.

    You can find the plugin right here:
    Get it on the Asset Store!

    Unity is a great engine, but out of the box it does not support any of the screen readers on Android, iOS or Windows, which means that making your game accessible isn’t that easy.

    This is where the UAP comes in and does the job for you. Making your UI accessible is super simple and just requires a few clicks.

    In a nutshell, here is what you get with the plugin:
    • Reach visually impaired players!
    • Supports Android and iOS
    • Works like VoiceOver/TalkBack
    • Easy setup with a few clicks!
    • Full C# source code!
    • UGUI & NGUI compatible!
    • Excellent support: Documentation, examples, tutorials!

    To get an idea of the basic workflow, check out the basic video tutorial:

    What type of games does this plugin support?

    The UAP can be used to make any 2D UI accessible. To get an idea of what an accessible game will play like you can check out mobile games such as Crafting Kingdom, which uses the UAP to make the UI accessible.

    Documentation and Support
    The plugin ships with full C# source code, so you do not have to deal with any DLLs and can take a look at all the code yourself.
    The plugin also comes with a fully documented API, examples and tutorials. You can find the documentation right here:

    Documentation

    If you need help with the integration or any other support with the plugin, just head over to our dedicated support forum.

    To the support forum

    Supported Platforms

    UAP 1.0 officially supports iOS and Android. With the initial version released, we want to focus on supporting Windows and Mac next. The released version actually already contains Windows support, but because it’s lacking in a few places we don’t want to officially support it yet.

    UGUI and NGUI

    Both the Unity GUI and NGUI are supported by the plugin. The workflow to make your UI accessible is very similar and you can make most UIs accessible really quickly.

    Roadmap
    We are working on a lot of stuff. You can take a look at the roadmap for the plugin:

    Roadmap

    If you are missing any features or have any other suggestions, just let us know!







     
  2. samizzo

    samizzo

    Joined:
    Sep 7, 2011
    Posts:
    441
    This looks great! Does it support a way to hook in to the text size accessibility settings in iOS (and I assume Android has something similar)? In the iOS settings you can set the default text size, and also even enable "larger text" in accessibility settings to make text even bigger.

    Thanks,
    Sam
     
  3. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    Hi, and sorry for the late reply - somehow my answer never showed up here.
    At the moment, the text size cannot be changed via the plugin. The feature is on the road map, but it is merely a research task right now.

    Changing text size comes with a lot of complications. In Unity, all text on the screen is inside text labels with a fixed width and height. Just changing the font size will not work, since the text might then no longer fit inside the box. It would either be truncated, overflow and obstruct other UI elements, or it might even disappear completely. The only way to increase text size sensibly would be to move the other UI elements to give the text label more space. And that is something that cannot be done automatically.
     
  4. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,232
    Maybe it just needs to be upgraded for the latest Unity (there's an API update warning when importing UAP from the Asset Store into Unity 2017.1.1) but it doesn't look like it's working. If I load UAP into a fresh project and run the Match 3 example with test accessibiility switched on, then navigate with the cursor keys I do hear clicks and see outlines around the widgets but no voices. When running the example on an iPhone (iOS 10.3) with voiceover on there's no outline or sounds.
     
  5. Holy-Manfred

    Holy-Manfred

    Joined:
    Nov 30, 2013
    Posts:
    6
    Thanks for reporting this issue. The update for 2017.1.1 is in the work and should hit the asset store soon. We haven't tested on 2017.1.1.p2 yet though. I'll try can reproduce the issue in the latest version and get to you asap.
     
  6. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,232
    OK, thanks! I know it's hard to keep up. with these Unity updates (even Unity IAP brings up the API update warning!).

    I've been meaning to try calling the text-to-speech function (Say?) directly to see if that works, but the app I want to add that to is still written in Unityscript so there's a long overdue rewrite that needs to happen.
     
  7. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    Hi technicat!
    I just tested the current store version of the UAP and the Match 3 example in 2017.1.1p2. Everything runs fine, including voice output. There must be an issue on your end that we need to figure out. Since you can see the frame and navigate with the keyboard, the plugin seems to turn on and function well in general. So the only problem left is that you have no voice output. As you already mentioned, you hear click sounds, which means the sound is not simply turned off - there is just no speech. My first question is whether you are working on a Window PC. Currently, the plugin doesn't yet support voice output on a Mac, only on Windows. Are you using a Mac by any chance?

    As for the second issue regarding the plugin on iOS: From the way you describe it, it seems to me that the plugin isn't turned on. It's not just that you do not have voice, you mentioned you didn't see an outline frame at all. That is a pretty clear indication that the plugin isn't actively running. This is good news. Most likely that plugin and voice will run fine, there might simply have been an issue detecting that it should turn itself on.
    Note that the plugin will only check whether VoiceOver is running when the app is freshly launched. If it is launched with VoiceOver turned off, then it will stay off, even if you turn it on later. This is because it assumes that you did so intentionally and the plugin doesn't want to interfere with what you're doing.

    So the problem could be that you started the app with VoiceOver off, and that's why the plugin is off. Alternatively it is possible that the latest iOS version introduced something that made the VoiceOver detection break inside our code base. This we will have to investigate.
    We have just updated the code base to work with Unity 2017, so there will no longer be an update warning. We also integrated a few other fixes as well. But we still have to do some testing before release, including on iOS devices with the latest OS. So we will definitely look into that and verify that it's working fine before releasing this update to the asset store.

    For your immediate problem solution, I would recommend giving the three finger triple tap a try, to check whether the plugin and voice output are working fine. Unless you disable this feature in the settings or code, tapping the screen three times with three fingers will turn the plugin on or off, while your app is running. This works on Android and iOS devices. If VoiceOver isn't running, the plugin will use the system voice to output text-to-speech. You should also see the outline around the UI elements and be able to navigate using swipes.

    Lastly, while you can use the Say() function directly, you should know that this function is designed to output accessibility announcements only. This means, if the plugin itself isn't running, or if it is in a paused state, any calls to Say() will be ignored. This is done on purpose, to make it easy to write accessibility announcements into you game code without constantly checking whether accessibility is enabled and not paused. Because this function is ignoring any calls if accessibility isn't really active, you can write you game announcements without any "if" checks and they will only trigger for people actively using accessibility mode.

     
  8. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,232
    I am using a Mac, so that explains that. I'll try again on the device with your suggestions after I update the package and let you know. Thanks for the clarification on Say(). I was hoping to use it independent of accessibility, but it should still be useful (I was going to try it in a typing game).
     
  9. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    Wonderful, that is one mystery solved!
    I also want to say that Voice output on Mac is coming and it's on the official road map (just without an ETA yet).

    If you want to use voice output independent of the plugin, take a look at the function TTS_Speak() inside UAP_AudioQueue.cs. You can probably reuse the code in there to create your own function that you can use independently of the plugin.
     
    technicat likes this.
  10. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,232
    Great, thanks! By the way, I should mention the reason I just started looking into accessibility is that I had a user tell me he's a fan of one of my games but recently lost his vision, so he was asking about Voiceover support and I feel obligated to see what I can do. So it was good to see this product on the Asset Store!
     
  11. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    That's great, that is exactly what the plugin is meant for. And it's wonderful to see more developer taking an interest in accessibility.
     
    technicat likes this.
  12. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,232
    Not sure why, but if I stub hasSavedState to false in the accessibility manager script then it's working as expected.
     
  13. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,232
    Now that I have it running, I have a new problem: I have a uGUI button that brings up a native iOS image chooser via

    https://github.com/thedoritos/unimgpicker

    which works fine with VoiceOver (and the image chooser naturally works with VoiceOver, also), but once I cancel/return from the image chooser, my menu is stuck in a state where any tap or swipe on the screen seems to be invoking the button again, i.e. brings up the image chooser, and I can't navigate to any other buttons on the menu.
     
    mikrima likes this.
  14. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    Hm, I have two different suspects that could be causing this one.

    It could be that the UI element is in an exclusive mode - that is a mode that is entered if you want to modify a complex UI element. A slider for example or a DropDown box. For things like those, double tapping activates a mode where swiping will then modify the slider, or select the next element in the DropDown - and another double tap returns back to normal mode. However, normal buttons don't have this exclusive mode, so this might not be the cause for what you are seeing.

    If it is a button, then I suspect that the plugin doesn't know that the native overlay is active (how could it) and is therefore still trying to react to all input (swipes and taps) - and ends up on a completely different UI element as a result.
    In that case, I suggest calling BlockInput(true) on the UAP_AccessibilityManager when you open the native dialog, and calling it again with a false parameter when it closes.
    (You can call this without checking whether accessibility is running - it will just be ignored if the game is running in sighted mode.)
     
  15. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,232
    Thanks, I tried BlockInput (should have looked at the class doc more closely!) with no luck, but I just realized it actually makes more sense for me to hide that button when in VoiceOver mode so it's not an important issue right now.
     
  16. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    If you have an UI elements that should only appear when accessibility mode is active or inactive, you could use the UAP Game Object Enabler component. This not only makes it easy to show/hide these elements, but it will also react when the accessibility mode is change while the game is running (for example with the three finger gesture).

    upload_2017-10-10_10-20-13.png

    Don't place this component on the button itself (obviously), but on your UI root/Canvas or similar.
     
  17. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,232
    Cool, I will try that! I used a little script that calls UAP_AccessibilityManager.IsEnabled() but this looks better.
     
  18. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    Yes, it's definitely preferable. You won't have to worry about enabling/disabling anything when the plugin is turned on/off, so it's a lot easier and safer to use.
     
    technicat likes this.
  19. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,232
    I started adding the Enabler to my HyperBowl games, turning off the camera and FPS graph menu options when VoiceOver is on. Works like a charm!
     
  20. Koru

    Koru

    Joined:
    Feb 14, 2012
    Posts:
    35
    Hi,
    your plugin sounds interesting, but I have one concern. You mentioned it "Works like VoiceOver/TalkBack" so I understand it does not use native accessibilities? How is your plugin working with talkback turned on?
     
  21. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    Yes, the functionality of VoiceOver or TalkBack is recreated by this plugin.
    This is necessary because of the way Unity works. The native operating system can not access or determine what user interface elements are on screen. This is why Unity apps are by default not accessible - the native accessibility systems and Unity renderings are inherently incompatible.

    TalkBack has to be paused while any Unity game is running, because it blocks the touch input from reaching Unity. This applies to any Unity game and has nothing to with whether this plugin is used. VoiceOver does not have this issue, as all input is automatically forwarded.
     
  22. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,232
    I ran into a compiler error in MacOSTTS.cs when building for the Windows Store, where that platform apparently doesn't recognize System.Diagnostics.Process. I worked around it by wrapping that code with #if !UNITY_WSA
     
  23. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    Thank you very much, that is good to know. I'll add something similar as well for the next update.

    EDIT: I'll wrap it in
    #if (UNITY_STANDALONE_OSX || UNITY_EDITOR_OSX)
     
    Last edited: Feb 12, 2018
    technicat likes this.
  24. visionman50

    visionman50

    Joined:
    Dec 19, 2017
    Posts:
    18
    hi. well will this plugin, work with unity and help with screen readers. did do a tutorial a roll a ball game, and did the coding in visual studio. as the script editor, maybe was not very accessible with jaws, and also did not read the tree view. so if i get my lecturer to purchase this for me, pity not free, will it give me keyboard short cuts and fix accessibility, and then how to set up the accessibility with keyboard short cuts and have a more accessible unity editor for the blind. i know the developers are working on this, but not sure when. thanks.
     
  25. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    Hi visionman50,

    unfortunately this plugin won't make the Unity Editor itself accessible. It works only on games that are built with it.
    I've been working on a plugin that will make the Editor accessible as well, but it isn't complete yet, and it only supports NVDA, not JAWS.
     
    technicat likes this.
  26. atitoHW

    atitoHW

    Joined:
    Nov 12, 2017
    Posts:
    3
    Hi,

    Are you looking to support TextMesh Pro as well as Unity Text?

    Cheers,
    Anna T.
     
  27. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    Hi Anna,

    yes, we are. It's a plugin we're using inhouse as well. We're currently looking into everything that would be needed for full support. I'm sorry I don't have more information for you yet, such as for example a release date!
     
    atitoHW and technicat like this.
  28. plasticwaterstudio

    plasticwaterstudio

    Joined:
    Nov 4, 2015
    Posts:
    3
    Hello,
    Is there a way to use UAP plugin for menu visualization without using inbuilt navigation???
    I have defined custom navigation in my app. Can I use UAP for visualization by passing the object or name of the object???
     
    Last edited: Aug 2, 2018
  29. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    Yes, you can.
    To turn off the automatic navigation, disable the checkbox "Handle UI" in the plugin's settings.

    To select a UI element, call this function from your own code and pass the GameObject along that you want to select:
    UAP_AccessibilityManager.SelectElement(GameObject obj);
     

    Attached Files:

  30. plasticwaterstudio

    plasticwaterstudio

    Joined:
    Nov 4, 2015
    Posts:
    3
    Thanks a lot. The main functionality is working fine. But how do I get the same functionalities like toggle (toggle on and off), Input field(on end edit), or slider to work???
     
  31. plasticwaterstudio

    plasticwaterstudio

    Joined:
    Nov 4, 2015
    Posts:
    3
    Is there any specific method to follow for building the app for ios ?? I tested the app on the editor it works fine... I changed the testing and debug value back to app controlled and built for iOS but neither the audio nor the focus for the UI elements are working.
     
  32. Skjalg

    Skjalg

    Joined:
    May 25, 2009
    Posts:
    98
    Hi there, I recently bought your asset to help visually impaired children to use a norwegian app that we've made.

    First off, I would like to express my gratitude for making such a great asset for sucn a niche market.

    That being said there is a few things I would like to see improved.

    1. When selecting an item that is named in Norwegian the hints are being told in a weird Norwegian-english because the hints are hardcoded english in UAP_AccessibilityManager. You should make is possible for me to change these hints in the prefab (or specify a localization key to use) - and not having to edit the code because that will get overriden when you create an update.
    2. Its a bad coding practice to name all your classes with a prefix like UAP_. This is what namespaces are for. So instead of
      Code (CSharp):
      1. public class UAP_SelectionGroup : MonoBehaviour
      you should instead do

      Code (CSharp):
      1. namespace UAP
      2. {
      3.    /// <summary>
      4.    /// This helper class sends a message to the GameObject when any accessibility items beneath it receive focus.
      5.    /// </summary>
      6.    [AddComponentMenu("Accessibility/Helper/Group Focus Notification")]
      7.    public class SelectionGroup : MonoBehaviour
    3. It would be very nice if you included a Assembly Definition File in your asset for those of us who use Unity 2018 or above.
    4. "Is localization Key" is visible in the inspector even if you don't have NGUI installed. Best case here would be the option to Localize easily with a custom Localization package. (I'm using Polyglot - https://github.com/agens-no/PolyglotUnity/)
     
    technicat likes this.
  33. MuhammadArslanRiaz

    MuhammadArslanRiaz

    Joined:
    Jun 22, 2018
    Posts:
    3
    Hi, kindly can you guide me, how to integrate it with webgl( including windows, android and iPhone web browsers). i read the document. it was mention that right now an external text-to-speech plugin is required and it varies from browser to browser, so is there a guide which tell how to integrate third part plugin with UAP.

    Thank you,
     
  34. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    Hi plasticwaterstudio!
    If you change the debug mode back to default (app controlled) then the plugin will NOT turn on on the device UNLESS it detects VoiceOver is running, or it is turned on via your game's menu. This is so that sighted users aren't surprised by the game talking at them, when they could just use the menus directly.
     
  35. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    Hi MuhammadArslanRiaz,
    I am just about to release an update which has a easy-to-use interface class for integrating with an external TTS plugin. The update is changing quite a few things on how the TTS is handled internally, so if at all possible, I would strongly suggest you to wait until the update is live. Not only will it contain an example and documentation on exactly what you're trying to do - but it will also prevent any TTS glue code you write now to become invalid or having to be adjusted.
     
  36. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    Hi Skjalg,
    sorry for the late reply! Since late August, my notification emails for this thread have disappeared.
    1. I'm still working on properly extracting the hint texts inside the UAP into a proper, separate table, so that it can be localized. This isn't done though, and currently, I'm sorry to say, you would unfortunately have to go through the manager class and replace all strings manually.
    2. That is very true, and I have put using namespaces on my TODO list.
    3. I just read up on them and that they can reduce compilation time - which is great. I've also added this to my road map.
    4. This is intentional. While "Is Localization Key" out-of-the-box currently only supports NGUI, all localization is handled through one single function. This means it's very easy to add a different localization backend into it. That's why the property is exposed.
    I heard of Polyglot before, and it seems to be somewhat popular. I think it might be a good candidate to offer a direct (optional) integration with, so it would be easy for anybody to use that (instead of NGUI localization). Thanks for pointing me to that!

    UPDATE 1: v1.0.5 now includes an assembly definition file
    UPDATE 2: There were some localization bugs in the previous release, which didn't use the correct generic localization function. These are now also fixed in v1.0.5
     
    Last edited: Oct 8, 2018
    technicat likes this.
  37. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    Hi again MuhammadArslanRiaz,
    the latest plugin update v1.0.5 is now live in the Unity Asset Store and includes the new and improved TTS bridge for third party plugins.
    Here is the direct link to the part of the documentation that explains how to integrate custom TTS into UAP:
    http://www.metalpopgames.com/assetstore/accessibility/doc/HowToGuides.html#ExchangeTheTTS
    Let me know if you need any help!

     
    technicat likes this.
  38. MuhammadArslanRiaz

    MuhammadArslanRiaz

    Joined:
    Jun 22, 2018
    Posts:
    3
    Thank you so much, i checked the code and documentation, "Unity Editor" was used which is not recognizable while creating build. kindly do something about it and as for webgl accessibility I really appreciate your work and the flexibility you provided to add third party tool in the plugin.

    And another thing, webgl build is not working on Iphone devices except iphone X. Build crashes during loading process. kindly look into it. thank you.
     

    Attached Files:

    Last edited: Oct 10, 2018
  39. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    Hi MuhammadArslanRiaz,

    thank you for the update. The scripts using UnityEditor are inside an Editor folder, which is supposed to be ignored/not included by Unity when creating builds. But it seems that at least Unity 2018 has an issue that when an assembly definition file is also present, it will include these files regardless.

    I am going to make an update today with a fix. In the meantime, you can delete the "UnityAccessibilityPlugin.asmdef" file from the UAP folder, and the build will work again.


     
    Last edited: Oct 10, 2018
    technicat likes this.
  40. MuhammadArslanRiaz

    MuhammadArslanRiaz

    Joined:
    Jun 22, 2018
    Posts:
    3
    Hi, thank you for guidance related to editor issue. Kindly do something about IOs related issue.
    Issue:
    • WebGl build is crashing on ios web browsers like chrome and safari.
    • How webgl build will detect that user's accessibilty is on /off, or we have to force-enable it for every user despite he/she is sighted or not.

    Thanks in advance
     
    Last edited: Oct 18, 2018 at 8:13 AM
  41. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    27
    Unity doesn't officially support mobile browsers with WebGL at the moment. It sometimes works and sometimes not, and it depends on whether you have a high end device or not. They described it as Hit and Miss.
    Here's the officical documentation page: https://docs.unity3d.com/Manual/webgl-browsercompatibility.html
    I've seen a forum post where Unity Staff members recommend using WASM export instead of asm.js, and also stripping down the build to make it as small as possible, to help improve compatibility and loading issues.

    Currently the plugin cannot auto-detect whether a user might want to use the accessibility features in a WebGL or not (since there is simply no way to tell from within a browser). It is up to the app to make a decision. Every app might want to do that differently - for example by adding an extra parameter to the URL. You could then distribute different links to the game, with or without accessibility on. Or by adding a link to the HTML file which embeds the player (regular HTML can be read with screen readers) - which sends a message to the Unity player to enable accessibility. Or maybe by simply adding a key combination in the game itself, which can be explained in the documentation or another HTML page. It depends highly on the app and how it is being used, embedded or distributed.
    I am however planning to investigate if there's something I could ship with the plugin itself to handle this in the future. It would probably be nice to offer developers who don't have any special needs for their app a standard way of enabling/disabling accessibility in WebGL.
     
    technicat likes this.