Search Unity

  1. We want you to join us at GDC this year! Take a peek at all of the events we will be hosting during the week of GDC.
    Dismiss Notice
  2. Tell us about your experience here and you’ll get early access to the 2018 Game Studios report + more goodies.
    Dismiss Notice
  3. Unity 2017.3 has arrived! Read about it here.
    Dismiss Notice
  4. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice
  5. We've closed the job boards. If you're looking for work, or looking to hire check out Unity Connect. You can see more information here.
    Dismiss Notice

[RELEASED] UI Accessibility Plugin - UAP V1.0

Discussion in 'Assets and Asset Store' started by Holy-Manfred, May 3, 2017.

  1. Holy-Manfred

    Holy-Manfred

    Joined:
    Nov 30, 2013
    Posts:
    6
    [​IMG]

    Hello everyone!

    We have just released our new UI Accessibility Plugin today (UAP). The plugin allows you to make your Unity game UI accessible to blind or visually impaired players with just a few clicks and make your UI ‘speak’.

    You can find the plugin right here:
    Get it on the Asset Store!

    Unity is a great engine, but out of the box it does not support any of the screen readers on Android, iOS or Windows, which means that making your game accessible isn’t that easy.

    This is where the UAP comes in and does the job for you. Making your UI accessible is super simple and just requires a few clicks.

    In a nutshell, here is what you get with the plugin:
    • Reach visually impaired players!
    • Supports Android and iOS
    • Works like VoiceOver/TalkBack
    • Easy setup with a few clicks!
    • Full C# source code!
    • UGUI & NGUI compatible!
    • Excellent support: Documentation, examples, tutorials!

    To get an idea of the basic workflow, check out the basic video tutorial:

    What type of games does this plugin support?

    The UAP can be used to make any 2D UI accessible. To get an idea of what an accessible game will play like you can check out mobile games such as Crafting Kingdom, which uses the UAP to make the UI accessible.

    Documentation and Support
    The plugin ships with full C# source code, so you do not have to deal with any DLLs and can take a look at all the code yourself.
    The plugin also comes with a fully documented API, examples and tutorials. You can find the documentation right here:

    Documentation

    If you need help with the integration or any other support with the plugin, just head over to our dedicated support forum.

    To the support forum

    Supported Platforms

    UAP 1.0 officially supports iOS and Android. With the initial version released, we want to focus on supporting Windows and Mac next. The released version actually already contains Windows support, but because it’s lacking in a few places we don’t want to officially support it yet.

    UGUI and NGUI

    Both the Unity GUI and NGUI are supported by the plugin. The workflow to make your UI accessible is very similar and you can make most UIs accessible really quickly.

    Roadmap
    We are working on a lot of stuff. You can take a look at the roadmap for the plugin:

    Roadmap

    If you are missing any features or have any other suggestions, just let us know!

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]
     
  2. samizzo

    samizzo

    Joined:
    Sep 7, 2011
    Posts:
    295
    This looks great! Does it support a way to hook in to the text size accessibility settings in iOS (and I assume Android has something similar)? In the iOS settings you can set the default text size, and also even enable "larger text" in accessibility settings to make text even bigger.

    Thanks,
    Sam
     
  3. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    18
    Hi, and sorry for the late reply - somehow my answer never showed up here.
    At the moment, the text size cannot be changed via the plugin. The feature is on the road map, but it is merely a research task right now.

    Changing text size comes with a lot of complications. In Unity, all text on the screen is inside text labels with a fixed width and height. Just changing the font size will not work, since the text might then no longer fit inside the box. It would either be truncated, overflow and obstruct other UI elements, or it might even disappear completely. The only way to increase text size sensibly would be to move the other UI elements to give the text label more space. And that is something that cannot be done automatically.
     
  4. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,163
    Maybe it just needs to be upgraded for the latest Unity (there's an API update warning when importing UAP from the Asset Store into Unity 2017.1.1) but it doesn't look like it's working. If I load UAP into a fresh project and run the Match 3 example with test accessibiility switched on, then navigate with the cursor keys I do hear clicks and see outlines around the widgets but no voices. When running the example on an iPhone (iOS 10.3) with voiceover on there's no outline or sounds.
     
  5. Holy-Manfred

    Holy-Manfred

    Joined:
    Nov 30, 2013
    Posts:
    6
    Thanks for reporting this issue. The update for 2017.1.1 is in the work and should hit the asset store soon. We haven't tested on 2017.1.1.p2 yet though. I'll try can reproduce the issue in the latest version and get to you asap.
     
  6. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,163
    OK, thanks! I know it's hard to keep up. with these Unity updates (even Unity IAP brings up the API update warning!).

    I've been meaning to try calling the text-to-speech function (Say?) directly to see if that works, but the app I want to add that to is still written in Unityscript so there's a long overdue rewrite that needs to happen.
     
  7. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    18
    Hi technicat!
    I just tested the current store version of the UAP and the Match 3 example in 2017.1.1p2. Everything runs fine, including voice output. There must be an issue on your end that we need to figure out. Since you can see the frame and navigate with the keyboard, the plugin seems to turn on and function well in general. So the only problem left is that you have no voice output. As you already mentioned, you hear click sounds, which means the sound is not simply turned off - there is just no speech. My first question is whether you are working on a Window PC. Currently, the plugin doesn't yet support voice output on a Mac, only on Windows. Are you using a Mac by any chance?

    As for the second issue regarding the plugin on iOS: From the way you describe it, it seems to me that the plugin isn't turned on. It's not just that you do not have voice, you mentioned you didn't see an outline frame at all. That is a pretty clear indication that the plugin isn't actively running. This is good news. Most likely that plugin and voice will run fine, there might simply have been an issue detecting that it should turn itself on.
    Note that the plugin will only check whether VoiceOver is running when the app is freshly launched. If it is launched with VoiceOver turned off, then it will stay off, even if you turn it on later. This is because it assumes that you did so intentionally and the plugin doesn't want to interfere with what you're doing.

    So the problem could be that you started the app with VoiceOver off, and that's why the plugin is off. Alternatively it is possible that the latest iOS version introduced something that made the VoiceOver detection break inside our code base. This we will have to investigate.
    We have just updated the code base to work with Unity 2017, so there will no longer be an update warning. We also integrated a few other fixes as well. But we still have to do some testing before release, including on iOS devices with the latest OS. So we will definitely look into that and verify that it's working fine before releasing this update to the asset store.

    For your immediate problem solution, I would recommend giving the three finger triple tap a try, to check whether the plugin and voice output are working fine. Unless you disable this feature in the settings or code, tapping the screen three times with three fingers will turn the plugin on or off, while your app is running. This works on Android and iOS devices. If VoiceOver isn't running, the plugin will use the system voice to output text-to-speech. You should also see the outline around the UI elements and be able to navigate using swipes.

    Lastly, while you can use the Say() function directly, you should know that this function is designed to output accessibility announcements only. This means, if the plugin itself isn't running, or if it is in a paused state, any calls to Say() will be ignored. This is done on purpose, to make it easy to write accessibility announcements into you game code without constantly checking whether accessibility is enabled and not paused. Because this function is ignoring any calls if accessibility isn't really active, you can write you game announcements without any "if" checks and they will only trigger for people actively using accessibility mode.

     
  8. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,163
    I am using a Mac, so that explains that. I'll try again on the device with your suggestions after I update the package and let you know. Thanks for the clarification on Say(). I was hoping to use it independent of accessibility, but it should still be useful (I was going to try it in a typing game).
     
  9. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    18
    Wonderful, that is one mystery solved!
    I also want to say that Voice output on Mac is coming and it's on the official road map (just without an ETA yet).

    If you want to use voice output independent of the plugin, take a look at the function TTS_Speak() inside UAP_AudioQueue.cs. You can probably reuse the code in there to create your own function that you can use independently of the plugin.
     
    technicat likes this.
  10. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,163
    Great, thanks! By the way, I should mention the reason I just started looking into accessibility is that I had a user tell me he's a fan of one of my games but recently lost his vision, so he was asking about Voiceover support and I feel obligated to see what I can do. So it was good to see this product on the Asset Store!
     
  11. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    18
    That's great, that is exactly what the plugin is meant for. And it's wonderful to see more developer taking an interest in accessibility.
     
    technicat likes this.
  12. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,163
    Not sure why, but if I stub hasSavedState to false in the accessibility manager script then it's working as expected.
     
  13. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,163
    Now that I have it running, I have a new problem: I have a uGUI button that brings up a native iOS image chooser via

    https://github.com/thedoritos/unimgpicker

    which works fine with VoiceOver (and the image chooser naturally works with VoiceOver, also), but once I cancel/return from the image chooser, my menu is stuck in a state where any tap or swipe on the screen seems to be invoking the button again, i.e. brings up the image chooser, and I can't navigate to any other buttons on the menu.
     
    mikrima likes this.
  14. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    18
    Hm, I have two different suspects that could be causing this one.

    It could be that the UI element is in an exclusive mode - that is a mode that is entered if you want to modify a complex UI element. A slider for example or a DropDown box. For things like those, double tapping activates a mode where swiping will then modify the slider, or select the next element in the DropDown - and another double tap returns back to normal mode. However, normal buttons don't have this exclusive mode, so this might not be the cause for what you are seeing.

    If it is a button, then I suspect that the plugin doesn't know that the native overlay is active (how could it) and is therefore still trying to react to all input (swipes and taps) - and ends up on a completely different UI element as a result.
    In that case, I suggest calling BlockInput(true) on the UAP_AccessibilityManager when you open the native dialog, and calling it again with a false parameter when it closes.
    (You can call this without checking whether accessibility is running - it will just be ignored if the game is running in sighted mode.)
     
  15. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,163
    Thanks, I tried BlockInput (should have looked at the class doc more closely!) with no luck, but I just realized it actually makes more sense for me to hide that button when in VoiceOver mode so it's not an important issue right now.
     
  16. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    18
    If you have an UI elements that should only appear when accessibility mode is active or inactive, you could use the UAP Game Object Enabler component. This not only makes it easy to show/hide these elements, but it will also react when the accessibility mode is change while the game is running (for example with the three finger gesture).

    upload_2017-10-10_10-20-13.png

    Don't place this component on the button itself (obviously), but on your UI root/Canvas or similar.
     
  17. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,163
    Cool, I will try that! I used a little script that calls UAP_AccessibilityManager.IsEnabled() but this looks better.
     
  18. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    18
    Yes, it's definitely preferable. You won't have to worry about enabling/disabling anything when the plugin is turned on/off, so it's a lot easier and safer to use.
     
    technicat likes this.
  19. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,163
    I started adding the Enabler to my HyperBowl games, turning off the camera and FPS graph menu options when VoiceOver is on. Works like a charm!
     
  20. Koru

    Koru

    Joined:
    Feb 14, 2012
    Posts:
    35
    Hi,
    your plugin sounds interesting, but I have one concern. You mentioned it "Works like VoiceOver/TalkBack" so I understand it does not use native accessibilities? How is your plugin working with talkback turned on?
     
  21. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    18
    Yes, the functionality of VoiceOver or TalkBack is recreated by this plugin.
    This is necessary because of the way Unity works. The native operating system can not access or determine what user interface elements are on screen. This is why Unity apps are by default not accessible - the native accessibility systems and Unity renderings are inherently incompatible.

    TalkBack has to be paused while any Unity game is running, because it blocks the touch input from reaching Unity. This applies to any Unity game and has nothing to with whether this plugin is used. VoiceOver does not have this issue, as all input is automatically forwarded.
     
  22. technicat

    technicat

    Joined:
    Nov 22, 2006
    Posts:
    1,163
    I ran into a compiler error in MacOSTTS.cs when building for the Windows Store, where that platform apparently doesn't recognize System.Diagnostics.Process. I worked around it by wrapping that code with #if !UNITY_WSA
     
  23. mikrima

    mikrima

    Joined:
    Dec 20, 2013
    Posts:
    18
    Thank you very much, that is good to know. I'll add something similar as well for the next update.

    EDIT: I'll wrap it in
    #if (UNITY_STANDALONE_OSX || UNITY_EDITOR_OSX)
     
    Last edited: Feb 12, 2018
    technicat likes this.