[RELEASED] UI Accessibility Plugin - UAP V1.0

​

Hello everyone!

We have just released our new UI Accessibility Plugin today (UAP). The plugin allows you to make your Unity game UI accessible to blind or visually impaired players with just a few clicks and make your UI ‘speak’.

You can find the plugin right here:
Get it on the Asset Store!

Unity is a great engine, but out of the box it does not support any of the screen readers on Android, iOS or Windows, which means that making your game accessible isn’t that easy.

This is where the UAP comes in and does the job for you. Making your UI accessible is super simple and just requires a few clicks.

In a nutshell, here is what you get with the plugin:

• Reach visually impaired players!
• Supports Android and iOS
• Works like VoiceOver/TalkBack
• Easy setup with a few clicks!
• Full C# source code!
• UGUI & NGUI compatible!
• Excellent support: Documentation, examples, tutorials!

To get an idea of the basic workflow, check out the basic video tutorial:

What type of games does this plugin support?

The UAP can be used to make any 2D UI accessible. To get an idea of what an accessible game will play like you can check out mobile games such as Crafting Kingdom, which uses the UAP to make the UI accessible.

Documentation and Support
The plugin ships with full C# source code, so you do not have to deal with any DLLs and can take a look at all the code yourself.
The plugin also comes with a fully documented API, examples and tutorials. You can find the documentation right here:

Documentation

If you need help with the integration or any other support with the plugin, just head over to our dedicated support forum.

To the support forum

Supported Platforms

UAP 1.0 officially supports iOS and Android. With the initial version released, we want to focus on supporting Windows and Mac next. The released version actually already contains Windows support, but because it’s lacking in a few places we don’t want to officially support it yet.

UGUI and NGUI

Both the Unity GUI and NGUI are supported by the plugin. The workflow to make your UI accessible is very similar and you can make most UIs accessible really quickly.

Roadmap
We are working on a lot of stuff. You can take a look at the roadmap for the plugin:

Roadmap

If you are missing any features or have any other suggestions, just let us know!

 

This looks great! Does it support a way to hook in to the text size accessibility settings in iOS (and I assume Android has something similar)? In the iOS settings you can set the default text size, and also even enable "larger text" in accessibility settings to make text even bigger.

Thanks,
Sam

 

Hi, and sorry for the late reply - somehow my answer never showed up here.
At the moment, the text size cannot be changed via the plugin. The feature is on the road map, but it is merely a research task right now.

Changing text size comes with a lot of complications. In Unity, all text on the screen is inside text labels with a fixed width and height. Just changing the font size will not work, since the text might then no longer fit inside the box. It would either be truncated, overflow and obstruct other UI elements, or it might even disappear completely. The only way to increase text size sensibly would be to move the other UI elements to give the text label more space. And that is something that cannot be done automatically.

 

Maybe it just needs to be upgraded for the latest Unity (there's an API update warning when importing UAP from the Asset Store into Unity 2017.1.1) but it doesn't look like it's working. If I load UAP into a fresh project and run the Match 3 example with test accessibiility switched on, then navigate with the cursor keys I do hear clicks and see outlines around the widgets but no voices. When running the example on an iPhone (iOS 10.3) with voiceover on there's no outline or sounds.

 

Thanks for reporting this issue. The update for 2017.1.1 is in the work and should hit the asset store soon. We haven't tested on 2017.1.1.p2 yet though. I'll try can reproduce the issue in the latest version and get to you asap.

 

OK, thanks! I know it's hard to keep up. with these Unity updates (even Unity IAP brings up the API update warning!).

I've been meaning to try calling the text-to-speech function (Say?) directly to see if that works, but the app I want to add that to is still written in Unityscript so there's a long overdue rewrite that needs to happen.

 

Hi technicat!
I just tested the current store version of the UAP and the Match 3 example in 2017.1.1p2. Everything runs fine, including voice output. There must be an issue on your end that we need to figure out. Since you can see the frame and navigate with the keyboard, the plugin seems to turn on and function well in general. So the only problem left is that you have no voice output. As you already mentioned, you hear click sounds, which means the sound is not simply turned off - there is just no speech. My first question is whether you are working on a Window PC. Currently, the plugin doesn't yet support voice output on a Mac, only on Windows. Are you using a Mac by any chance?

As for the second issue regarding the plugin on iOS: From the way you describe it, it seems to me that the plugin isn't turned on. It's not just that you do not have voice, you mentioned you didn't see an outline frame at all. That is a pretty clear indication that the plugin isn't actively running. This is good news. Most likely that plugin and voice will run fine, there might simply have been an issue detecting that it should turn itself on.
Note that the plugin will only check whether VoiceOver is running when the app is freshly launched. If it is launched with VoiceOver turned off, then it will stay off, even if you turn it on later. This is because it assumes that you did so intentionally and the plugin doesn't want to interfere with what you're doing.

So the problem could be that you started the app with VoiceOver off, and that's why the plugin is off. Alternatively it is possible that the latest iOS version introduced something that made the VoiceOver detection break inside our code base. This we will have to investigate.
We have just updated the code base to work with Unity 2017, so there will no longer be an update warning. We also integrated a few other fixes as well. But we still have to do some testing before release, including on iOS devices with the latest OS. So we will definitely look into that and verify that it's working fine before releasing this update to the asset store.

For your immediate problem solution, I would recommend giving the three finger triple tap a try, to check whether the plugin and voice output are working fine. Unless you disable this feature in the settings or code, tapping the screen three times with three fingers will turn the plugin on or off, while your app is running. This works on Android and iOS devices. If VoiceOver isn't running, the plugin will use the system voice to output text-to-speech. You should also see the outline around the UI elements and be able to navigate using swipes.

Lastly, while you can use the Say() function directly, you should know that this function is designed to output accessibility announcements only. This means, if the plugin itself isn't running, or if it is in a paused state, any calls to Say() will be ignored. This is done on purpose, to make it easy to write accessibility announcements into you game code without constantly checking whether accessibility is enabled and not paused. Because this function is ignoring any calls if accessibility isn't really active, you can write you game announcements without any "if" checks and they will only trigger for people actively using accessibility mode.

 

I am using a Mac, so that explains that. I'll try again on the device with your suggestions after I update the package and let you know. Thanks for the clarification on Say(). I was hoping to use it independent of accessibility, but it should still be useful (I was going to try it in a typing game).

 

Wonderful, that is one mystery solved!
I also want to say that Voice output on Mac is coming and it's on the official road map (just without an ETA yet).

If you want to use voice output independent of the plugin, take a look at the function TTS_Speak() inside UAP_AudioQueue.cs. You can probably reuse the code in there to create your own function that you can use independently of the plugin.

 

Great, thanks! By the way, I should mention the reason I just started looking into accessibility is that I had a user tell me he's a fan of one of my games but recently lost his vision, so he was asking about Voiceover support and I feel obligated to see what I can do. So it was good to see this product on the Asset Store!

 

That's great, that is exactly what the plugin is meant for. And it's wonderful to see more developer taking an interest in accessibility.

 

Not sure why, but if I stub hasSavedState to false in the accessibility manager script then it's working as expected.

 

Now that I have it running, I have a new problem: I have a uGUI button that brings up a native iOS image chooser via

https://github.com/thedoritos/unimgpicker

which works fine with VoiceOver (and the image chooser naturally works with VoiceOver, also), but once I cancel/return from the image chooser, my menu is stuck in a state where any tap or swipe on the screen seems to be invoking the button again, i.e. brings up the image chooser, and I can't navigate to any other buttons on the menu.

 

Hm, I have two different suspects that could be causing this one.

It could be that the UI element is in an exclusive mode - that is a mode that is entered if you want to modify a complex UI element. A slider for example or a DropDown box. For things like those, double tapping activates a mode where swiping will then modify the slider, or select the next element in the DropDown - and another double tap returns back to normal mode. However, normal buttons don't have this exclusive mode, so this might not be the cause for what you are seeing.

If it is a button, then I suspect that the plugin doesn't know that the native overlay is active (how could it) and is therefore still trying to react to all input (swipes and taps) - and ends up on a completely different UI element as a result.
In that case, I suggest calling BlockInput(true) on the UAP_AccessibilityManager when you open the native dialog, and calling it again with a false parameter when it closes.
(You can call this without checking whether accessibility is running - it will just be ignored if the game is running in sighted mode.)

 

Thanks, I tried BlockInput (should have looked at the class doc more closely!) with no luck, but I just realized it actually makes more sense for me to hide that button when in VoiceOver mode so it's not an important issue right now.

 

If you have an UI elements that should only appear when accessibility mode is active or inactive, you could use the UAP Game Object Enabler component. This not only makes it easy to show/hide these elements, but it will also react when the accessibility mode is change while the game is running (for example with the three finger gesture).



Don't place this component on the button itself (obviously), but on your UI root/Canvas or similar.

 

Cool, I will try that! I used a little script that calls UAP_AccessibilityManager.IsEnabled() but this looks better.

 

Yes, it's definitely preferable. You won't have to worry about enabling/disabling anything when the plugin is turned on/off, so it's a lot easier and safer to use.

 

I started adding the Enabler to my HyperBowl games, turning off the camera and FPS graph menu options when VoiceOver is on. Works like a charm!

 

Hi,
your plugin sounds interesting, but I have one concern. You mentioned it "Works like VoiceOver/TalkBack" so I understand it does not use native accessibilities? How is your plugin working with talkback turned on?

 

Yes, the functionality of VoiceOver or TalkBack is recreated by this plugin.
This is necessary because of the way Unity works. The native operating system can not access or determine what user interface elements are on screen. This is why Unity apps are by default not accessible - the native accessibility systems and Unity renderings are inherently incompatible.

TalkBack has to be paused while any Unity game is running, because it blocks the touch input from reaching Unity. This applies to any Unity game and has nothing to with whether this plugin is used. VoiceOver does not have this issue, as all input is automatically forwarded.

 

I ran into a compiler error in MacOSTTS.cs when building for the Windows Store, where that platform apparently doesn't recognize System.Diagnostics.Process. I worked around it by wrapping that code with #if !UNITY_WSA

 

Thank you very much, that is good to know. I'll add something similar as well for the next update.

EDIT: I'll wrap it in
#if (UNITY_STANDALONE_OSX || UNITY_EDITOR_OSX)

 

hi. well will this plugin, work with unity and help with screen readers. did do a tutorial a roll a ball game, and did the coding in visual studio. as the script editor, maybe was not very accessible with jaws, and also did not read the tree view. so if i get my lecturer to purchase this for me, pity not free, will it give me keyboard short cuts and fix accessibility, and then how to set up the accessibility with keyboard short cuts and have a more accessible unity editor for the blind. i know the developers are working on this, but not sure when. thanks.

 

Hi visionman50,

unfortunately this plugin won't make the Unity Editor itself accessible. It works only on games that are built with it.
I've been working on a plugin that will make the Editor accessible as well, but it isn't complete yet, and it only supports NVDA, not JAWS.

 

Hi,

Are you looking to support TextMesh Pro as well as Unity Text?

Cheers,
Anna T.

 

Hi Anna,

yes, we are. It's a plugin we're using inhouse as well. We're currently looking into everything that would be needed for full support. I'm sorry I don't have more information for you yet, such as for example a release date!

 

Hello,
Is there a way to use UAP plugin for menu visualization without using inbuilt navigation???
I have defined custom navigation in my app. Can I use UAP for visualization by passing the object or name of the object???

 

Yes, you can.
To turn off the automatic navigation, disable the checkbox "Handle UI" in the plugin's settings.

To select a UI element, call this function from your own code and pass the GameObject along that you want to select:
UAP_AccessibilityManager.SelectElement(GameObject obj);

 

Thanks a lot. The main functionality is working fine. But how do I get the same functionalities like toggle (toggle on and off), Input field(on end edit), or slider to work???

 

Is there any specific method to follow for building the app for ios ?? I tested the app on the editor it works fine... I changed the testing and debug value back to app controlled and built for iOS but neither the audio nor the focus for the UI elements are working.

 

Hi there, I recently bought your asset to help visually impaired children to use a norwegian app that we've made.

First off, I would like to express my gratitude for making such a great asset for sucn a niche market.

That being said there is a few things I would like to see improved.

1. When selecting an item that is named in Norwegian the hints are being told in a weird Norwegian-english because the hints are hardcoded english in UAP_AccessibilityManager. You should make is possible for me to change these hints in the prefab (or specify a localization key to use) - and not having to edit the code because that will get overriden when you create an update.
2. Its a bad coding practice to name all your classes with a prefix like UAP_. This is what namespaces are for. So instead of
   
   Code (CSharp):

   1. public class UAP_SelectionGroup : MonoBehaviour
   you should instead do
   
   
   Code (CSharp):

   1. namespace UAP
   2. {
   3.    /// <summary>
   4.    /// This helper class sends a message to the GameObject when any accessibility items beneath it receive focus.
   5.    /// </summary>
   6.    [AddComponentMenu("Accessibility/Helper/Group Focus Notification")]
   7.    public class SelectionGroup : MonoBehaviour
3. It would be very nice if you included a Assembly Definition File in your asset for those of us who use Unity 2018 or above.
4. "Is localization Key" is visible in the inspector even if you don't have NGUI installed. Best case here would be the option to Localize easily with a custom Localization package. (I'm using Polyglot - https://github.com/agens-no/PolyglotUnity/)

 

Hi, kindly can you guide me, how to integrate it with webgl( including windows, android and iPhone web browsers). i read the document. it was mention that right now an external text-to-speech plugin is required and it varies from browser to browser, so is there a guide which tell how to integrate third part plugin with UAP.

Thank you,

 

Hi plasticwaterstudio!
If you change the debug mode back to default (app controlled) then the plugin will NOT turn on on the device UNLESS it detects VoiceOver is running, or it is turned on via your game's menu. This is so that sighted users aren't surprised by the game talking at them, when they could just use the menus directly.

 

Hi MuhammadArslanRiaz,
I am just about to release an update which has a easy-to-use interface class for integrating with an external TTS plugin. The update is changing quite a few things on how the TTS is handled internally, so if at all possible, I would strongly suggest you to wait until the update is live. Not only will it contain an example and documentation on exactly what you're trying to do - but it will also prevent any TTS glue code you write now to become invalid or having to be adjusted.

 

Hi Skjalg,
sorry for the late reply! Since late August, my notification emails for this thread have disappeared.
1. I'm still working on properly extracting the hint texts inside the UAP into a proper, separate table, so that it can be localized. This isn't done though, and currently, I'm sorry to say, you would unfortunately have to go through the manager class and replace all strings manually.
2. That is very true, and I have put using namespaces on my TODO list.
3. I just read up on them and that they can reduce compilation time - which is great. I've also added this to my road map.
4. This is intentional. While "Is Localization Key" out-of-the-box currently only supports NGUI, all localization is handled through one single function. This means it's very easy to add a different localization backend into it. That's why the property is exposed.
I heard of Polyglot before, and it seems to be somewhat popular. I think it might be a good candidate to offer a direct (optional) integration with, so it would be easy for anybody to use that (instead of NGUI localization). Thanks for pointing me to that!

UPDATE 1: v1.0.5 now includes an assembly definition file
UPDATE 2: There were some localization bugs in the previous release, which didn't use the correct generic localization function. These are now also fixed in v1.0.5

 

Hi again MuhammadArslanRiaz,
the latest plugin update v1.0.5 is now live in the Unity Asset Store and includes the new and improved TTS bridge for third party plugins.
Here is the direct link to the part of the documentation that explains how to integrate custom TTS into UAP:
http://www.metalpopgames.com/assetstore/accessibility/doc/HowToGuides.html#ExchangeTheTTS
Let me know if you need any help!

 

Thank you so much, i checked the code and documentation, "Unity Editor" was used which is not recognizable while creating build. kindly do something about it and as for webgl accessibility I really appreciate your work and the flexibility you provided to add third party tool in the plugin.

And another thing, webgl build is not working on Iphone devices except iphone X. Build crashes during loading process. kindly look into it. thank you.

 

Hi MuhammadArslanRiaz,

thank you for the update. The scripts using UnityEditor are inside an Editor folder, which is supposed to be ignored/not included by Unity when creating builds. But it seems that at least Unity 2018 has an issue that when an assembly definition file is also present, it will include these files regardless.

I am going to make an update today with a fix. In the meantime, you can delete the "UnityAccessibilityPlugin.asmdef" file from the UAP folder, and the build will work again.

 

Hi, thank you for guidance related to editor issue. Kindly do something about IOs related issue.
Issue:

• WebGl build is crashing on ios web browsers like chrome and safari.
  
• How webgl build will detect that user's accessibilty is on /off, or we have to force-enable it for every user despite he/she is sighted or not.

Thanks in advance

 

Unity doesn't officially support mobile browsers with WebGL at the moment. It sometimes works and sometimes not, and it depends on whether you have a high end device or not. They described it as Hit and Miss.
Here's the officical documentation page: https://docs.unity3d.com/Manual/webgl-browsercompatibility.html
I've seen a forum post where Unity Staff members recommend using WASM export instead of asm.js, and also stripping down the build to make it as small as possible, to help improve compatibility and loading issues.

Currently the plugin cannot auto-detect whether a user might want to use the accessibility features in a WebGL or not (since there is simply no way to tell from within a browser). It is up to the app to make a decision. Every app might want to do that differently - for example by adding an extra parameter to the URL. You could then distribute different links to the game, with or without accessibility on. Or by adding a link to the HTML file which embeds the player (regular HTML can be read with screen readers) - which sends a message to the Unity player to enable accessibility. Or maybe by simply adding a key combination in the game itself, which can be explained in the documentation or another HTML page. It depends highly on the app and how it is being used, embedded or distributed.
I am however planning to investigate if there's something I could ship with the plugin itself to handle this in the future. It would probably be nice to offer developers who don't have any special needs for their app a standard way of enabling/disabling accessibility in WebGL.

 

Thank you for the update. i also checked this and got the game answer about about accessibility through webgl. Just want to check on your browser.

 

Finding on Windows that once I run the app once it locks into the voice default at that time only. If I change the voice later in the windows default or even NVDA I am not seeing that update in the app when I run it again. This appears to only happen on Windows. Did I choose something in the setup that would cause this? I am surprised its not updating to the default text-to-speech voice when I change it. Thanks in advance.

 

That's an interesting observation. NVDA and Windows SAPI are handled differently from the plugin - in Windows it just uses the first voice it finds - I have voice selection for Windows SAPI on my road-map for a future update however. I'll have to check how easy it would be to use the voice that is set to default instead of using the first one. It might not be a big change at all.

For NVDA, the plugin doesn't actually select the voice at all - it just tells NVDA to speak. NVDA however stores app specific voices and speech settings. I remember changing this for specific apps like Chrome and Visual Studio, because in different apps I wanted different voices and different level of pronunciation (in Visual Studio for example, I set it to speak out all punctuation as well). I don't remember the steps to set that up from the top of my head though - I think it might have been as easy as changing the voice while the app was in the foreground, but I am not sure.

 

Hi mikrima, thanks for taking the time to reply to my lengthy post.

If you need any help with integrating your system with Polyglot, just let me know.

 

I was able to use a workaround in the project with NVDA. If you can take a look.

 

Hallo, I'm managing to purchase your plugin to integrate it in an Android application. My question is: does this plugin speak using device's main language or does it works only in english? Best regards.

 

Hi sb2,
for all game related texts the plugin uses the system's set voice language. This means if your buttons, label etc are labeled in a different language that matches the device's language, they will be read out correctly.

However, there are a few internal instructional texts which are not currently localized. These texts include the UI element type and instructions on how to use it.

Example:
After the text of a button is read out aloud, the plugin will wait for a short moment, then announce "Button, double tap to select".

Similar texts exist for sliders, dropdown lists etc. These texts currently only exist in English.
I am working on supporting localization for these texts, and provide both support for developers to add in additional languages, as well as localized texts for the most common languages out of the box. I do not have a definite release date for this update yet, though it will be in either the next update or the one after that.

EDIT: We have released the game Crafting Kingdom despite this limitation, and from the feedback we've gotten, it seems that most blind users do not need the instructional texts all that much. The plugin will soon be supporting localization regardless, but I just thought I'd share that bit of feedback we got with you.

 

Hallo again and thanks for your reply.

According to your customers feedbacks, those instructional texts seem to be unecessary. Could I disable them through your plugin?

I read that I can edit instructional texts source strings: "I'm still working on properly extracting the hint texts inside the UAP into a proper, separate table, so that it can be localized. This isn't done though, and currently, I'm sorry to say, you would unfortunately have to go through the manager class and replace all strings manually."
Is that 'manager class' scalable so I can add one or more sets of strings? (i.e.: I'd like to keep the existing set of english strings and add spanish and italian sets).

Kind regards,
Stefano

 

Hi Stefano,
it's even easier:
In the latest release of the plugin (v1.0.6) supports localization. All strings are properly extracted into a separate, editable table. It only contains English and German out of the box at this time, but adding more languages is as easy as adding a new column into the table. The plugin will parse and load all available translations automatically. It will also try to pick the language that the device is set to, if possible. You can also set which language the plugin should use via a simple code call, of course.

I'll post a direct link to the new page in the documentation which explains how to add languages and deal with localization in general: http://www.metalpopgames.com/assetstore/accessibility/doc/HowToGuides.html#Localization

As for your question to disable the instructional texts - these are called 'hints' inside the plugin. There's no switch to disable them, but you can create one very easily. The UAP_AudioQueue, which handles all the accessibility announcements, defines an Audio Type called Element_Hint. Whenever it receives a call to the function ::SayAudio() with this audio type you could just ignore it. You'll might want to consider creating a setting / bool variable to switch this behavior, so that players can choose themselves whether they want the hints to be read out.

The same has also been requested for element types (which reads out 'slider' or 'button' after an element). This one is not something I'd disable from the get go, since new players need it. But once players know your game, they no longer need that little bit of extra information.