[RELEASED] UI Accessibility Plugin - UAP V1.0

EDIT: Wait, having the UAP_BackCloseEventListener in the scene completely stops UAP from working! Is this a known bug?

Original message:
Voice over stopped working for us on iOS. Or at least, the Rotor, required to enable direct touch is not showing up. It seems apple has made some changes that impact UAP :/

Anyone else experiencing this on iOS 14.5++ ?

 

Curious about the behavior of UAP_AccessibilityManager.Say() - my assumption would be that if canBeInterrupted=true, if I send another message to Say() while another one is currently being played, the first message would be interrupted and the next would be read. This doesn't seem to be the case when I test this out - is there any way to allow speech to be interrupted?

 

Hi guys, have you guys ever encounter when you test UAP on unity it's perfectly fine, but when I actually build it, the speech system never worked, however the green boxes and all the other sound effects works fine.
I am sure that I turned enabled UAP in my script, is there anything else that I am missing?

 

Hi @mikrima . I'm using your UAP now and found some bugs via WindowsTTS. If I send a unicode string to read, then I will get a incorrect character reading, because I'm using Chinese. I think this bug also may exist in Japanese or any other languages which require Unicode.
I have download your DLL code from Git and make a little bit changes which will make everything works fine:


DLL_API void __cdecl AddToSpeechQueue(wchar_t* text);
void AddToSpeechQueue(wchar_t* wText)
{
if (wText)
{
theMutex.lock();
theSpeechQueue.push_back(wText);
theMutex.unlock();
}
}


and a little bit changes in WindowsTTS.cs:

public static extern void AddToSpeechQueue([MarshalAs(UnmanagedType.LPWStr)] string s);

 

Hi @mikrima,

Great to see this plugin. One of our clients is using JAWS and we are curious if you are planning JAWS support?

Thanks!

 

I'm using this on OS X, but after I build and run the example the voiceover is not working and I have no idea why?

Answer - I found the reason is because you have to use Mono and not IL2CPP as the scripting backend on a Mac.

 

Hi,

Great job on the plugin! Is there a way to adjust the volume of the text to speech? I'm developing an audio game for windows, and would like to make a slider the user can use to adjust the volume.

Best regards,
Emil

 

I'm not familiar with Windows, but isn't the speech volume based on the Voice Over engine? At least on iOS that is how it works..

 

Hi @NoodleNation,
there's no option to adjust the volume of the speech output in the UAP. The reason for this is simply that the voice output is generated via various TTS options. On iOS for example, it could be using VoiceOver, or the system voice (both of which have their own settings in the OS). On Windows, UAP uses NVDA if it is detected and running, or Window SAPI. All handle their speech generation, including volume separately, usually allowing the user to set these options in the settings of the corresponding app or screen reader. Similarly, UAP allows the devs to even hook their own 3rd party speech generation/TTS engines up, so there's no way to generically support every system.

That said, once upon a time, this feature was actually available - and you might be able to add this functionality to the SAPI TTS back in yourself if you want to stay on Windows exclusively. If you look into the file WindowsTTS.cs, you should see that there actually is a function to set the speech volume, although it is currently commented out. You could try to comment this function back in and test.
Unfortunately, I don't remember if the code in the WindowsTTS.dll that handles the volume still works, since this feature was disabled fairly early on. But it's definitely worth a try.

If it isn't enough to comment the volume function back in inside the C# file, you can modify and compile your own WindowsTTS.dll. The source code for the Window SAPI TTS implementation is included in the github repository, and can be modified. You can find it in the NativePlugins~ folder.
Here's the direct link:
https://github.com/mikrima/UnityAccessibilityPlugin/tree/main/Assets/UAP/NativePlugins~

 

Hi there!
Is using UAP outside the unity editor possible?
Was wanting to create a steam workshop mod to add an accessible UI to an existing game (bepInEx + harmony).

It looks like UAP_AccessibilityManager loads somehow automatically, at least from what I've tried to research.
https://github.com/mikrima/UnityAccessibilityPlugin/search?p=2&q=UAP_AccessibilityManager
Any resource online would be appreciated as well.
Thanks!

 

Figuring out where to place the Resources/Accessibility Manager.prefab
in a mod is the next thing I need to figure out.
due to this line in UAP

Code (CSharp):

1.             GameObject newInstance = Instantiate(Resources.Load("Accessibility Manager")) as GameObject;
2.  

 

Hi @gurgalex ,
that is certainly an interesting use of the UAP!
If you can manage to make a mod that runs the UAP you would be able to make your new (mod) UI accessible, but the game's original UI would still be inaccessible (unless the game allowed you to replace the game's UI with your own prefabs).

Depending on how you have to go about creating a mod for this particular game, you can just change the UAP code to load the accessibility manager prefab from wherever you have it, including from within other prefabs. Do you have any documentation on how mods are created for this game? I could take a quick look and see if there's an obvious best way to do this. No promises I'll find something, but what you're attempting to do is very intriguing!

 

Hi @eric97168,
We had that problem too. You need to turn on NVDA manually.
It is explained in post #126 9 February 2021.

 

Hi @mikrima,

Thank you for continuing to support this plug-in after you made it free and published the source code. I’ve been working with the Royal Society For Blind People in South Australia and they really enjoy being able to read a visual novel.

We have an issue with an audio overlap issue on the Windows PC. The UAP speech continues to describe the return button while the narrator speaks. The blind testers find it hard to hear the narrator.

 

Just here to say: thx for creating and sharing the source of UI Accessibility Plugin (UAP) !
Really well done and needed.

 

Hi all,
Just let you know if you having issues with missing classes in Android you should add these lines in the proguard-user
If its activated...(Hope its will helps someone)

-keep class com.metalpopgames.** {
*;
}

 

I just wanted to say that the audio overlap bug I am experiencing may be my own fault because I deleted second Audio Sources from a number of assets. I will reinstall UAP and report back.

 

I've hit an issue on MacOS Big Sur, Unity 2021.2.7 where it gives the error;

Assets/UAP/Scripts/Core/UAP_AudioQueue.cs(427,5): error CS0103: The name 'MacOSTTS' does not exist in the current context

Its working fine on Windows with the same version of Unity has something changed with Big Sur that would break it?

 

@krs3 It sounds like a file couldn't be found, or the class was renamed. The same error would not show up under Windows, because the code wouldn't be looking for MacOSTTS on that platform (it would be looking for WindowsTTS instead).
Do you know if this is this something that only happens on Big Sur, but doesn't occur on other MacOS versions?
Can you please verify that the file exists? Is it possible it just was deleted by mistake?
The file should be here:
Assets/UAP/Plugins/MacOS/MacOSTTS.cs
(and the file should contain the class MacOSTTS, which is a MonoBehaviour)

 

@kathy12345 Were you able to find out whether the second AudioSource was causing the overlap issues?

 

@erangindi1976 Thank you for that suggestion. I am not experienced with Proguard, so I will have to look into where those lines have to go. It's probably not as simple as adding a proguard-user.txt file into the plugin's Android folder.
In case you're reading this and happen to have any quick doc links handy, they'd be appreciated

 

Thanks @mikrima I'd accidentally overwritten the Mac package with files from the PC one re-importing sorted it, and thanks for developing UAP its been so useful for adding accessibility to ios / android apps

 

Hi, i was wandering if there is a way to use the text to speach functionality even when the plugin is disabled.
Thanks

 

Hi,

I am using this accessibility feature in my game, but when I test it on a physical device, I do not get the highlighted options even when I am using a screen reader. Can you help me out?

 

Hi @mikrima, thanks for asking. The second AudioSource was NOT causing the overlap issues. We still don't know what is, but it drives our testers crazy, because while they are trying to listen to the voice actor, the screen reader is intoning: "Next. Button. Double tap to select."

 

Hi, I'm using this for an interactive game and it's awesome that you provide it.
My Problem is, that the German TTS is barely understandable since it's not reading the Umlaute (ÄÖÜ) or compinations like ck or sch or ch correctly.
Did I miss something, since you say it's supporting german out of the box.
Cheers

(Using it on windows atm and will compile for android also)

 

Hi,

is there a way to get what UI GO is currently selected?
Similarly to es.currentSelectedGameObject (es is instance of EventSystem)? (this retrieves null after implementing UAP)

I really appreciate any help you can provide.

EDIT: instead of es.currentSelectedGameObject use EventSystem.current.currentSelectedGameObject

 

Hi there, we're using your plugin in our mobile app and for the most part works great, thanks! But we do have some longer labels that are getting cut off on iOS. Any chance this is a known issue?

 

Hi, i am trying to use this plugin for implement accessibility for my app, however, by my testing, this plugin would only work when device TalkBack is disabled. If the device system TalkBack was enabled before launch the app, this will only able to read 'GameView' for some reason, am i missing some setting ? or am i did something wrong ?

 

Hello.
After installing UAP and installing the sample scene on the Android device, I turned on Talkback and ran it.
However, it doesn't seem to be recognized at all.
When clicked, only the voice "Game View" is heard.
Only the buttons do not come out with a green line.
It works fine in unity editor.
How do I fix something that doesn't work on my device?
I went through the documentation you provided.
please answer about my question. thank you

 

I appreciate you making this. Unfortunately it doesn't work at all on Unity 2021.3.6f1 on Mac M1 with metal.