Tools for blind

We'd like to create a tool helping blind people to orient themselves in a real 3d world.
For prototyping reasons we consider implementing the same tool using JS and Unity.

 

Is this a question? A suggestion for Unity? A general statement?

 

It's a project they want to make. OP made a post in general discussion asking for partners (without giving any information about what they want to do) and it was removed. I gave them a cue to provide more details and they PM'd me an extended version of this post instead.

In general they don't seem to understand how the forum works.

 

We don't delete that question in General discussion. May be mod. I am not sure.

We'd like to create a tool helping blind people to orient themselves in a real 3d world.
For prototyping reasons we consider implementing the same tool using JS and Unity.

- some real world scene (we have some fbx files)
- Then implementing 3 different ways to raycast the environment eg distance by cylinder coordinates.
- Then turn the result of the 'distance map' into sound by summing up cos/sin

The spectator should be able to move and rotate in all directions eg also go upstairs.
Maybe some additional triggers such as move head up down or left right.

At the beginning the Web target is good enough. Later we also eventually build an app.
But this also depends on sensors etc. threejs can use a 'distance shader'.

If the initial test is fine we need some training like environmenst such as
- road with cars
- .. whatever game like 'learn how to use this tool'

As input sensors of the real world we could use
- depth camera (?)
- intel real sense
- stereo analysis of two cameras
- sound (like bats)
- maybe kind of wifi analysis
However we'd probably start with the easy.

Thus if it happen that you own a modern phone with depth sensor the better.

Of course later we want to add typical features like image recognition etc.
But this you'll already find on the store.

If we offered this as SAS (eg monthly subscription) this small project could
return some revenue.

 

Basically I am "Looking for a cofounder knowing unity, sound generation from wav like stuff, and raytracing" to build prototype for blind people to see with the ears.

 

I am looking for a cofounder of my Unity project.

 

What are you putting on the table, besides the idea? Do you have funds for this project? Do you or someone from your team have experience working with blind people?

 

I think that accessing the camera images directly is faster than UnityOur focus shifted from being complicated to trying to create a prototype for
the real world.

The initial idea was to create a virtualized environment to play with many
things to see what works best which also can be presented on the internet
eg as web app. Unity would have done this.

Now I think it's too much effort because all we need for a prototype is
1) access depth image
2) turn into sound

I can do with Python and RealSense myself.

If it happens that you know Android/Gradle (multiple versions)/ or Iphone dev
well of course we'd prefer having an app.
But one step at a time.

What we can put no the table:
- initial reserach that there is no such product
- probably prototpe (eg written in Python with Intel Real Sense) soon
No money/no investment yet.

If you charged fully blind people 30 EUR/month to create SAS product
it coudl make 20K/month or much more easily.
The open question is how many blind people really are interested in it.
There are some which regularly go abroad and learn other languages.
Some are happy staying at home.

There is an alternative to unity: threejs
It even has a shader which renders distances.

In a later phase what we're looking for is kind of small game with some challenges such as
1) walk around a house
2) find the door
3) exit it find the neighbour house
4) find the stairs, walk up
5) Enter a floor, there are 5 doors. try to enter the room.
Try to experience them. We'll put you in any room and you tell us in which
you are.
6) same as 5) but outside with some trees whatever
7) maybe even some fun stuff like throw a baseball

-> if people turn confident in that the system helps them they are more likely to buy.

Thus because you're unity experte you might judg the effort to get such done using unity.

WHat I know
- Unity can do raytracing to get dsitance to objects
- Unity can render sound from samples
- the demo is an ego shooter, thus comes close to what we actually need.

Thus if it turns out to be a case I am eventually even looking for a tech savy
manager cause in the future many more features need to be integrated such as
image recognition (gogole /amazon API) etc.


Image to sound - what we want to do:
- draw multiple horicontal lines on a depth image
- walk along each line from left to right imagining the distance getting
represented as volume and the distance to left (x position) as pitch

-> there are multiple ways to do this thus we need some experimentation to find
out what might be working best.

depth sensors will only look ahead of you. But maybe having 360 degree view
would be fun - don't know yet.
That's whe we're looking for a virtualized environment

Yes, we already found some .fbx files to represent kind of outdoor landscape.