Question Using signals to animate an edge

Hello. I would like for a game to have three vertices. One would be at a longitude and latitude. The other two would form an edge. The edge would move according to signals that begin from, reach, and/or are reflected off of parts on my elbow and wrist (as part of a positioning system). To be more accurate, to make up for the line of sight problem, I’d like for there to be more of those parts (that aren’t on me), and the computer to know the distance from a part on my shoulder to the part on my elbow. I’d like to send the size of an angle, formed by (two rays formed by) those three vertices, to an Arduino. I’m going to continue by reading “Unity game development cookbook : essentials for every game”. Do you have any advice, such as an easier/inexpensive/better way to do it? Thank you.

 

I'm guessing you're talking about XR here. Do you have experience working with the XR framework already?

 

No, I don’t.

Maybe I should have mentioned that I don’t need a screen or a headset. Still, I think it is better to have too much than too little.

 

Okay. Then maybe you could give a bit more info about what exactly you're doing.

 

How do you know where your elbow is

 

It's generally somewhere between your wrist and your shoulder, though it may be offset by a value no greater than the length of your humerus. If these conditions are not met, you should seek medical attention at your nearest convenience.

 

I’m not sure if these are the best ways, but they are the strength of signals, the time of flight of signals, and maybe the angle of arrival of signals. I may be wrong, but I think (in this case) making up for the line of sight problem is better than making up for the problem of drift (which I think I will have with inertial sensors). If the signal can’t get to my elbow with one external reference, (if that is what I need) I might need one more external reference. And, if that doesn’t work, and signals can get to parts on my wrist and shoulder, and the computer knows (ONE) the distance from my elbow to my wrist (and (TWO) the distance from my elbow to my shoulder), I’m trying to figure out if where two spheres (formed by two radii, each of which has a distance equal to one of those two distances) intercept would be one of the best things to use.

An Arduino would use the differences between the sizes of the angles (which the vertices are used to form) to determine how often lights (on a dance costume) flash, and how bright the lights get.

Please let me know if I can provide more information.

Thank you.

 

So, maaaaaybe we shoulid back up a bit here.

Are you using the Unity game engine for this project?

Because I'm starting to think you're doing a project on the Arduino that's wired up to a wearable apparatus of some sort with sensors and potentiometers and such, and you're having trouble writing the code for that.

If that's the case, I'm sorry to tell you you're as lost as a goose in a snowstorm. This forum is specifically for topics related to the Unity game engine. Questions about writing code -- what is it, python? Javascript? -- for the Arduino would be rather off-topic. You'd probably have far better luck getting help from the Arduino forums.

But if I'm totally misunderstanding things and you are here for help with a project made in Unity, you're definitely in the right place! But you're gonna have to give us waaaaaay more information about the technical aspects of your project (share code, screenshots of elements in your scene, error messages, etc) and be much more specific in your requests. Pretend like we're all complete strangers on the Internet who have no idea about anything relating to your project and need to have it fully explained to us.

 

I’m not sure if I can.

Can (information about) three points in three-dimensional space be translated from Arduino into Unity fast enough? Can one of those points be a longitude and latitude? Instead of getting the longitude and latitude from Arduino, (if possible) do I have to put them (the longitude and latitude) in Unity? Can Unity make two of those points into an edge? Then, can Unity use the edge with a point to make an angle? If so, as Unity gets more points from Arduino, can more angles be made? Can the sizes of the angles be translated from Unity into Arduino fast enough? Can Unity do any of that without Arduino, or is there a better way?

 

Well, first of all, Unity is a game engine. It's made for making games. Yeah, you can make things like architectural visualizations and even UI-focused apps, but the point is that it's a collection of tools and software used for managing an asset pipeline, processing user input, managing game state, and rendering 2D and 3D elements to a display. The fact that you're working with 3D math isn't enough to justify (IMHO) using a full-featured game engine if you're not utilizing the rest of it.

I'm actually just really curious as to why you're posting here, on the Unity forums, if you're not already using Unity. Like... are you trying to switch to it? It doesn't sound like it based on your last message. Did you just think there'd be a bunch of people here who knew 3D math well enough to help you figure your project out? Because like I said, you'll be much better off going to an Arduino-specific forum for that.

But just for fun, I'll answer your questions anyway.

Where's that information coming from in the first place? You still haven't really described what you're doing all that well. Is it sensors wired up to your Arduino? If so, why on Earth would you get information from your Arduino, send it out to a device running a Unity player instance, and then send it back? Your Arduino has a processor in it, and your work should probably all be done on that.

Do you literally mean latitude and longitude here? Because if you're trying to use GPS devices on your hands and elbows and stuff to try to get accurate location data, I suspect you're going to be sorely disappointed. I believe most commercial GPS units provide location data with as high as 3 meter accuracy, so you'd have to have some veeeeery long arms to be able to see any kind of useful information from that.

This makes me think you might not actually be using lat/long, because you wouldn't be getting that information from an Arduino. Maybe an accelerometer attached to it or something, but not the microcontroller itself. But once more, if you have that data on the Arduino already, it makes no sense to move it somewhere else to process.

We generally call that a "line," and yes, any 3D engine is going to be able to determine information about a line between two points.

I'm not really sure what you're asking here, because you need two lines to make an angle. If you're referring to the vector between a sequence of points, then yes, Unity is capable of determining that, as is any 3D engine or library.

See previous answer.

Fast enough for what?

The tough thing about this question is I'm still not really clear what you're doing. Something about lights on a dancing suit, and there's an Arduino involved. We don't know how you're getting the point data, what your (app?) is running on, what you have written already, and what specifically you want to do. I've tried multiple times to get this information from you, but you're being very vague with your responses. Maybe it's a secret project for the government (next-gen military light-up dancing suits!) or maybe you're just a bit of a novice with development, but if we don't know what you're doing, it's really difficult to help you do it.

 

Thank you for explaining all that to a novice.

I would like for lights on a dance costume to give hints (such as with how often they flash, and how bright they get) when part of the dance costume (such as part of a sleeve) is pointing more towards an important location, though a lot more could go on. If you want me to be more specific, tell me a location that you think should be more important to people, and I’ll cater by describing a dance. If that is not enough, tell me how many letters/words I can reply with (or visit my website at https://dancing4locations.org/, which isn't for small phones yet). I’m hoping, with that information, someone might be able to help me find a way (to get point data) that is good enough, though I’m not sure if the answer belongs here. The closest thing I found is at

, which uses a lot more than two vertices (which is all I need).

I’m worried that (if I have a part on my elbow, and a part on my wrist) if I twist my arm then I’ll wind up with a shorter/longer line, instead of one with the same length (which is what I want). What's puzzling, to me, is I thought the length of line (segments) can be different, but the length of an edge can't. Though I may be wrong, arms don't grow or shrink that quickly. I was attracted to Unity because I thought Unity might be the best way to keep it the same length. Is it okay to ask what vocabulary people (who use 3D engines) use to keep it the same length? It seems like I would have found the vocabulary by now. Having that vocabulary might make it easier for me to go back to Wikipedia (and the Arduino forum, and the Sparkfun forum), though I don’t know if the answer belongs here. I'd rather find a library than a mathematician.

One (very small) line would be defined by a point near my elbow and a point near my wrist. The other (very long) line would be defined by a point near my elbow and a point near the important location (which I have been calling a latitude and a longitude, I haven't thought about enough, and I didn't think is my biggest problem now). Sorry. When those angles are smaller, the lights are supposed to do what was described above, and more.

I’ve heard about persistence of vision, but I’m not sure yet. I was hoping experts would know.

This might be more like a game if it has different levels of difficulty. For example, though I haven't tested this, the lights could only give hints when the part (of the costume) isn't pointing towards the important location as much, the speed with which the part moves towards the important location could be used (to influence the lights), and (near the end of a dance, to make it harder) what the lights do could become more random. I thought it was a game, and O'Reilly (a publisher) has very few books on gaming (one of which is on Unity), according to my search results.

I'm trying hard to answer your questions, so I appreciate you sticking with this.

 

I'm gonna try to avoid using a lot of extra words, because I feel like that's distracting from answering the important questions.

What does your application run on? Like what kind of hardware?

Where are you getting the data for how your arms and body are positioned? Again, what is the hardware doing this? Like is it being tracked by cameras or something? Or sensors on your body? Be specific with your setup.

What code have you written so far? What's your experience with code in general?

 

I don’t know what would be best to use yet, and if it will be compatible with Arduino, Unity, and/or something else. Is there a way to teach myself?

In a college, I passed two courses in Java, and one in data structures. I haven’t made anything using the books I’ve read outside of school, so I don’t know if I should say what the books were about. For this project, I haven’t written any code yet, and I don’t know what language(s) to learn.

I’m really excited about this project. I feel like there are two things that I’m not sure if I can do, and I hope it is okay that I try to tackle the two first. I hope that is okay. If not, let me know.

First of the two, if I’m using lines (that are made using parts that receive signals at different times) then those lines might not be the same length as each other. (I’ve been using the word “part” in different ways. One is part of a dance costume, and the other is a part that involves signals, involved in a positioning system. The two parts could be attached to each other.) They might not be the same length as each other if the parts slide up and down my arm (because the costume fits too loosely). I want them to be the same length, instead. Are there names for ways of making them the same length, with code?

Second, the other thing I’d like to find a library for is forming angles using the three points/vertices I mentioned (which can be used to form two lines).

Unless there is a better way.

If that is too ambitious for now, I’d like to get more books that will help me get there.

Or, is this type of game not a type of game that is made using Unity?

Thanks.

 

What do people think of my strategy of finding libraries before finding a language that has those libraries, whether or not that language is Unity? It would be different if I thought those types of libraries are popular, thought they might be popular, and I just don’t know where to find them.

 

I'm at a loss for how to help you. I'm still not even sure what it is you're making, how it works (or if nothing actually exists yet, how you THINK it should work), and what your plan is for writing and executing the code.

It doesn't sound like you're rendering to a screen of any sort, which makes me think there's absolutely zero reason you should be considering a game engine. But despite asking you a few times, I feel no closer to an answer. It's like I'm a car salesman trying to get an idea for what kind of car you need, and I say "How many kids do you have?" and you respond with "I'd like the car to have titanium-plated wheel hubs, please, as I've heard they're needed for the best braking."

 

Same question as previously, where do you know where your elbow is. Do you mean your actual elbow? Or do you mean a virtual character. Are you using a depth camera or other on-person tracking device? What signals are you referring to? UDP packets from an Arduino device? "I don't need a screen or a headset". So there is no visualization at all in this project? If not, they you probably don't need Unity which is a very visual game engine.

 

Someone is helping me say what I want. They said I need to identify the location of specific points in space so I can determine their relative positions, and that (some of these) points are always on someone's body. Before I add more, is that good enough?

I'm not sure how visual this is, but the frequency with which a light flashes (and how bright it gets) would react to what a user does. Unity might be too much, but I can’t find anything else that is enough. If it is wrong to ask what the types of libraries I need do at this forum, do people think I’ll find what I’m looking for in the book “Unity game development cookbook : essentials for every game” (by the publisher O’Reilly), which I mentioned in post #1?

 

Do you mean a real person? How are you tracking them, if so. And what flashing light are you referring to, do you mean a real light flashing on an actual person? It sounds like you are referring to augmented reality, can you confirm?

 

Yes.

If people knew the output that I want, would they be able to help me find the best way to track them?

Yes.

I just started learning what augmented reality is, so I can’t answer that question yet. The only output is the light mentioned.

 

"If people knew the output that I want, would they be able to help me find the best way to track them?" I personally use a StereoLabs ZED2 3D depth camera, it does tracking as you suggest https://www.stereolabs.com/zed-2/ There is also Unity integration https://github.com/stereolabs/zed-unity

 

Thank you. It looks like, in that video, there is one point representing the approximate location of each player. And, each point is used to make a curve. However, I would like only two points at a time, representing the location of one player’s elbow and wrist. And, I need the basketball hoop to be at Greenwich, England. Though, most players in the world can’t throw that far, because they don’t play in Greenwich.

 

Yes, that is correct. You can use the StereoLabs SDK to get the values that you need from the point cloud. It also does skeleton recognition, so you can obtain specific body part locations like elbow and wrist. You can place the basketball hoop anywhere in the world using AR

 

Thank you. How can the light connect?

 

You could connect the light to a Raspberry Pi. Then Unity could communicate to the Pi via WiFi with TCP or UDP, for example. Or use ROS, probably best, and what I use. I use a Jetson Nano instead of a Raspberry Pi. They both would work, but Pi is a bit cheaper.

 

Thank you. Does AR mean augmented reality? I just want to make sure.

 

Yes, and is what you seem to be talking about here. Perhaps best if you described the purpose of this project so we don't get off track. You mention a "game" so that implies Unity and a visual representation either in a AR headset or a computer screen. And you also mention tracking an actual person, so may imply AR. Who is "playing the game", the person being tracked, or another person?

 

Please let me know if I should shorten this.

Suppose there is a store. On their receipts, they invite their customers to a dance. A professional would be dancing, not customers. They have a store at a few places. At the end of the dance, a few of the customers will (be randomly selected to) win gift certificates that can be used at only one of the stores. The game would involve trying to figure out which store the gift certificates can be used at. To figure that out, a light on the dance costume would give hints. The light would give hints by flashing more often when part of the dance costume (such as a part covering a forearm) is pointing more towards the place (where the store is) where the gift certificates can be used. For example, if the store sold vacations in different places, the vacations could be in Greenwich, England (thought the audience wouldn’t be able to figure out that it is in Greenwich until near the end of the dance, when the hints become easier).

Also, if the store is going to donate to two nonprofits (each of which does work in one place, which is different from the place where the other one does work), one nonprofit after the other nonprofit, the dance could reveal which one of the two places more work will be done at first (thanks to the store’s donation). This type of dance might make people in the audience want to donate/volunteer (more), even if a store didn’t help plan the dance.

So, a dancer is being tracked. And, there is only one team (which is made up of an audience, customers, and/or potential donors) playing (by trying to figure out which place a dance is about). I could describe more possibilities, if that wouldn’t make this too long.

 

(I'm hoping the reason no one is replying is because post #27 is too long. So, I'm going to shorten it. Please let me know if no one is replying for a different reason, so I can make the necessary improvements.)

Suppose there is a store. On their receipts, they invite their customers to a dance. A professional would be dancing, not customers. They have a store at a few places. At the end of the dance, a few of the customers will (be randomly selected to) win gift certificates that can be used at only one of the stores. The game would involve trying to figure out which store the gift certificates can be used at. To figure that out, a light on the dance costume would give hints. The light would give hints by flashing more often when part of the dance costume (such as a part covering a forearm) is pointing more towards the place (where the store is) where the gift certificates can be used.

So, a dancer is being tracked. And, there is only one team (which is made up of customers) playing (by trying to figure out which place a dance is about).

 

Got it! Hopefully the advice so far has been helpful. You're going to need to purchase some hardware!

 

Have you seen any videos (like the video at https://www.stereolabs.com/zed-2/) that look like they are close to what I am looking for?

 

I already mentioned I'm using the ZED camera myself on a project, please try to read this thread a bit more carefully https://forum.unity.com/threads/using-signals-to-animate-an-edge.1294317/#post-8256321

 

Thank you. I did. Maybe you could tell me a keyword to search your github page with.

 

I don't have a github page for the project. Start with the StereoLab examples and get them working first.