First test to aquire data for motion capture from arduino to unity.

This time only testing with an accelerometer, but next test will be with a full imu with no drift at all. It'll have accelerometer, gyro and magnetometer.

 

awesome.

 

Very good work man!... keep working

 

and release your source

 

Thats the purpose somehow, but first I need to have source worth releasing

This demo above is very nice but currenly useless for motion capture since it only uses 1 accelerometer and thats absolutely not enough sensors to capture motion.

But... its a good start

 

Looks like you've sorted out your serial comms issue.

 

Not completely yet. There are still a few minor glitches but working on those.

 

I wrote an app a while back that sends ASCII data out the serial port, worked fine under Windows 7. Haven't tried receiving ASCII yet, what issue(s) are you experiencing? Not that I can help fix it, but it would be nice to know in case I need to do this in the future.

 

My program currently is not reading from a serial port. Instead I let my electronics connect over serial to another proxyprogram on the pc which serves as a gateway between serial and networking,so once that program receceived the data I can connect with sockets to it.

Currently I am looking to do this with binary values instead of strings for performance reasons.
And I need to cleanup the first data which I receive after connection is made.

But....it currently looks good and can only improve

 

Yup, looks very good!

 

Did some improvements + now tested with a real model instead of a box.
Now its waiting time for the real sensors to arrive. In the meantime I'm also thinking out possibilities for absolute positioning of the root bone, because IMU sensors can only track very accurately the rotations; but not absolute position.

 

webcam and an led?

 

This looks pretty awesome to me. Yo could place this on a gun and with a projector this could be quite inmmersive

 

redmacmanz : webcam + led would be a way, but its very hard to have 2 webcams looking at same time + 2 is even little.

I'm more looking for a way to track the pelvis root bone with 4 infrared lightsources and 4 or 8 Wiimote camera's atached to some special electronics.

The purpose of this whole thing is realtime motion capture and preferably without needed postprocessing. Therefore the tracking of the rootbone will also have to be done in realtime. I don't see a webcam capable of doing that.

At this moment the bones will be trackable at speeds upto 60Hz (60 updates per second) and is I use Wii camera modules then these are capable of going 100Hz, so a synchronised 60Hz is very possible.


Pulov. I dunno what this would give to guns. I think that there are other techniques for that

 

I meant that you could handle the aiming by simply aiming the gun.

 

I've worked with the phase space motion capture system for my job, it's an infrared beacon tracking motion capture system, but as you state, it requires multiple cameras, no post processing though, just capture, if the data is good.

This is very interesting work, I wonder what you will be able to get out of an IMU.

 

Well I'm pretty sure the IMU is going to give me what I want. Normally I should get around 60Hz results directly computed in euler angles and without any drift, since we'll use a 9 degrees of freedom IMU using Accelerometer, Gyroscope + magnetometer.
It makes the stuff more expensive that way, but I don't want to go supercheap and get bad results.

1 IMU will probably come around 70 to 75 euros and for upper body capture you need 11 of them. You also need flex sensors for the fingers + then you need to pelvis tracking stuff.

Its not damn cheap, but if you look at the price of motion capture devices, it'l still be damn cheap.

 

New test, better formula and now also YAW measurement with very high precision and angle compensation

 

Looks awesome but have you researched into magnetic tracking?

 

impressive

 

Very interested in this for work. IMU's as you said are cheap compared to many other systems... nicely scalable too, especially for what I'm interested in. I can't really see what components you have on that board specifically, but is this still 1 accelerometer?

 

This system uses gyroscope data, accelerometer data + MAGNETOMETER , so yes its doing magnetic tracking too. That way the oriëntation in real world (earth) space is always spot on.

This does however implement 1 limitation and it means you should be in a place where a normal compas is capable of finding north.
If a compas cannot find north because of magnetic interference (for example from a pc within 1 meter), then you can get some yaw error because of that.

However, that yaw error isn't always very visible because in this demo I tested within 30cm of my lcd screen and 50cm of a pc consuming at least 700watts, so it did not interfere too much.



The current model is still wired.



Next step is now to make the sensor wireless, add 5 bend sensors and mount them to a dataglove.
And the step after that is adding more sensors to have upper body motion capture in realtime without postprocessing.
Last step would be full body.

 

what radios are going to use for wireless?

 

Normally it would be Xbee modules.

I prefer not tu use blutooth, because these Xbee/Zigbee modules work very well also and can attain 250kbps speed transfer speed.

I'm now defining a protocol to minimize bandwidth and still be able to have 50Hz sampling.

Thats of course not a problem for 1 IMU, but for full body motion capture, thats 17 IMUs, 10 flexsensors and that at 50 times per second, so you can easily fill a hell lot of bandwidth with that.

 

thats what I was thinking. I've run into trouble sending data from multiple multiplexed sensors and 1 xbee.

 

Actually at 50Hz 17IMUs and 10 flex sensors shouldn't be a problem

1 IMU generates directly a quaternion on the electronics hardware, so I get 4 singles to send over the data connection

1 single = 4 bytes
4 singles = 16 bytes

17 IMUs = 17 x 16 bytes = 272 bytes of data
10 flex sensors should do with 20 bytes (2 per sencor)


So thats 292 bytes for 1 shot. Multiply that with 50 shots per second and you get

292 * 50 = 14600 bytes or 146000 baud (10bits per pyte serial transmission)

xbee has 250000 kbps capability so there is quite some room for protocol implementation and encoding.



The real trick is to synchronize all sensors to the exact same timestamp and then send them over as 1 big block.
The xbee should handle it.

 

So, are you going to store all info in memory, and then read/transmit as one block... OR attempt to sync all sensors and just send together when read?

 

Of course. You don't send each sensor individually, thats a disaster to synchronize while you colledt all sensors data over a bus system by 1 piece of electronics and then fire it as 1 big datapackage to the pc. Advantage: very synchronised and modular.

 

dj.sumdj@gmail.com

This is very similar to what I'm trying to do.

Basically, I want to be able to track 3d position and orientation from a reference point in the real-world.
Just starting to investigate arduino, rtls, etc...

Any help is greatly appreciated... trying to do this on a shoestring if possible.

 

Hey Educasoft, did you ever get this working on a mac, or were you working with windows all along?

 

Windows all along.

 

since you're using sockets, can you advise me on the best way to open comms?

I want to send data with the ethernet shield for arduino...

any tips?, where in ref to look?

 

Ok, Arduino is fantastic, and I use it for reading in special sensors, in real-life / museums context,
but IMO to move a virtual puppet, the best bet would be a data glove. Isn't it ?

 

hey, what you have done is awesome. I have developed a Billiards game and I want to control the cue using the 9dof module. I wanted to know how did you use the accelerometer, gyro and magnetometer data to translate into Unity. Did you use the raw data (in g's) or you converted it into actual m/sec2 measurements before using it in Unity? I have no clue how to use this data to achieve the desired effect. Any help would be extremely appreciated. I am using wireless Shimmer 9dof devices on virtual com ports. Thanks!