Search Unity

First test to aquire data for motion capture from arduino to unity.

Discussion in 'Made With Unity' started by EducaSoft, May 10, 2011.

  1. EducaSoft

    EducaSoft

    Joined:
    Sep 9, 2007
    Posts:
    650
    This time only testing with an accelerometer, but next test will be with a full imu with no drift at all. It'll have accelerometer, gyro and magnetometer.

     
  2. holyjewsus

    holyjewsus

    Joined:
    Mar 7, 2011
    Posts:
    624
    awesome.
     
  3. Kokumo

    Kokumo

    Joined:
    Jul 23, 2010
    Posts:
    416
    Very good work man!... keep working :D
     
  4. holyjewsus

    holyjewsus

    Joined:
    Mar 7, 2011
    Posts:
    624
    and release your source :)
     
  5. EducaSoft

    EducaSoft

    Joined:
    Sep 9, 2007
    Posts:
    650
    Thats the purpose somehow, but first I need to have source worth releasing :)

    This demo above is very nice but currenly useless for motion capture since it only uses 1 accelerometer and thats absolutely not enough sensors to capture motion.

    But... its a good start
     
  6. bigkahuna

    bigkahuna

    Joined:
    Apr 30, 2006
    Posts:
    5,435
    Looks like you've sorted out your serial comms issue. ;)
     
  7. EducaSoft

    EducaSoft

    Joined:
    Sep 9, 2007
    Posts:
    650
    Not completely yet. There are still a few minor glitches but working on those.
     
  8. bigkahuna

    bigkahuna

    Joined:
    Apr 30, 2006
    Posts:
    5,435
    I wrote an app a while back that sends ASCII data out the serial port, worked fine under Windows 7. Haven't tried receiving ASCII yet, what issue(s) are you experiencing? Not that I can help fix it, but it would be nice to know in case I need to do this in the future.
     
  9. EducaSoft

    EducaSoft

    Joined:
    Sep 9, 2007
    Posts:
    650
    My program currently is not reading from a serial port. Instead I let my electronics connect over serial to another proxyprogram on the pc which serves as a gateway between serial and networking,so once that program receceived the data I can connect with sockets to it.

    Currently I am looking to do this with binary values instead of strings for performance reasons.
    And I need to cleanup the first data which I receive after connection is made.

    But....it currently looks good and can only improve
     
  10. bigkahuna

    bigkahuna

    Joined:
    Apr 30, 2006
    Posts:
    5,435
    Yup, looks very good! :)
     
  11. EducaSoft

    EducaSoft

    Joined:
    Sep 9, 2007
    Posts:
    650
    Did some improvements + now tested with a real model instead of a box.
    Now its waiting time for the real sensors to arrive. In the meantime I'm also thinking out possibilities for absolute positioning of the root bone, because IMU sensors can only track very accurately the rotations; but not absolute position.

     
  12. holyjewsus

    holyjewsus

    Joined:
    Mar 7, 2011
    Posts:
    624
    webcam and an led?
     
  13. Pulov

    Pulov

    Joined:
    Feb 20, 2010
    Posts:
    824
    This looks pretty awesome to me. Yo could place this on a gun and with a projector this could be quite inmmersive :p
     
  14. EducaSoft

    EducaSoft

    Joined:
    Sep 9, 2007
    Posts:
    650
    redmacmanz : webcam + led would be a way, but its very hard to have 2 webcams looking at same time + 2 is even little.

    I'm more looking for a way to track the pelvis root bone with 4 infrared lightsources and 4 or 8 Wiimote camera's atached to some special electronics.

    The purpose of this whole thing is realtime motion capture and preferably without needed postprocessing. Therefore the tracking of the rootbone will also have to be done in realtime. I don't see a webcam capable of doing that.

    At this moment the bones will be trackable at speeds upto 60Hz (60 updates per second) and is I use Wii camera modules then these are capable of going 100Hz, so a synchronised 60Hz is very possible.


    Pulov. I dunno what this would give to guns. I think that there are other techniques for that :)
     
  15. Pulov

    Pulov

    Joined:
    Feb 20, 2010
    Posts:
    824
    I meant that you could handle the aiming by simply aiming the gun.
     
  16. holyjewsus

    holyjewsus

    Joined:
    Mar 7, 2011
    Posts:
    624
    I've worked with the phase space motion capture system for my job, it's an infrared beacon tracking motion capture system, but as you state, it requires multiple cameras, no post processing though, just capture, if the data is good.

    This is very interesting work, I wonder what you will be able to get out of an IMU.
     
  17. EducaSoft

    EducaSoft

    Joined:
    Sep 9, 2007
    Posts:
    650
    Well I'm pretty sure the IMU is going to give me what I want. Normally I should get around 60Hz results directly computed in euler angles and without any drift, since we'll use a 9 degrees of freedom IMU using Accelerometer, Gyroscope + magnetometer.
    It makes the stuff more expensive that way, but I don't want to go supercheap and get bad results.

    1 IMU will probably come around 70 to 75 euros and for upper body capture you need 11 of them. You also need flex sensors for the fingers + then you need to pelvis tracking stuff.

    Its not damn cheap, but if you look at the price of motion capture devices, it'l still be damn cheap.
     
  18. EducaSoft

    EducaSoft

    Joined:
    Sep 9, 2007
    Posts:
    650
    New test, better formula and now also YAW measurement with very high precision and angle compensation

     
  19. BlackHawkStudios

    BlackHawkStudios

    Joined:
    May 1, 2011
    Posts:
    107
    Looks awesome but have you researched into magnetic tracking?
     
  20. holyjewsus

    holyjewsus

    Joined:
    Mar 7, 2011
    Posts:
    624
    impressive
     
  21. holyjewsus

    holyjewsus

    Joined:
    Mar 7, 2011
    Posts:
    624
    Very interested in this for work. IMU's as you said are cheap compared to many other systems... nicely scalable too, especially for what I'm interested in. I can't really see what components you have on that board specifically, but is this still 1 accelerometer?
     
  22. EducaSoft

    EducaSoft

    Joined:
    Sep 9, 2007
    Posts:
    650
    This system uses gyroscope data, accelerometer data + MAGNETOMETER , so yes its doing magnetic tracking too. That way the oriëntation in real world (earth) space is always spot on.

    This does however implement 1 limitation and it means you should be in a place where a normal compas is capable of finding north.
    If a compas cannot find north because of magnetic interference (for example from a pc within 1 meter), then you can get some yaw error because of that.

    However, that yaw error isn't always very visible because in this demo I tested within 30cm of my lcd screen and 50cm of a pc consuming at least 700watts, so it did not interfere too much.



    The current model is still wired.



    Next step is now to make the sensor wireless, add 5 bend sensors and mount them to a dataglove.
    And the step after that is adding more sensors to have upper body motion capture in realtime without postprocessing.
    Last step would be full body.
     
  23. holyjewsus

    holyjewsus

    Joined:
    Mar 7, 2011
    Posts:
    624
    what radios are going to use for wireless?
     
  24. EducaSoft

    EducaSoft

    Joined:
    Sep 9, 2007
    Posts:
    650
    Normally it would be Xbee modules.

    I prefer not tu use blutooth, because these Xbee/Zigbee modules work very well also and can attain 250kbps speed transfer speed.

    I'm now defining a protocol to minimize bandwidth and still be able to have 50Hz sampling.

    Thats of course not a problem for 1 IMU, but for full body motion capture, thats 17 IMUs, 10 flexsensors and that at 50 times per second, so you can easily fill a hell lot of bandwidth with that.
     
  25. holyjewsus

    holyjewsus

    Joined:
    Mar 7, 2011
    Posts:
    624
    thats what I was thinking. I've run into trouble sending data from multiple multiplexed sensors and 1 xbee.
     
  26. EducaSoft

    EducaSoft

    Joined:
    Sep 9, 2007
    Posts:
    650
    Actually at 50Hz 17IMUs and 10 flex sensors shouldn't be a problem

    1 IMU generates directly a quaternion on the electronics hardware, so I get 4 singles to send over the data connection

    1 single = 4 bytes
    4 singles = 16 bytes

    17 IMUs = 17 x 16 bytes = 272 bytes of data
    10 flex sensors should do with 20 bytes (2 per sencor)


    So thats 292 bytes for 1 shot. Multiply that with 50 shots per second and you get

    292 * 50 = 14600 bytes or 146000 baud (10bits per pyte serial transmission)

    xbee has 250000 kbps capability so there is quite some room for protocol implementation and encoding.



    The real trick is to synchronize all sensors to the exact same timestamp and then send them over as 1 big block.
    The xbee should handle it.
     
  27. holyjewsus

    holyjewsus

    Joined:
    Mar 7, 2011
    Posts:
    624
    So, are you going to store all info in memory, and then read/transmit as one block... OR attempt to sync all sensors and just send together when read?
     
  28. EducaSoft

    EducaSoft

    Joined:
    Sep 9, 2007
    Posts:
    650
    Of course. You don't send each sensor individually, thats a disaster to synchronize while you colledt all sensors data over a bus system by 1 piece of electronics and then fire it as 1 big datapackage to the pc. Advantage: very synchronised and modular.
     
  29. sumner

    sumner

    Joined:
    Jul 1, 2008
    Posts:
    38
    dj.sumdj@gmail.com

    This is very similar to what I'm trying to do.

    Basically, I want to be able to track 3d position and orientation from a reference point in the real-world.
    Just starting to investigate arduino, rtls, etc...

    Any help is greatly appreciated... trying to do this on a shoestring if possible.
     
  30. holyjewsus

    holyjewsus

    Joined:
    Mar 7, 2011
    Posts:
    624
    Hey Educasoft, did you ever get this working on a mac, or were you working with windows all along?
     
  31. EducaSoft

    EducaSoft

    Joined:
    Sep 9, 2007
    Posts:
    650
    Windows all along.
     
  32. holyjewsus

    holyjewsus

    Joined:
    Mar 7, 2011
    Posts:
    624
    since you're using sockets, can you advise me on the best way to open comms?

    I want to send data with the ethernet shield for arduino...

    any tips?, where in ref to look?
     
  33. LitobyteSoftworks-RD

    LitobyteSoftworks-RD

    Joined:
    Sep 15, 2011
    Posts:
    7
    Ok, Arduino is fantastic, and I use it for reading in special sensors, in real-life / museums context,
    but IMO to move a virtual puppet, the best bet would be a data glove. Isn't it ?
     
  34. fueldown

    fueldown

    Joined:
    Aug 21, 2012
    Posts:
    15
    hey, what you have done is awesome. I have developed a Billiards game and I want to control the cue using the 9dof module. I wanted to know how did you use the accelerometer, gyro and magnetometer data to translate into Unity. Did you use the raw data (in g's) or you converted it into actual m/sec2 measurements before using it in Unity? I have no clue how to use this data to achieve the desired effect. Any help would be extremely appreciated. I am using wireless Shimmer 9dof devices on virtual com ports. Thanks!