Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Virtual puppetry

Discussion in 'Made With Unity' started by JeDi, Oct 3, 2011.

  1. JeDi

    JeDi

    Joined:
    Jun 16, 2010
    Posts:
    52
    Hi all,

    We are developing a "serious" game for children between 8 and 10, in which they will learn about the world of puppetry. Using the game and its custom-built controller, they will learn how to do basic movements with a marionette, while solving very simple puzzles that teach them about material properties of the puppets. The level itself progresses through the history of a certain type of puppet, from traditional art to modern, more absurd applications of the art form. There are three types of puppets (a wooden puppet, a metal puppet and a puppet made of fabric), but the children can use a puppet editor (separated from the game itself) to customize their puppets (adding accessories to them for example).

    In this Belgian research project (GameHUB), we are developing the first level as a demonstrator for the concept. The level involves the world of "rod puppets", going from the traditional Wayang Golek puppets to the modern puppets of Handpring Puppet Company.

    We are only just producing the actual scenery and puzzles for the game, but I wanted to show our current results anyway, so we can maybe get some critique on the general idea. The innovative about this game is that we are using purely physically based character animation for the player-controller marionette. So no animations on that character whatsoever.

    Normally, the game is played using a custom-built controller that uses the same wooden "cross" that real puppeteers use. In stead of attaching the strings of the wooden cross to a puppet, we use bend and distance sensors and an Arduino board to capture the movement of the controller. We changed the USB stack of the Arduino board to act as a Joystick HID device, so we can use the normal Unity input system to capture the input of the sensors. For demonstration purposes however, I implemented a keyboard controller to control the puppet without the custom controller.

    Here are some screenshots of the current state of the level. Note that we are still tweaking this very much, especially in terms of lighting, but all comments are more than welcome.








    A picture of the controller:


    Here are some links to the playable versions:

    I also created a very simple demo to show all three puppets at once. Walking is disabled, but the keyboard controls are the same as in the other demonstrator.


    The keyboard controls:
    • Left/right arrow keys: Pull left/right leg up
    • Left shift + Left/right arrow keys: Pull left/right arm up
    • Down arrow: Squat
    • Up arrow: jump
    • Up+Down arrows: bend over
    • Escape: Restart level

    You can also drag the puppet and other physical objects around with the left mouse button.

    I feel that I wrote too much text already, so if anyone wants more information about something, just post here. And again, all comments are very welcome. We have a thick skin ;)

    Greetings,
    JeDi
     
    Last edited: Oct 24, 2011
  2. jister

    jister

    Joined:
    Oct 9, 2009
    Posts:
    1,749
    looks awesome ;-)
     
  3. kenshin

    kenshin

    Joined:
    Apr 21, 2010
    Posts:
    940
    amazing!
     
  4. JeDi

    JeDi

    Joined:
    Jun 16, 2010
    Posts:
    52
    Just one note: the first puppet that you can select is kinda broken. It was a prototype puppet that isn't updated in a long time. Today, I almost finished the third puppet, made of fabric. I will update the demo probably tomorrow with the prototype puppet removed, and the fabric puppet added.
     
    Last edited: Oct 5, 2011
  5. JeDi

    JeDi

    Joined:
    Jun 16, 2010
    Posts:
    52
    We added the final puppet, and it has a very funny walk :)

    Here are the new links for the demonstrator:

    This is the concept for the three puppets that are now in the game. Note that we are still working on the accessory system, so the puppets in the game are "nude".



    Greetings,
    JeDi

    P.S. We are still having an annoying problem with substances and version control. If someone has an answer to this thread, we would be very grateful!
     
  6. JeDi

    JeDi

    Joined:
    Jun 16, 2010
    Posts:
    52
    I edited the first post with links to the latest builds, which have a lot of tweaks but still the same part of the level. I removed the prototype puppet because it was broken anyway.

    We host a symposium tomorrow about this research project, and after that we will finish the new part of the level so the demonstrator will become longer. In the new part of the level, we will move from the era of classical Wayang Golek to the style of Richard Teschner (a bit less Asian and a bit more renaissance type of style).

    I find it strange that almost nobody seems to comment on this thread. Has nobody tried this yet, or doesn't it work? We designed this with a modern PC in mind, so we didn't really optimise for performance (yet). Does it run smoothly on your computers? What do you think of the art style? Again, we are happy with any constructive criticism because we are still beginners ourselves.

    Greetings,
    JeDi
     
  7. BooBi

    BooBi

    Joined:
    Jan 18, 2010
    Posts:
    534
    Great job and great idea ! Greetings from a compatriot ;)
     
  8. wccrawford

    wccrawford

    Joined:
    Sep 30, 2011
    Posts:
    2,039
    Nothing has ever made me want to learn to be a puppeteer before. Congrats on at least having and awesome concept! I don't want to play it without the controller, though. :(
     
  9. JeDi

    JeDi

    Joined:
    Jun 16, 2010
    Posts:
    52
    Thanks for the comments. The keyboard controls actually work fairly well. Especially the fabric puppet is a lot of fun to play with. Don't forget to press and hold the down key from time to time, so he falls down on the ground.

    Edit:
    I also created a very simple demo with all three puppets. Walking is disabled, but the keyboard controls are the same as in the other demonstrator.

     
    Last edited: Oct 24, 2011
  10. jister

    jister

    Joined:
    Oct 9, 2009
    Posts:
    1,749
    the project is finished :( but we had a great boost in the end and got two more levels done (the last one is not complete unfortunately).
    don't know if JeDi updated to our "final" build?
    If we find spare time I'm sure we'll also finish level 3 ;-)
    anyway it was a nice project to work on. hope more like that come along in the future.
     
  11. JeDi

    JeDi

    Joined:
    Jun 16, 2010
    Posts:
    52
    Yes, the research project (GameHUB) is finished. We had a blast working on it, and gained a lot of experience doing so.

    I will update the first post with more info as soon as I find the time, but you can find the latest builds here:

    The zip files contain a small readme file with the keyboard controls in it (normally, the game is played using a custom controller based on the controls of a real marionette).

    Mind that the game is a prototype, not a finished project. We used it to do research about completely physically based character animation, and I will surely use that technique (minus the marionette strings) for a game in the future.

    You can find the two other prototypes created during this research project at http://gamehub.be/games.
    • Unizo Limburg is a game insprired by Theme Hospital, where you run an ice cream salon. The game is used to teach (dutch speaking) students how to start and manage a business
    • New Belgica is a game insprired by the Monkey Island series, where you play through a parody of the real journey to the Southpole during a belgian expedition at the end of the 19th century.

    Note that all text in the games had to be in Dutch, so it can be a bit hard to follow for people using other languages ;)

    Of course, any feedback is appreciated!

    Greetings,
    JeDi
     
  12. jister

    jister

    Joined:
    Oct 9, 2009
    Posts:
    1,749
    and some more screen goedies




     
  13. zoxmax1

    zoxmax1

    Joined:
    Dec 18, 2010
    Posts:
    154
    it is very nice and interesting work
     
  14. steviewondrs

    steviewondrs

    Joined:
    Apr 8, 2013
    Posts:
    3
    At the moment I'm working on a similar project which uses my handmovement as input for the puppets. Could you explain how you achieved the 'string-pulling' effect? (Meaning: the way the string attaches to the joint and can move up and down, pulling the limbs up or down)
     
  15. JeroenDierckx

    JeroenDierckx

    Joined:
    Apr 18, 2013
    Posts:
    1
    View attachment $Rope.cs

    In attachment you can find the Rope script we used. I think the code is fairly clean, readable and commented, but in short:

    We use a "maximum distance constraint" using a ConfigurableJoint to simulate simplified behaviour of a non-elastic rope. We don't do actual rope physics: the rope is always straight and objects don't collide with the rope. It was enough for our simulation, and very stable.

    Getting the simulation to behave as we intended involved some tricks including positioning the rope's end at the attachment point, then create the constraint, and then move it back into its original position. There are some comments in the script explaining it a bit.

    With our Rope script, we can attach a rope to any rigid body, optionally at a certain attachment point, and move the upper end of the rope (the transform of the rope object, which contains a kinematic rigid body) using our controls as input.

    If you need more information, just ask! By the way, your project sounds very interesting. Could you elaborate a bit about the techniques you will be using (kinect, physically based animations, Mecanim, ...), the genre of simulation (game, pure simulation, ...)?

    Greetings,
    JeDi
     
    Last edited: Apr 18, 2013
  16. steviewondrs

    steviewondrs

    Joined:
    Apr 8, 2013
    Posts:
    3
    I'm doing an internship in Finland with a friend of mine (We're students from Hasselt by the way ;) ). It's at an software engineering lab at the RAMK university in Rovaniemi called pLAB. We were asked to experiment with a few different technologies which we ultimately have to combine inside Unity.
    We're using Kinect to capture full body movement and then we received this neat little piece of hardware called "Cyber Glove."
    It's basically a glove with sensors, which captures the angles between fingers and joints.

    We were asked to invent something with this applying only one rule: be creative! My friend and I decided to build a little puppeteer game. The current version has got the Cyber Glove implemented, which moves a model of my hand. I almost got a puppet/ragdoll attached to it.
    Not sure yet which genre of game we're going to create yet, as we're still experimenting (and learning how to work with the different tech.)

    Thanks for the Rope script! I'm going to start testing if it suits our project needs. I'll let you know if it works.

    Cheers,
    Steven
     
  17. steviewondrs

    steviewondrs

    Joined:
    Apr 8, 2013
    Posts:
    3
    The script worked like a charm! :D
    Thank you so much!
     
  18. ram_uitda

    ram_uitda

    Joined:
    Apr 25, 2013
    Posts:
    1
    Hello Jeroen,

    Congrats on the project. I am completely new to unity. And I would like to explore your project in detail. Is there anywhere the assets/scene or the project is available for learning??
     
  19. JoeVoxel

    JoeVoxel

    Joined:
    Sep 28, 2012
    Posts:
    127
    This is a great project.
     
  20. priyankaborar

    priyankaborar

    Joined:
    Aug 17, 2013
    Posts:
    1
    Great idea!
    I am working on something similar with puppets and strings. The rope script is very useful. Thanks. :)