Vision-based Gestural Interaction Using Plush Toys

We are currently developing gestural interaction techniques to control computing devices remotely using everyday objects such as plush toys, without any special devices attached to the objects. As one example, we developed a prototype system to control appliances in a living room, equipped with a single camera and processor for each appliance, using computer-vision techniques. We chose everyday objects that exist naturally in a living environment. This approach is appealing to end users since they can use their favorite object as a controller.

In our prototype, we decided to use plush toys since their soft body is easy to grasp and safe to shake, though of course it is technically possible to track other everyday objects. Plush toys tend to have vivid colors, making tracking easier; also, as other researchers have pointed out, they are especially pleasing to children.

Additional Resources

UIST 2005 Demo, Paper (PDF), Video (MOV, 22.8MB), Poster/Handout


by Yoshinori Kawasaki (luvtechno.net)