Innovative Approaches to Human-Computer Interfaces

Research Projects

Results from this project have included both software and hardware. The Anna Xtra may be downloaded directly. You may also request a copy of any other game by contacting either Lori Scarlatos or Tony Scarlatos.

This Xtra, for Macromedia Director, provides a set of functions for tracking visual tags in applications with tangible user interfaces. Four types of tags are supported: orientation, space-filling, binary attribute, and tri-colored bi-directional barcodes. The paper on Tangible Math (see Publications) has code samples. This Xtra currently runs only in the Windows operating system.

We use speech recognition to "listen" to a parent who is reading aloud to his or her hearing-impaired child. The interface shows a video clip of the American Sign Language (ASL) sign, the word being read, and an illustration. This application can be used to teach deaf children to read, or improve their speaking abilities. It can also be used as a tool for learning ASL.

This application encourages students to step on a square respresenting the answer to a mathematics question. A DDR dancepad is used as the input device. A separate interface allows teachers to customize the questions posed by the application.

Early grade school students get practice with their spelling and their handwriting using the tablet interface.

Dichotomous Sorting
Students create sortings of objects provided by the teacher. A separate instructor interface allows teachers to define the objects, their properties, and a set of hints for the students.

Tower of Hanoi
Students learn about recursion and problem solving as they work out this classic puzzle. An overhead camera, watching the movement of the pieces, drives a display that represents the sequence of moves as a path in Sierpinski's triangle.

A custom-made glove equipped with touch and bend sensors is used to recognize all the letters of the alphabet. Spelled words appear on the screen; a word-completion feature makes finger spelling even faster.

Floor Math
Two custom-made floor mats were built. Each floor mat is equipped with an array of binary touch sensors that feed into a keyboard emulator. One floor mat, with a linear arrangement, can be used as a number line. The other has sensors arranged in a 4x6 array with spaces for walking inbetween.

TICLE: Tangible Interfaces for Collaborative Learning Environments
This is the project that started it all. We have built a spcial table - with a camera looking up - for playing with math manipulatives such as the tangram, Cuissenaire rods, and pattern blocks. The computer uses the camera image to determine the state of the puzzle, which helps to determine what feedback should be given to the students.