Computer Vision and The Towers of Hanoi:
A Tangible Math Learning Interface

Faculty Advisor:
Tony Scarlatos
Lecturer, Computer Science
SUNY at Stony Brook

Brian Scammon
Adam De Nyse
Kurt Simbron

Traditional educational multimedia applications have used video, animation, audio and non-linear interaction to enhance learning. But while the use of multimedia content has been shown to expedite knowledge acquisition and increase retention, the traditional user interface of keyboard and mouse has two deficits when it comes to how children learn, according to pedagogical experts and cognitive psychologists. Traditional multimedia applications are typically single-user, and only provide a simulation of problem solving. Yet children learn best collaboratively and by tangibly solving problems, according to experts.

In this project we designed a computer-mediated learning task based on the traditional mathematics puzzle, the Towers of Hanoi. Targeted at middle and high school students, the application will help them to learn advanced concepts, like recursion and graph coloring, as they physically try to solve the puzzle in real time, working in groups of two or three.

The Prototype:
Our approach involves the use of computer vision to track the location of puzzle pieces on a physical game board. Based on the state of board the application computes the optimal solution, and can provide audio/visual clues to the student, while explaining the principle behind the solution.

The student interface consists of a graphic display of the current state of the puzzle, and a graphic of Sierpinski's Triangle which represents all of the possible moves in the game. As the pieces are moved both the board state and the triangle are updated. Students can look at the triangle to see where in the sequence of moves they are toward solving the puzzle.

The ToH application is written in Macromedia Director MX’s scripting language, Lingo. An extension to Director, TrackThemColors, enables Director MX to process DV camera input in real time.

The prototype runs on either Mac OSX or Windows XP platforms.

ToH was presented at CTE at Stony Brook in January 2005.

This work is supported by an NSF CRCD grant (#0203333).