Augmented reality system for learning chess

An ordinary webcam, a chess board, a set of 32 pieces and custom software are the key elements in the final degree project of the telecommunications engineering students Ivan Paquico and Cristina Palmero, from the UPC-Barcelona Tech's Terrassa School of Engineering (EET). The project, for which the students were awarded a distinction, was directed by the professor Jordi Voltas and completed during an international mobility placement in Finland.

The system created by Ivan Paquico, the 2001 Spanish Internet chess champion, and Cristina Palmero, a keen player and federation member, is a didactic tool that will help chess clubs and associations to teach the game and make it more appealing, particularly to younger players.

The system combines augmented reality, computer vision and artificial intelligence, and the only equipment required is a high-definition home webcam, the Augmented Reality Chess software, a standard board and pieces, and a set of cardboard markers the same size as the squares on the board, each marked with the first letter of the corresponding piece: R for the king (rei in Catalan), D for the queen (dama), T for the rooks (torres), A for the bishops (alfils), C for the knights (cavalls) and P for the pawns (peons).

Learning chess with virtual pieces

To use the system, learners play with an ordinary chess board but move the cardboard markers instead of standard pieces. The table is lit from above and the webcam focuses on the board, and every time the player moves one of the markers the system recognises the piece and reproduces the move in 3D on the computer screen, creating a virtual representation of the game.

For example, if the learner moves the marker P (pawn), the corresponding piece will be displayed on the screen in 3D, with all of the possible moves indicated. This is a simple and attractive way of showing novices the permitted movements of each piece, making the system particularly suitable for children learning the basics of this board game.

Making chess accessible to all

The learning tool also incorporates a move-tracking program called Chess Recognition: from the images captured by the webcam, the system instantly recognises and analyses every movement of every piece and can act as a referee, identify illegal moves and provide the players with an audible description of the game status. According to Ivan Paquico and Cristina Palmero, this feature could be very useful for players with visual impairment -- who have their own federation and, until now, have had to play with specially adapted boards and pieces -- and for clubs and federations, tournament organisers and enthusiasts of all levels.

The Chess Recognition program saves whole games so that they can be shared, broadcast online and viewed on demand, and can generate a complete user history for analysing the evolution of a player's game. The program also creates an automatic copy of the scoresheet (the official record of each game) for players to view or print.

The technology for playing chess and recording games online has been available for a number of years, but until now players needed sophisticated equipment including pieces with integrated chips and a special electronic board with a USB connection. The standard retail cost of this equipment is between 400 and 500 euros.