Machines Can't Replicate Human Image Recognition, Yet

"Our goal is to seek a better understanding of the fundamental differences between humans and machines and utilize this in developing automated methods for distinguishing humans and robotic programs," said James Z. Wang, associate professor in Penn State's College of Information Sciences and Technology.

Wang, along with Ritendra Datta, a Penn State Ph.D. recipient, and Jia Li, associate professor of statistics at Penn State, explored the difference in human and machine recognition of visual concepts under various image distortions.

The researchers used those differences to design image-based CAPTCHAs (Completely Automated Public Turing Test to Tell Computers and Humans Apart), visual devices used to prevent automated network attacks.

Many e-commerce web sites use CAPTCHAs, which are randomly generated sets of words that a user types in a box provided in order to complete a registration or purchasing process. This is done to verify that the user is human and not a robotic program.

In Wang's study, a demonstration program with an image-based CAPTCHA called IMAGINATION was presented on imagination.alipr.com. Both humans and robotic programs were observed using the CAPTCHA.

Although the scope of the human users was limited, the results, presented in the September issue of IEEE Transactions on Information Forensics and Security, proved that robotic programs were not able to recognize distorted images. In other words, a computer recognition program had to rely on an accurate picture, while humans were able to tell what the picture was even though it was distorted.

Wang said he hopes to work with developers in the future to make IMAGINATION a CAPTCHA program that Web sites can use to strengthen the prevention of automated network attacks.

Even though machine recognizability does not exceed human recognizability at this time, Wang says that there is a possibility that it will in the future.

"We are seeing more intelligently designed computer programs that can harness a large volume of online data, much more than a typical human can experience in a lifetime, for knowledge generation and automatic recognition," said Wang. "If certain obstacles, which many believe to be insurmountable, such as scalability and image representation, can be overcome, it is possible that one day machine recognizability can reach that of humans."