Where can I buy a chair like that? This app will tell you

Given a photo of a chair, lamp or some other item, a new service will tell you who makes it and where to buy it, and show you pictures of how it might look in various rooms.

"It seems a lot of people want to buy things they see in someone else's home or in a photo, but they don't know where to look," said Sean Bell, a doctoral candidate in computer science. Bell and Kavita Bala, professor of computer science, describe their method for "learning visual similarity for product design" in a paper presented at the 2015 SIGGRAPH conference and published in ACM Transactions on Graphics.

The system relies on "deep learning," a neural network that enables a computer to match a submitted photo with a vast database of "iconic images" from manufacturers' catalogs or specialized websites devoted to home furnishings.

A neural network is a computer program inspired by the working of neurons in the human brain. As data is passed through the network, locations in memory that are activated repeatedly are increased in value, just as a biological brain forms synapses. "Deep learning" combines several layers of neurons that represent different aspects of the data -- earlier layers typically represent edges and lines, middle layers represent parts and shapes, and later layers represent entire objects and concepts.

The researchers used crowdsourcing to prepare a collection of images to train the neural network. On the Amazon Mechanical Turk service, where home workers can be paid a few cents at a time to perform simple microtasks, they showed workers scene photos and asked them to draw boxes around objects. The resulting collection of boxes, along with their matching iconic images, was used to train the computer.

People using the service won't want to wait for results. Rather than force the computer to go through the entire database looking for a match, the system begins by using the neural network to generate a "fingerprint" of a submitted image, based on very broad characteristics of how the pixels are arranged. Then the computer can search just a local area of the database, analogous to searching for a phone number in just one area code.

Bala and Bell have formed a startup company, GrokStyle, to offer the service on a subscription basis to retailers and design professionals, with support from the Small Business Innovation Research Program through which the National Science Foundation and other government agencies encourage the commercialization of new technology to advance the economy.

"I'm excited by the importance of this for the design industry," said Bala, who considers herself a hobbyist-level interior designer.

In the future, the researchers said, similar systems might be developed for other kinds of products, such as clothing and fashion.