Crowd review: Using the wisdom of the crowd to improve design

Now, research published in the International Journal of Knowledge and Web Intelligence demonstrates how consumer product reviews published on the internet could be analyzed through data mining techniques and allow designers to find ways to improve a given product or even add features that had not occurred to the manufacturer.

Ismail Art Yagci and Sanchoy Das of the Department of Mechanical and Industrial Engineering, at New Jersey Institute of Technology, in Newark, USA, explain how web reviews are a readily available source of product intelligence and suggest that such reviews might contain significant pointers regarding the pros and cons of a design or features that are redundant, unwanted or missing. In light of this, the team introduces a design-feature-opinion-cause relationship (DFOC) method that can extract design intelligence from unstructured web reviews.

Product development managers are constantly challenged to learn about the consumer's experience with their product and to uncover the specifics of how well or how badly a product is performing in the hands of those consumers. Conventional market research, which might involve prototype testing, field tests and independent assessment are costly, time consuming and limited in the amount of data they can return. The advent of the internet and in particular the idea of interactive web 2.0 sites, online social media and networking and a boom in our all-round need to share our opinions has opened up an entirely new approach to such testing based around what one might called "crowd review."

The researchers explain how their DFOC method first builds a sentence-based web review database and then mines that database to identify design features that are of interest to both designers and users. Next it extracts and estimates the significance and polarity of customer opinion and identifies the underlying factors that may have given rise to a particular opinion. The DFOC system used an association rule-based opinion mining procedure for capturing and extracting noun-adjective and noun-verb relationships.

Opinion mining is not entirely new, but the team has demonstrated a strong proof of principle with their DFOC method on car design on characteristics such as engine power, fuel economy, safety, comfort, exhaust, sound system, interior design etc. The DFOC approach identified 42 features of interest many of which are not commonly specified in car design and yet are of interest or concern to drivers. "This implies that a DFOC type analysis expands the design feature analysis set, and hence makes the intelligence is even more valuable," the team says. They have also demonstrated how it can overcome several of the problems inherent to earlier techniques.

They point out that their DFOC method could enhance creative efficiency by identifying negative opinions in the early stages of the design process, and so help channel effort to areas where designers can have the greatest impact on consumer opinion of future products. The approach could also improve competitiveness by shortening the lead-time for the introduction of new products or services, lowering design costs, and improving quality and reliability of products and services.