2009 OvercomingTheJshapedDistrib

• (Hu et al., 2009) ⇒ Nan Hu, Jie Zhang, Paul A. Pavlou. (2009). “Overcoming the J-shaped Distribution of Product Reviews.” In: Communications of the ACM, 52(10). doi:10.1145/1562764.1562800

Keyword: Product Review, Product Rating.

Quotes

Abstract

• While product review systems that collect and disseminate opinions about products from recent buyers (Table 1) are valuable forms of word-of-mouth communication, evidence suggests that they are overwhelmingly positive. Kadet notes that most products receive almost five stars. Chevalier and Mayzlin also show that book reviews on Amazon and Barnes & Noble are overwhelmingly positive. Is this because all products are simply outstanding? However, a graphical representation of product reviews reveals a J-shaped distribution (Figure 1) with mostly 5-star ratings, some 1-star ratings, and hardly any ratings in between. What explains this J-shaped distribution? If products are indeed outstanding, why do we also see many 1-star ratings? Why aren't there any product ratings in between? Is it because there are no "average" products? Or, is it because there are biases in product review systems? If so, how can we overcome them?
• The J-shaped distribution also creates some fundamental statistical problems. Conventional wisdom assumes that the average of the product ratings is a sufficient proxy of product quality and product sales. Many studies used the average of product ratings to predict sales. However, these studies showed inconsistent results: some found product reviews to influence product sales, while others did not. The average is statistically meaningful only when it is based on a unimodal distribution, or when it is based on a symmetric bimodal distribution. However, since product review systems have an asymmetric bimodal (J-shaped) distribution, the average is a poor proxy of product quality.
• This report aims to first demonstrate the existence of a J-shaped distribution, second to identify the sources of bias that cause the J-shaped distribution, third to propose ways to overcome these biases, and finally to show that overcoming these biases helps product review systems better predict future product sales.
• We tested the distribution of product ratings for three product categories (books, DVDs, videos) with data from Amazon collected between February--July 2005: 78%, 73%, and 72% of the product ratings for books, DVDs, and videos are greater or equal to four stars (Figure 1), confirming our proposition that product reviews are overwhelmingly positive.
• Figure 1 (left graph) shows a J-shaped distribution of all products. This contradicts the law of "large numbers" that would imply a normal distribution. Figure 1 (middle graph) shows the distribution of three randomly-selected products in each category with over 2,000 reviews. The results show that these reviews still have a J-shaped distribution, implying that the J-shaped distribution is not due to a "small number" problem. Figure 1 (right graph) shows that even products with a median average review (around 3-stars) follow the same pattern.

References

1

Admati, A. R. and Pfleiderer, P. Broadcasting opinions with an overconfident sender. International Economic Review 45, 2, (2004), 467--498.

2

Chatterjee, P. Online reviews: Do consumers use them. Advances in Consumer Research 28, 1, (2001), 129--133.

3

Chevalier, J. and Goolsbee, A. Measuring prices and price competition online: Amazon and Barnes and Noble. Quantitative Marketing and Economics 1, 2, (2003), 203--222.

4

Chevalier, J. and Mayzlin, D. The effect of word of mouth on sales: Online book reviews. J. of Marketing Research 43, 3, (2006).

5

Clemons, E. K., Gao, G., and Hitt, L. M. When online reviews meet hyper differentiation: A study of craft beer industry. J. of Management Information Systems 23, 2, (2006), 149--171.

6

Dellarocas, C. The digitization of word-of-mouth: Promise and challenges of online reputation mechanisms. Management Science 49, 10, (2003), 1407--1424.

7

Dellarocas, C., Awad, N., and Zhang, X. Exploring the value of online reviews to organizations: Implications for revenue forecasting and planning. Proceedings of the 24th International Conference on Information Systems, 2004, Washington, D.C.

8

Godes, D. and Mayzlin, D. Using online conversations to study word of mouth communication. Marketing Science 23, 4, (2004), 545--560.

9

Kadet, A. Rah-Rah ratings. SmartMoney Magazine, (Feb. 23, 2007) 116.

10

Li, X. and Hitt, L. M. Self selection and information role of online product reviews. Workshop on Information Systems and Economics, 2004, Washington, DC.

11

Liu, Y. Word-of-mouth for movies: Its dynamics and impact on box office revenue. J. of Marketing 70, 3, (2006) 74--89.

12

Reichheld, F. The one number you need to grow. Harvard Business Review 81, 12, (2003), 46--54.

---

,

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2009 OvercomingTheJshapedDistrib	Nan Hu Jie Zhang Paul A. Pavlou			Overcoming the J-shaped Distribution of Product Reviews		Communications of the ACM		10.1145/1562764.1562800		2009