Aggregation processes in customer rating systems - Insights from an economic decision experiment

Abstract

The aggregation of rating metrics in reputation systems is crucial for mitigating information overload by condensing customer rating distributions into singular valence scores. While platforms typically employ technical aggregation functions, such as the arithmetic mean to capture product quality, it remains unclear whether these functions align with customers’ innate aggregation patterns. To address this knowledge gap, we designed a controlled economic decision experiment to elicit customers’ aggregation principles by analyzing their product ranking decisions and contrasting these with various reference functions. Our findings indicate that the majority of customers aggregate rating information in accordance with the arithmetic mean. However, a granular analysis at the individual level reveals significant heterogeneity in aggregation behavior, with a substantial cluster exhibiting binary patterns that focus equally on negative (1–2 star) and positive (4–5 star) ratings. Additional clusters concentrate on negative feedback, particularly 1-star ratings or 1–2 star ratings collectively. Notably, these inherent aggregation patterns exhibit stability across variations in numerical information presentation and are largely not significantly influenced by individual characteristics, such as online shopping experience or demographics. The only exception is a plausible relationship between risk attitudes and aggregation functions that focus on extreme ratings, such as 1-star, 1–2 star, and 5-star ratings. Our findings suggest that while the arithmetic mean captures the behavior of the majority of consumers, platforms could benefit from offering customizable aggregation options to better cater to diverse user preferences for processing rating distributions. By doing so, platforms can enhance the effectiveness of their reputation systems and improve the overall quality of decision-making for consumers.