Assessing outliers and their influence on consumer data

Challenge

Ensuring data quality underpins the reliability of insights that guide product development and strategic decisions. Aberrant or inconsistent data can distort findings, especially in online studies and home use tests, where disengaged respondents or even AI bots may influence results.

Approach

A study was conducted in France with n=103 consumers evaluating sliced roast pork products. A range of questions were posed including product liking, organoleptic diagnosis, purchase intent, and quality checks.

To identify irregular responses, various detection strategies were applied—such as outlier detection, reverse scaling, response time analysis, pattern recognition, logical statements, and open-ended queries.

Thresholds were defined to flag potential issues, for example: respondents spending less than one-third of the median completion time or using the same response for over 80% of Likert scale questions.

A penalty score system was used to classify respondents based on these indicators.

Outcome

The analysis revealed that the choice of thresholds significantly affects the identification of aberrant data. However, in this specific study, the presence of outliers had minimal impact on the overall conclusions, particularly regarding liking questions and perceived product defects.

Increased engagement through interactive and gamification elements can enhance user experience and reduce the occurrence of aberrant data, leading to more reliable insights and better-informed decisions.

Conclusion - To ensure robust and relevant insights, it’s vital to detect and manage outliers effectively. Combining smart data screening techniques with engaging research formats helps maintain data integrity and supports the development of more successful products.

If you are interested to see the presentation we delivered at Pangborn 2025, please get in touch.