Do Crowds Validate False Data? Systematic Distortion and Affective Polarization

This research note examines how socio-cognitive influences can systematically distort crowdsourced ground truth in event-centric data through subgroups. The "wisdom of the crowd" is based on the assumption that consensus drives accuracy. While existing research addresses tendencies of the overall crowd, this research note shows identifiable subgroups within the crowd can systematically influence crowdsource validation. We conducted an immersive experiment to investigate whether crowd consensus can be systematically distorted by subgroup-based socio-cognitive influences, such as affective polarization. In the experiment, raters from a range of subgroups with varying levels of affective polarization were asked to view and validate crisis data from a violent public riot in the year 2020. Relying in part on double debiased machine learning techniques, we analyze heterogeneous treatment effects across subgroups. The results show that affective polarization and more extreme raters, via the constructs of loyalty and betrayal, distort consensus-based ground truth in different ways. This research note demonstrates how subgroup-based socio-cognitive influences can systematically distort the results of consensus-based crowdsourced validation. Additionally, it provides guidance for research and practice on how to account for identifiable subgroups in the crowd. These findings challenge key assumptions about the wisdom of crowds and the accuracy of crowdsourced ground truth in event-centric situations.