Real-Effort Incentives in Online Labor Markets: Punishments and Rewards for Individuals and Groups

In stock

Publication History

Received: September 6, 2017
Revised: March 31, 2019; August 9, 2021; September 9, 2022; May 9, 2023
Accepted: June 2, 2023
Published Online in Issue: March 1, 2024

Downloadable File

Online labor markets and the humans that power them serve a critical role in the advancement of artificial intelligence and supervised machine learning via the creation of useful training datasets. The use of human effort in online labor markets is not enough, however, as a key factor is understanding the possible interventions that market operators can leverage to incentivize human effort among their labor force. We propose that platforms could implement mechanisms such as rewards or punishments at individual or group levels to incentivize real-effort and output. We apply our interventions using a collaborative image tagging experiment—a folksonomy—and the results provide interesting insights and nonobvious consequences. On average, interventions applied at the group level outperformed interventions applied at the individual level. Punishing the group provided the most controversial incentive strategy and provided a nonobvious significant improvement in effort. Rewarding or sanctioning an individual had similar effects on average, with both treatments leading to significant increases in effort post-intervention. In contrast to predictions, sanctioning appears to have significantly motivated those that were punished. Overall, the interventions applied in our real-effort collaborative image tagging experiment had a significant impact on behavior, which provides guidance for online labor market operators and the use of incentives in the creation of labeled machine learning training datasets.

Additional Details
Author Matthew J. Hashim and Jesse C. Bockstedt
Year 2024
Volume 48
Issue 1
Keywords Online labor market, free riding, incentive mechanism, economic experiment, economics of IS, collaborative image tagging
Page Numbers 299-320
Copyright © 2023 MISQ. All rights reserved.