When the Machine Meets the Expert: An Ethnography of Developing AI for Hiring

In stock
Downloadable File
The introduction of machine learning (ML) in organizations comes with the claim that algorithms will produce insights superior to those of experts by discovering the “truth” from data. Such a claim gives rise to a tension between the need to produce knowledge independent of domain experts and the need to remain relevant to the domain the system serves. This two-year ethnographic study focuses on how developers managed this tension when building an ML system to support the process of hiring job candidates at a large international organization. Despite the initial goal of getting domain experts “out the loop,” we found that developers and experts arrived at a new hybrid practice that relied on a combination of ML and domain expertise. We explain this outcome as resulting from a process of mutual learning in which deep engagement with the technology triggered actors to reflect on how they produced knowledge. These reflections prompted the developers to iterate between excluding domain expertise from the ML system and including it. Contrary to common views that imply an opposition between ML and domain expertise, our study foregrounds their interdependence and as such shows the dialectic nature of developing ML. We discuss the theoretical implications of these findings for the literature on information technologies and knowledge work, information system development and implementation, and human–ML hybrids.
Additional Details
Author Elmira van den Broek, Anastasia Sergeeva, and Marleen Huysman
Year 2021
Volume 45
Issue 3b
Keywords Machine learning, artificial intelligence, knowledge production, knowledge work, mutual learning, human–ML hybrids, information system development, learning algorithms, ethnography, hiring, human resources
Page Numbers 1557-1580; DOI: 10.25300/MISQ/2021/16559
Copyright © 2023 MISQ. All rights reserved.