ICIS Panel 1995: Judging Qualitative Research In Information Systems: Criteria For Accepting And Rejecting Manuscripts

Criteria and Conventions used for judging manuscripts in the area of ethnography

Comments by Michael D. Myers

1. Introduction

There is no single ethnographic method

I would like to begin with the qualification that there is no single ethnographic method within the source discipline - there are different schools within anthropology. These schools differ in their procedures, purposes and evaluation criteria, and there is great variety within each one (see Harvey and Myers, 1995). Therefore, in such a short time I cannot possibly review all of these various approaches and the appropriate evaluation criteria for each. What I propose to do, instead, is to focus on just one approach which I advocate, which is

Critical hermeneutics

For a critical hermeneutic approach to ethnography, see Forester (1992). Hermeneutics is concerned with the interpretation of a text or text-analogue (such as an organization). Critical hermeneutics goes beyond "pure" hermeneutics - instead of taking a text at face value, and attempting to understand a text in its own terms, critical hermeneutics also looks at the dialectical relationship between the text and the interpreter (see Myers 1994; 1995).

2. Judging ethnographic manuscripts - Theory

The following questions need to be asked:

Is this a contribution to the field? This is the key question. Some ways of helping us to answer this question are as follows:

Has the author developed or applied new concepts or theories? They don't have to be completely new, the key thing is that they are new to the IS research community eg. Orlikowski (1991) adapted Giddens' structuration theory from sociology.

Does the author offer rich insights?

Does this manuscript contradict conventional wisdom? eg. Bentley, Hughes et al (1992) found that their ethnographic studies contradicted conventional thinking in systems design. They found that the conventional principles normally thought of as 'good design' could be inappropriate for cooperative systems.

Are the findings significant for researchers and/or practitioners?

3. Judging ethnographic manuscripts - Data

In terms of data, the first thing I would look out for is, is this a "real" ethnography? Whereas both case studies and ethnographies use interviews, the distinguishing feature of ethnographic research is participant observation. The researcher must have been there and lived there for reasonable length of time.

This means that a sufficient mass of data must have been collected for significant insights to emerge.

Does the real complexity of the organization as a social, cultural and political system come out? Does the manuscript tell us about real people with their different agendas, problems etc?

Is the subject matter set in its social and historical context? In interpretive research approaches, context is crucial. It is the context which constitutes and gives meaning to the situation.

Is there sufficient information about the research method and the research process?

Is the researcher aware of his/her historicality? eg. Zuboff (1988) realized that "history would offer only a brief window of time during which such data could be gathered." How did the research develop?

Are multiple viewpoints/ alternative perspectives presented? For an anthropologist, there is nothing more suspicious than being presented with the "official line". Unfortunately, many IS case studies are of this kind, where the only people who have been interviewed are from management.

Does the author adopt a critical perspective?

(this follows from the previous point). Does the author take their informants' comments at face value? Or are there hidden agendas? Are there unintended consequences of people's actions?

4. Writing Ethnographic manuscripts

A key problem is the average length journal paper in IS (approx. 20 pages).

Ethnographic work by definition is holistic. The ethnographer tries to understand the whole picture, and the parts in terms of that whole. In anthropology, the main way of presenting the research findings is in an ethnography, in book form. It is only in a book that a reader can get a sense of the whole.

In IS however, the journal article is regarded as THE best way of reporting research findings. There is an expectation that singular findings will be presented in individual papers.

A suggested solution - each paper is treated as part of the whole - each paper is only a microcosm of the whole. You can't tell the whole story in one paper. Therefore, the question becomes: which part of the story are you going to tell (in this particular paper)? One advantage of this approach is that you should be able to publish a number of papers from the one ethnographic study.

5. Conclusion

In conclusion, these are the things which I look for:

i. Is this a contribution to the field? This is key.

ii. Is the story convincing? There are two parts to this:

a. Does it make sense? (this is the hermeneutic imperative) and b. Is it believable?

Return to Qualitative Research in IS


This work was published in MISQ Discovery on May 20, 1997. This is the original archival version. It is currently maintained by Michael D. Myers. Corrections, clarifications, and suggested modifications should be directed to him at m.myers@auckland.ac.nz. Serious problems should be referred to the Editor-in-Chief.