Research Transparency

Following the research transparency approach as explained in the editorial by Burton-Jones, Boh, Oborn, and Padmanabhan (MISQ, 2021), MISQ expects authors to take account of the following guidelines upon submission of manuscripts. Full details regarding each step in the process are described below.

Effective January 1, 2022

Background Information

Guidelines like these are now common across the sciences. While specific practices vary among journals, the spirit is similar. For MISQ, transparency materials refer to materials posted on external research repositories that provide more details, beyond what is described in the paper, about how a piece of research has been undertaken and its potential implications. Examples of such materials, and external repositories, are included later in these instructions.

Research Transparency Guidelines

In their cover letter, authors must declare if and how they will follow these guidelines, and if they will not follow a given element, what alternative approaches they plan to follow. Authors’ declaration requires that they understand the following aspects:

1. If their paper passes the minor revision stage, authors will need to post a set of transparency materials (agreed to with their Senior Editor) on an external research repository. While they do not have to post the materials until that time, the initial cover letter should indicate their willingness to post the materials, and the research repository where they will post them.
2. The list of transparency materials and the external repository that authors propose in their cover letter reflects an initial commitment. While this commitment should provide a good guide for the final materials to provide, it is subject to change based on conversation with the paper’s Senior Editor. During review cycles, the Senior Editor will comment on the suitability of the proposed materials and may suggest changes. Authors should communicate with their Senior Editor the extent to which they can accommodate these changes.
3. Until a paper is accepted, when authors voluntarily post their transparency materials on an external research repository and provide the link to the materials in their cover letter, they must post them in an anonymous manner (to avoid breaching blind review). If the paper is accepted, this anonymity on the materials is removed after the decision is given, as done with the paper.
4. When the transparency materials are provided after the minor revision stage, they are not subject to peer review. Rather, the Senior Editor and Transparency Editor will assess the materials to determine if:
  1. they include all the information required by the Senior Editor (as committed to in the cover letter and in subsequent communication with the Senior Editor)
  2. they are presented clearly and understandably, to a standard that MIS Quarterly readers should expect.
  If the Senior Editor accepts the paper for publication, this acceptance decision will only be granted if the Senior Editor agrees that the transparency materials meet these two criteria.
5. If the paper is accepted, the transparency materials posted on the external repository will remain available as a permanent, accessible reference. A link to these materials will be provided in the published paper. Changes (such as corrections/additions) can still be made to such materials over time, if required/useful, but with appropriate version control to show when changes are made and to retain the original version. See the Author FAQ for details.
6. If the paper is accepted, the transparency materials will have a disclosure added to them to:
  1. clarify the scope of MISQ’s review process (i.e., stating that the review process is limited to the paper, not the transparency materials
  2. indicate that the transparency materials are provided for academic purposes only
  3. indicate that users of the materials must cite the paper appropriately

Research Transparency Commitment

The concept of research transparency applies to any research. Accordingly, these guidelines apply to all submission categories. Nonetheless, norms for research transparency are more advanced for some genres of research than others. As a result, we only offer guidelines for some genres. For research that uses genres not covered here, and for research that cannot implement our proposed guidelines in whole or in part, please use the following resource to state how you will implement research transparency in a way that befit your research.

Step 1: If your research falls within one/more of the categories in the transparency guidelines, please use this table to indicate how you will enact those guidelines.

*Section of Guidelines*	*Guideline (see section for details)*	*I will follow the guideline*	*Comments (if needed)*
Quantitative studies
Data collection procedures and materials	Code used to automate the collection/storage of data	Yes/No/NA
Data:	Data sets for the main tests. If data cannot be provided, give: (i) reason, (ii) supporting evidence, (iii) alternatives.	Yes/No/NA	If data not provided, describe alternative provided
Data analytic methods:	Description of the data analysis, including intermediate steps, and program code for analysis	Yes/No/NA
Quantitative primary data	Data collection procedures and materials	Yes/No/NA
Quantitative Secondary data	Data collection procedures and important intermediate data sets	Yes/No/NA	If data not provided, describe alternative provided
Quantitative design/computational research	Design artifact/analysis, and program code	Yes/No/NA	If code not provided, describe alternative provided
Qualitative studies	Research design and data collection	Yes/No/NA
	Tracing the analytics process	Yes/No/NA
Qualitative (strong social construction assumptions)	Ontological Assumptions	Yes/No/NA
	Transparency of Relationships	Yes/No/NA
	Tracing the Analytic Process	Yes/No/NA
Relevant Constraints	Licenses for data or code	Yes/No/NA	If license will not be provided, justify and specify implications
	Embargo	Yes/No/NA	If requesting embargo, justify
	General ethics requirements	Yes/No/NA
	Contact details for data	Yes/No/NA	If relevant, provide contact details
	Multiple studies	Yes/No/NA	If relevant, provide details

Step 2: If your research does not fall in the categories in the transparency guidelines, please describe the genre/category of your research (e.g., theory/review, analytical, simulation), and state how you propose to enact research transparency in a manner that befits that genre/category.

*Genre/category of your research in your words*	[Describe/label the genre/category]
*Your proposal for enacting research transparency to fit the genre/category of your work*	[Enter proposal]

Step 3: Please indicate the research repository you plan to use for depositing your research transparency materials. See Section 5 of the author instruction guidelines for details.

Name of proposed research repository and URL:

[Enter details]

Guidelines for Transparency Materials

The main guideline for providing transparency materials is to embrace the spirit of research transparency, not merely the letter of the commitment. For instance, novel materials may be provided beyond the list below. The list below just provides a guide. The list is subdivided by research genre.

There are no page lengths for the transparency materials because the relevant materials for a given paper could vary widely. For instance, the full experimental materials for a lab experiment could vary from one to many pages. While we do not offer page-length guidelines, authors must present the material clearly, concisely, and in a well-organized manner so that readers can understand and use them. Authors need not provide every piece of detail behind a study, as this would be impossible.

When we refer to providing program code below, we give authors the option of providing the actual code or pseudocode. Pseudo-code has some advantages over actual code. It is not language-specific, is not tied to specific runtime environments, and can sometimes result in the same (or higher) level of clarity as the actual code. When implementation of said pseudo-code is obvious (e.g., “sort the tweets based on number of likes”) it offers the same benefits that sharing code does. However, in many cases implementation of the pseudo-code itself might be complex, and providing code therefore will help others understand exactly what was done and how, and this can better facilitate the goals of use, extensions, and reproducibility. Of course, providing both pseudo code and code can be done. Since every case is likely to be different, we will leave it to the authors to determine the best combination of pseudo-code and/or code to satisfy the transparency goals, and to articulate this clearly in the cover letter when the work is submitted.

Quantitative Studies - Generic Guidance

Data collection procedures and materials:

If program code is used to automate the collection and storage of data, provide the executable code, the software command file, or pseudocode, with sufficient comments/explanations, to allow readers to reproduce the process.
For data collection procedures and materials that generate data for multiple studies, please see the ‘constraints’ section later.

Data:

The data sets used to conduct the main tests reported in the paper with meta-data describing the fields.
The license (if applicable) for data being provided (please see the ‘constraints’ section)
If data cannot be provided, please provide:
- a clear and justifiable reason for not providing the data (e.g., ethical constraints, non-disclosure agreements, contracts)
- supporting evidence for the constraint (e.g., advice from university ethics committee, clauses from the non-disclosure agreement or contract)
- alternatives (e.g., disguised data, a random subset of the data, simulated data with similar properties to the real data, or worked examples). The aim is to help readers to understand the data and how they can obtain the same or similar data.

Data analytic methods:

Sufficient description of the data analytic methods (beyond that reported in the body of the paper) to allow researchers to reproduce the analysis in the paper. This includes intermediate stages of data preparation and manipulation (e.g., transformations, treatment of missing values) to enable readers to understand how the author(s) arrived at the final tests.
If program code is used in the analysis, provide the executable code, the software command file, or pseudocode, with sufficient comments/explanations, to allow readers to reproduce the process.

Additional Guidance for Quantitative Studies Collecting Primary Data (e.g., Lab Experiments and Surveys):

Data collection procedures and materials:

Sufficient details of the setting, sample, materials, and procedures, to allow, at least in principle, a literal replication.
If a questionnaire is used, include the full questionnaire with instructions and response options. If it is not in English, provide both the original version and the English translation.
Institutional (Ethical) Review Board approval number for human subjects research and the approved (blank) consent form that human subjects used to give consent.

Additional Guidance for Quantitative Studies Collecting Secondary Data (e.g., Archival data):

Data collection procedures:

Sufficient description of the data source and the data extraction process to enable researchers, where possible (e.g., where not constrained by non-disclosure agreements), to access the data source and reproduce the data collection.
If the secondary data reflects the outcomes of field experiments that affect human subjects, Institutional (Ethical) Review Board approval number for human subjects research and the approved (blank) consent form that human subjects used to give consent.

Data:

Descriptions of important intermediate data sets (e.g., where multiple data sets need to be merged and manipulated to create the final data set).

Additional Guidance for Quantitative Studies of a Design Science or Computational Genre

Design Artifact and Analysis:

Extended details of the system artifacts, algorithms, models, assumptions, or data to enable reproduction, e.g.:

A working prototype and/or a detailed architecture
Exact specification of parameters in all the algorithms that can enable researchers to reproduce the code and run it on similar data, or in a scenario similar to the one outlined in the paper.

Code:

Code, or code fragments, that enable researchers working in the area to understand what was done in a manner detailed enough for them to implement it. If exact code cannot be provided, please provide:

a clear and justifiable reason for not providing the code (e.g., ethical constraints, non-disclosure agreements, contracts)
supporting evidence for the constraint (e.g., advice from university ethics committee, clauses from the non-disclosure agreement or contract)
alternatives that can help readers to understand the code and how they can obtain the same or similar data.

Where suitable, executable code in a virtual containerized infrastructure such as CodeOcean that enables researchers to run the code and examine the results under different scenarios. Authors may be asked to sign a release form to release MIS Quarterly from liability resulting from the use of code.

Qualitative Studies – Generic Guidance

The research transparency literature has progressed more slowly in this genre than in others because of controversies over the meaning and purpose of transparency. As a result, authors are encouraged to think creatively about the guidelines as they apply them to their study. We begin with generic guidance for all types of qualitative research and then provide additional guidance for qualitative genres in which issues of social construction, meaning, and interpretation are central.

Research Design and Data Collection:

Further details of research design or approach beyond those reported in the paper, such as:
- Interview templates, and how the questions changed over time during the study.
- Extended descriptions of the case to show authenticity and to help readers to understand the nature of the data (e.g., where deep knowledge of the context, such as social or language dimensions, is needed to understand the research design choices or the nature of the data obtained).
- Institutional (Ethical) Review Board approval number for human subjects research and the approved (blank) consent form that human subjects used to give consent.

If program code is used to automate the collection and storage of qualitative data (e.g., from apps or social media), provide the executable code, the software command file, or pseudocode, with sufficient comments/explanations, to allow readers to reproduce the process.

Tracing the Analytic Process:

Instructions or training materials for research assistants involved in the analytic process (if applicable)
Worked examples of data coding
Intermediate results or representative artifacts of the theorizing process that help the reader to understand how the final theoretical insights were derived

Qualitative Studies – Specific Guidance for Studies with Strong Social Construction Assumptions

Ontological Assumptions:

Extended discussion of the study’s philosophical assumptions to enable readers to understand the implications of the chosen assumptions, provoking re-examination of readers’ assumptions.

Transparency of Relationships:

Extended discussion of the relationships between the researcher and the researched, beyond those discussed in the paper, such as researcher subjectivity, biases, positionality, ethical challenges, and risks to the human participants and/or the researcher

Tracing the Analytic Process:

Extended insight into the interpretations and analysis process, such as legitimating atypical situations or contestable assertions, and providing worked examples of data coding.

Relevant Constraints

Licenses for data or code:

The data or code used in a study may be protected under a license. Also, if authors release data or code with no license, by default they could be retaining all rights and preventing others from extending the data or code in any way. For such reasons, authors should select a license to enable reuse. For details on choosing appropriate licenses please see,:

Embargo:

While the default commitment is to make transparency materials available immediately upon publication, there may be rare circumstances where it is appropriate to place an embargo of up to one year on a portion of the materials, e.g., when the data or code reflect an especially complex or sensitive case where the researcher or public would benefit from more time before it is made available. If this applies to your study, please disclose this in the cover letter for the Senior Editor’s consideration.

System dependencies (e.g., when collecting data from platforms/websites)

Researchers may use code to collect quantitative or qualitative data from platforms or websites. In such cases, the ability to execute the code may depend on the characteristics of the other system. For instance, if the API changes, the code may no longer work. As a result, it is infeasible for researchers to guarantee the suitability of the code for all time. Rather, they just need to provide a plausible case that their code worked for the platform/site at the time of data collection. Sufficient details of the characteristics of the platform/site/API need to be given to help readers to understand the plausibility of the data collection at that time.

General ethics requirements:

As noted earlier, authors may sometimes be unable to provide data and code. In such cases, authors are still required to abide by traditional research ethics expectations to maintain the data and code and cooperate with reasonable requests for access they may receive.

Organizational data (contact details):

In cases where data is obtained from a private organization, if the authors cannot make the data available, they must provide contact information for a representative from either the focal organization or the authors’ university’s research office, who can confirm the nature of the data obtained and the authors’ right to use that data in the study. This information will not be made public but will be maintained by the editors in the confidential files for the paper. It is possible that MISQ could use those contact details to confirm the details.

Multiple studies:

Authors might split up a large data set or a large amount of code into different segments, with each segment targeted to a different journal. In such cases, the MISQ guidelines are designed to strike a balance between respecting:
- authors’ prerogative to only reveal the parts of the study relevant for this paper
- biases that can arise when stakeholders (authors, reviewers, editors, or future readers) make inferences regarding the paper without full transparency of the larger picture
To illustrate biases that can arise in multi-paper studies, consider questionnaire studies. Participants responses to questions about some constructs in a questionnaire can depend on the responses they give to other questions. As a result, if a paper does not reveal all the constructs in the questionnaire, it is impossible for readers to draw accurate inferences from the reported results.
Because of such biases (which can arise with any research genre), authors must provide a high-level summary of the data sources and code used across the multiple studies (whether published or not). For examples for how this can be achieved, we recommend that authors include a transparency table, as shown here: https://www.apa.org/pubs/journals/apl/data-transparency-appendix-example.
Example: Applying this approach to a questionnaire study, authors are required to:

- provide the full questionnaire for the relevant constructs included in the paper
- use a transparency table (example at link above) to provide an overview of the other items in the full questionnaire that may be submitted or planned for other papers
- respond to requests from the SE for details on these other items if required

Choosing Where to Archive your Data

In the Research Transparency Commitment, authors need to list the research repository/repositories where they agree to post their transparency materials at a later time. This is an initial commitment; the decision can be changed in consultation with the Senior Editor during the review process.

Research repositories differ from general repositories. See: Suggested Information for Data and Code Hosting | Data and Code Guidance by Data Editors (social-science-data-editors.github.io

We do not dictate the use of any one repository because different research repositories have different features that may suit different studies. Authors may choose from any repository that supports ‘FAIR’ principles (Findability, Accessibility, Interoperability and Reusability) and that provides for permanent archiving and preservation of material with persistent identifiers, so that the material can last as long as the journal paper lasts, and readers can be assured that the original form will remain available and any changes can be identified through version control. Such repositories include, but are not limited to:

Open Science Framework: https://osf.io/
DRYAD: https://datadryad.org/
Figshare: https://figshare.com/
Harvard Dataverse: https://dataverse.harvard.edu/
Kaggle: https://www.kaggle.com/datasets
Zenodo: https://zenodo.org/

In research that utilizes computational techniques, related platforms are also available for packaging code and making it available for transparency and reproducibility purposes. Well-supported platforms include, but are not limited to:

Code Ocean: https://codeocean.com/

External research repositories have many instructions to help authors. For instance, the OSF provides instructions for authors at the following links to help them with:

Sharing data: https://help.osf.io/hc/en-us/sections/360003569654-Handling-Data
Version control: https://help.osf.io/hc/en-us/articles/360019971574-Version-Control
Licensing: https://help.osf.io/hc/en-us/articles/360019739014-Licensing

For more details on suitable repositories, please see:

Benefits

These new guidelines will enhance the value of authors’ research in at least two main ways:

Immediate benefits: Use and impact

The immediate benefit of following these guidelines is that it will enhance the value of the research. Papers that provide effective transparency materials will be easier for readers to understand, trust, test, extend, and apply. Such papers are then likely to have greater impact. The June 2021 Editor’s Comments provides a detailed review of these benefits. For some additional research on these benefits and the cost/benefit equation, please see:
- Chang, X., Gao, H., and Li, W. 2021. P-Hacking in Experimental Accounting Studies (September 5, 2017). Nanyang Business School Research Paper No. 21-01, http://dx.doi.org/10.2139/ssrn.3762342
- Miguel, E. 2021. Evidence on Research Transparency in Economics, Journal of Economic Perspectives (35:3), pp. 193-214.

Secondary benefits: Transparency tags and further impact

To enhance the promotion of your paper, we will use ‘transparency tags,’ i.e., searchable tags that highlight that your paper makes data available, material available, code available, etc. These tags will provide an additional way for readers to search for your paper and provide an additional way for MISQ to signal the value of your work for readers. Over time, we aim to expand these initiatives (e.g., by linking them with the AIS Transactions on Replication Research “replication badges”) so that even richer tags can be used (e.g., ‘results replicated’). We will continue to look for ways to promote and celebrate your work.