Enhancing the Reliability of NIMH-Supported Research through Rigorous Study Design and Reporting
The NIMH mission is to transform the understanding and treatment of mental illnesses through basic and clinical research, paving the way for prevention, recovery, and cure. For the Institute to continue fulfilling this vital public health mission, it must foster innovative thinking and ensure that a full array of novel scientific perspectives are used to further discovery in the evolving science of brain, behavior, and experience. In this way, breakthroughs in science can become breakthroughs for all people with mental illnesses.
High-quality and reproducible studies that are reported to the scientific community in a transparent manner are an essential cornerstone of the research enterprise. Attention to principles of study design and transparency are essential to enable the scientific community to assess the quality of scientific findings and for peer reviewers to advise NIMH appropriately on funding decisions, and for NIMH Program staff to monitor the progress of funded awards.
To ensure the highest standard of rigor and reproducibility of studies, investigators are encouraged to consider the following points in designing their research studies:
Fundamental Information on Vertebrate Animal Species (if applicable)
- Species and strain
- Vendor and vendor’s location (or if bred on-site)
Fundamental Information on Human Subjects (if applicable)
- Rationale for the models and endpoints chosen
- Rationale for and adequacy of the control groups
- Justification of sample size, including power calculations, number of subjects per condition, and a clear definition of “a subject” (e.g., in electrophysiological studies, is “one subject” the recording from one cell or the recording from one animal?)
- Statistical methods used to analyze and interpret results
- For studies using or assessing multiple behavioral outcomes, rationale for and controlling for the order of testing
- For pharmacological studies, rationale for dose, route, and timing of intervention delivery/dosing (if appropriate)
- Methods of blinding (e.g., allocation concealment and blinded assessment of outcome) or discussion as to why blinding is not feasible in the proposed study
- Strategies for randomization and/or stratification
- Criteria for exclusion or attrition of data
- Reporting of all results (negative and positive)
- Independent validation/replication of key preliminary data or plans for future follow-up studies for discovery research
- Robustness and reproducibility of the observed results
- Dose-response results
- Verification that interventional drug or biologic reached and engaged the target
Interpretation of Results
- Alternative interpretations of the experimental data and consideration of methodological limitations
- Relevant literature in support of or in disagreement with the results
- Discussion of effect size in relation to potential clinical impact, if applicable
Reporting Guidelines for Publication of Results
For animal studies, refer to Landis, S.C. et al., A call for transparent reporting to optimize the predictive value of preclinical research. Nature 490: 187-191, 2012.
For phase II and III randomized controlled trials, refer to the CONSORT statement and CONSORT checklist http://www.consort-statement.org/
- Please see NOT-MH-14-004 in the NIH Guide.
- Matters of significance. Nature Methods 10: 805, 2013.
- Krzywinski, M., Altman, N., Points of significance: Importance of being uncertain. Nature Methods 10: 809-810, 2013.
- Krzywinski, M., Altman, N., Points of significance: Error bars. Nature Methods 10: 921-922, 2013.
- Krzywinski, M., Altman, N., Points of significance: Significance, P values and t-tests. Nature Methods 10: 1041-1042, 2013.
- Krzywinski, M., Altman, N., Points of significance: Power and sample size. Nature Methods 10: 1139-1140, 2013.
- Krzywinski, M., Altman, N., Points of significance: Visualizing samples with box plots. Nature Methods 11: 119-120, 2014.