Changing NIMH Clinical Trials: Efficiency, Transparency, and Reporting
Nitin Gogtay, M.D., and Tom Insel, M.D., National Institute of Mental Health
Recent reviews have painted a dismal picture of treatment development for mental disorders.1 Industry has reduced its investment in clinical trials, and the pipeline of new compounds appears depleted. At the same time, the public health burden of mental disorders is becoming ever more apparent, emphasizing the need for new and better interventions.2 Given the unmet public health need, the National Institute of Mental Health (NIMH) has been exploring how federal investments might stimulate research to deliver the next generation of therapeutics.
While promising scientific opportunities are emerging,3 there are also some concerns about the current clinical trial enterprise. These concerns were raised in a 2010 Institute of Medicine (IOM) report critical of the National Cancer Institute's (NCI) approach to clinical trials, noting, among other issues, the slow pace of completion of NCI clinical trials and the lack of scientific innovation in the context of direct, tangible benefits to patients.4 A subsequent report in the British Medical Journal showed that fewer than half of NIH-funded trials resulted in papers published in peer-reviewed journals within 30 months of trial completion, and a third still had no published results after a median of 51 months.5 The U.S. Congress, concerned about the need for new treatments, created the Cures Acceleration Network within the Patient Protection and Affordable Care Act to find more efficient pathways to new treatments.6
What changes did Congress call for? First, improved efficiency, measured in time for approval, recruitment, and completion while maintaining the quality of subject selection and trial design. Second, increased transparency, as witnessed through the registering of trials, monitoring of human subject protection issues, and posting of results in a public database. And third, more comprehensive reporting if needed, to address concerns about the failure to publish, publication bias, or lack of access to raw data. As one indication of the changing expectations, the European Medicines Agency recently announced a draft policy requiring sharing of patient level data for all clinical trials in the European Union; in addition, the U.S. Food and Drug Administration (FDA) has announced similar plans to expand sharing of data submitted in regulatory applications.7
All of these issues, which were directed at NIH-funded trials, are relevant to NIMH-supported clinical trials. For this essay, the definition of a clinical trial is based on definitions found at ClinicalTrials.gov , and used by the World Health Organization and International Clinical Medical Journal Editors (ICMJE), as "A clinical study in which participants are assigned to receive one or more interventions (or no intervention) so that researchers can evaluate the effects of the interventions on biomedical or health-related outcomes." Elsewhere we have described the need for next generation trials focused on target validation using an experimental medicine approach.3 The issues of efficiency, transparency, and reporting are independent of the experimental medicine approach. This essay reviews the "how" rather than the "what" of NIMH's investments in clinical trials for treating mental disorders.
Clinical Trials at NIMH
In 2012, NIMH supported over 250 clinical trials for a total investment of roughly $100M. Almost half of these trials (49%) were R01 grants, with many other exploratory R34 or R21 grants (16%). The trials covered a range of therapeutics including small molecules, devices, psychosocial and behavioral interventions, and single- as well as multi-site trials. Over half of the NIMH portfolio was focused on psychological or psychosocial treatments, reflecting the traditional role of NIMH as developer of psychosocial interventions, in contrast to the traditional role of industry as developer of medications for mental illness.
How are these projects performing? For trials with over 150 subjects, the NIMH tracks recruitment every 4 months. This captures roughly two-thirds of trials from R01 grants and nearly all U01 (cooperative agreement) and contract projects. A check of recruitment in 2012 revealed that the majority of these projects had difficulty enrolling subjects within the first year of the Notice of Grant Award. Measured as 80% of target recruitment, for all trials with over 150 subjects, 64% were behind schedule in 2012. In contrast to most trials in industry, which are completed within 3 years, the mean time to reach target enrollment for NIMH trials approximates 4 years.
What about registration of NIMH trials? Like grantees from other Institutes and industry, NIMH grantees are subject to two broad policies. The first policy, originally part of the FDA Modernization Act (FDAMA) of 1997, mandated the registration of Investigational New Drug application trials for serious and life-threatening diseases and conditions. A public registry, ClinicalTrials.gov , was created by the NIH through the National Library of Medicine to house this important new database. Subsequently, the policy was significantly expanded in 2007, through the FDA Amendments Act (FDAAA), which required that all intervention trials (e.g., drugs, biologics, devices, etc.), including early phase and behavioral trials, be registered in ClinicalTrials.gov. Observational studies were exempted. Importantly, FDAAA required that results also be shown within 12 months from primary completion within ClinicalTrials.gov.
A second policy, developed in 2004 by the ICMJE, required prospective registration of intervention trials of all types in ClinicalTrials.gov as a prerequisite for publication in one of its member journals. ICMJE policy exempted observational studies but included behavioral interventions. As of this writing, about 150 studies are listed as open for recruitment in ClinicalTrials.gov with NIMH as lead funder or co-sponsor.
Are NIMH trials being published promptly? The definition of prompt, of course, must consider not only the time to analyze and report data but also the journal's time for review and publication. As of March 2012, less than half of NIMH trials ending in 2008 and 2009 (36 and 24 months, respectively) had reported results, which is consistent for NIH generally.5
Modifying NIMH Trials
The results cited above suggest that NIMH is no different than other NIH Institutes in the conduct of its clinical trials: There is a clear need for improvement in efficiency, transparency, and reporting. A workgroup of the National Advisory Mental Health Council (NAMHC) reviewed the state of our clinical trials portfolio and recommended a number of changes.8 Additional conversation with both the NAMHC and stakeholder groups have stressed that "time matters," given the urgency for new and better treatments and the new ecosystem in which large pharmaceutical companies are reducing investments in treatments for mental illness. With this recent retreat of industry, NIMH recognizes the need for shifting the balance in its portfolio, from traditional efficacy trials to focus more on target validation and experimental therapeutics for both biomedical and psychosocial interventions. The goal of this new approach is to identify new targets for treatment development, or identify therapies that directly influence disease outcome or knowledge of the disease. In this context, it will be important that trials be designed such that negative as well as positive outcomes will be equally informative. But this new approach will be accompanied by several changes in our expectations for the conduct of clinical trials.
A common complaint from investigators is that many of the regulations for the conduct of clinical trials add time without clearly improving rigor or safety. As one example, multi-site trials require approval from individual Institutional Review Boards (IRBs). This delays the launch of trials in all sites without necessarily improving the quality or safety of the trial. Several experiments are underway within the Clinical Translational Science Award (CTSA) program as well as within individual NIH Institutes to accelerate review without reducing the quality of the process.9 Going forward, for multi-site trials, NIMH will mandate the use of a single IRB of record.
In the past, many NIMH-funded investigators have applied for extensions or competitive renewals of their clinical trials without demonstrating the performance metrics noted above. NIMH will require accounting for time to enrollment, rate of recruitment, and time to completion before funding extensions or renewals for all trials. Additionally, NIMH will monitor these milestones, as a requirement for continuation of funding and will consider past performance as an additional criterion during the grant review process.
The new Funding Opportunity Announcements (FOAs) announced in the fall of 2013 reflect the shift of NIMH portfolio to the experimental medicine model. NIMH will also move more of its clinical trials portfolio from research project grants (i.e., R01s and related mechanisms) to contracts and cooperative agreements. The New Experimental Medicine Studies: Fast-Fail Trials (FAST) for psychotic spectrum disorders, autism, and mood and anxiety disorders were awarded as contract trials in fall 2012. Also, the Rapidly-Acting Treatments for Treatment-Resistant Depression (RAPID) contracts, awarded in 2013, reflect a new mechanism for supporting clinical trials for compounds with potentially rapid antidepressant activity with a priority on efficiency and quality. Contract trials are monitored carefully for their progress to ensure that the stated goals and objectives are met on time.
Registration and Regulatory Oversight
In accord with FDAMA and ICMJE guidelines, all NIMH-funded trials (except observational studies) must be registered in ClinicalTrials.gov prior to receiving funding. This applies to studies of medications, devices, psychological, and psychosocial interventions in both our extramural and intramural programs. Registration will be a requirement specified in every notice of grant award for future clinical trials and will be monitored by extramural program officers and the intramural Office of the Clinical Director.
Publication and Data Sharing
It is not clear if the low and slow rates of publication reflect failed trials, publication bias, or delays in the publishing process. Whatever the reason, NIMH will continue to monitor the rate of publishing results from our clinical trials portfolio. Reducing the trial duration, monitoring progress, and encouraging data sharing should also help improve the publication rate.
Currently, a data sharing plan is expected for all projects over $500K/year in direct costs. Moving forward, NIMH will expect a data sharing plan for all clinical trials. The investment of public funds as well as research participation of subjects in clinical trials entails a commitment to the public good. We are developing a process for standardizing, integrating, and sharing clinical trials data that will be expected of all grantees.
NIMH could improve the performance, the transparency, and the reporting of our clinical trials but still fail to reduce the burden of mental illness. We have heard from our stakeholders that too many of our trials are scientifically rigorous and academically interesting but clinically irrelevant. The public is looking for impact, defined by either the development of new treatments or the demonstration of the best use of current treatments. The 2010 review of our portfolio by the NAMHC Workgroup raised questions about the impact of many of our trials, irrespective of their results.
The current dire situation for treatment development can be traced largely to the absence of new targets, based on an insufficient understanding of pathophysiology. Clinical trials have heretofore validated targets and demonstrated the importance of target engagement for modifying disease. The next generation of NIMH trials, supported by new Funding Opportunity Announcements (FOAs), will test for target engagement itself as a potential mechanism of disease. These studies, with either biomedical or psychosocial interventions, should prove informative even when the intervention does not prove efficacious because the goal of the trial is to learn about the disorder and not simply to demonstrate efficacy of the intervention. Information for applicants and reviewers what NIMH expects to be a new wave of clinical trials of great impact can be found here: http://www.nimh.nih.gov/funding/opportunities-announcements/clinical-trials-foas/index.shtml.
Doing More with Less
The NIMH budget cannot replace what industry has invested historically in drug development for mental disorders. Our role is different and our goals extend well beyond drug development. What we have done and will strive to do in the future is improve our understanding of pathophysiology so that both federal and private investment in treatment development can be targeted. But the kinds of trials described above are expensive, adding the costs of mechanistic studies to the already high costs of trials due to regulatory requirements and the challenges of recruitment. As a result, NIMH may well support fewer trials than in the past, but we expect greater yield because each trial should advance the field of therapeutics.
As federal support for research is being reduced,10 how can we propose to take on even more expensive trials? Going forward, we will need to leverage investments whenever possible. As just one example, the previously mentioned CTSA program is an investment of almost $500M to support the infrastructure of clinical research and clinical trials at academic health centers across the nation. The CTSAs have already developed a range of tools for electronic data capture, patient recruitment, and statistical support that should be considered by every NIMH applicant. Another opportunity for "doing more with less" is the use of large health care systems, often outside of academic medical centers. Bringing "practice to research" is increasingly popular for studies of services and epidemiology. Large health care systems can also be used for pragmatic trials, with cost-efficiency, speed, and impact that might not be possible in traditional academic research settings.
The future of clinical research requires a modification of how we do trials as well as what we study. In 2012, NIMH funded over 250 trials, investing about $100M. We are implementing significant changes to increase efficiency, transparency, reporting, and impact of our trials. We are mindful that clinical trials research is already challenged by regulatory burden and increasing costs on those whom we trust to perform trials. Some of the new tools developed by the CTSA network can facilitate the early stages of trials without increasing costs. We are developing mechanisms to ensure that trials are completed with efficiency, within budget, and to streamline the process for clearer, transparent, and uniform monitoring. The comprehensive and prospective use of ClinicalTrials.gov should provide greater transparency for our trials and a standard database for monitoring the duration to completion and publication of results.
There are extraordinary opportunities for innovation to develop the next generation of therapeutics.3 However, to ensure that we deliver on this promise, we need a better system for launching, completing, and reporting trials of both biomedical and psychosocial interventions. Clearly, the cost of inaction would be unacceptably greater than these steps towards developing more efficient and informative trials.
1. Hyman SE. Revolution stalled . Sci Transl Med. 2012 Oct 10. 4(155): 155cm11.
2. Fineberg NA et al. The size, burden and cost of disorders of the brain in the UK . J Psychopharmacol. 2013 Sep;27(9): 761-70.
3. Insel TR. Next-generation treatments for mental disorders . Sci Transl Med. 2012 Oct 10; 4(155): 155ps19.
4. Nass SJ et al. A national cancer clinical trials system for 21st century: Reinvigorating the NCI cooperative group program . Institute of Medicine Report. 2010.
5. Ross JS et al. Publication of NIH funded trials registered in ClinicalTrials.gov: Cross sectional analysis . BMJ. 2012 Jan 3; 344: d7292.
6. Olson S, Clairborne A. Accelerating the development of new drugs and diagnostics: aximizing the impact of the cures accelertion network . Institute of Medicine Report. 2012.
7. Mello MM et al.Preparing for responsible sharing of clinical trial data . N Engl J Med. 2013 Oct 24;369(17): 1651-8.
8. From Discovery to Cure: Accelerating the development of new and personalized interventions for mental illness. National Advisory Mental Health Council's Workgroup Report. 2010 Aug.
9. Marsolo K. Approaches to facilitate institutional review board approval of multicenter research studies . Med Care. 2012 Jul;50 Suppl: S77-81.
10. Wadman M. The NIH faces up to hard times . Nature. 2012 Sep 26.