This post was originally posted on the Cochrane Methods website
In January 2018, the FDA announced a pilot aimed at enhancing the transparency of the Agency's drug approval process and decisions. As part of the pilot, sponsors of up to nine drug applications to the FDA were asked, on a voluntary basis, if they would approve the public release of portions of the CSRs, which would be posted within an Action Package for the drug that also includes FDA application review files and letters. The parts of the CSRs made publicly available included information from the study report body, the protocol and amendments, and the statistical analysis plan for each study.
The FDA is calling for comments on this pilot as part of the ‘New Drugs Regulatory Program Modernization: Improving Approval Package Documentation and Communication’ call for public comment (deadline 26 August 2019).
Karla Soares-Weiser, Editor-in-Chief of the Cochrane Library, and members of the Cochrane Clinical Study Report Working Group have submitted a response to this pilot, which is reproduced below as an open letter. The Cochrane Clinical Study Report Working Group is a recently announced, open group that aims to support and facilitate the increased use of CSRs in Cochrane Reviews.
The Clinical Study Report Working Group also invites, and welcomes, any sponsors or regulators who would like to reach out and initiate discussions with us to help facilitate CSR use in Cochrane Reviews. To date, only two published Cochrane Reviews have used CSRs (here and here) and overcoming challenges with access and terms of use will be key to increasing this number.
If you are interested in joining the Clinical Study Report Working Group, or have any questions or feedback on the project, please email methods@cochrane.org (emails to the attention of Ella Flemyng, Methods Implementation Coordinator). You may also be interested in the recent post ‘Using Clinical Study Reports as a data source for Cochrane Reviews: consultation meeting report and next steps’.
Submitted to the FDA via the online portal:
Covering note: The attached letter was posted on the Cochrane Methods website on 22 August 2019 (https://methods.cochrane.org/news/open-response-FDA-Clinical-Data-Summary-Report-Pilot) in response to the FDA’s invitation for public comment on the Clinical Data Summary Report Pilot program (Docket No. FDA-2019-N-2012 for the “New Drugs Regulatory Program Modernization: Improving Approval Package Documentation and Communication”). We, the undersigned, Editor-in-Chief of the Cochrane Library and members of the Cochrane Clinical Study Report Working Group, are formally submitting our letter as part of the public comment process. Institutional affiliations, for identification purposes, and competing interest statements were current as of 22 August 2019. Submitted by Ella Flemyng on 22 August 2019.
Background
Cochrane exists so that healthcare decisions get better. We produce high-quality, relevant, up-to-date systematic reviews and other synthesized research evidence to inform health decision making.
Many projects in the field, including a recent Cochrane Methods Innovation Funded project, have highlighted that journal article reporting of randomized controlled trials (RCTs) is often incomplete and can therefore bias results of systematic reviews. Clinical study reports (CSRs) are increasingly recognised as a source of RCT information that can enhance the value of systematic reviews as they are likely to be the most comprehensive and complete single source of information about a RCT, particularly when compared to a journal article. CSRs help systematic reviewers gain a more detailed picture of the methods, and results for under-reported outcomes from a trial, especially adverse effects.
Feedback on the Clinical Data Summary Report Pilot program
By the end of the FDA Clinical Data Summary Report Pilot, only one sponsor out of a potential nine opted-in and approved the release of this information (in February 2018).
The released Erleada CSR contains substantially more information than would be available via a journal article, which is of huge benefit for systematic reviewers; the methods are better described, there is more information about the intervention and co-interventions, and the results tables and descriptions of adverse events are relatively comprehensive.
Redactions in the Erleada CSR are relatively light, but perhaps some are still unnecessary. For example, at the bottom of page 31, information on storage and use of the drug could be important. Redacted participant IDs means information across different tables cannot be linked. However, even with redactions, there is a lot of useful information in the document that would be valuable for researchers and systematic reviewers.
In addition to releasing information from the main body of the CSR, the protocol and amendments, and the statistical analysis plan for each study, we ask the FDA to provide access to the appendices, such as the complete tables and line listings of adverse events. Members of our CSR Working Group have found these can provide additional valuable information for the assessment of benefit and risk of a treatment.
We would also like to comment that the information on the ‘Drug Approval Package: ERLEADA (apalutamide)’ page is clearly identified and therefore easy to use, including the CSR, protocol and statistical analysis plan information. However, any integrated review provided by the FDA should be in addition to the individual reviews and materials that are already made publicly available in the FDA Drug Approval Packages, and not replace them. The individual reviews and other documentation in the packages can be used for multiple purposes, including the assessment of the safety and efficacy of medicines and the synthesis of data in systematic reviews. Publicly available CSRs and the full FDA Drug Approval Package can thus complement each other, and neither is a substitute for each other. Whilst a CSR is necessary for transparency, providing both will enhance the ability to independently assess benefit and risk of medicines.
The Erleada CSR and action package does not appear to have clear copyright or reuse information. We assume that by making the CSRs publicly available that there are no restrictions on use and the data presented in the CSR can be used freely within Cochrane Reviews. We encourage the FDA to include positive statements about re-use of data for all available CSRs.
We urge the FDA to make CSRs publicly available
Given the benefits of using CSRs, Cochrane is exploring how more could be used within relevant Cochrane Reviews. Cochrane held a consultation meeting in May 2019 to identify practical ways forward to facilitate CSR use within Cochrane review teams and we are developing a strategy based on these recommendations (summary post and meeting report can be found here).
Cochrane has always supported initiatives to improve reporting of, and access to, RCTs and other primary research. The FDA states, “The Agency is.. considering whether to focus its efforts to better communicate the basis for drug approvals on the development of new integrated review documents, rather than on the release of CSRs” [see section ‘Update (6/26/2019)’] and we urge the FDA not to go down this route. For CSR disclosure, we feel that pilots are no longer needed and we urge all sponsors to make CSRs publicly available, not only all new drugs coming onto the market but also for past RCTs that have CSRs.
CSRs are a complete record of a trial. They contain essential information about the benefits and harms of medications, and about the credibility of that information. Independent researchers and systematic reviewers need to be able to access CSRs to verify the presented summary evidence from RCTs and assess how it was generated, and if necessary, to re-evaluate conclusions. Access to CSRs when compared with other sources of data (journal articles, registry records, etc.) reduces the potential for mistakes, misinterpretations, bias, evidence distortion, corruption and fraud.
We support and applaud the efforts by the FDA to improve the transparency, usability and accessibility of information used in the drug review process. We urge the FDA to make it a mandatory policy to publicly disclose CSRs, not a voluntary choice of the sponsors. We also urge all sponsors with CSRs for new, or previously approved, drugs or biologics to publicly release them to help facilitate their use in systematic reviews and health decisions.
Signatories:
Karla Soares-Weiser
Editor-in-Chief
Cochrane
Isabelle Boutron
Co-convenor Bias Methods group
Director of Cochrane France
Prof of Epidemiology at Université de Paris
Yuan Chi
Research Assistant
Centre for Evidence-Based Chinese Medicine
Beijing University of Chinese Medicine, China
Kerry Dwan
Statistical Editor
Cochrane
Ella Flemyng
Methods Implementation Coordinator
Cochrane
Kit Gilchrist
Editorial and Communications Officer
Cochrane
Marius Goldkuhle
Research Associate
Cochrane Haematological Malignancies, University of Cologne, Faculty of Medicine and University Hospital Cologne, Cologne, Germany
Toby Lasserson
Senior Editor (Methods Support and Development)
Cochrane
Carol Lefebvre
Lead Convenor, Cochrane Information Retrieval Methods Group
Independent Information Consultant, Lefebvre Associates Ltd, UK
Anne Littlewood
Information Specialist
Cochrane Oral Health, the University of Manchester
Evan Mayo-Wilson, MPA, DPhil
Associate Professor, Department of Epidemiology and Biostatistics
Indiana University School of Public Health-Bloomington
Lorri Puil, MD, PhD
Editor, Cochrane Hypertension
Department of Anesthesiology, Pharmacology & Therapeutics
Faculty of Medicine, University of British Columbia
Teo Quay
Managing Editor (Cochrane Anaesthesia and Cochrane Emergency and Critical Care)
Cochrane
Tarang Sharma
Editorial Officer
Cochrane
Farhad Shokraneh
Information Specialist
Cochrane Schizophrenia Group, Institute of Mental Health, Division of Psychiatry and Applied Psychology, School of Medicine, University of Nottingham, UK
Nicole Skoetz
Senior Editor Cochrane Cancer
Department I of Internal Medicine
University Hospital of Cologne
Professor Lesley Stewart
Director Centre for Reviews and Dissemination (CRD), University of York
Co-convenor of the Cochrane Individual Patient Data Methods Group
Emma Thompson
Advocacy and Partnership Officer
Cochrane
Sheila Wallace
Cochrane Information Specialist
Cochrane Incontinence, Institute of Health & Society, Newcastle University, UK
Competing interests:
Cochrane: Cochrane does not accept commercial or conflicted funding.
Karla Soares-Weiser: Karla Soares-Weiser is a full-time employee of Cochrane and receives a salary.
Isabelle Boutron: Isabelle Boutron is Prof of Epidemiology at Université de Paris and receives a salary from université de Paris and Assistance Publique de Hôpitaux de Paris. She is a member of the Cochrane CSR Working Group.
Yuan Chi: Yuan Chi has no employment relationship with Cochrane and is a member of the CSR Working Group in Cochrane.
Kerry Dwan: Kerry Dwan is a full-time employee of Cochrane and receives a salary, and is a member of the CSR Working Group in Cochrane.
Ella Flemyng: Ella Flemyng is a full-time employee of Cochrane and receives a salary. She is leading a project in Cochrane that aims to assess how more clinical study reports could be used in Cochrane Reviews and is setting up an open CSR Working Group in Cochrane.
Kit Gilchrist: Kit Gilchrist is a full-time employee of Cochrane and receives a salary, and is a member of the CSR Working Group in Cochrane.
Marius Goldkuhle: Marius Goldkuhle is not employed by Cochrane. He is a member of Cochrane Haematological Malignancies and the CSR Working Group within Cochrane.
Toby Lasserson: Toby Lasserson is a full-time employee of Cochrane and receives a salary, and is a member of the CSR Working Group in Cochrane.
Carol Lefebvre: Carol Lefebvre serves Cochrane part-time as Lead Convenor, Cochrane Information Retrieval Methods Group but receives no salary. She is an independent information consultant and receives fees for consultancy and teaching in information retrieval for evidence synthesis. She is a member of the CSR Working Group in Cochrane.
Anne Littlewood: Anne Littlewood is a salaried employee of Cochrane Oral Health, the University of Manchester. She is a member of the Cochrane CSR Working Group.
Evan Mayo-Wilson: In the last three years, Evan Mayo-Wilson has received support from the Laura and John Arnold Foundation, Patient Centered Outcomes Research Institute, National Institutes of Health, US Food and Drug Administration, and Robert Wood Johnson Foundation. He has no other interests to declare. He is a member of the Cochrane CSR Working Group.
Lorri Puil: In the last three years, Lorri Puil has received support from the Canadian Institutes of Health Research, the British Columbia Ministry of Health via the Therapeutics Initiative, Faculty of Medicine, University of British Columbia, and a Michael Smith Foundation for Health Research Reach grant. She is also part of a research group receiving funding from the Australian National Health and Medical Research Council, and a member of the CSR Working Group in Cochrane.
Teo Quay: Teo Quay is a full-time employee of Cochrane Anaesthesia and Cochrane Emergency and Critical Care. She is a member of the CSR Working Group in Cochrane.
Tarang Sharma: Tarang Sharma is a full-time employee of Cochrane and receives a salary, and is a member of the CSR Working Group in Cochrane. She has previously undertaken research using CSRs as source documents for reviews.
Farhad Shokraneh: Farhad Shokraneh is a member of the Cochrane CSR Working Group. No other competing interests to declare.
Nicole Skoetz: Nicole Skoetz a member of the Cochrane CSR Working Group. No other competing interests to declare.
Lesley Stewart: Lesley Stewart is employed by the University of York and has no financial conflicts of interest. She advocates data sharing and has previously used data from CSRs in her research. She is a member of the Cochrane CSR Working Group.
Emma Thomson: Emma Thomson is a full-time employee of Cochrane and receives a salary, and is a member of the CSR Working Group in Cochrane.
Sheila Wallace: Sheila Wallace is a full-time employee of Cochrane Incontinence (employed via the Group’s host Newcastle University) and receives a salary, and is a member of the CSR Working Group in Cochrane. The single largest funder of Cochrane Incontinence is the National Institute for Health Research (NIHR). The views and opinions expressed herein are those of the signatory and do not necessarily reflect those of Newcastle University, the Systematic Reviews Programme, NIHR, NHS or the Department of Health.