When IRBs Say No to Participating in Research about Single IRBs
(Click DOI to download directly)
In response to a policy of the National Institutes of Health and requirements in the revised Common Rule, a protocol for a multisite study must be reviewed by a single institutional review board (IRB), rather than by the IRB at each study site. The goal of the single IRB approach is to increase the efficiency of IRB review of multisite research without jeopardizing protections for research subjects. Yet the extent to which these joint goals are being achieved is unclear. To better understand how single IRBs function, we recruited academic, government, and commercial single IRBs (N = 49) to participate in a study involving observation of protocol review meetings and/or interviews with their members, chairs, and administrators. Twenty (40.8%) agreed to participate, of which 50% agreed to both interviews and observation. While 81.8% (9/11) of academic and 50% (4/8) of government single IRBs participated in some way, only 23.3% (7/30) of commercial single IRBs did so. The four largest commercial single IRBs declined to participate. Because evaluation of single IRBs is important to inform development, implementation, monitoring, and refinement of federal policies, single IRBs should be encouraged to participate in research that examines how they function.
In response to a policy of the National Institutes of Health (NIH)1 and requirements in the revised Common Rule,2 a protocol for a multisite study must be reviewed by a single institutional review board, rather than by the institutional review board (IRB) at each study site. This single IRB approach aims to increase the efficiency of IRB review of multisite research without jeopardizing protections for research subjects. Because the extent to which these goals are being achieved is unclear, we sought to enroll commercial (that is, for‐profit) single IRBs and single IRBs at academic institutions and government agencies in an observational and interview study to assess challenges they face in implementing the new single IRB mandates. In previous publications, we reported findings about how single IRBs address issues about conflicts of interest,3 local knowledge,4 and reliance agreements.5 Here we report findings about single IRBs’ willingness or unwillingness to participate in research that investigates how they function.
Our Study about Single IRBs
As discussed in other papers,6 to identify eligible single IRBs to recruit for our study, we relied on lists of relevant IRBs from the Association for the Accreditation of Human Research Protection Programs (AAHRPP),7 the Office for Human Research Protections (OHRP) (using its registration portal for IRBs),8 Citizens for Responsible Care and Research (which compiles a list of commercial IRBs),9 Public Responsibility in Medicine and Research (PRIM&R) (using its website),10 the relevant published literature, and personal communications with various IRBs. We excluded IRBs that did not currently serve as single IRBs for multisite studies, were inactive, were not registered with OHRP, or were outside the United States. The IRBs at the University of Massachusetts Medical School and the New York State Psychiatric Institute approved the study.
From March 2015 until August 2017, we emailed an introductory letter to a senior administrator of all identified single IRBs self‐characterized as academic, government based, or commercial (which appeared to represent all U.S. single IRBs), explaining that the purpose of our study was to investigate single IRBs’ “deliberations as they review multi‐site studies” and that the study would involve “observing two protocol review meetings and conducting interviews with senior administrative personnel and chairs, primary reviewers of proposals, and staff members involved in the processing of applications at those IRBs.” The letter also said we would appreciate the opportunity to tell them more about the study to help them determine if they would be willing to be involved and that we would call them “in the near future to talk more about this.” If, during a subsequent phone call with the contact person, we were informed that the IRB did not want to participate in the part of the study involving observation of single IRB protocol review meetings, we then offered the possibility of participating only in the interview portion of the study. We explained to potential interviewees that the interviews would last 30 to 60 minutes, that they concerned “your experiences of reviewing research studies,” and that interviewees would receive $20.00. We also attached a copy of a fact sheet that provided additional details about the study, including its procedures for protecting confidentiality of participants and their responses.
For the one‐on‐one interviews, we obtained informed consent from individual interviewees. For observations of meetings, we obtained informed consent from the administrator or chair of the single IRB and subsequently provided an information sheet to all present in the room and described the study. We stated that any individuals in the room who did not want to be study participants could tell us in person or by email and that we would redact any comments they made. However, no individuals requested such redaction. We assured participants of confidentiality, including removal of identifying information. We also asked administrators who refused to allow us to observe single IRB meetings and/or to conduct interviews to indicate why.
We performed a three‐way comparison of participation among commercial, academic, and government‐funded single IRBs (using a two‐sided Pearson chi‐square test) and two‐way comparisons (using two‐sided Fisher exact tests) of commercial versus noncommercial (i.e., academic and government‐funded) single IRBs and of academic versus government‐funded single IRBs. For all analyses, we used SPSS Version 24, with P < .05 indicating statistical significance.
Of the 49 single IRBs contacted, 11 were academic, 8 government, and 30 commercial, including the four
commercial single IRBs that were the largest at that time. Twenty (40.8%) agreed to participate, and ten of these (50%) agreed to both interviews and board meeting observation (see table S1, which is available online; see the “Supporting Information” section at the end of this article). While 81.8% (9/11) of academic and 50% (4/8) of government single IRBs participated in some way, only 23.3% (7/30) of commercial single IRBs did so. The four largest commercial single IRBs all refused to participate. Participation differed significantly among the three types of single IRBs (p < 0.003) and between commercial and noncommercial single IRBs (p < 0.003). Academic and government single IRBs did not differ significantly (though the sample sizes are relatively small for statistical analysis).
The reluctance of single IRBs—particularly large commercial single IRBs—to participate in research raises critical concerns about researchers’ ability to evaluate how these boards function and how effectively they protect research participants.
Of the 39 single IRBs refusing full participation (board observations and individual interviews), 18 (46.2%) gave no reason. Reasons offered by the others included privacy or confidentiality concerns and insufficient time or resources (see table 1). Several commercial single IRB administrators expressed interest in participating but reported that they first needed permission from their companies’ owners, who then refused, citing a lack of any financial or other business advantages to be gained from participation. Of these single IRBs, 25% (5) and 75% (15) reviewed high and low annual volumes of protocols, respectively; and 30% (6) reviewed protocols only about certain specific diseases, while 70% (14) did not.
|Rejection by the IRB board of the invitation to participate||4||10.3%|
|Privacy or confidentiality concerns (e.g., “propriety information,” “loss of clientele”)||4||10.3%|
|Insufficient time or resources||7||17.9%|
|Other (e.g., no gains and some risks from participation)||6||15.3%|
IRBs as Participants in Research
Several studies have provided limited data on the structure and functioning of IRBs, generally relying on self‐reported quantitative information, and focusing only on a few metrics (such as sociodemographics of members, numbers of protocols reviewed, costs, and time to completion of review).11 Yet self‐reported data, in general, may be incomplete or biased: affected by time and resource pressures, social desirability bias (reflecting respondents’ desire to “look good” to researchers), and competitive concerns.12 In a review of 14,275 health‐related studies, for example, only 0.2% assessed social desirability bias, and 43% of these found that this bias had indeed affected the study's results.13
Important questions remain about many aspects of how IRBs make decisions and view their roles. However, IRBs have been difficult to study observationally, as suggested by the relatively few studies of them over the nearly half‐century of their existence. In a prior study of local IRBs through observations of IRB meetings and interviews with personnel (chairs, members, and staff members), we encountered some resistance to participation and ultimately succeeded in recruiting IRBs at ten medical centers, partly by compensating a local investigator at each medical center for assisting with recruitment. But approximately half the IRBs we approached declined to participate.14 We did not inquire about or collect reasons for refusal in that study. Other researchers attempting to observe more than two or three IRBs have encountered similar difficulties,15 and little information is available about why IRBs do not participate in research about IRBs.
To our knowledge, our study reported here was the first to examine issues regarding single IRBs’ participation in research. Of the 49 single IRBs we recruited, only 40.8% participated in some way in this study of their functioning. The fact that only 50% of government single IRBs participated in our study is concerning, as transparency is vital when public funds are used in the review and oversight of research with humans. Commercial single IRBs were significantly more likely than academic or government single IRBs to refuse to participate. Yet, importantly, commercial single IRBs conduct a large proportion of the single IRB reviews of multisite clinical trials, with one large commercial single IRB alone reviewing 89% of the studies of drugs approved by the U.S. Food and Drug Administration in 201516 and reporting that it has more than 2,700 academic medical center‐associated partners.17 Given the complexities of establishing, staffing, and managing single IRBs for multiple study sites, many academic medical centers are using commercial single IRBs.
The reluctance of single IRBs—particularly large commercial single IRBs—to participate in research raises critical concerns about transparency and the ability of researchers to evaluate these boards’ functioning and effectiveness in protecting research participants. Changes in the nature of commercial single IRBs heighten these concerns. As John Isidor, former chief executive officer of Schulman Associates, one of the largest commercial single IRBs, pointed out in 2014, “[R]ight now, you really only see four major IRBs: WCG [WIRB‐Copernicus Group], Schulman, Quorum … , and Chesapeake.” “The family‐owned nature of the IRBs,” he observed, “is dying rather quickly with the private equity firms entering.”18 In 2017, Shulman and Chesapeake merged to create Advarra; in 2019, Advarra acquired Quorum, leaving only two very large commercial IRBs.19
Research on how single IRBs function is crucial in assessing how best to develop, implement, monitor, and refine policies to improve the single IRB system, whether through institutional and governmental policies or regulatory changes. Moreover, understanding the effectiveness of single IRBs in protecting human subjects is essential to furthering the intent for which IRBs were created and the overall ability of the current human research protection system to optimally achieve those goals. Refusals by IRBs to participate in research about IRBs also run counter to the ethos of science and medicine more broadly—an ethic of transparency and of sharing information to advance science, aid patients, and improve care.20 Any aspect of the research enterprise that is not firmly committed to this critical principle and to efforts to enhance the current system of oversight should be of concern.
Single IRBs may fear participation in research on their functioning because of worries that the results might reveal shortcomings. Thus, researchers conducting studies of single IRBs must provide full confidentiality for the individual participants and their organizations, just as IRBs routinely require researchers to provide such protection to participants in other types of human subjects research. Particular protection is needed to avoid disclosure of proprietary or sensitive information.
Our difficulty in getting single IRBs to participate in research about their review of protocols for multisite studies was limited to a single study. However, refusal to participate in our single study was common, even though participation required, at minimum, only one relatively brief interview. Single IRBs may have been willing to participate in other studies, although we identified no published reports of findings from such studies.
Evaluation of single IRBs, as major components now of the human research protection system, is crucial to inform development, implementation, monitoring, and refinement of federal policies regarding the protection of human subjects. OHRP and accrediting bodies such as AAHRPP should encourage single IRBs to participate in research studies about IRBs as much as possible. Moreover, the NIH and the U.S. Department of Health and Human Services (which has jurisdiction over the Common Rule) should consider requiring that single IRBs undergo outside review about their functioning. Possible other means of increasing the participation of these IRBs in research should also be considered.
IRBs have been criticized for working “behind closed doors.”21 Critics have argued that IRBs need to be more transparent to investigators whose work is being reviewed and to study participants, sponsors, regulators, the public at large, and researchers studying IRBs.22 Concerns about transparency, efficiency, and effectiveness apply to all IRBs, but are especially important regarding single IRBs since these new entities play a critical role in reviewing multisite studies and ensuring the protection of research participants in those studies.
ACKNOWLEDGMENTS AND DISCLAIMER
We thank Sayantanee Das, Jiseop Kim, and especially Patricia Contino for their assistance with the preparation of this manuscript. This work was supported by the National Institute of General Medical Sciences through grant 5R01GM113640‐03 (with Charles Lidz as the principal investigator). The funders or sponsors of this work had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Klitzman, Lidz, Pivovarova, and Murray had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
SUPPORTING INFORMATION—TABLE S1
Table S1 is available in the “Supporting Information” section for the online version of this article and via Ethics & Human Research's “Supporting Information” page: https://www.thehastingscenter.org/supporting-information-ehr/.
|eahr500041-sup-0001-S1.pdf50.5 KB||Supporting Information|
Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
- 1“ Single IRB Policy for Multi-site Research,” National Institutes of Health, accessed November 5, 2019, https://grants.nih.gov/policy/humansubjects/single-irb-policy-multi-site-research.htm.
- 2“ Federal Policy for the Protection of Human Subjects: Final Rule,” Federal Register 82, no. 12 (2017): 7149– 7274.
- 3, et al., “How Single Institutional Review Boards Manage Their Own Conflicts of Interest: Findings from a National Interview Study,” Academic Medicine (April 16, 2019) (epub ahead of print), doi:10.1097/ACM.0000000000002762.
- 4, et al., “ Local Knowledge and Single IRBs for Multisite Studies: Challenges and Solutions,” Ethics & Human Research 41, no. 1 (2019): 22– 31.
- 5, et al., “ Reliance Agreements and Single IRB Review of Multisite Research: Concerns of IRB Members and Staff,” AJOB Empirical Bioethics 9, no. 3 (2018): 164– 72.
- 6Ibid.; Pivovarova et al., “ How Single Institutional Review Boards Manage Their Own Conflicts of Interest”; Klitzman et al., “Local Knowledge and Single IRBs for Multisite Studies.”
- 7 Association for the Accreditation of Human Research Protection Programs, 2018, http://www.aahrpp.org/.
- 8“ IRB Registration,” Office for Human Research Protections, 2018, https://www.hhs.gov/ohrp/register-irbs-and-obtain-fwas/irb-registration/index.html.
- 9 Citizens for Responsible Care and Research, https://www.circare.org.
- 10 Public Responsibility in Medicine & Research, 2018, https://www.primr.org/.
- 11, et al., “ A Scoping Review of Empirical Research Relating to Quality and Effectiveness of Research Ethics Review,” PLoS One 10, no. 7 (2015): e0133639.
- 12, , and , “ Measuring Bias in Self-Reported Data,” International Journal of Behavioural and Healthcare Research 2, no. 4 (2011): 320– 32; Althubaiti, A., “Information Bias in Health Research: Definition, Pitfalls, and Adjustment Methods,” Journal of Multidisciplinary Healthcare 4, no. 9 (2016): 211-17.
- 13, “ Faking It: Social Desirability Response Bias in Self-Report Research,” Australian Journal of Advanced Nursing 25, no. 4 (2008): 40– 48.
- 14, et al., “ IRBs: How Closely Do They Follow the Common Rule?,” Academic Medicine 87, no. 7 (2012): 969– 74.
- 15, written communication to Robert Klitzman, December 2018.
- 16, “In Clinical Trials, For-Profit Review Boards Are Taking Over for Hospitals. Should They?,” STAT, July 6, 2016, at https://www.statnews.com.
- 17 WIRB-Copernicus Group, https://www.wcgclinical.com/about/.
- 18“ The Changing World of Independent IRBs,” Relias Media, October 1, 2014, https://www.reliasmedia.com/articles/21652-the-changing-world-of-independent-irbs.
- 19“ Chesapeake IRB and Schulman IRB Merge to Establish Premier Independent Institutional Review Board for Research,” Advarra News, November 7, 2018, https://www.advarra.com/about-advarra/news/chesapeake-irb-and-schulman-irb-merge-to-establish-premier-independent-institutional-review-board-for-research/; “About,” Arsenal Capital Partners, https://www.arsenalcapital.com/about; “IRB Consolidation Continues with Advarra Gobbling Quorum Review IRB & Kinetiq Research,” TrialSite News, March 6, 2019, https://www.trialsitenews.com/irb-consolidation-continues-with-advarra-gobbling-quorum-review-irb-kinetiq-research/.
- 20, “ The Normative Structure of Science,” in The Sociology of Science: Theoretical and Empirical Investigations, ed. R. K. Merton ( Chicago: University of Chicago Press, 1973), 267– 78.
- 21, Behind Closed Doors: IRBs and the Making of Ethical Research ( Chicago: University of Chicago Press, 2012).
- 22, The Ethics Police: The Struggle to Make Human Research Safe ( New York: Oxford University Press, 2015); Fernandez Lynch, H., “Opening Closed Doors: Promoting IRB Transparency,” Journal of Law, Medicine & Ethics 46, no. 1 (2018): 145-58.