Summary Table of FY2020 ORI Grants

Conferences on Research Integrity
Principal Investigator	Grantee	Conference Title
*Bettie M. Steinberg, PhD, MS	The Feinstein Institutes for Medical Research	Driving Responsible Conduct of Research During a Pandemic
Research Integrity Program Development and Evaluation
Principal Investigator	Grantee	Title
*Geeta K. Swamy, MD	Duke University	Affecting Culture Change: Building Unit-Level Infrastructure to Mitigate Research Integrity Risk
Daniel E. Acuña, PhD, MS	Syracuse University	Large-scale High-Quality Labeled Datasets and Competitions to Advance Artificial Intelligence for Computational Research Integrity
Research on Research Integrity
Principal Investigator	Grantee	Study Title
Christopher W. Jones, MD	Cooper University Hospital	Biased COVID-19 Study Reporting in Preprints Versus Peer Reviewed Publications
*Erin Rothwell, PhD, MS	University of Utah	The Roles of Targeted Education and Moral Responsibility for Responsible Conduct of Research Training
Sergio G. Litewka, MD, MPH	University of Miami School of Medicine	Inventory and Analyses of Latin American Policies on Research Integrity

* First-time ORI Awardee

Abstracts

Conferences on Research Integrity

Number of Grants: 1

Conference Title: Driving Responsible Conduct of Research During a Pandemic
Dr. B. Steinberg
The Feinstein Institutes for Medical Research

Abstract: Conducting research in healthcare settings during a national public health emergency, such as a pandemic, presents unique challenges. Ongoing studies may need to be modified, suspended, or terminated, and protocol deviations are likely to occur due to re-prioritization of resources and on-site-restrictions. A typical healthcare environment will often transform into a research focused setting with an urgent need for research on new treatments, especially when responding to novel pathogens. Other challenges pertain to consenting critically ill patients in research and ensuring efforts include diverse populations. Rapid publication efforts can raise concerns over data integrity and far reaching impacts of false information. The best way to mitigate these challenges is to identify the risks unique to research integrity during pandemics and then establish a clear set of guidelines that can be used to prevent or manage them effectively. Implementing these guidelines can maximize the value of the research by promoting practices that reinforce scientific rigor while maintaining high ethical standards. Ensuring the responsible conduct of research will maintain the public’s trust in data produced by healthcare organizations and provide meaningful knowledge that can be used to inform other researchers, healthcare providers and regulatory agencies (such as the Food and Drug Administration) on approaches to future research and healthcare delivery. The Feinstein Institutes for Medical Research, Northwell Health, in partnership with other health systems, academic medical centers and healthcare related organizations in the greater New York metropolitan area, will present a virtual, interactive conference dedicated to this topic, using lessons learned during the current COVID-19 pandemic. The goals of the conference are to: 1) Identify the major areas of vulnerability to ethical and responsible conduct of research during a pandemic or other public health care emergencies; 2) Identify a set of best practices to promote the responsible conduct of research that could minimize likelihood of deviations; 3) Disseminate the findings of the conference regionally, nationally and internationally through multiple mechanisms.

Research Integrity Program Development and Evaluation

Number of Grants: 2

Study Title: Affecting Culture Change: Building Unit-Level Infrastructure to Mitigate Research Integrity Risk
Geeta Swamy
Duke University

Abstract: To more effectively and comprehensively foster Duke’s culture of research integrity, this project utilizes and expands two recent additions to our integrity infrastructure, the Research Quality Management Program (RQMP) and the Corrective Action Reviews (CAReS) initiative. This will enable the institution to 1) promote Responsible Conduct of Research (RCR) programming at the unit level, rather than at the level of individuals or based on the participants’ role/position, 2) encourage within-unit dialogue and communication about institutional values and expectations, 3) facilitate assessment of group cultures and their research practices, 4) help identify individuals or groups at risk of integrity failures, and 5) support development and implementation of early interventions as needed. The Empowerment Phase will focus on providing unit-level Research Quality Teams (RQTs) with resources and hands-on training to better enable their active participation in detecting, mitigating, and preventing detrimental research practices (DRPs) and research misconduct. This will also serve to develop the collaboration between the Duke Office of Scientific Integrity (DOSI) and the RQTs, allowing for more coordinated efforts when integrity concerns arise. The Implementation Phase will focus on establishing a more rigorous implementation strategy for CAReS activities which involve the discharge of corrective actions, as well as building a proactive component in the CAReS team to assist individuals or groups at risk of integrity failures. In addition, completion of Part 1 will enable facile collaboration between the CAReS team and RQTs to strengthen the corrective action review and implementation process.

Study Title: Large-scale High-Quality Labeled Datasets and Competitions to Advance Artificial Intelligence for Computational Research Integrity
Daniel Acuña
Syracuse University

Abstract: Authors, publishers, and funders agree that scientific misconduct is a problem and that automated tools could alleviate it. Tools can help in scalability and accuracy. As an example, text reuse detection tools have demonstrated that automated methods do help with scalability. We have all become accustomed to receiving immediate feedback on whether a manuscript has text reuses across millions of scientific papers and the web. But text reuse detection tools might need significant improvements to increase accuracy—i.e., to claim that they have detected plagiarism, a serious accusation. In general, plagiarism tools tell us how much text or image patches appear elsewhere, but it does not tell us whether those appearances are problematic. When teaching an Artificial Intelligence (AI) agent to classify whether a text or image reuse is problematic, we need labeled data from experts who have contextualized the cases. These types of labeled data for scientific misconduct, however, are difficult to come by.  Moreover, given how complex misconduct verdicts are, we suspect that these AI systems would need dramatically richer and larger datasets that ever built before, and perhaps incorporate full texts, citations, captions, and past author collaborations. The overall objective of this application is the development of a large-scale high-quality labeled dataset of known potential misconduct cases of scientific image manipulations, violations of graphical integrity, and text plagiarism. Simultaneously, another objective is to organize an open challenge where researchers compete to build AI models to achieve the highest performance when predicting these labeled datasets. Aim 1. Create a large-scale high-quality labeled dataset with machine readable format of common research misconduct cases. Aim 2. Organize a community challenge where researchers compete to develop Artificial Intelligence methods to detect misconduct.

Research on Research Integrity

Number of Grants: 3

Study Title: Biased COVID-19 Study Reporting in Preprints Versus Peer Reviewed Publications 
Christopher W. Jones
Cooper University Hospital

Abstract: The Coronavirus disease 2019 (COVID-19) pandemic triggered a massive effort from the biomedical research community aimed at rapidly identifying and testing multiple potential therapeutic and preventative interventions. In an effort to disseminate results from clinical research studies as quickly as possible to other researchers, policy makers, and healthcare providers, many study investigators have chosen to release study findings as preprints either prior to, or instead of, submitting manuscripts for peer review by a scientific journal. Because preprints do not undergo peer review or evaluation through a traditional editorial process, use of this format can substantially speed public release of study results. This is particularly important early in a pandemic caused by a novel pathogen such as COVID-19, where very few interventions are known to be effective. This increased speed is only beneficial, however, if the information being disseminated is reliable. By publicly sharing clinical research findings without peer review, there is risk that preprints may compromise the integrity of the research process by allowing poorly conducted or inaccurately reported research to influence clinical practice and policy. The magnitude of this risk has not been previously defined. This study addresses this knowledge gap by proposing a cross-sectional analysis of preprints reporting results from clinical studies assessing interventions for COVID-19. Both observational studies and clinical trials will be included. Investigators will determine the prevalence of misleading language (i.e., “spin”) within preprint abstracts, as well as the prevalence of inconsistencies between prospectively registered primary study outcomes and primary outcomes reported in preprints. Additionally, we will search for peer reviewed publications corresponding to each included preprint, and will compare the prevalence of both spin and outcome inconsistencies between preprints and peer reviewed publications. The overarching goal of this work is to advance our understanding of the presence of compromised research reporting within the rapidly expanding domain of medical preprints. In doing so, the proposed study will inform interpretation of the medical literature relevant to COVID-19, will provide the scientific community with knowledge critical to helping ensure rapid and accurate dissemination of clinical research findings during the early phases of subsequent pandemic events, and will help guide interpretation of medical preprints in general.

Study Title 2 (PI; Grantee): The Roles of Targeted Education and Moral Responsibility for Responsible Conduct of Research Training Erin Rothwell
University of Utah

Abstract: There is growing empirical evidence that the research climate influences the way research teams conduct their research on a regular basis. This evidence describes the impact of the research climate on appropriate and questionable research practices (QRPs). It is important to note that broader, more common research misbehaviors (QRPs) are equally or even more harmful to the integrity of the research process than research misconduct (i.e., fabrication, falsification and plagiarism). Education in responsible conduct of research (RCR) is supported as a mechanism to educate, train, and promote integrity throughout the research process. However, there is a lack of consensus on what RCR training should entail, how it should be delivered, and what constitutes effectiveness in the delivery of information. This high variability about the content, method of delivery, and outcomes for RCR raises significant barriers for how institutions should be delivering research education. The typical format for RCR training has been the traditional classroom, but online courses or informal mentoring are also used. Recent evidence suggests RCR training needs to be targeted and conducted in the context in which the research is occurring to address the research climate and to contextualize appropriate research behaviors. Further, there are different conceptual models for how to deliver RCR. A research ethics framework addresses what is a scientist’s moral responsibility in the honest, accurate, and fair conduct of their research. A research integrity framework uses professional standards to identify practices, knowledge, and adherence to RCR. Some scholars, however, argue that addressing moral responsibility may be more effective in the specific context of the research activities, but the impact of these two difference conceptual approaches to promoting RCR and reducing QRPs is unknown. This proposal will assess how 1) targeted RCR education within research teams may differ from traditional classroom formats and 2) RCR education about moral responsibility compares to RCR education about professional standards on perceptions of the research climate, QRPs and ethical decision making. Outcomes of this research will provide empirical evidence for institutions to improve their research climate and promote the integrity of the research process for investigators.

Study Title: Inventory and Analyses of Latin American Policies on Research Integrity
Sergio Litewka
University of Miami School of Medicine

Abstract: The overarching goal of this project is to identify and evaluate governmental and institutional policies for protecting and fostering research integrity in academic biomedical research in Latin America, specifically in those Spanish-speaking countries that receive U.S. federal research funds, and to compare such policies and instruments with standards and existing policies in the United States. This project will bring together upper-level administrators, research educators, and investigators from Latin America’s leading research universities and representatives from the region’s national funding agencies. This project has three specific aims: 1) To create an inventory of regulations and policies for fostering scientific integrity and preventing and managing research misconduct in Argentina, Colombia, Costa Rica, Mexico, and Peru, and develop an online repository of these materials; 2) To compare these regulations and institutional policies to those present in the U.S.; and 3) To build on the multidisciplinary network of Latin American academic researchers, educators, and administrators actively engaged in our previous ORI-funded activities in the region. Latin American collaborators from the target countries, all researchers and educators active in promoting research integrity, will work with the PI and the study team to review policy documents and relevant statements from their home institutions, national and international professional societies, research funders, and governmental agencies. Together we will create a repository of Latin American regulations and policies on research integrity, analyze the documents to identify Latin American concerns about misconduct and questionable research practices, and compare these norms and issues with existing U.S. policy standards.