Responsible Conduct of Research (RCR) Instruction Delphi Panel Results
Method
Panel 3 convened RCR and research experts to address the following question:
Within RCR instructional programs, what specific topics should be taught and assessed in the core areas of:
1. Mentor/trainee responsibilities,
2. Collaborative science, and
3. Conflicts of interest and commitment?
The panel used a version of the Delphi method to achieve consensus. Panelists were asked to complete three successive online questionnaires. All responses were anonymous.
Round 1 consisted of an open-response format. Participants were directed to a website which asked them to list, in corresponding text boxes, at least five specific topics in each of the core areas under consideration. After each participant had completed round 1, their responses were carefully condensed, re-worded and organized into topics and subtopics to enhance clarity and prevent redundancy.
Round 2 involved presenting participants the lists of topics they had generated and asking them to evaluate the importance of teaching each topic in an RCR course. Participants were asked to rate the importance of teaching each topic or subtopic on a four-point scale (1 = Unimportant, 2 = Less important, 3 = Important, 4 = Very important). Participants were also asked to make additional comments about each item concerning wording or clarity. Topics receiving a vote of “Important” or “Very Important” from at least two-thirds of participants were deemed to meet consensus criteria and were presented to participants in the next round, after they were revised according to the participants comments. Topics not meeting consensus are nevertheless displayed in the tables below, with their corresponding consensus values and mean scores.
Round 3 added to Round 2 by re-asking participants the importance of teaching each item, and also asked participants to rate the importance of assessing each item within an RCR course. Assessment rankings followed the same four-point scale used in the previous round. For each question (one on teaching, one on assessing), each topic receiving a vote of “Important” or “Very Important” from at least two-thirds of participants was deemed to meet consensus criteria, and is labeled with an asterisk in the table below. The consensus value and the mean score for each topic and subtopic are also shown.
Results: Panel 3
0BCore Area II. Mentor/trainee Responsibilities
| Topic (Subtopics indented) | Percentage of participants rating item as “important” or “very important” (Mean score) | |
|---|---|---|
| Teaching | Assessing | |
| 1. Definitions and expectations of the mentor/trainee relationship | 100* (3.75) | 36 (2.36) |
| a. Defining research advisors, mentors, and trainees—across a variety of settings including degree programs, postdoctoral training, and jobs | 42 (2.50) | N/AP= |
| b. Boundaries of the mentor/trainee relationship | 100* (3.58) | 45 (2.36) |
| 2. Power relationships and the potential problems they involve | 100* (3.58) | 40 (2.50) |
| a. Power structures and hierarchical relationships within science and the mentor-trainee relationship | 92* (3.33) | 25 (2.08) |
| b. Friendships and mentoring relationships | 42 (2.50) | N/AP= |
| c. Harassment, sexual and other types | 67* (3.08) | 42 (2.42) |
| 3. Scientific responsibilities of the mentor | 100* (3.42) | 50 (2.58) |
| a. Promoting professional research skills, including identifying research questions, writing proposals, conducting research, and publishing | 92* (3.17) | 33 (2.42) |
| b. Fostering research compliance (IRB, IUCUC, etc.), RCR, and integrity | 100* (3.58) | 75* (3.08) |
| c. Finding funding and negotiating grants and contracts | 33 (2.25) | N/AP= |
| d. Sharing discipline-specific wisdom on how to operate in the field | 33 (2.33) | N/AP= |
| 4. Non-scientific responsibilities or roles of the mentor | 67* (2.92) | 42 (2.08) |
| a. Career counseling, including trainees with science and non-science career goals | 42 (2.42) | N/AP= |
| b. Conflict resolution | 67* (3.00) | 25 (2.00) |
| c. Fostering autonomy with trainees while accomplishing mentor’s goals | 67* (2.92) | 25 (2.17) |
| d. Management skills | 42 (2.50) | N/AP= |
| 5. Responsibilities of trainees within the mentor-trainee relationship | 100* (3.42) | 45 (2.45) |
| a. Work with integrity | 100* (3.42) | 42 (2.42) |
| b. Willingness to blow whistle or challenge misconduct and questionable conduct | 100* (3.50) | 36 (2.36) |
| 6. How to get the most out of the mentor/trainee experience | 58 (2.67) | N/AP= |
| a. Optimal characteristics of mentors and trainees | 58 (2.58) | N/AP= |
| b. Effective mentoring strategies and characteristics | 83* (3.08) | 27 (2.27) |
| c. Contracting for a good mentoring relationship | 33 (2.33) | N/AP= |
| 7. Addressing challenges and problems in the mentor-trainee relationship | 100* (3.25) | 27 (2.27) |
| a. Conscientious refusal | 58 (2.58) | 8 (1.83) |
| b. Importance of clear communication of expectations | 100* (3.25) | 33 (2.42) |
| c. Dealing with diversity of cultures, races, and other personal traits | 92* (3.25) | 42 (2.33) |
Legend:
* = Item achieved a “consensus” by receiving a rating of important or very important from two-thirds of participants
P=P = Not applicable because these items were eliminated after round 2 and their importance of being assessed was not measured
1BCore Area V. Collaborative Science
| Topic (Subtopics indented) | Percentage of participants rating item as “important” or “very important” (Mean score) | |
|---|---|---|
| Teaching | Assessing | |
| 1. The nature and advantages of successful collaborations | 83* (3.17) | 50 (2.50) |
| a. Reasons for collaborating | 58 (2.83) | N/AP= |
| b. Risks and benefits of collaborations | 75* (3.08) | 42 (2.17) |
| c. Identifying a good collaborator | 83* (3.08) | 33 (2.33) |
| 2. Types of collaboration | 63 (2.73) | 22 (2.00) |
| a. Collaboration within an institution | 67* (2.75) | 17 (2.00) |
| b. Collaboration between institutions | 58 (2.67) | 8 (1.83) |
| c. International collaboration | 58 (2.83) | N/AP= |
| 3. Working well with others | 92* (3.25) | 27 (2.27) |
| a. Identifying the authority and procedures for establishing collaborative relationships | 92* (3.00) | 33 (2.25) |
| b. Defining and clarifying roles, responsibilities, and expectations in a collaboration | 100* (3.42) | 33 (2.42) |
| c. Identifying mechanisms for ongoing decision-making | 75* (2.92) | 25 (2.17) |
| d. When are written agreements necessary, and what should be addressed in contracts | 92* (3.25) | 75* (2.75) |
| e. Knowing how and when to end collaborative relationships | 83* (3.00) | 33 (2.17) |
| 4. Dealing with challenges in collaborative relationships | 100* (3.40) | 40 (2.50) |
| a. Addressing failures in RCR or research integrity | 83* (3.33) | 82* (2.73) |
| b. Allocating rewards such as credit, authorship, ownership, and rights of use | 100* (3.58) | 83* (3.08) |
| c. Dealing with competition | 50 (2.58) | N/AP= |
| d. Addressing power discrepancies when junior scientists collaborate with senior scientists | 75* (3.00) | 50 (2.58) |
| 5. The role of institutions in collaborative science | 58 (2.67) | N/AP= |
| a. Working with appropriate officers | 50 (2.58) | N/AP= |
| b. Knowledge of institutional policies | 83* (3.08) | 50 (2.58) |
Legend:
* = Item achieved a “consensus” by receiving a rating of important or very important from two-thirds of participants
P=P = Not applicable because these items were eliminated after round 2 and their of being assessed was not measured
2BCore Area IX. Conflicts of Interest and Commitment
| Topic (Subtopics indented) | Percentage of participants rating item as “important” or “very important” (Mean score) | |
|---|---|---|
| Teaching | Assessing | |
| 1. The significance of conflicts of interest | 100* (3.73) | 75* (2.83) |
| a. Historical examples of conflicts of interest in science | 50 (2.58) | N/AP= |
| b. Psychology and conflicts of interest, i.e., how conflicts of interest may cloud judgment or influence decisions | 83* (3.33) | 50 (2.42) |
| c. The pervasiveness of conflicts of interest, including sponsored research | 83* (3.25) | 33 (2.17) |
| d. Consequences of conflicts for researchers, institutions, students and research participants | 92* (3.33) | 33 (2.33) |
| e. Why conflicts of interest are pervasive and not always bad | 83* (3.42) | 50 (2.50) |
| 2. Types, definitions, and examples of conflicts of interest | 100* (3.55) | 50 (2.50) |
| a. Financial conflicts of interest, including gifts and honoraria, patents, spin off companies, SBIR/STTRs, personal investments, funding contracts with industry | 92* (3.58) | 67* (3.00) |
| b. Non-financial conflicts of interest (e.g., recognition, publications, promotions) | 58 (2.67) | N/AP= |
| c. Role conflicts (e.g., physician-researcher or teacher-researcher) and conflicting duties to self, clients, institutions and society | 75* (3.08) | 50 (2.42) |
| d. Conflicts of interest are objective relationships—they do not imply actual or intended wrong doing | 83* (3.00) | 33 (2.25) |
| 3. Conflicts of commitment (i.e., dividing one’s percent effort within a job)—definition, examples, and management | 58 (2.67) | N/AP= |
| a. Effort reporting rules | 58 (2.67) | N/AP= |
| b. Balancing sponsored research with other duties | 58 (2.67) | N/AP= |
| c. The perils of becoming over extended | 67* (2.92) | 33 (2.17) |
| 4. Institutional conflicts of interest | 50 (2.58) | N/AP= |
| a. Conflicted oversight (e.g., IRB and IACUC members are employees who review work of peers) | 42 (2.50) | N/AP= |
| b. Institutional investments and profits from research | 42 (2.42) | N/AP= |
| 5. Managing conflicts of interest | 100* (3.50) | 67* (2.83) |
| a. Avoiding or eliminating conflicts of interest | 100* (3.25) | 58 (2.75) |
| b. Disclosing conflicts of interest / conflicts of interest and informed consent | 100* (3.58) | 83* (3.00) |
| c. Management plans, including, e.g., role separation | 42 (2.67) | N/AP= |
| 6. Conflicts of interest law and policy | 50 (2.58) | N/AP= |
| a. Regulatory and statutory laws | 50 (2.75) | N/AP= |
| b. Institutional policies on conflicts of interest | 82* (3.27) | 55 (2.55) |
Legend:
* = Item achieved a “consensus” by receiving a rating of important or very important from two-thirds of participants
P=P = Not applicable because these items were eliminated after round 2 and their importance of being assessed was not measured
Recruitment and Panelist Biosketches
Recruitment:
Recruitment began with (1) a literature search to identify authors actively researching and publishing in RCR, (2) a review of ORI Annual Reports from 2000 through 2005 to identify those who received ORI contracts and grants, and (3) review of recent research administrative and RCR conference programs to identify those who had presented on relevant topics. Lastly, certain subgroups such as current research students, research administrators and journal editors, were identified for specific panels. From the resulting list of possible participants, the Project Director, in consultation with ORI, selected those who were both qualified to serve on a particular panel and represented diverse backgrounds. For Panel 3, experts with knowledge of and experience in the related core areas were recruited along with a select group of general RCR experts. Recruitment letters were sent to these individuals, asking them to volunteer without compensation for a total of 1.5 hours (30 minutes for each round) over approximately nine months. Those who declined participation, but represented a subgroup of interest, were asked to provide a recommendation for another possible participant. Unless otherwise noted, each of the following individuals participated in at least 2 of the 3 rounds.
Panelists:
Ken Bensch, B.S. is a student in the M.D./Ph.D. program at the Saint Louis University School of Medicine. He received is B.S. in Biological Sciences and French from Marshall University in West Virginia in 1998, and also has studied in France and England. He has worked in basic sciences research for four years prior to starting the M.D./Ph.D. program and currently conducts graduate research at the Saint Louis University School of Medicine.
David Clark, M.S., M.B.A is the Director and Research Compliance Officer for the Office of Research Assurances at Washington State University since 2002. He received his M.S in Geology from Florida State University and his M.B.A. from Augusta College. He manages aspects of compliance with state and federal laws in the university’s research policies and procedures for the Use of Human Subjects (IRB), Institutional Animal Use and Care (IACUC), Biohazards and Biosafety (IBC), and Conflict of Interest (COI). In addition to developing and implementing these policies, he also provides training and support for the university’s research community.
John E. Kaplan, Ph.D. is a Professor in the Center for Cardiovascular Sciences at Albany Medical College and Professor of Biomedical Sciences at the State University of New York School of Public Health. Dr. Kaplan received his Ph.D. from Albany Medical College in 1976. Since 1995 he has taught Introduction to Scientific Integrity and Discussions in Scientific Integrity. He is also the course director for Research Ethics: Discussions, an online graduate course, and Responsible Conduct of Research at the School of Public Health. He serves as a grant reviewer for the Office of Research Integrity’s RCR Education Resource Development Program and the RCR Program for Academic Societies of the Association of American Medical Colleges.
Gerald Koocher, Ph.D. is Professor of Psychology and Dean of the School for Health Studies at Simmons College in Boston as well as the 2006 President of the American Psychological Association. Dr. Koocher received his Ph.D. from the University of Missouri in Columbia in 1972 in Clinical and Developmental Psychology. He serves as Senior Associate in Psychology at Children’s Hospital in Boston as well as a member of the Data Safety Monitoring Board in Autism Research within the National Institute of Mental Health. Among his major research interests are ethical and public policy issues in delivery of mental health services, including malpractice in the mental health professions, ethical misconduct in research, academic integrity, quackery in mental health, and functioning of Institutional Review Boards (IRBs).
Greg Koski, M.D. Ph.D. is currently Associate Professor of Anesthesia in the Cardiac Anesthesia Group, Department of Anesthesia and Critical Care, and Senior Scientist of the Institute for Health Policy at the Massachusetts General Hospital. He received his Ph.D. in 1977 and his M.D. in 1978 from Harvard Medical School. Dr. Koski was the first director of the United States Office for Human Research Protections (OHRP) from 2000 to 2002. In addition to his appointments at Harvard Medical School, Dr. Koski serves as President-Elect of the Academy of Pharmaceutical Physicians and Investigators, where he is also vice-president for Ethics and Professional Conduct, and chairs the Ethics Advisory Board of the National Heart, Lung and Blood Institute's Framingham Heart Study.
Michelle Mello, J.D., Ph.D. is Director of the Program in Law and Public Health and Associate Professor of Health Policy and Law in the Department of Health Policy and Management at the Harvard School of Public Health. She received her Ph.D. in 1999 from the University of North Carolina, Chapel Hill and her J.D. in 2000 from Yale Law School. At Harvard University, Dr. Mello co-chairs the Human Subjects Committee in the School of Public Health and serves as a member of the Standing Committee on Ethics and Conflict of Interest in Continuing Education in the Medical School.
Wayne Patterson, Ph.D. is currently serving as the Program Manager for International Science and Engineering in Developing Countries at the National Science Foundation. Dr. Patterson received his Ph.D. in Mathematics from the University of Michigan in 1971 in differential topology. He was named Dean in Residence at the national office of the Council of Graduate Schools in Washington, D.C. Since the year 2000, he has been the Senior Fellow for International Programs and Academic Program Review in the Graduate School at Howard University. From December 2003 to July 2006, he also served as Associate Vice Provost for Research at Howard. Notably, Dr. Patterson received a grant from ORI in 2005 for his project Mentoring Relationships for Multi-cultural Populations.
Kenneth D. Pimple, Ph.D. is Director of Teaching Research Ethics Programs and Assistant Scholar at the Poynter Center for the Study of Ethics and American Institutions at Indiana University-Bloomington. He also serves as Director of H"Scientists and Subjects: A Web-based Seminar on the Ethics of Research with Human Subjects"H and H"Teaching Research Ethics: A Workshop at Indiana University."H Dr. Pimple received his Ph.D. from Indiana University-Bloomington in Folklore in 1991. In 2002, he completed his NIH grant project, H"Web-based Ethics Course in Research with Human Subjects."H His research and expertise is diverse and includes animal research, human cloning, teaching research ethics, research misconduct, treatment of human subjects, and data management.
Larry Rhoades, Ph.D. is Associate Director for Education and Integrity at the Office of Research Integrity (ORI) since 2000 and as Director of the Division of Education and Integrity at ORI since 1993. He received his Ph.D. in 1973 from the University of Michigan in Sociology. Prior to work with scientific misconduct, he worked in the Office of Policy Analysis and Coordination for the National Institute of Mental Health and Executive Associate for Programs at the American Sociological Association.
John Ruffolo, Ph.D. is Associate Vice President for Research at South Dakota State University and Professor in the Department of Biology and Microbiology. Dr. Ruffolo received his Ph.D. in Zoology from the University of Iowa in 1972. In his current position, he Heads the Office of Research & Sponsored Programs providing mostly pre-award services to the University community and serves as the Research Integrity Officer responsible for compliance in the areas of research integrity and conflict of interest. As a research administrator, Dr. Ruffolo’s areas of experience include developing and monitoring research policies and assuring compliance with federal regulations with a strong interest in information processing and electronic research administration.
Patricia Tereskerz, J.D., Ph.D. is the Director of the Program in Ethics and Policy in Healthcare Systems and Associate Professor at the Center for Biomedical Ethics at the University of Virginia School of Medicine, and Chair of the University of Virginia Conflict of Interest Committee. She received her J.D. in 1992 from the University of Virginia School of Law and her Ph.D. in 1994 from the University of Virginia in Research Studies. In 2005, Dr. she completed a project entitled “Research Integrity and Financial Conflicts of Interest”, for which she was the principal investigator and was co-sponsored by ORI. She is currently the principal investigator for a project entitled, Ethics and Data Monitoring Committees, sponsored by the Greenwall Foundation. (Round 1 only)
Mark Tumeo, J.D., Ph.D. is the Vice Provost for Research and Dean of the College of Graduate Studies at Cleveland State University and a faculty member of the Department of Civil and Environmental Engineering. He received his Ph.D. in 1988 from University of California, Davis in Civil and Environmental Engineering, and his J.D. in 2002 from Cleveland-Marshall University. Since 2000, Dr. Tumeo has coordinated all aspects of research and graduate programs across all colleges at Cleveland State University. He oversees the Technology Transfer and Intellectual Property operations of the University, Regulatory Compliance on research issues, including animal care and institutional review boards, and management of three University-wide Research Institutes. He received a grant in 2002 from ORI for his project Research Integrity Training Program: Conflicts of Interest and Commitment Module.
Sarah Vollmer, Ph.D. is Research Assistant Professor in the Department of Philosophy and the Ethics Center at the University of Alabama at Birmingham. She received her first Ph.D in 1982 from the University of Nebraska in Chemistry and a second Ph.D. in 1999 from the University of Maryland in Philosophy. In 2002, she was a co-principal investigator with Dr. Harold Kincaid in a project sponsored by ORI, entitled A Documentary Film: A Round Table on Mentoring and Authorship. In Fall 2006, she presented another ORI sponsored project for which she was the principal investigator, “Statistics, Images, and Perceptions of Truth: Detecting Research Bias and Misconduct.” Currently, she is co-principal investigator with Dr. Kincaid on another ORI-sponsored project, “Teaching Research Integrity in Analysis and Reporting: A Web-site with Case-Based Vignettes.”
Wendy Williams, Ph.D. is the Director of Research Education at the Children’s Hospital of Philadelphia and Adjunct Professor of Biology at Temple University. She received her Ph.D. in 1999 from Johns Hopkins University in Biology. Dr. Williams oversees the design, delivery, and evaluation of education and training programs for the Children’s Hospital research community and chairs the Committee for Post-doctoral Affairs. In 2004, she received a grant from ORI for her project Mentoring International Postdocs: Working Together to Advance Science and Careers. She received another grant as a co-principal investigator from ORI in 2005 for her project Guidebook for Teaching Selected Responsible Conduct of Research Topics for a Culturally Diverse Trainee Group.
Acknowledgements:
This project was funded by a RCR Resource Development contract from the Office of Research Integrity. James DuBois served as the project director and Jeff Dueker as the research assistant. Kathleen Wyrwich served as a methodology consultant. Courtney Andrews served as a technology consultant and created the online questionnaires.
This report was submitted to the Office of Research Integrity by the project director on July 12, 2007.
