Selective Reporting of Methodology
Replication of others’ research is one of the hallmarks of the scientific enterprise. As such, scientists and scholars have a professional obligation to inform others about the specific procedures used in their research. This information is found in the methods section of a research paper, the purpose of which is to provide other researchers with sufficient details about the study so that anyone who wishes to verify the results will have the necessary information to do so. In the methods section we identify the subjects of our study (e.g., select clinical population, specific species of animals) and provide important details about characteristics of the sample, such as how subjects were recruited, that are relevant to the variables that are being manipulated and measured.
The Methods section also contains description of instrumentation or other observational and measuring techniques that are used to obtain the outcomes reported. Whether data were collected using sophisticated instrumentation, such as a positron emission tomography or via a simple paper-and-pencil questionnaire, scientists must describe these materials with sufficient detail to allow others to carry out the study and verify the results.
PPerhaps the most important part of a Methods section is the description of the actual procedure used to carry out the study. Here, investigators must explain in clear language the series of steps used to establish, observe, and/or manipulate all relevant variables. They must offer a complete description of the testing conditions and all of the other necessary details that would allow an independent investigator to carry out the exact same study again. Admittedly, some studies may include several highly complex components that are carried out by different members of a research team. Nonetheless, it is essential that all key details and steps be described in this section in a most clear manner. The inadvertent omission or ambiguous presentation of a single step or piece of information may doom to failure the replication efforts of other researchers, thus needlessly wasting valuable time and resources. Obviously, a more serious offense occurs when an author intentionally leaves out an important detail about the procedure or fails to report a crucial event that altered the conditions of the study. There may be several reasons why some authors will knowingly leave important details out of a research report (e.g., assumed irrelevance, perceived minimal impact). Perhaps an extraneous variable was inadvertently introduced late into the study while it was still in progress leading to biased results. Thus, for the sake of expediency, rather than discarding the biased results and starting all over again, the investigator may inappropriately leave that major detail out of the report. The important point here is that authors have an ethical obligation to describe all of the important aspects of the research conducted, even if some of those details reflect poorly on the abilities of any member of the research team.
Because of the concern that some investigators may at times omit important details of the methodology used, guidelines have been formulated to help authors write better research reports. For example, for reports describing randomized control trials authors are advised to consult Moher, Schultz, and Altman’s (2001) Consort statement, which is a set of guidelines designed to improve the quality of such reports.