6b. Data collection

There is no one best way to collect data. Different types of research call for different collection techniques. There are, however, four important considerations that apply to all data collection and that will help ensure the overall integrity of both the process and the information collected.

     Appropriate methods. Reliable data are vitally dependent on reliable methods. If you use a test that can detect an effect in one of every 100 samples to find an effect that may not occur more frequently than 1 in every 1,000 cases, your results will not be reliable. Failure to find the effect could be due to either your experimental design or the lack of an effect, but you will not know which is true. The common saying, “garbage in, garbage out,” applies to research methods.

Although the need for appropriate methods might seem obvious, studies have suggested that researchers sometimes use inappropriate statistical tests to evaluate their results (see articles by DeMets and Gardner, Additional Reading). Methods can also be compromised by bias—choosing one method or set of experimental conditions so that a particular conclusion can be drawn—or sloppy technique. Whatever the origin, the use of inappropriate methods in research compromises the integrity of research data and should be avoided. Responsible research is research conducted using appropriate, reliable methods.

     Attention to detail. Quality research requires attention to detail. Experiments must be set up properly and the resultsaccurately recorded, interpreted, and published. A failure to pay attention to detail can result in mistakes that will later have to be corrected and reported. Correcting the record takes time and resources that are better spent on the research itself.

Obviously, it is not possible to avoid all mistakes in research. However, take a look at the errata section of any scientific journal and ask yourself how the mistakes reported could have been avoided. Did the authors check to make sure that each figure was correctly labeled? Were the calculations double checked? Did someone check to make sure the authors were properly listed? Since others rely on their work, researchers have a responsibility to make sure their work is carefully undertaken and reported. Sloppy research wastes funds and should be avoided.

      Authorized. Many types of data collection need to be authorized before they can proceed. Typically permission is needed to use:

• human and animal subjects in research;
• hazardous materials and biological agents;
• information contained in some libraries, databases, and archives;
• information posted on some Web sites;
• published photographs and other published information; and
• other copyrighted or patented processes or materials.

Researchers have a responsibility to know when permission is needed to collect or use specific data in their research. If you are not sure whether permission is needed, check before proceeding with data collection.

     Recording. The final step in data collection is the physical process of recording the data in some type of notebook (hard copy), computer file (electronic copy), or other permanent “record” of the work done. The physical formats for recording data vary considerably, from measurements or observations to photographs or interview tapes. However data are recorded, it is important to keep in mind that the purpose of any record is to document what was actually done and the results that were achieved.

In recording data, keep two simple rules in mind to avoid problems later, should someone ask about or question your work:

• Hard-copy evidence should be entered into a numbered, bound notebook so that there is no question later about the date the experiment was run, the order in which the data were collected, or the results achieved. Do not use loose-leaf notebooks or simply collect pages of evidence in a file. Do not change records in a bound notebook without noting the date and reasons for the change.
• Electronic evidence should be validated in some way to assure that it was actually recorded on a particular date and not changed at some later date. It is easy to change dates on computers and thereby alter the date a particular file seems to have been created. If you collect your data electronically, you must be able to demonstrate that they are valid and have not been changed.

As you record your data, it may be helpful to think about them as the legal tender of research—the currency researchers cash in when they apply for grants, publish, are considered for promotion, and enter into business ventures. To have and hold their value, research data must be properly recorded.