Chapter 6 – Evaluating Samples when Researchers Generalize

One of the most consequential decisions that researchers make about their studies is the selection of participants (or aggregate units such as schools, hospitals, or cities). Hence, sampling, or the process of selecting study subjects, is extremely important to pay attention to when evaluating studies described in research articles. The goal of most studies is to generalize the results from a sample to the population. This chapter helps make sense of various aspects of sampling and the impact they have on the study results and their generalizability. Through evaluation questions and examples, this chapter discusses the advantages and weaknesses of random (or probability) and non-random sampling, how nonresponse and self-selection biases can distort a sample, how sample size and diversity matter, why total population studies may still be useful for generalizing, and how opt-in panels resolve the issue of decreasing response rates. This chapter is organized around the following main sections: (1) Gold Standard of Sampling, (2) Biasing Influences, (3) Balancing Influences, (4) Characteristics of the Sample, and (5) Ethical Considerations. Exercises at the end of the chapter help reinforce the acquired knowledge and skills.