One of the bigger tasks the communication researcher faces is obtaining the data to answer the research question or test the hypothesis that motivated the research in the first place. In the field of communication, data comes primarily from two sources – human beings, or some media form (which, it could be argued, ultimately are produced by human beings, so perhaps these are not distinct sources). Human beings may be asked by a researcher to fill out a survey or participate in an experiment. Alternatively, a researcher might seek out political advertisements, newspaper editorials, or speeches to include in a content analysis.
Regardless, practical considerations typically require the researcher to limit his or her data collection to a sample drawn from a larger population of interest. For instance, a researcher interested in examining the effectiveness of a communication campaign in reducing unsafe sexual practices among high-school students will not be able to get all high-school students to participate in the study to evaluate its effectiveness. At best, he or she will only be able to get a certain fraction of students who are enrolled in a particular set of high schools, probably in a restricted location of the country, to provide data relevant to studying the effectiveness of the program. Those who participate in the study constitute the sample (those students enrolled in the high schools where data collection occurs) drawn from the broader population of interest (all high school students). Similarly, a content analyst interested in describing themes politicians have used recently in their television campaign advertisements is probably not going to analyze every campaign advertisement produced recently (the population). Instead, he or she may restrict the content analysis to most advertisements that aired during a national network evening news broadcast between 2001 and 2007 (the sample).
The Nature Of Sampling
Researchers rarely care much about the sample per se – what particular people in the study say or do, or the content of those advertisements that the investigator happened to content analyze. Instead, researchers are usually interested in making some kind of an inference from the data obtained from the sample – a “generalization” of some sort. The inference or generalization often focuses on using information from the sample to infer characteristics of the population that the sample hopefully represents – a “population inference.” For example, the investigator might want to make an inference about the fraction of the kids in the population who would be likely to change their behavior as a result of the communication campaign, or what percentage of advertisements in the population include a fear appeal.
The ability to make a population inference is going to depend in large part on how the sample was obtained, for the method chosen influences how representative the sample is of the population of interest – how similar the sample is to the population on all dimensions, characteristics, or features that are likely to influence or be related to the measurement of the variables in the study. When population inference is the goal (which it may not be; see Frick 1998; Hayes 2005; Mook 1983), the researcher is well advised to employ some kind of random sampling method.
Random sampling (also called “probability” or “probabilistic” sampling) requires that the process through which members of the population end up in the sample be determined by chance. Furthermore, for each member of the population, it must be possible to derive the probability of inclusion in the sample (even if you never actually calculate that probability). Random sampling is extremely important when the goal of the research is population inference, for it is the random sampling process that will, over the long haul, produce a sample that represents the population. Although it is possible that, just by chance, a specific sample will be unrepresentative of the population as a whole on one or more relevant dimensions, random sampling ensures that no conscious or unconscious biases the investigator brings into the sampling will influence who ends up included in the sample. This is good, for such biases, when they exist, can limit the generalizability of any study results and limit the ability to make a sound population inference.
Simple Random Sampling
The most basic form of random sampling is “simple random sampling.” With a simple random sample, each member of the population must have an equal probability of being included in the sample. In order to conduct a simple random sample, the researcher must have some means of identifying who is in the population in order to implement a method for making sure that each member has an equal chance of being included. Thus, simple random sampling requires that the investigator have some kind of list of the population prior to sampling – the “sampling frame.” An example of simple random sampling would be sampling members of the International Communication Association (ICA) by obtaining a membership list from the headquarters of the association and then randomly selecting names from the list. If there are 3,000 members and you wanted a sample of 100 members, you might assign each member a unique number between 1 and 3,000 and then have a computer randomly select 100 numbers between 1 and 3,000 to identify who to include in the sample.
Many of the statistical methods that communication students learn about in introductory statistics classes and books assume simple random sampling, although simple random sampling is rarely actually done. The problem with simple random sampling is that it requires that all members of the population can be identified and enumerated so that a simple random sampling plan can actually be implemented. But for many populations that communication researchers would be interested in sampling from, such lists do not exist. Instead another method of sampling would have to be used. An exception would be media content of certain newspapers, magazines, television shows, or other things that are printed, published, or broadcast regularly and frequently. For example, we know that the Los Angeles Times is printed every day. So if we wanted to analyze the content of the front page of the Los Angeles Times between 1998 and 2007, it would be possible to enumerate the days of the year (there are 3,655 days during that period), assign a unique number between 1 and 3,655 to each day, and then use a computer to randomly select a few hundred days (i.e., a few hundred numbers between 1 and 3,655), thereby ensuring that each day has an equal chance of being included in the sample.
Stratified Sampling
There are reasons not to use simple random sampling even when it is possible. For instance, it might be particularly important that you not leave it to chance whether the sample is representative of the population on certain variables that you know are likely to be related to what you are measuring. For example, if the goal is to estimate the average number of peer-reviewed publications of ICA members employed at universities in the US, you can be pretty sure that how many publications a person has is related to his or her academic rank. If you were to collect a simple random sample of ICA members in this population, it is possible that your sample, just by chance, may include assistant professors in greater proportion than they exist in the population, which could lead to a substantial underestimate of the quantity of interest.
To minimize this likelihood, you could conduct a stratified random sample, using academic rank as the stratification variable. When conducting a stratified random sample, the population is first split into groups (“strata”) that are homogeneous on the stratification variable. Then a simple random sample of each stratum is taken. So in this example, academic rank would be the stratification variable. The population is then divided up into strata (lecturers, assistant professors, associate professors, full professors). A simple random sample of lecturers is then taken, as is a simple random sample of assistant professors, and so forth. The final sample is then constructed by aggregating the simple random samples of each stratum into a single sample.
When collecting a stratified random sample, the sample will contain as many members of population in each stratum as you desire, with that number being a function of whether the stratified sampling is done proportionally or non-proportionally. The distinction relates to whether the researcher attempts to sample the strata in proportion to their size in the population. For example, if 75 percent of the members of the population are in stratum A and 25 percent are in stratum B, proportional stratified sampling would require the researcher to apportion to the total sample in such a way that 75 percent of the total sample is taken from members of stratum A and 25 percent is taken from stratum B. By contrast, non-proportional sampling allows the investigator to concentrate the sampling in a manner disproportionate to the population distribution on the stratification variable.
For example, even though 25 percent of the population may be in stratum B, the investigator might intentionally apportion half of the total sample to stratum B. This is called “oversampling,” and can increase estimation precision when oversampled strata are more variable on whatever is being measured compared to the other strata. After the sampling, cases from oversampled strata might then be mathematically underweighted so that cases in the sample from those strata do not “count” as much when deriving the study results statistically, to compensate for their increased likelihood of being included in the sample relative to members of other strata.
Cluster Sampling
Both simple and stratified random sampling require a list of the population prior to sampling. For instance, to conduct a simple random sample of ICA members, you would need to have a list of all ICA members. Stratified sampling imposes the additional requirement that each member of the population be placed into one and only one of perhaps several subpopulations defined by the stratification variable. In order for that to happen, information about each member’s value on the stratification variable must be available. For example, to stratify by academic rank, you would need to know not only who is a member of ICA, but also each member’s academic rank. This information might not be available.
A related method easily confused with stratified sampling is “cluster sampling.” To conduct a cluster sample, it must be possible for members of the population to be classified into groups (“clusters”) in some fashion. However, there is no requirement, as there is when doing stratified sampling, that these groups be defined by a measured variable (such as academic rank), or even that you know before sampling which members of the population are in which group. Indeed, you do not even need to know how large the population is, so long as each and every member of the population can be said to be a member of one and only one cluster. When you cluster sample, all you need to have available is the universe of clusters. You randomly sample clusters from the universe of clusters, and for those clusters that are randomly selected, you include each and every cluster member in the sample.
For example, suppose you wanted to sample the residents of a multistory apartment building to measure their attitudes about building management. For privacy purposes, the manager of the building might be reluctant to give the names and contact information for everyone living there. But she or he might be comfortable giving you limited access to the building, allowing you to knock on doors and talk to the residents. In the absence of prior information about who lives in the building and how to get in touch with them, you could not collect a simple or stratified random sample. But knowing that the building contains 30 floors, you could treat the floors as clusters, and then randomly select perhaps five floors. Once those five floors are randomly selected (probably through a simple random sample), you then approach everyone who lives on those five floors and include them in the sample. This would produce a bona fide random sample of residents of the building, even though you did not even know the specific identities of members of this population in advance of sampling from it.
A communication researcher interested in sampling the advertising content of newspapers could use cluster sampling quite easily. Perhaps the population is defined as all advertisements published in the London Times in 2007. No doubt it would be difficult to obtain a list of all members of this population (i.e., every single advertisement published during this period), making it impossible to conduct a simple random sample or a stratified random sample. But the investigator could easily conduct a random cluster sample, defining each day of the year as a cluster, randomly selecting perhaps 30 days in the calendar year, and then scanning the paper on those days, including every advertisement that appears on those days in the sample. See Lacy et al. (2001) and references cited within for advice on how to approach the sampling of media content.
Random Digit Dialing
The advent of the telephone and its penetration into most households, at least in industrialized countries, has made sampling of people much easier than in the past. By randomly dialing telephone numbers, it is possible to collect random samples of large populations of people who are geographically dispersed. This approach does not require an enumeration of the members of the population in advance of sampling because it relies on the assumption that most people are attached to at least one phone number. Numbers need not be dialed completely randomly, and indeed often are not.
For instance, to sample a particular region of a country, one might restrict the dialing to certain area codes or phone exchanges. And because many phones are not connected to residences, it would be advantageous to do list-assisted random digit dialing by purchasing a list of random phone numbers from a company that has already been culled of numbers that are disconnected or assigned to businesses or fax machines, for instance. Many companies exist that are in the business of constructing specialized lists of phone numbers to sell to researchers interested in sampling populations varying in size and specificity, from entire countries to people with specific occupations or interests.
Multistage Sampling
In practice, random sampling plans are often “multistage,” mixing sampling methods of different types that are conducted at different stages during the sampling process. For example, a researcher who wanted to collect data by doing face-to-face interviews of a random sample of urban city dwellers of an entire country would find if very difficult to collect a simple random or stratified sample of that population. Even if it were possible to enumerate the population, it might be cost-prohibitive to travel to the residences of, say, 1,000 different people dispersed across an entire country.
But it might be possible to divide the population up into clusters such as cities, then randomly select cities, perhaps stratified by number of residents (small, medium, and large, defined in some justifiable manner). Once a small number of cities is randomly selected from the population of cities, the researcher could get a residential phone directory for those cities and conduct a simple random sample of pages of each city’s directory. With pages randomly selected, the researcher could then randomly sample phone numbers from those pages that were randomly selected. To reduce the likelihood of excluding people from the sample who are not listed in the phone directory, the researcher might permute the last two digits of the phone numbers randomly selected from each page. Once this is done, calls are made to these numbers to set up a face-to-face interview of whoever answers the phone. As a result of this process, the researcher would not have to travel the entire country, thereby substantially reducing data-collection costs. And yet this approach would produce a bona fide random sample of the entire population of interest (or at least be very close to one).
Caveats Of Random Sampling
It is important to acknowledge that even if the selection of members of the population for inclusion in a sample is governed by a random process, nonrandom processes can adulterate random samples, and this can disrupt the ability to make an accurate population inference. For instance, an investigator might select a sample of people randomly from a population of interest, but certain people who are approached for inclusion in the study are likely to choose not to participate. The process that drives that choice may not be a random one. An example would be people who have a relatively low level of education being more likely to refuse to participate in the study. This is called “nonresponse bias,” and it is very difficult to avoid entirely. In this case, the sample of people who ultimately provide data to the investigator will over-represent those in the population who are more educated, and that might be very important if the measurement of the variables of interest to the researcher is likely to vary systematically as a function of level of education of the participants. So random sampling does not guarantee a representative sample, although compared to nonrandom sampling methods, it is superior when representativeness is the goal.
When thinking about population inference, it is important to keep in mind whether the population is dynamic or static, for this determines how time invariant the inference is. A static population is one that does not change over time, whereas a dynamic population changes over time. For example, the political advertisements broadcast on the major television networks prior to the 2004 presidential election represent a static population both in size and any study-relevant feature one can conceive. The number of these advertisements does not change, and the characteristics of those advertisements that a researcher might be interested in measuring are fixed. But the population of adult citizens of a specific nation is dynamic in size and features. The number of adult citizens of a nation fluctuates in size daily, as people die and adolescents “come of age” and become adults. As the members of a population change, so too will aggregates of the features of this population, some perhaps more than others. It is unlikely that there will be dramatic shifts in time of the distribution of men versus women, for example, in a large dynamic population of people. However, the attitudes of the members of this population may shift considerably with time, some attitudes perhaps more than others.
Inferences about a dynamic population must necessarily be conditioned at least to some extent on the time the data was collected. Any pollster would know, for instance, that one cannot infer much about a politician’s current approval from data collected more than a few weeks prior, as national and world events, even a single event, can move such judgments quickly. Other features of a population, even if dynamic, change much more slowly. It would be fairly safe to infer that if sample data suggests that the population distributes itself equally among a few political parties, for example, that distribution is not likely to shift much in a matter of weeks, months, or perhaps even a year or so. So inferences from “old” data about some features of a population are more reasonably made, even if the population is dynamic on that feature. The point is that random sampling from a dynamic population affords an inference only about the characteristics of that population at the time of data collection. The inference may be ephemeral, depending on what the researcher is measuring and attempting to make an inference about. For a good overview of the details of the methods of random sampling described here, see Stuart (1984).
References:
- Frick, R. W. (1998). Interpreting statistical testing: Process and propensity, not population and random sampling. Behavior Research Methods, Instruments, and Computers, 30, 527–535.
- Hayes, A. F. (2005). Statistical methods for communication science. Mahwah, NJ: Lawrence Erlbaum.
- Lacy, S., Riffe, D., Stoddard, S., Martin, H., & Chang, K.-K. (2001). Sample size for newspaper content analysis multi-year studies. Journalism and Mass Communication Quarterly, 78, 836–845.
- Mook, D. G. (1983). In defense of external invalidity. American Psychologist, 38, 379–387.
- Stuart, A. (1984). The ideas of sampling. New York: Macmillan.