Surveys are one of the standard forms of collecting data on individuals. The process is uniquely suited to the collection of knowledge, attitudes, and opinions, but there are many circumstances where it is the only way to obtain information about behaviors as well. Surveys are sometimes distinguished from polls because they are more likely to be conducted by academic than commercial researchers or because they tend to be longer. This distinction is not applied here.
Applications
In political research, for example, there are many ways to obtain information about voter turnout – how many people went to the polls – including official government statistics. But governments do not collect information on candidate or party preferences, and surveys are required to obtain such information, as well as on other personal characteristics such as age, race, or sex, and attitudes about issues of the day or specific policies. The only way that the impact of such personal characteristics on voting behavior and preferences can be pursued is through surveys. In the area of economic research, as another example, sales data can indicate the preference of consumers for one product over another or the relative market share of each. But the only way to obtain systematic data on the nature of preferences underlying these purchases or the demographic characteristics is through surveys. In these instances, the turnout data and the purchase and market share data from an external source can provide an external validation of some of the measures derived from surveys, a useful check on the overall quality of the data.
Surveys are typically used to collect data in areas where other sources of information are unavailable. In the area of consumer research, for example, economists now understand that expectations have just as important a role in determining purchases as do available economic resources. So the measurement of consumer expectation or the national economy as well as consumers’ own resources is an important element of economic forecasting. Another area in which surveys are especially relevant is in the measurement of trends and reactions to them; surveys can be used to assess reactions and responses to things that might occur in the future or to different policy options that the government might pursue. This applies equally well to the possibilities of new technologies and how they might affect our daily or working lives.
One special and frequent application of surveys is the collection of data by media organizations. Their primary interest is to obtain information that is useful for producing news content to add to their coverage of current events. But their dissemination of such information provides important social and political cues about public opinion to elites as well as a small segment of the general population that is interested in what their fellow citizens think. This can have important effects on subsequent measurements of opinion as well as consequences for specific policies that the government pursues.
Surveys have been an important and growing part of news gathering and social science research for more than 70 years. And increasingly, data from a variety of sources have been saved in data archives where they are available for secondary or extended analysis. As a consequence, more and better data are available for the analysis of trends and the comparison of public opinion across political and media systems for relatively long periods of time. Because of data archiving, it is now possible to trace the development of party systems and party identification across this period in individual countries, across countries that differ in their forms of government, or across countries that differ in their stage of democracy, as just one example. Because of the preservation of surveys in this way, there is an added value to the data for such analyses that is greater than the value for which any single survey was conducted.
General Survey Methodology
Surveys are appropriately seen as a system for data collection that involves several steps or stages. Among the major steps are selection of a mode of interviewing, sampling, questionnaire design and pre-testing, training of interviewers, preparation of the final questionnaire, fieldwork for data collection, and data analysis. In a general sense, a survey involves a sequential process of completing these steps, but there are often iterative stages of the process such as when pre-testing of the questionnaire suggests changes that should be made to it, necessitating an additional pre-test of the new instrument.
Mode Of Data Collection
The mode of data collection comes at the front of the sequence because so many other issues follow as a result of this decision. Data can be collected through face-to-face interviews where the interviewer and the respondent engage in direct personal contact; this is a useful approach when objects such as handout cards with measurement scales, illustrations, or photographs can be given to the respondent. It is an expensive form of data collection because of travel costs, especially to revisit or recontact respondents who were not available the first time. Data can also be collected on the telephone, which is generally a much less expensive form of data collection because of decreasing phone charges. Telephone interviews are usually shorter and consist primarily of closed-end questions. Interviewers have to read the scale options to the respondents, however, and handouts are obviously impossible. Telephone interviewing can usually be done in a shorter period of time, making it a preferred form of data collection for many applications, because it significantly reduces the possibility of the occurrence of external events impeding data collection in a way that makes the later interviews different from the earlier ones.
Pencil-and-paper surveys can be administered in person or by mail. Mail surveys are often employed because they are inexpensive and are sometimes preferred for asking about sensitive topics, because there is no interaction between an interviewer and the respondent. One problem with a pencil-and-paper survey is that the respondents can move back and forward through the questionnaire, trying to bring some consistency to their responses, often changing earlier responses to conform to later ones; this is not possible with faceto-face and telephone interviews. The fact that an interviewer is not present can be a problem if the respondent has a question or needs additional information, as no one is available to answer such a query. Handouts are also a difficulty in this mode for the same reason.
Surveys on the Internet are another mode; they permit rapid data collection and are very amenable to the use of examples and illustrations. They can also incorporate the use of color and different kinds of fonts to guide the respondent through the questionnaire. In fact, they can also include embedded video and audio segments, which is not possible with most other forms of data collection. A significant push toward data collection on the Internet is currently underway, primarily because of cost and reduced field periods; but there are also significant methodological issues about other aspects of this mode, primarily because of sampling issues.
Many surveys are increasingly employing a multimode design. Studies begin with a face-to-face or mail design, for example, but at the end of the process respondents who are difficult to reach are pursued on the telephone in order to obtain an interview. In some instances, the questionnaire is actually shortened for these late interviews as another accommodation to obtaining the most important measures in the survey.
Sampling
One reason that the mode of data collection is a primary concern is that so many additional choices are affected by it. Sampling is a prime example. In face-to-face interviews, multistage area probability samples typically have to be designed, where clusters of households in the same neighborhood are selected in order to reduce travel costs. The design of these samples is laborious and time-consuming.
For telephone surveys, on the other hand, a list of phone numbers can be developed or purchased from a commercial vendor, often on the same day a decision is made to field a study. These samples can have true random dispersion across a nation, for example, because the call rates are cheaper and do not vary with distance to the same extent that travel costs typically do. Procedures have to be followed to insure that unlisted numbers are incorporated into the sample, but these are now pretty well standardized. The use of mobile phones as the only means of contact is a growing problem in many countries.
In mail surveys, lists of addresses sometimes have to be located and often have to be checked independently for currency. Specific procedures have been developed for pursuing mail surveys that can maximize response rates, often making them comparable with other modes of data collection.
Total Survey Error Perspective
Another advantage of seeing a survey as a process is that it provides the basis for reviewing and controlling potential data quality through a total survey error (TSE) perspective. A common misperception is that the main source of error in surveys comes from sampling, including design and coverage issues. This view is fostered by the fact that sampling error is the most easily quantified form of error.
But the TSE approach acknowledges that errors can also derive from poorly worded questions, inadequate training of interviewers, or poor supervision of the interviewing process. The approach also permits available resources to be invested effectively at different stages of the process, often based upon the nature of the topics being covered, in order to reduce likely errors. Every survey involves a series of cost–benefit calculations, usually involving decisions about how much of a limited or fixed budget should be devoted to various steps in the process. In the simplest case, a researcher faces a decision about how many questions to ask (improvement of the measurement or the number of relationships to be investigated) as opposed to how many respondents to interview (greater precision in the estimation process). But the TSE approach adds other possibilities: how much to invest in the training and monitoring of interviewers, as opposed to how often to pre-test the questionnaire, as opposed to how persistent the staff should be in trying to raise the response rate.
The Management And Conduct Of Surveys
As the preceding discussion suggests, the organization and management of surveys is a complex and expensive process. As a result, there are very few organizations within any country that can offer on-site services that encompass under their direct supervision all of the steps in the process for different modes of data collection; the capitalization and administrative costs are just too great. This has produced two trends in the survey industry: specialization among organizations, by mode and in the production of services, and a movement toward telephone surveys because of an increased ability to cover capital and administrative costs.
Surveys can be pursued in a number of ways, especially since most researchers do not belong to organizations that engage in their own data collection. A researcher can construct a general design of all the elements of a survey and then put the study out for bids, eventually contracting with a firm that offers the best quality within the constraints of the available budget. The services offered range from having the original researcher interact with the survey organization all the way through the process to simply having a final report and a dataset delivered at the end. Many firms offer a full range of services, although they may not in fact have all of the facilities on site or under their direct supervision. Some firms have developed that specialize in the provision of samples, starting with telephone numbers and increasingly moving toward supplying email addresses from recruited panel participants. Other firms specialize in the production and fielding of questionnaires by operating telephone facilities, direct mail operations, or questionnaire design for computeraided telephone interviews (CATI) or web-based formats. Smaller numbers of firms offer services such as advanced data analysis facilities.
References:
- Biemer, P. B., & Lyberg, L. E. (2003). Introduction to survey quality. Hoboken, NJ: John Wiley.
- Dillman, D. A. (2007). Mail and internet surveys, 2nd edn. Hoboken, NJ: John Wiley.
- Groves, R. M., Fowler Jr., F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2004). Survey methodology. Hoboken, NJ: John Wiley.
- Kish, L. (1995). Survey sampling. Hoboken, NJ: John Wiley.
- Weisberg, H. (2005). The new science of survey research: The total survey error approach. Chicago: University of Chicago Press.