The term “extra-media data” describes a methodological approach to assessing the quality of media content. The phrase was coined in the early 1970s by Swedish scholar Karl Erik Rosengren (1970) during a controversy about the criteria needed to assess bias in the news. Scandinavian researchers Galtung and Ruge (1965) had criticized the coverage of foreign countries in the Norwegian press for highlighting certain news factors and thus creating a distorted picture of these countries. The researchers’ conclusion was based exclusively on the occurrence of news factors like personalization and negativism. Rosengren instead suggested that researchers should evaluate the performance of news media by comparing their coverage to external, primarily statistical indicators. In his own study Rosengren investigated how newspapers in three different countries covered parliamentary elections in countries around the world. He then related the coverage to information about the countries and about the respective elections as listed in contemporary archives. By doing this, he was able to compare media coverage of the events with their “objective” news value according to, e.g., country size, GNP, or whether an election brought about a new majority.
In communication research today we can find at least three different approaches to assessing the quality of media content by sources from outside: statistical data (as suggested by Rosengren), participant observation of events, and the use of expert opinions. Funkhouser (1973) compared the number of news articles on several political issues in the sixties in the USA with statistical indicators for their real salience. For instance, in the case of the Vietnam War Funkhouser used the number of US soldiers fighting in Vietnam; in the case of the labor market he used the number of unemployed Americans. He found almost no correlation between the number of news stories about these issues and their salience as indicated by these statistical data. However, he did find a strong correlation between media coverage and public opinion. In a similar manner, Kepplinger (1992) compared the coverage of the ecological condition of air and water in Germany with statistical data on air and water pollution over a period of 20 years. Adams (1986) developed a “news geography” of US media by comparing the death tolls of natural disasters with the emphasis given in the news.
Almost 20 years before Rosengren wrote about extra-media data, Lang and Lang (1953) compared the impressions of an event when seen on television with the impression of the same event when participating in it. The event was General MacArthur’s parade in the streets of Chicago. People who watched the parade on television received the impression that there was much more excitement in the streets than there actually was. The authors attributed this discrepancy to a reciprocal effect created by the presence of the television camera itself. Many years later Donsbach et al. (1993) applied the same approach in an experimental study on a campaign speech. While one group of subjects attended the event live, another watched several versions of a television broadcast of the event. The impressions received of the event, the speaker, and his speech when watching him live and when seeing a television broadcast differed significantly, with the television versions usually creating much worse ratings.
Finally, experts can be used as an external benchmark for evaluating the quality of media content. Lichter et al. (1986) surveyed experts on nuclear energy about the potential risks of this technology and compared the result to the opinions expressed by experts cited or interviewed in the media. In a similar study, Snyderman and Rothman (1988) surveyed experts on human intelligence and compared this to the coverage of the IQ controversy in the US media. According to both studies, the media misrepresented the distribution of opinions among experts.
These examples show that the use of such reality indicators for an assessment of media coverage by extra-media data is problematic. In most cases the concrete indicators used either represent only a certain aspect of the issue or event they are supposed to indicate (e.g., the number of American GIs in Vietnam), or may be biased themselves (as in the case of the experts). Further, there are many areas of media coverage where no such extramedia data exist to be used to assess media content. Schulz (1976) even went so far as to negate the possibility of any benchmark for media content. He argued that each system had its own criteria for constructing reality and each of these would necessarily differ from the dominant criteria used by the news media, i.e., the value of news factors. Nevertheless, if one assumes that some scientific measure of the quality of reality representation in the media is important, the comparison of media content with extra-media data is probably the strongest tool.
- Adams, W. C. (1986). Whose lives count? TV coverage of natural disasters. Journal of Communication, 36(2), 113 –132.
- Donsbach, W., Brosius, H.-B., & Mattenklott, A. (1993). How unique is the perspective of television? A field experiment on the perception of a campaign event by participants and television viewers. Political Communication, 10(1), 41–57.
- Funkhouser, R. (1973). The issues of the sixties: An exploratory study in the dynamics of public opinion. Public Opinion Quarterly, 37(1), 62–75.
- Galtung, J., & Ruge, M. H. (1965). The structure of foreign news. Journal of Peace Research, 2(1), 64 –91.
- Kepplinger, H. M. (1992). Artificial horizons: How the press presented and how the population received technology in Germany from 1965–1986. In S. Rothman (ed.), The mass media in liberal democratic societies. New York: Paragon House, pp. 147 –176.
- Lang, K., & Lang, G. E. (1953). The unique perspective of television and its effect. American Sociological Review, 18(1), 3 –12.
- Lichter, S. R., Rothman, S., & Lichter, L. S. (1986). The media elite: America’s new power brokers, Bethseda, MD: Adler and Adler.
- Rosengren, K. E. (1970). International news: Intra and extra media data. Acta Sociologica, 13, 96 –109.
- Schulz, W. (1976). Die Konstruktion von Realität in den Nachrichtenmedien [The construction of reality in the news media]. Freiburg and Munich: Alber.
- Snyderman, M., & Rothman, S. (1988). The IQ controversy: The media and public opinion. New Brunswick, NJ: Transaction Press.