The term media performance has a broad reference to the assessment of mass media according to a range of evaluative criteria and primarily employing “objective” methods. In practice most attention has been given to the product of mass media, its content as sent and received. The criteria applied are mainly derived either from professional goals and standards or from considerations of the public interest as specified in certain evaluative concepts. The methods are mainly those of the social sciences, aiming to be systematic, reliable, and leading to some degree of generalization. We are thus speaking of a particular field of media research with several different aims and purposes. Apart from general description, these purposes include the wish to improve the media in some way, to support a line of criticism of media, to explain variations in quality of performance, and to identify causal factors.
Origins And History
Although the origin of the term media performance, as used here, is uncertain, it was employed for many years to classify topics of articles (mainly content analyses of news) in Journalism Quarterly, the official publication of the American Association for Education in Journalism (AAEJ). This positions media performance in close relation to journalism education and journalism criticism. James Lemert’s book Criticizing the media (1989) is a key work in establishing an identity for media performance research. It is essentially a plea for empirical criticism, deploying various concepts of quality of media structure and performance. Lemert distinguished empirical criticism from three other theoretical schools: a Marxist-oriented critique; cultural/critical studies; and social-responsibility-oriented criticism. He also advised against exclusiveness and recommended cooperation between theory and empirical inquiry. His own suggested topics for inquiry included the effects of chain ownership on editorial independence and activity, the coverage of environmental problems, news coverage of rape, and the consequences of relying on official sources. Each of these could result in recommendations for improvement, reflecting a positivistic inclination, although not necessarily a preference for “administrative” over critical research approaches.
It is not easy to draw boundaries around performance research, since many other kinds of research can help in assessing performance. This applies to: ratings and readership research; economic and market evaluations of particular media and their products; and criticism in the form of reviewing of other media or the very large literature that has emerged dealing (mainly) with the many alleged social, cultural, political, moral, and aesthetic failings of mass media. Despite this approximate boundary, for certain purposes of media performance assessment both economic and audience data can be relevant and even “subjective” forms of evaluation can be taken into account. Generally, the field of media performance has expanded and diversified in method, especially where attention is given to the full range of output of a (national) media system (or a general television network), as has been the case in the tradition of cultivation analysis (Signorielli & Morgan 1990).
The earliest phase of media performance research was, as noted, shaped by a concern to improve the professional quality of journalism, with primary reference to various ideas about news objectivity. The main criteria applied were those of accuracy and completeness in reporting relevant facts, reflection in news of the external “reality” being reported, lack of personal or political bias in the news report, and avoidance of sensationalism in tone and presentation. The more complex topic of editorial or journalistic independence was also covered, as well as certain media ethical issues.
From the late 1960s onwards, media performance research was more influenced by external social and political criticism than internal professional criteria, following the rise of radical and antiwar movements in North America and Europe. The rise of television news to a central position in the media system by the 1970s was also a factor, not only because of its reach and believed impact but also because of high public expectations of neutrality, truthfulness, and informative power. Much effort was made on both sides of the Atlantic to assess the balance of attention in news between various competing actors, political parties, and points of view, with strong suspicion of hidden ideological and manipulative motives and equally strong denials. This trend in research was associated with the intensification of efforts to create and manage news by competing economic or political interest groups and also with a critique of monopoly tendencies in media. For the most part, this research was done by way of content analysis (e.g., Efron 1971; Westerstahl 1980; Glasgow Media Group 1976), but there was also a significant growth during the 1970s in studies of news organizational practices that shed light on the shaping of news (e.g., Tuchman 1978) and thus indirectly on “media performance.”
An important advance in media performance research was the program directed at measuring media quality under the auspices of the Japanese Broadcasting Corporation (NHK) (see Ishikawa 1996). This program worked with a framework of three different levels at which problems of media quality can be identified: the whole media system; the channel (or equivalent); the content (e.g., program). It also identified four different perspectives from which media could be viewed and evaluated: of the state, of the society, of the audience, and of media professionals. This framework can encompass many criteria and find a way of integrating different kinds of evidence. For the first time, perhaps, systematic account was taken of the views of professionals about qualities that might escape social scientific methods. At about this time, Shoemaker and Reese published their first edition of Mediating the message (1991), which systematically examined influences on the production of news content, drawing on empirical research examples. The main relevant causal factors for news performance (quality) were identified as stemming from the personal characteristics of journalists, the organizational routines of news-making, external economic and institutional influences, or ideology and power distributions in society.
Focus And Themes Of Research
One of the main roots of critical research, as noted by Lemert, lies in ideas about the social responsibility of media to society, first given expression by the American Commission on Freedom of the Press (Hutchins 1947) and then taken up in postwar Europe, where both press and broadcasting were made subject to public criticism, inquiry, and policymaking on grounds of their claimed social obligations. This is the background to another effort to erect a framework for research, McQuail’s Media performance (1992), which assembled a number of basic criteria of media performance from theory and practice held to be “in the public interest” and reviewed relevant methods and examples of performance research.
Other perennial topics for media evaluation inspired by critical theory of one sort or another have included: the representation of women and of gender roles in the media; the portrayal of ethnic and other minorities; the distribution of attention in foreign news to places and countries outside the circle of rich and developed, mainly western, countries; the reporting of insurgent terrorism and war. The case of terrorism raised rather complex issues, with competing journalistic norms of freedom and responsibility. The case of war reporting is not so different, since there is great pressure from government and military to insure favorable coverage and deny any propaganda success to the enemy. Inevitably, also, the independence of journalists is compromised. These issues have taken research beyond the scope of news journalism and often into the territory of fiction and entertainment.
Media performance research has at times been inspired by acute public concern about its potentially harmful effects. For instance, it played a part in inquiries into the causes of crime and violence (Baker & Ball 1969), into urban rioting in the United States in the 1960s (Kerner 1968), into pornography (Einsiedel 1988). Most recently there has been heightened concern about the availability of pornography on the Internet, especially where it involves children. In many of these cases, there are conflicting norms for identifying quality.
An abiding focus of performance research is to be found in the field of political communication, dating back at least to the study of the 1940 US presidential election (Lazarsfeld et al. 1944). A core element in election research has been the measurement of time and space allotted to different candidates, parties, or issues, according to criteria of amount and direction (e.g., Hofstetter 1976). More recently, performance research has been stimulated by the complaint that the mass media are failing the democratic political system by not providing information of substance, by presenting politics in a negative light, and by diverting citizens from active participation (Capella & Jamieson 1997).
Research Methods And Problems
The extension of evaluative attention from printed newspapers to television and nonnews content has long called for new thinking and new methods, especially methods for dealing with audiovisual content. Researchers had already encountered difficulty in dealing with news photographs and other images, whose precise meaning cannot be either known or interpreted with any certainty. In assessing print news content, the assumption had usually been that the meaning of words in natural languages is knowable and thus interpreted by its audience as intended by news writers. Readers were also assumed to be familiar with conventions of presentation (space and headlines, etc.) that are guides to interpretation.
Even so, there are considerable obstacles to inferring “meaning” or “learning” that have undermined the claim to objectivity of assessment research. There have also been practical problems of accessing television news after the event (gradually eased by better recording methods) and of devising reliable coding frames for multimedia messages. Theoretical developments in linguistics, discourse analysis, and semiology posed fundamental challenges to the very notion of assigning fixed meanings to media content, almost vitiating the whole enterprise. Despite these difficulties, a variety of strategies and research approaches have been developed to achieve some practical results. These developments include: the invention of software for qualitative analysis of content; the specification of recurrent visual codes in news; new ways of assessing the direction of evaluation of content, based on knowledge of language; the application of the notion of “framing” (and also of “schemata” as employed by journalists and audiences) to understanding the intention and direction of news content; the use of ethnographic methods to record variations in the pattern of audience interpretation. All this has tended to make media performance evaluation more possible, but also more costly.
Criteria Of Evaluation
Although media performance can be evaluated according to many different standards, depending on circumstances and expectations, in practice there is a fairly limited set of criteria that recur, most of them indicated above. They emerge from diverse sources, but especially from one or other of the following: media professionals themselves; government, law and policy; public opinion; various interest groups and institutions affected by the media; the public as audience; various social critics and moral guardians. The most commonly encountered evaluative concepts to be found in the field include the following: objectivity, with its component elements as described above; diversity, which is a key value in most pluralistic democracies and underlies expectations that media will pay attention, or give access and expression, to a range of persons, groups, ideas, and events that are broadly reflective of the social, cultural, and political environment in which they operate; cultural quality, with reference to accepted aesthetic or ethical standards or the prevailing tastes and interests of the public; freedom, which here means mainly the independence of media as reflected in a willingness to speak out, to be critical or original, without deference to the power of government, business interests, or (in some cases) the media’s owners.
It should be stressed that such higher-level concepts are not applied directly but by way of a variety of more precise indicators related to these concepts. There are, of course, many negative criteria that are either the observed absence (or reversal) of the standards mentioned or relate to undesirable effects.
The Future Of Media Performance Research
The last decade or two of media and social transformation have been typically characterized by a reduction in external control of mass media and also by a great expansion of existing and new media, with attendant commercial pressures. The greater reliance on media self-regulation has increased the salience of media performance evaluation, sometimes as a substitute for public policy. The criteria of “good performance” have not, however, changed greatly and the same research apparatus can still serve, but there are significant new challenges. These arise most obviously from the enormous volume of supply transmitted by the Internet (and other new means). Such content is also often multimedia in character and may not conform to established genres and the conventions of presentation of traditional media. It is difficult to sample, to generalize about, and to codify, making the methods developed for mass communication inappropriate. New indicators of performance are called for. Crude forms of assessment of Internet flow are already appearing, but the task of performance assessment has barely begun.
- Baker, S., & Ball, S. (1969). Mass media and violence. Washington, DC: GPO.
- Capella, J., & Jamieson, K. (1997). The spiral of cynicism. New York: Oxford University Press.
- Efron, E. (1971). The news twisters. Los Angeles: Nash.
- Einsiedel, E. F. (1988). The British, US and Canadian pornography commissions and their use of the social sciences. Journal of Communication, 38(2), 108 –121.
- Glasgow Media Group (1976). Bad news. London: Routledge.
- Hofstetter, C. R. (1976). Bias in the news network television coverage of the 1972 election campaign. Columbus: Ohio State University Press.
- Hutchins, R. (ed.) (1947). A free and responsible press: A general report on mass communication: Newspapers, radio, motion pictures, magazines, and books (Commission on Freedom of the Press). Chicago: Chicago University Press.
- Ishikawa, S. (1996). Quality assessment of television. Luton: University of Luton Press.
- Kerner, O. (ed.) (1968). Report of the National Advisory Commission on Civil Disorders. New York: Bantam Books.
- Lazarsfeld, P. F., Berelson, B., & Gaudet, H. (1944). The people’s choice. New York: Sloan, Duell and Pearce.
- Lemert, J. B. (1978). Media criticism. Beverly Hills, CA: Sage.
- Lemert, J. B. (1989). Criticizing the media. Newbury Park, CA: Sage.
- McQuail, D. (1992). Media performance. London: Sage.
- Shoemaker, P., & Reese, S. (1991). Mediating the message. New York: Longman.
- Signorielli, N., & Morgan, M. (1990). Cultivation analysis. Newbury Park, CA: Sage.
- Tuchman, G. (1978). Making news: A study in the construction of reality. New York: Free Press.
- Westerstahl, J. (1980). Objective news reporting. Communication Research, 10, 403 – 424.