Visual perception is a seemingly effortless process for most people. However, it is not the case that the human brain receives information just as it exists in the environment. Vision is an active process, and while information from the outside world becomes transformed into information sent to the visual cortex, there are a number of decisions the visual system makes in order to solve inherent ambiguities. For example, objects in the physical world are three-dimensional, while images stored on the retina are represented in only two dimensions. Consequently, it is almost impossible to resolve the information about size and distance with complete accuracy. Consider the case of a scuba diver with limited visual cues – the shark in her visual field could be seen as a very large fish 10 meters away, or as a very small fish just 1 meter away. Much research in psychology has focused on how people resolve ambiguous information using various visual cues including object size, motion, color, segregation (i.e., determining when one object begins and another one ends), and the distance of the object from the perceiver.
A separate line of research seeks to understand how people form and process mental images internally. Evidence suggests that the manner in which people store and utilize visual memories is different than that for verbal memories, because the former are largely contingent on spatial principles, while the latter are stored in a propositional format. Researchers have utilized various paradigms to study the manner in which people store and use images. For example, work by Shepard and colleagues in the early 1970s established a paradigm of mental rotation (e.g., Shepard & Metzler 1971). By having experimental subjects rotate objects in their mind (e.g., rotate the letter “K” 90 degrees counterclockwise) and comparing the imagined results to the actual results, researchers learned about the accuracy and malleability of mental representations. In addition, by running experiments in which subjects scanned mental maps and recalled features of imagined images, Kosslyn and colleagues demonstrated that people used mental imagery often and accurately (e.g., Kosslyn 1973). Moreover, certain types of visualization are qualitatively different from others in terms of richness of representation; evidence suggests that people process human faces more substantially than other images.
When people communicate with others, either face-to-face or through some mediated context, vision usually plays a major role. Of course, how substantial that role is obviously depends on context – watching television relies more on vision than listening to the radio. However, communication scholars have relied on theories and methodologies created by cognitive and perceptual psychologists to guide their work (see Smith et al. 2005 for a review).
One fruitful area of work has been examining the manner in which visual information interacts with verbal information. The now classic “McGurk effect” had demonstrated that people rely on visual information when listening to another person speak. McGurk and McDonald (1976) created a video clip with the speaker mouthing the sound “pa-pa” but coupled with the sound “na-na”. People watching the clip integrate the two conflicting cues and hear the phoneme “ma-ma.” In other words, visual information changes the way a communicator hears information.
Many communication scholars have studied the effect that visualization has on verbal processing, when the two channels are either redundant (i.e., consistent) or inconsistent. For example, Edell and Staelin (1983) have provided support for a model in which the degree to which visual information improves or detracts from a verbal advertisement depends on the type of message presented in the advertisement and on the congruence of the verbal and visual information. Subsequent work has examined the degree to which visual information augments verbal information in newscasts, political flyers, and many other forms of mediated messages with visual information.
The effect of visual processing is significant enough that people may be unknowingly influenced in various important decisions made in society. The realm of politics serves as a good example. Modern politics depends heavily on the development of communication media, which heighten the visual aspect of political campaigns. Work has demonstrated that in low-information situations people often turn to visual affective cues and base their electoral choices on the appearance of the candidate. One study demonstrated that manipulating images to increase the facial similarity of the candidates to the voters’ own facial features increased the likelihood the voter would support that candidate (Bailenson et al. 2006). Of course, well-informed voters have the ability to filter such intuitive attractions out of their decision-making processes, but the influence of the visual information above and beyond substantive issues is significant.
Infinite possibilities for visual processing await discovery in the field of communication. In the future, it will become increasingly important for scholars and the public alike to constantly question the ethics of innovative methods to influence the human mind through visual manipulation.
- Bailenson, J. N., Garland, P., Iyengar S., & Yee N. (2006). Transformed facial similarity as a political cue: A preliminary investigation. Political Psychology, 27, 373–386.
- Edell, J. A., & Staelin, R. (1983). The information processing of pictures in print advertisements. Journal of Consumer Research, 10, 45–61.
- Iyengar, S., & McGrady, J. (2006). Media and politics. New York: W. W. Norton.
- Kosslyn, S. M. (1973). Scanning visual images: Some structural implications. Perception and Psychophysics, 14, 90–94.
- McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746–748.
- Shepard, R. N., & Metzler, J. (1971). Mental rotation of three-dimensional objects. Science, 171, 701–703.
- Smith, K., Moriarty, S., Bartatsis, G., & Kenney, K. (2005). Handbook of visual communication: Theory, methods, and media. Mahwah, NJ: Lawrence Erlbaum.