In the typical demonstration of dual coding, a list consisting of an equal number of pictures and words is presented to study participants. On encountering an item in the list, the study participant is asked to read the word or name the picture. Later, when the items are recalled, twice as many pictures than words are recalled. This picture-superiority effect, or the better recall of pictures over words, can be attributed to dual coding, which is the activation of traces in visual and verbal memory systems. While the verbal system is dedicated to processing of linguistic information, such as words and sentences, which are represented as discrete units or “logogens” and processed sequentially, the visual or imagistic system is dedicated to the processing of images, which are processed more holistically and represented through units referred to as “imagens” (Paivio 2006). The complementary role of two distinct, but interconnected, visual and verbal systems is at the heart of dual coding theory (DCT).
According to DCT, pictures benefit from dual activation. First, the visual system has to interpret or decode the picture. Second, the verbal system has to generate a verbal label for the picture that has been activated in the visual system. Because the systems are interconnected, the activations are pooled, thus enhancing memory for the pictures. The pooling of activations from the visual and verbal systems is referred to as the additivity hypothesis of DCT, which explains the picture-superiority effect.
Unlike the naming of pictures, the reading of a word activates mainly the verbal system. However, because of the interconnection between the two systems, some leakage or incidental activation of the visual system may occur automatically. For example, when reading the word “fire,” a mental image of fire may be triggered. However, because such activations are only incidental, they are weak in comparison to the direct activation created by a picture.
The automatic or incidental activation of the visual system from verbal inputs is the basis for another robust experimental finding in DCT, which is the better recall of concrete items over abstract items. When study participants are presented with a list of concrete and abstract nouns, concrete nouns enjoy a decisive advantage, though their performance is slightly below the recall of pictures. The better performance of the concrete items can be attributed to their imagery evoking properties. A concrete noun, such as “lobster,” has greater imagery evoking potential than, say, an abstract noun, such as “truth.” In other words, just the thought of a lobster automatically conjures up visual images in memory, whereas an abstract construct, such as truth, does not have the same imagery evoking potential. This effect of concreteness has been demonstrated for a range of verbal stimuli, including concrete phrases, sentences, and even passages.
Logogens and imagens, which are the foundational elements of DCT, are not limited only to visual inputs, but can be extended to other senses. For instance, when we hear the word lobster, as opposed to seeing the word, the auditory input can be combined with the visual input. This is an important extension because communication researchers have applied DCT mainly to television news, in which the verbal information is streamed through the audio channel.
Finally, it is important to note that DCT is not without its detractors. Those opposed to DCT contend that a singe code system is more parsimonious than a dual code system. They argue that the better performance for pictures and concrete items could be attributed to the vividness of the picture or richer encoding of concrete items. Such explanations discount the need for two specialized systems that deal separately with pictures and words. However, evidence from recent research, including brain imaging studies, supports the existence of multiple specialized systems that work in concert to decode and integrate sensory information, thus providing support for DCT.
Both effects of the DCT – the additivity hypothesis and the concreteness effect – have implications for news design and consumption. Researchers from a number of areas, including marketing, education, and communication, have focused on the additivity hypothesis. The emphasis of these efforts has been to examine how the dual coding through the audio and the video channels can be maximized to improve learning. On the surface, it would appear that inputs from the two channels should be better than separate inputs from either the audio or video channel. But a key limiting factor is human processing capacity. Streaming television video can easily overwhelm human processing capacity if the audio and video are not strategically edited to reinforce one another (Lang 1995). Communication researchers have examined this topic under the rubric of audio-video (AV) redundancy.
The concreteness effect, too, has implications for news practice. An important characterization of news is as episodic and thematic frames. While episodic news focuses on the concrete details of the four Ws – who, what, when, and where – thematic news focuses on the “why” aspects of a story and tends to emphasize analysis. Thematic stories tend to be abstract, lack compelling visuals, and have low imagery evoking potential. Because of these differences between the two types of stories, episodic news items are better recalled than thematic news items (David 1998). However, thematic news can be concretized through various techniques such as through a metaphor, a conceptual peg, or multimedia elements. Such solutions would require a concerted effort by news producers, but may be well worth the efforts in the interests of maintaining an informed citizenry.
References:
- David, P. (1998). News concreteness and visual–verbal association: Do news pictures narrow the recall gap between concrete and abstract news? Human Communication Research, 25, 180 –201.
- Lang, A. (1995). Defining audio/video redundancy from a limited-capacity information processing perspective. Communication Research, 22, 86 –115.
- Paivio, A. (2006). Mind and its evolution: A dual-coding theoretical interpretation. Mahwah, NJ: Lawrence Erlbaum.