Effects of Nonverbal Signals

Human communication is a multichannel reality comprising verbal, paraverbal, and nonverbal signals. Although some authors subsume paraverbal aspects, like pitch, tone of voice, etc., under the heading of nonverbal behavior, it is most common to preserve the term “nonverbal” for those aspects of communicative behavior that are transmitted visually, such as gestures, body posture and movement, facial displays, gaze , and interpersonal distance.

There is ample empirical evidence that the visual communication channel plays a prominent role, not only in face-to-face interactions but in mediated communication as well (Burgoon et al. 1989). Beyond so called discourse and dialog functions closely tied to speech production and conversation, such as illustrative gestures or turn-taking signals, nonverbal behavior constitutes the major basis for the perception of persons and the formation of social impressions (see Bente & Krämer in press). Based on nonverbal cues, humans are prone to make inferences about the emotional states, intentions, and even personality attributes of other people. Impressions on the basis of nonverbal behavior are formed within extremely short time periods, and even minimal variations in posture, movement, or facial expression can have a deep impact on the emotional attitudes of an observer.

Based on the observation that nonverbal cues, like all visually perceptible stimuli, possess an overwhelming suggestive force, Frey (1999) introduces the concept of inferential communication. Referring to Helmholtz’s concept of unconscious conclusions, he highlights the fact that the inferences based on visual cues are not subject to conscious cognitive control but leave us defenseless, while affecting our judgments immediately and sustainably. With regard to the neuro-biological bases of inferential communication, recent neuro-imaging studies could identify specialized brain areas for the processing of visual communication cues, especially sensitive for human faces and gaze behavior. Further, the discovery of the so-called mirror-neurons enforced the assumption of a hardwired empathy system, which takes advantage of the fact that observed movement produces the same neural activation patterns as the production of this behavior.

The conclusion, however, that this neural reflection leads to a simulation of comparable emotional states in the observer is still a hypothesis. This hypothesis also draws upon another implicit assumption, namely that emotional states more or less involuntarily lead to specific expressive behaviors on the part of the sender that can then be informative for an observer (see Manstead et al. 1999 for a review). This position, however, has been challenged in recent years. Referring to empirical findings in evolutionary psychology, Fridlund (1991) argues that it is not functional to disclose emotions to an observer, and that it makes much more sense to assume that nonverbal cues are used independently from the affective state in a socially purposeful or manipulative way. Based on recent findings, Manstead et al. (1999) suggest that both emotional and social contexts can determine nonverbal behavior and that both views are thus not mutually exclusive.

Signals And Behavioral Qualities

In an attempt to systematize nonverbal cues according to their communicative functions, Mehrabian (1972) identified three basic effect dimensions and associated signal categories: (1) the evaluation dimension (liking) is communicated by so-called immediacy cues such as smiling, forward lean, close proximity, and touch; (2) the potency dimension (power) is conveyed by relaxation cues such as asymmetry, side lean, relaxed hands, and backward lean, and moreover by staring, averted gaze, and expansive gestures; and (3) the activity dimension is associated with activity cues comprising vivid and extensive use of gestures, frequent facial displays, etc. A broad literature exists dealing with those functional categories, the identification of particular cues or signals, and the proof of their distinct effects. In recent years, however, scholars discovered that dynamic characteristics or movement qualities, such as speed, acceleration, dimensional complexity, symmetry, etc., may have an even stronger impact on the observers’ impressions than the so-called semantic aspects (Grammer et al. 1997; Krumhuber & Kappas 2005). These aspects exert a subliminal influence on the observer and thus are rarely identified as a cause of social judgments and interpersonal feelings. Against this background, Grammer et al. (1997) suggest a view of nonverbal communication that differs radically from traditional categoryoriented “body language” approaches and that has been labeled the analogous communication perspective.

Experts of propaganda and political campaigns long since discovered the power of televised images and the crucial role of visual presence, physical appearance, and nonverbal behavior in influencing the attitudes of a mass audience, as in televised presidential campaigns. In fact, it has been shown that politicians’ facial expressions (the presentation of happiness/reassurance, anger/threat or fear/evasion displays) are even more decisive for the perception of the actor than the observers’ political partisanship or ideological beliefs (Sullivan 1996). Referring to the facial feedback theory, the socioemotional effects of facial expressions have been interpreted in this context as vicarious instigation (Vaughan & Lanzetta 1980). This view holds that the observation of the actor’s facial displays leads to the automatic mimicry of facial movements and subsequently – due to the interlinkage of facial muscles and brain regions associated with emotions – to the corresponding feelings in the observer. This affective resonance is assumed to be the basis of empathic reactions to the screen actors, which may also account for emotional involvement and identification processes in fictional offerings.

Nonverbal Behavior In Traditional Media

It is evident that the interpretation of mediated nonverbal behavior depends heavily on the context information presented. The Kulechov effect describes the fact that even neutral faces can lead to such different emotional attributions as terror, joy, or contentment when embedded into different stories (Pudowkin 1961). Moreover, the staging of nonverbal behavior implies numerous influences by the medium itself, including camera perspectives, production techniques, or journalists’ decisions about the selection and placement of nonverbal material. As visual attention is a valid indicator of power and status in social groups, presence on-screen not only reflects the relative importance of the personalities presented, but can also subtly influence the audience’s perception of leadership and dominance (Masters et al. 1991).

Physical distance is a highly influential nonverbal cue in interpersonal communication. It is also an intriguing example of how staging variables can simulate or substitute variations in nonverbal behavior. Meyrowitz (1979) coined the term “para-proxemics” to account for the fact that camera distance, judged by the relative size of an actor on the screen, is capable of triggering psychological mechanisms similar to the real distance to another person. A close-up could thus create intimacy but could also be threatening when penetrating virtual personal space abruptly, whereas a wide shot could create a psychological distance between the observer and the screen actor.

The camera perspective also determines the observer’s field of view and thus directs the attention of the audience to particular nonverbal cues. For instance in close-ups the focus is on facial activity, which stimulates inferences about the emotions and intentions of the actor. On the other hand in half or wide shots the view is opened up to communicative gestures, interpersonal constellations, and information about context. It is likely that these perceptual constellations also influence the direction of the audience’s causal attributions, so that the causes of observed behavior are located within either the actor or the particular situation.

With regard to angle of view, the results are equivocal. Kepplinger (1982), for example, found politicians presented on TV from a straight angle to be the most likeable, while deviations in either direction (up or down) led to less positive evaluations. Other studies (Kappas et al. 1994) show the low-angle shot to be preferred. Messaris (1997) states that low camera angles can make actors appear more powerful, provided that they are already recognized as high in status, as in the case of known political leaders. It has to be noted that the camera angle as a surrogate for the observer’s perspective and gaze direction interacts with the gaze direction of the actor portrayed. Here, looking directly into the camera can reduce the impression of an actor’s power. In line with results from gaze research, powerful people are expected to be looked at by inferiors rather than to look at observers themselves.

The subtle interplay between nonverbal behavior and presentation variables is also reflected in the concept of parasocial interaction (Horton & Wohl 1956). Although in mass communication large audiences are present, each member might still feel they are being addressed in relative privacy and thus get the illusion of a face-to-face encounter with the performer or persona. To achieve this, the persona attempts to duplicate the gestures, conversational style, and milieu of a small informal gathering, and turns on eye contact by facing the camera. This behavior is supported by formal presentation techniques. While the persona is facing the camera, zoom activity can enhance intimacy by making the persona appear actually to approach. Whether the emotional response of the viewer will be positive or negative depends on the communication behavior of the persona.

Nonverbal Communication In The New Media

So-called “cues-filtered-out theories” of computer mediated communication (CMC) consider the partial or complete lack of nonverbal signals as a deficit with particular socio-emotional consequences. On one hand it is considered as detrimental, in the sense of resulting both in emotional impoverishment and in the facilitation of antisocial and deviant behaviors. On the other hand the lack of nonverbal signals, in particular the lack of status and dominance cues, is expected to foster a balance of power and influence, and to equalize participation and mutual attention in net-based communications (Kiesler et al. 1984). Both positions use face-toface encounters as normative reference. In his “social information processing” (SIP) theory, Walther (1996) challenges the deficit conception of mediated communication and highlights the creative processes in the users. These processes might not only find substitutes for the nonverbal channels (e.g., by using so-called emoticons or avatars), but also result in new forms of human interaction that are not possible in face-to-face meetings at all. It is still an open question, though, under what circumstances the inclusion of nonverbal channels in CMC (e.g., by video or avatar representations) will enhance the experience of social presence (Biocca et al. 2003) and lead to beneficial socioemotional effects and better task performance. As in face-to-face interactions, the outcome will depend more on what the actors do than on which channels are available. This also holds for the implementation of so-called embodied conversational agents (ECAs) in human–computer interaction (HCI). By decoding but also by showing expressive nonverbal behaviors, ECAs are expected to personalize the interactions between humans and electronic devices and to facilitate acceptance and intuitive usage (Cassell et al. 2000). As Blascovich (2002) states in his “threshold model of social influence,” the social influence of virtual interlocutors depends on at least two factors: the perceived agency (whether the behavior is directly or indirectly human in origin or is computer-generated) and the behavioral realism. With regard to the latter, it is assumed that verification of the entity being a social actor depends largely on the processing of subtle nonverbal cues and transient movement dynamics.

References:

1. Bente, G., & Krämer, N. C. (in press). Virtual gestures: Embodiment and nonverbal behaviour in computer-mediated communication. In A. Kappas (ed.), Emotion in the Internet. Cambridge: Cambridge University Press.
2. Biocca, F., Harms, C., & Burgoon, J. (2003). Towards a more robust theory and measure of social presence: Review and suggested criteria. Presence: Teleoperators & Virtual Environments, 12(5), 456 – 480. At www.mindlab.org/images/d/DOC716.pdf, accessed September 11, 2006.
3. Blascovich, J. (2002). Social influence within immersive virtual environments. In R. Schroeder (ed.), The social life of avatars: presence and interaction in shared virtual environments. New York: Springer, pp. 127–145.
4. Burgoon, J., Buller, D. B., Woodall, W. G. (1989). Nonverbal communication: The unspoken dialogue. New York: Harper and Row.
5. Cassell, J., Sullivan, J., Prevost, S., & Churchill, E. (eds.) (2000). Embodied conversational agents. Cambridge: Cambridge University Press.
6. Frey, S. (1999). Die Macht des Bildes. Bern: Huber.
7. Fridlund, A. J. (1991). Evolution and facial action in reflex, social motive, and paralanguage. Biological Psychology, 32(1), 3 –100.
8. Grammer, K., Filova, V., & Fieder, M. (1997). The communication paradox and a possible solution: Toward a radical empiricism. In A. Schmitt, K. Atzwanger, K. Grammer, & K. Schäfer (eds.), New aspects of human ethology. New York: Plenum, pp. 91–120.
9. Horton, D., & Wohl, R. R. (1956). Mass communication and para-social interaction: Observations on intimacy at a distance. Psychiatry, 19(3), 215 –229.
10. Kappas, A., Hess, U., Barr, C. L., & Kleck, R. E. (1994). Angle of regard: The effect of vertical viewing angle on the perception of facial expression. Journal of Nonverbal Behavior, 18, 263 – 280.
11. Kepplinger, H. M. (1982). Visual biases in television campaign coverage. Communication Research, 9, 432 – 446.
12. Kiesler, S., Siegel, J., & McGuire, T. W. (1984). Social psychological aspects of computer-mediated communication. American Psychologist, 39(10), 1123 –1134.
13. Krumhuber, E., & Kappas, A. (2005). Moving smiles: The role of dynamic components for the perception of the genuineness of smiles. Journal of Nonverbal Behavior, 29(1), 3 –24.
14. Manstead, A. S. R., Fischer, A., & Jacobs, E. B. (1999). The social and emotional functions of facial displays. In P. Phillipot, R. S. Feldman, and E. J. Coats (eds.). The social context of nonverbal behavior. Cambridge: Cambridge University Press, pp. 287–316.
15. Masters, R., Frey, S., & Bente, G. (1991). Dominance and attention: Images of leaders in German, French and American TV news. Polity, 23, 373 –394.
16. Mehrabian, A. (1972). Nonverbal communication. Chicago, IL: Aldine.
17. Messaris, P. (1997). Visual persuasion: The role of images in advertising. London: Sage.
18. Meyrowitz, J. (1979). Television and interpersonal behavior: Codes of perception and response. In G. Gumpert & R. Cathcart (eds.), Inter/media: Interpersonal communication in a media world. New York: Oxford University Press, pp. 253 –272.
19. Pudowkin, W. I. (1961). Über die Filmtechnik. Zurich: Arche.
20. Sullivan, D. G. (1996). Emotional responses to the nonverbal behavior of French and American political leaders. Political Behavior, 18(3), 311–325.
21. Vaughan, K. B., & Lanzetta, J. T. (1980). Vicarious instigation and conditioning of facial, expressive and autonomic responses to a model’s expressive displays of pain. Journal of Personality and Social Psychology, 38, 909 – 923.
22. Walther, J. B. (1996). Computer-mediated communication: Impersonal, interpersonal, and hyperpersonal interaction. Communication Research, 23, 3 – 43.