• Custom Writing Services
  • How to Write a Research Paper
  • Research Paper Topics
  • Research Paper Examples
  • Order

Communication

iResearchNet

Custom Writing Services

Custom Writing Services
Communication » Communication Theory and Philosophy » Modality and Multimodality

Modality and Multimodality




The modalities of communication – speech, song, writing, images, etc. – constitute general registers of human expression and experience. As such, they lend themselves to different forms of technological mediation, both individually and in combination as multimodal communication.

The various modalities are at once biologically conditioned and culturally shaped. On the one hand, the human senses enable people to identify particular affordances (Gibson 1979) or possibilities for action in their environment. On the other hand, representations of this environment are realized in culturally conventional forms, in embodied as well as technologically mediated communication. Such representations, further, come to constitute affordances in their own right. Research on, for example, computer-mediated communication has examined media as cultural artifacts with specific affordances for social interaction (Norman 1990). Hutchby (2001) identified distinctive affordances of interaction that takes place either face-to-face, via telephone, or in online chat. In a perspective of media and communication theory, the modalities can be considered potentials of the human senses for meaningful discourse in and about the natural and cultural environment – potentials that are extended through information and communication technologies.




While everyday language, like Aristotle, distinguishes five main senses – sight, hearing, smell, taste, and touch – communication research has focused special attention on aspects of sight and hearing in social interaction, even while recognizing, for example, the role of touch in interpersonal communication and experiments integrating smell into cinema (“odorama”). A key issue is how to typologize modalities in relation to sight and hearing: how many modalities are there? With reference to face-to-face interaction, Stivers and Sidnell (2005) employed a main distinction between a vocal–aural and a visuospatial modality, sub-dividing the latter into gesture, gaze, and body posture and the former into lexico-syntactic, vocal, and prosodic channels, because each of these could be said to make a difference in the process and outcome of communication. From a perspective of technologically mediated communication, Kress and Van Leeuwen rather suggested that language as such is a modality or mode, “because it can be realised either as speech or as writing,” adding that writing is itself a semiotic mode that can be realized as “engraving in stone [or] print on glossy paper” (Kress & Van Leeuwen 2001, 6).

In a historical perspective, modalities can be understood as an intermediate category between the material vehicles of communication – human bodies and information technologies – and its discursive vehicles – texts or messages. Technologies – from writing to the Internet – have separated and recombined modalities in specific ways. As brought home by McLuhan (1964), the material and modal forms of messages affect their interpreted contents. In an oral culture, face-to-face communication is, by definition, multimodal, involving several sensory registers at once. Writing and print focus the vision of the individual reader into a communicative gaze on texts, although reading aloud and subvocal reading persist. Radio reproduces a different, secondary orality (Ong 1982), and simultaneously integrates multiple modalities of language, music, and soundscapes. Cinema and television, in public and private contexts of reception, recreate a rich experience of natural and social settings, and of other humans, in sound as well as vision. Digital media, such as online virtual worlds, arguably, take multimodal communication to the next degree, simulating features of face-to-face interaction.

While it may, thus, be argued that communication history has witnessed a trajectory from multimodal, via tendentially monomodal, and back toward multimodal media, the configuration of modalities in any given period has been complex, both within the available technological media – for instance, illustrated books and magazines – and in the multistep flows involving such media and face-to-face communication (Katz & Lazarsfeld 1955). All along, language has been a master modality of sorts, providing both a model for other communications and the means by which all other modalities could be redescribed and explained. As noted by Émile Benveniste (1969/1985, 236), “the signs of society can be interpreted integrally by those of language, but the reverse is not so.” Also in this regard, language can be considered a key component of individual psychological development as well as of self-reflective cultures.

The concept of modalities remains relatively undeveloped in media and communication research. The coming of more explicitly and emphatically multimodal digital media technologies may be an opportune moment to reconsider the definition and operationalization of modality, also in historical studies of analog media. A renewed round of sound media studies has raised questions, for instance, of whether music and speech should be considered media, or modalities that are realized in different media (Cook 1998; Jensen 2006). In retrospect, “new” multimodal media can throw light on “old” media, their aspects and degrees of monoand multimodality.

References:

  1. Benveniste, É. (1985). The semiology of language. In R. E. Innis (ed.), Semiotics: An introductory anthology. Bloomington, IN: Indiana University Press, pp. 228 –246. (Original work published 1969).
  2. Cook, N. (1998). Analyzing musical multimedia. Oxford: Oxford University Press.
  3. Gibson, J. J. (1979). The ecological approach to visual perception. Boston, MA: Houghton Mifflin.
  4. Hutchby, I. (2001). Conversation and technology: From the telephone to the Internet. Cambridge: Polity.
  5. Jensen, K. B. (2006). Sounding the media: An interdisciplinary review and a research agenda for digital sound studies. Nordicom Review, 27(2), 7–33.
  6. Katz, E., & Lazarsfeld, P. F. (1955). Personal influence. Glencoe, IL: Free Press.
  7. Kress, G., & Van Leeuwen, T. (2001). Multimodal discourse: The modes and media of contemporary communication. London: Arnold.
  8. McLuhan, M. (1964). Understanding media. New York: McGraw-Hill.
  9. Norman, D. A. (1990). The design of everyday things. New York: Doubleday.
  10. Ong, W. (1982). Orality and literacy. London: Methuen.
  11. Stivers, T., & Sidnell, J. (2005). Introduction: Multimodal interaction. Semiotica, 156(1/4), 1– 20.

Back to Communication Theory and Philosophy.




Communication Research

Communication Research

  • Communication Theory and Philosophy
    • Aesthetics
    • Roland Barthes
    • Cognitive Science
    • History of the Idea of Communication
    • Communicology
    • Constructivism
    • Critical Rationalism
    • Critical Theory
    • Cultivation Theory
    • Cultural Studies
    • Culture
    • Cybernetics
    • Deduction, Induction, and Abduction
    • Discourse
    • Emic and Etic Research
    • Fiction
    • Functionalism
    • Genre
    • Jürgen Habermas
    • Hermeneutics
    • Idiographic vs Nomothetic Science
    • Information
    • Information Science
    • Harold Innis
    • Interaction
    • Interactivity
    • Knowledge Interests
    • Harold D. Lasswell
    • Paul F. Lazarsfeld
    • Linguistics
    • Walter Lippmann
    • Marshall McLuhan
    • Meaning
    • Medium Theory
    • Modality and Multimodality
    • Models of Communication
    • Objectivity
    • Paradigm
    • Phenomenology
    • Postcolonial Theory
    • Postmodernism
    • Pragmatism
    • Psychology in Communication Processes
    • Realism
    • Remediation
    • Semiotics
    • Structuralism
    • Symbolic Interactionism
    • Systems Theory
    • Texts and Intertextuality
    • Verstehen vs Erklären
    • Development, Gender, and Communication

Custom Writing Services

Custom Writing Services