Scarica Discourse analysis capitolo 9 e più Dispense in PDF di Lingua Inglese solo su Docsity!
A9. MULTIMODAL DISCOURSE ANALYSIS – focuses on tools or ‘modes’ of communication. Multimodal discourse analysts see discourse as involving multiple modes which often work together. In a face-to-face conversation, for example, people do not just communicate with spoken language. They also communicate through their gaze, gestures, facial expressions, posture, dress, etc. Similarly, ‘written texts’ rarely consist only of words, especially nowadays. They often include pictures, charts, or graphs. Even the font that is used and the way paragraphs are arranged on a page or screen can convey meaning. The point of multimodal discourse analysis is to understand how different modes, including speech and writing, work together in discourse. Halliday’s definition of texts: We can define a text, in the simplest way perhaps, by saying that it is language that is functional. By functional we simply mean language that is doing some job in some context, as opposed to isolated words or sentences that I might put up on the blackboard (…) So any instance of living language that is playing some part in a context of situation, we shall call a text. It may be either spoken or written, or indeed in any other medium of expression that we like to think of. (Hallyday, 1989) MULTIMODAL TEXTS A multimodal text may be something written, spoken or a combination of written and oral discourse, but it may also extend beyond the linguistic semiotic to include other meaning-making modalities, and in so doing, may not necessarily include language. RESOURCE INTEGRATION PRINCIPLE In multimodal texts of different semiotic systems (=resources) intertwine (connect) to make meaning. This is called Resource Integration Principle (Baldry, 2006) Semiotic modalities, such as language, gesture, gaze, and so on, are described as resource systems. Multimodal texts integrate selections from different semiotic resources to their principle of organization. Ex. A written page makes use of depiction, written language, spatial positioning, arrangement of items and so on. The resource integration principle refers to the ways in which the selection from the different semiotic systems in multimodal texts refer to, and affect each other, in many complex ways across many different levels of organisation. According to the resource integration principle, texts are never monomodal. Texts of all kinds are always multimodal, combining different semiotic
resources in ways that show generic (i.e. standardised) and text-specific (i.e. individual, even innovative) aspects. Examples Consider a telephone conversation: no visual contact, apparently only spoken language. But we attend to many aspects of the other person’s spoken voice, that are not necessarily part of language: voice quality, breath control, rate of speaking, hesitations and pauses. Speakers and listeners are not always aware of these resources, even that they may be considered as meaning-making resources. Verbal and non-verbal phenomena Language used to be the focus of reflection about human communication. What lay around language was deemed non-verbal : image, gesture, gaze, music, posture, colour, layout. The label non- seems to convey a sense inferiority with respect to verbal language. Is this really the case? The use of “non-verbal” phenomena – which convey meaning – is strictly connected to the context and culture. Consider, for example, the way we say “no” moving the forefinger sideways: it is as effective as pronouncing the word: “no”. So, the gesture conveys meaning as well as verbal communication. But it is essential that the speakers agree on the interpretation of gestural signs. Cultural interpretation Colours don’t have an intrinsic meaning. When communities agree as regards interpreting such meanings associated to colours, the communication is successful. Consider the traffic lights colours or the bride’s colours in western and eastern societies. When we interpret texts, we are involved in a process of cultural interpretation. MODES, MEDIA, AND ACTION Communicative mode or semiotic mode is a system for making meaning: (ex. Modes of speech, writing, gestures, colour, dress…) Any system of signs that is used in a consistent and systematic way to make meaning can be considered a mode. Modes should not be confused with media, which are the material carriers of modes (ex. Telephones, radios and computers are media that can carry the mode of spoken language). They can also carry other modes, such as music, and, in the case of computers and some mobile phones, many other modes like written text and pictures. Multimodal discourse analysis can generally be divided into two types:
Furthermore, with the increasing popularity of interactive text-based forms of communication like instant messaging, blogs and social networking sites, discourse analysts often find that they need to focus both on patterns and structures in the organization of elements in texts and on the sequentiality and simultaneity of actions as people interact using these texts. DIGITALLY MEDIATED COMMUNICATION As new forms of media are developed that allow people to mix modes of communication in new ways over time and space, the whole idea of what we mean by a text, or a conversation is beginning to change. The idea of cohesion – how the elements are connected together – may change and include hyperlinks and even the whole web. Digitally mediated conversations involve modes such as writing, using emojis, images, vocal recordings… And may extend over days on different platforms, such as Facebook, Instagram, WhatsApp, Facetime. Conversation travels across communication media and modes. RESEMIOTISATION Because different modes (and media) alter the kinds of meanings we can make, we need to learn to adjust our discourse in different ways every time we move from one mode to another. This phenomenon is known as resemiotisation. The meanings we make are shaped by the different modes we use. The Australian discourse analyst Rick Iedema (2001) gives an example of resemiotisation : the way meanings associated with the building of a hospital wing were expressed orally in planting meetings, later in the written language of reports, later in the graphic language of architectural drawings, and finally in the materiality of brick and mortar.
- Modes can never really be analysed in isolation from other modes.
- Modes always interact with other modes in texts and interactions.
- Authors and speakers often shift from foregrounding one mode or set of modes to foregrounding one mode or set of modes and in doing so, alter the meaning potential of the communicative environment. The analysis of how multiple modes interact when we communicate can be divided into two branches approaches:
- One focuses on texts such as web pages, films, adverts...
- The other focuses on real time interactions. Both approaches share the idea that different modes have different affordances and different constraints. Different modes have different sets of meaning potential.
B9 MODES, MEANING AND ACTION
When we say that different modes have different affordances and constraints, we mean both that they have different sets of ‘meaning potential’ and that they allow us to take different kinds of actions. (Written text and spoken language: we must present information in a sequential way governed by the logic of time; images are governed by the logic of space.) The fact that different modes make different kinds of meanings more possible and others less possible is one of the reasons why people strategically mix different modes when they are communicating, so that the constraints of one mode are balanced out by the affordances of others. Communicative functions of modes: According to Halliday, language has three basic functions: 1) Ideationa l function: language is used to represent our experience of the world. 2) Interpersonal function: way to communicate something about the relationship we have with the people we are communicating with. 3) Textual function: way to organise ideas so that people can make sense of them. Kress and Leeuwen believe that the three functions originally conceived for understanding language can provide a useful starting point for studying other modes. IDEATIONAL FUNCTION The ideational function of language is accomplished through the linking of participants (nouns) and processes (verbs) – ‘whos doing whats’. In images, participants are portrayed as figure s and the processes are portrayed visually. Images can be: Narrative – representing figures engaged in actions or events. Classificatory – representing figures in ways in which they are related to one another in terms of similarity or differences or as representatives of ‘types’. Analytical – representing figures in ways in which parts are related to wholes. In narrative images, action processes are represented through vectors that indicate directionality. Many images represent multiple processes simultaneously. Mental processes are represented through facial expressions. INTERPERSONAL FUNCTION Any mode has to create and maintain certain kind of relationships between the producer of the message and its recipient.
of clauses, and new information tends to appear closer to the end of clauses, they posit that, similarly, the left side of an image is more likely to contain ‘given’ information and the right side to contain ‘new’ information. This is based on the assumption that people tend to ‘read’ images in the same way they read texts, starting at the left and moving towards the right. The distinction between the upper part of an image and the lower part is related to the strong metaphorical connotations of ‘up’ and ‘down’ in many cultures. According to Kress and van Leeuwen, the top part of the image is often used for more ‘ideal’, generalized or abstract information, and the bottom for ‘real’, specific and concrete information. Multimodality in interaction – modes in face-to-face interaction The mode of gaze, has an obvious interpersonal function , creating a relationship between the gazer and whomever or whatever is the object of the gaze. It also carries ideational meaning, conveying that the gazer is looking at, watching or paying attention to something. Finally, gaze is often an important textual resource, helping people to manage things like turn-taking in conversations.
- One of the problems with analysing real time, face-to-face interactions is that participants have so many modes available to them to make meaning:
- There are what Norris calls ‘embodied’ modes such as gaze, gesture, posture, head movement, proxemics (the distance one maintains from his or her interlocutor), spoken language and prosody (features of stress and intonation in a person’s voice).
- And there are also ‘disembodied’ modes like written texts, images, signs, clothing, the layout of furniture and the architectural arrangement of rooms and other spaces in which the interaction takes place. All of these different modes organise meaning differently:
- Some, like spoken language and gaze tend to operate sequentially
- Others like gesture and prosody tend to operate globally, often helping to create the contest in which other modes like spoken language are to be interpreted. Different modes are likely to take on different degrees of importance at different times.
- Another problem with analysing multimodality in face-to-face interactions is that the spatial boundaries of interactions are not always as clear as the spatial boundaries of texts. While the frame of an image clearly marks what should be considered as belonging to the image and what should be considered external to it, a conversation in a coffee shops not so clearly bounded. In analysing such an interaction, how much of the surrounding modes should be taken into account?
- Norris solves these two problems by adopting the practice of mediated discourse analysis and taking action as her unit of analysis. In determining which modes to focus on, the analyst begins by asking what actions participants are engaged in and the attempts to determine which modes are being used to accomplish these actions. Actions are always made up of smaller actions and themselves contribute to making up larger actions. Norris divides actions into three types:
- Lower-level actions, the smallest pragmatic meaning of units of communicative modes (including things like gestures, postural shifts, gaze shifts, and tone units)
- Higher-level actions (such as ‘having a cup of coffee’)
- Frozen actions (previously performed actions that are instantiated in material modes) One of the goals of multimodal interaction analysis is to understand how participants in interaction work cooperatively to weave together lower-level actions like gesture, glances and head and body movements into higher-level actions, and, in doing so, help to create and reinforce social practices, relationships and identities.
