Multimodal Discourse: A Visual Design Analysis of Two Advertising Images

Abstract

The area of discourse analysis has long neglected the value of images as a semiotic resource in communication. This paper suggests that like language, images are rich in meaning potential and are governed by visual grammar structures which can be utilized to decode the meanings of images. Employing a theoretical framework in visual communication, two digital images are examined for their representational and interactive dimensions and the dimensions' relation to the magazine advertisement genre. The results show that the framework identified narrative and conceptual processes, relations between participants and viewers, and symbolic attributes of the images, which all contribute to the sociological interpretations of the images. The identities and relationships between viewers and participants suggested in the images signify desirable qualities that may be associated to the product of the advertiser. The findings support the theory of visual grammar and highlight the potential of images to convey multi-layered meanings.

1. INTRODUCTION

Since its emergence in the early 1970’s, the study of discourse analysis has been focused primarily on language and its forms [1], [2]. Consequently, semiotic resources such as images, space and architecture had been largely ignored and resulted in what O’Halloran [1] called an “impoverished view” of discourse (p. 1). Recently, however, there has been a shift in discourse that has begun to acknowledge the value of these modes of communication for making meaning. One such mode that has received greater attention is the use of images in visual communication. Perhaps the most widely recognized theoretical framework for studying images is Kress and van Leeuwen’s [2] visual semiotic theory. Following Halliday and Hasan’s [3] systemic functional approach to language, Kress and van Leeuwen [2] propose that images, like language, have grammar structures that can be analyzed for meaning.

This paper aims to analyze the visual grammar of two advertising images using Kress and van Leeuwen’s [2] theoretical framework. A semiotic analysis will be conducted to examine the visual grammar of the advertisements and decode their meaning potential. The first part of the paper will attempt to describe the grammar of visual communication by looking at the components of the framework. The second part of the paper will attempt an analysis of the advertisements according to the following three points of evaluation:

The three points of evaluation are based on Halliday’s [4] proposition that all modes of semiotics, including visual design, consists of three major functions. The first two points explore the ideational and interpersonal functions of the images, respectively, and the final point examines the contribution of these two functions to the communicative effect and purpose of the images. The following section will briefly discuss the contribution of Michael Halliday to social semiotics and its influence on Kress and van Leeuwen’s [2] theoretical framework.

 

2. LITERATURE REVIEW

The underpinnings of Kress and van Leeuwen’s framework in visual communication come from the seminal work of linguist Michael Halliday. In his book ‘Language as social semiotic’ [4], Halliday proposed that language is a semiotic system that uses semiotic resources to create meaning. He described language as being structured in three configurations that operate simultaneously where each configuration represents a different function, or ‘metafunction’, in meaning-making. The first metafunction of Halliday’s [5] systemic functional framework is the ideational function, which expresses the speaker’s experience of the world. It is the content function of language and includes statements such as, ‘clouds are white’. The second metafunction is the interpersonal function of language. It expresses the role of relationships and identities of individuals in social interactions between people. The third metafunction, textual, connects linguistic elements (e.g. clauses) together into whole unified texts. Jewitt and Oyama [6] describe it as the metafunction that “brings together the individual bits of representation and interaction into the kind of wholes we recognize as specific kinds of text or communicative event” (p. 140). Kress and van Leeuwen [2] believe that visual design, like all semiotic modes, fulfils these three metafunctions, and incorporate Halliday’s ideas into their theoretical framework of semiotic analysis. Although the framework is comprehensive, Jewitt and Oyama [6] describe it as a descriptive framework that alone does not “offer all that is needed for the sociological interpretation of images” (p. 154). They argue that “it can only ever be one element of an interdisciplinary equation which must also involve relevant theories and histories” (p. 138). The following sections will provide a brief summary of the framework’s components that are relevant to this paper’s analysis.

2.1 The Representational Dimension

This dimension stems from Halliday’s [4] ideational metafunction where it seeks to represent the relationships that exist between things in the world and within us. There are two processes under this dimension: Narrative processes and conceptual processes. Kress and van Leeuwen [2] describe narrative patterns as dynamic and serving to “present unfolding actions and events, processes of change, transitory spatial arrangements,” whereas conceptual patterns are static and “represent participants in terms of their class, structure or meaning” (p. 59). Kress and van Leeuwen [2] refer to two types of participants in their framework. The first type, represented participants, refers to the people, places and things that are represented in images and texts. The second type, interactive participants, refers to those who produce the images and texts (e.g. painters, photographers) and those who view and read them.

2.1.1 Narrative process: In narrative processes, represented participants in an image are connected by a vector, a line formed by elements in the image. Vectors can be formed by bodies, limbs or tools in action when participants are represented as doing something to or for each other [2]. They can be “eye-lines or gestures indicating a line of force in a particular direction” [7]. The participant that creates the vector is called the ‘Actor’ and the participant that receives it is the ‘Goal’. One type of narrative processes called ‘reactional processes’ occur in images when a vector is formed by an eyeline and creates a reaction rather than an action [6]. The participant doing the looking is referred to as the ‘Reacter’ and the object or person receiving the gaze is the ‘Phenomenon’. A reaction in an image can be either transactional, where the Reacter and Phenomena are present, or non-transactional, where only the Reacter is present [2].

2.1.2 Conceptual process: In contrast to narrative processes, conceptual processes are static (no vectors) and represent participants in terms of their “more generalized and more or less stable and timeless essence, in terms of class, or structure or meaning” [2]. In other words, conceptual processes are concerned with the representation of ideas in images where participants can be analyzed, classified or defined (e.g. charts, tree structures and scientific diagrams). In this paper’s analysis, the two main types of conceptual processes are analytical and symbolic processes.

Analytical processes depict visual elements in a partwhole structure. The two components of the structure are ‘Possessive Attributes’ (the parts) and ‘Carrier’ (the whole). Fashion shots and posed photographs in advertisements can be analytical where the entire outfit as a whole is the Carrier and the parts of the outfit are the Possessive Attributes [2]. Symbolic processes, on the other hand, are concerned with defining the meaning or identity of a represented participant. In Symbolic Attributive Processes, an image can include two participants: the ‘Carrier’, whose meaning or identity is established in the relation with the ‘Symbolic Attribute’, which represents the meaning or identity itself. Symbolization in images is primarily achieved by Symbolic Attributes, which tend to be more salient (e.g. exaggerated size, color), be pointed out in the image by a gesture, look out of place or have conventional symbolic values [2]. Human participants in such images usually pose for the viewer. They are shown sitting or standing without reason instead of being involved in some action.

2.2 The Interactive Dimension

The interaction between the producer and the viewer of the image (interactive participants) is the second dimension presented by Kress and van Leeuwen [2] and it is based on Halliday’s [4] interpersonal function. According to them, producers visually encode social meanings into images through the gaze of the represented participant, the distance of the participant from the viewer and the angle from which the participant is seen by the viewer.

2.2.1 The gaze: In images where represented participants look at the viewer, vectors are formed between the participants and viewer, connecting both parties at an imaginary level [2]. These images make ‘demands’: “The participant’s gaze demands something from the viewer, demands that the viewer enter into some kind of imaginary relation with him or her” [2]. In such cases, the type of relation is dependent on the facial expression and gesture (if present) of the participant (see Table 1).

Table 1.Types of demand relations [6], [2]

In images where represented participants do not look directly at the viewer, Kress and Van Leeuwen [2] call them ‘offer’ images because they offer “the represented participants to the viewer as items of information, objects of contemplation, impersonally, as though they were specimens in a display case” (p. 119).

2.2.2 Social distance: The second structure is related to the different relations that the distance between a represented participant and the viewer can suggest. Kress and van Leeuwen refer to Edward T. Hall’s [8] work in ‘proxemics’ to describe how social relations are determined by different fields of vision in images (see Table 2).

Table 2.Social distance and relations [8]

Therefore, according to Hall [9], the shorter the distance between the participant and viewer the more intimate the relation becomes.

2.2.3 Angle: A third structure that produces relations between represented participants and the viewer is the angle or point of view. There are two angles discussed by Kress and van Leeuwen [2]: Horizontal and Vertical. The horizontal angle is “a function of the relation between the frontal plane of the image-producer and the frontal plane of the represented participants” [2]. The angle can express whether the imageproducer and viewer are involved with the represented participants or not; a frontal angle indicates involvement whereas an oblique angle indicates detachment. On the other hand, the vertical angle can be related to power. If a represented participant is seen from a high angle by the viewer, then the viewer is depicted as more powerful. However, if the represented participant is seen from a low angle, the represented participant is viewed as holding the power in the relationship. In cases where the picture is at eye level, the relationship between participant and viewer is one of equality and there is no power difference involved. Kress and van Leeuwen [2] also point out that the level of involvement, detachment and power is graded and depends on the degree of the angle.

2.2.4 Modality: Image modality is another aspect of the interactive dimension in the framework and is related to how viewers judge realism in images. Kress and van Leeuwen [2] describe modality judgements as “social and dependent on what is considered real in a social group for which the representation is primarily intended” (p. 156); and for this reason they view modality as interactive rather than representational. The truthfulness or credibility of an image is determined by modality markers that are listed in Table 3.

Table 3.Modality markers [2]

However, the realism of the image is dependent on the context in which it is used. Kress and van Leeuwen [2] argue that different domains of representation such as scientific, abstract, naturalistic and sensory have different coding orientations and thus different definitions of high modality (realism). An example they provide is that of blue prints, a colorless visual. When blue prints are judged in the standard domain, naturalistic coding orientation, they are considered as having low modality, but in the technological coding orientation, the colorless blue prints are effective for their purposes and regarded as having high modality. This paper’s analysis of the images will follow Kress and van Leeuwen’s [2] classification of advertising images under the sensory coding orientation.

2.3 The Compositional Dimension

The third dimension of the framework is related to Halliday’s [4] textual metafunction. The focus here is on “the composition of the whole, the way in which the representational and interactive elements are made to relate to each other, the way they are integrated into a meaningful whole” [2]. The image elements can be analyzed according to three systems: information value, salience and framing. These systems will not be further discussed in this paper since the advertising images will not be evaluated according to the compositional dimension.

 

3. METHODS

The advertisements in this analysis were selected from a Korean fashion magazine titled ‘CeCi’, which is published monthly by Joongang M & B. The two advertisements were published in the September 2010 issue as part of an advertising campaign by Evisu, a Japanese clothing company. Advertisement 1 depicts a male model, and Advertisement 2 depicts a female model.

An analysis of the advertisements will be attempted according to the three points of evaluation: 1) the ways the participants are represented; 2) the relationships suggested between the viewer and represented participants; 3) and how these relationships relate to the purposes of the magazine advertisement genre. The analysis of the first and second points of evaluation will be based on the representational and interactive dimensions of Kress and van Leeuwen’s [2] framework, respectively. The analysis of the third point will draw on sources from advertising and communication theory.

 

4. FINDINGS

Kress and van Leeuwen [2] state that their “grammar is a quite general grammar of contemporary visual design in ‘Western’ cultures” (p. 3). However, “in many parts of the world, Western visual communication exists side by side with local forms” (p. 4) where Western forms might be used in advertising. In a study on American and Korean magazine advertisements, Hovland et al. [10] found that there were Western elements in Korean advertisements and suggest that “the Westernization of Korean advertising seems to be firmly at work” (p. 897). It is likely that these Western forms are used in Ceci magazine, which features a large number of Western brands and Western-looking advertisements.

4.1 Representation of Participants

In Fig. 1, there is only one represented participant: the male model. He appears to be dancing, which would be considered as ‘doing something’, fitting into Kress and van Leeuwen’s [2] description of narrative processes. Since the participant’s eyes appear to be looking at something the viewer cannot see, it can be assumed that the eye-line formed is a vector emanating from the model to the unknown object. Therefore the process is reactional and the participant can be identified as a Reacter because he is reacting to the unknown object. The absence of a Phenomenon further classifies the image as a non-transactional reactional process; it leaves the viewer to imagine what the Reacter is looking at or thinking about, and “this can create a powerful sense of empathy or identification with the represented participant” [2]. The identity of the participant can be represented by his reaction: he appears to be doing a celebratory dance as indicated by his ‘fist-pump’, conveying the idea of success. He is also alone and appears engaged in his own thoughts while dressed in a ‘trendy’ outfit. Perhaps the image producer is attempting to represent the participant as an independent, fashionable, successful, young man; a ‘winner’ who possesses qualities that the viewer would like to identify with.

Fig. 1.Evisu Advertisement 1 (CeCi, September 2010)

Similarly in Fig. 2, there is also only one participant; however, in this case, the female model is not ‘doing something’ and appears static. This appears to be an example of a conceptual process. At first glance it may seem to be, more specifically, an analytical process where the model is the Carrier and the parts of her outfit the Possessive Attributes. However, the advertisement does not contain any detailed labels or descriptions of her outfit. Perhaps the more appropriate process is the Symbolic Attributive Process which is concerned with what a participant symbolizes. As mentioned above, in such processes, human participants usually display themselves in a standing or sitting pose for the viewer without reason. In Fig. 2, the model appears to follow this description closely, standing and slightly leaning on the large round concrete disc without any apparent purpose. The concrete disc might be the Symbolic Attribute due to its salience created by its large size, the fine detail of its texture, and its portrayal of a mythological symbol, a full moon. In mythology, the moon can be a symbolic representation of a lunar deity, and in the case of Fig. 2, the ‘moon’ may function as an Attribute that establishes the model’s identity as a ‘goddess’. Thus, it may be the image producer’s intent to represent the participant as a young desirable idol.

Fig. 2.Evisu Advertisement 2 (CeCi, September 2010)

4.2 Relations between the Viewer and Participants

This section will briefly look at the gaze, social distance and angle between the represented participants and viewer as well as the modality of the advertising images.

4.2.1 The gaze: The eyes of the male model in Fig. 1 are directed not at the viewer but at something outside of the image frame. The represented participant becomes the object of what Kress and van Leeuwen [2] call “the viewer’s dispassionate scrutiny” (p. 116) and the viewer becomes an “invisible onlooker” (p. 116). As a result, Fig. 1 can be categorized as an ‘offer’ image where the represented participant offers himself to the viewer as an object for contemplation. Without eye contact, the viewer may see the participant in an impersonal and detached way, perhaps as someone who is preoccupied in his own world, which the viewer is excluded from.

This separation can be analogous to the division between a winner and loser further supporting the participant’s proposed identity as a ‘winner’ in the previous section. In Fig. 2, the female model looks directly at the viewer and ‘demands’ that the viewer enter an imaginary relation with her. The kind of relation she demands can be determined by her facial expression. An examination of the model’s face reveals that she has a cold stare perhaps asking the viewer to relate to her as an inferior would to a superior [2]. This relation is congruent with the love and respect commanded from ordinary people by a goddess and provides support to the suggested identity of the model.

4.2.2 Social distance: In both images the represented participants’ figures are entirely visible. According to Hall [8], the entire figure of a participant is visible in close and far social distances. The difference between the two fields of vision is that there is space around the figure at far social distance. In the images, this space is not present above and below the participants, which suggests that they are at close social distance. This distance is where “impersonal business occurs” [9] and defines a social relation of acquaintanceship. Thus the close social distance might indicate that the viewer can become acquainted with the ‘winner’ and ‘goddess’ but cannot get close enough to form a friendship with them. This relation further defines the participants as superior to the viewer.

4.2.3 Angle: The represented participant in Fig. 1 is shown from the side at an oblique horizontal angle. As a result, the viewer in this case is detached from the participant and his world. What the viewer sees is not part of his or her world; this world belongs to the participant and the viewer is not involved with it [2]. From the vertical angle perspective, the participant is seen from a low angle by the viewer which indicates that the power in the relation is with the participant. Both the oblique and low angles seen from the viewer’s point of view further establishes the suggested relation between the viewer and the participant: the viewer and participant are not in the same group, and the participant is superior.

In Fig. 2, the represented participant is shown from the frontal angle which indicates involvement with the viewer. The image producer says, “What you see here is part of our world, something we are involved with” [2]. Similarly to Fig. 1, the vertical angle depicts the model as more powerful where the viewer looks at her from a low angle. According to Kress and van Leeuwen [2], models in magazine advertisements generally look down on the viewer and are depicted as exercising symbolic power over the viewers. This explanation provides further support to the proposed symbolic identity of the model as a ‘goddess’. In addition, the frontal angle may indicate that the viewer is a part of the participant’s world, thus establishing the participant as the viewer’s ‘goddess’ not someone else’s.

4.2.4 Modality: As mentioned above, Kress and van Leeuwen place the advertising context under the sensory coding orientation. High modality (realism) in this orientation is achieved with amplified modality marker values (e.g. full color saturation) that makes an image appear ‘more than real’ when compared to the standard naturalistic coding orientation. In Fig. 1 and 2, the colors are not vibrant and the depth is natural (no ‘fish–eye’ perspective) which lowers modality. However, contextualization, representation, illumination and brightness values are greater than those of naturalism and increase the modality of the images. As a result, both images have varying degrees of realism, or credibility, but they appear to have higher modality than not in the advertising context. The image producer, by presenting the images as somewhat ‘realistic’, may be suggesting to the viewer that the identities and attitudes of the participants are real, and that they can be identified with.

4.3 Relations and the Magazine Advertising Genre

According to Dyer [11], the primary function of advertising is to introduce a wide range of consumer goods and services to the public. In most cases advertisers use social images and constructs to sell particular products [12]-[15]. However, over the years advertising has gradually become “more involved in the manipulation of social values and attitudes, and less concerned with the communication of essential information about goods and services” [11]. This is evident in Fig. 1 and 2 where the only information that is provided is the name and logo of the brand and its website address. Therefore a possible conclusion that can be drawn is that one of the main functions of the advertisements is to establish social relations between the represented participants and the viewer.

The analysis of the relationships according to gaze, social distance and angle suggests that the participants are depicted as being superior to the viewers and possess almost a mythical aura to them. This is supported by Dyer [11] who believes that “the images of men and women in ads are usually considered to be mythic rather than real” (p. 116). These desirable symbolic identities and attitudes of the models can be used by advertisers to promote their products by transferring these qualities to the products. As Dyer [11] states, “The meaning of one thing is transferred to or made interchangeable with another quality, whose value attaches itself to the product” (pp. 116-117). This is important since potential buyers identify with these images and link certain moods, sensations, and images of themselves with particular products [16]. As a result, the desirable identities and attitudes of the models can be obtained through the acquisition of the outfits advertised. Additionally, the high modality of the advertisements may further contribute to the promotion of the products by stating to viewers that the models’ qualities are real and obtainable. By establishing social relations between the represented participants and the viewer, the advertiser can improve the promotion of their goods leading to increased sales.

 

7. CONCLUSION

The aim of this paper was to apply Kress and van Leeuwen’s [2] theoretical framework of visual design on two advertising images. The advertisements were analyzed according to the representational and interactive dimensions of the framework. Although this analysis was attempted on advertisements from a Korean fashion magazine, the framework appeared capable of decoding the meaning potential of the images. From the analysis it was found that 1) the represented participants seem to be represented as superior figures that possess desirable qualities; 2) the social relations established between the participants and viewer appear to support the identities represented; and 3) the identities and relations suggested in the advertising images can be used to promote the products of the advertiser.

These findings suggest that images have grammar structures that can be analyzed for meaning and support Kress and van Leeuwen’s [2] theory of visual grammar. However, to interpret the visual elements of the images, this analysis referred to external sources on mythology, advertising and communication. The use of these sources may provide support to Jewitt and Oyama’s [6] argument that the framework alone is insufficient for sociological interpretation of images and needs to be used in cooperation with other interdisciplinary theories. Nonetheless, one point that has been made clear from this analysis is that images have the potential to convey multilevelled meanings. With further development in visual semiotic theory, images, as a semiotic resource, may have the potential to take a more prominent role in communication.