The concept of multimodal communication—conveying messages through multiple modes of communication, such as visual, linguistic, spatial, aural, and gestural modes—stands as fundamental to a comprehensive text that interacts with readers who hold many differentiated identities. Despite the prevalence of multimodal communication, many of our decisions in creating multimodal text are unintentional (Arola et al. 3).
Multimodal communication operates in complex manners that can connect with all five senses despite the process not always being visible to the reader. For instance, examine the features of the advertisement below, a 1.7-metre-tall banner placed on a board on central campus grounds, which I decided to analyse. Despite the numerous modes present throughout the text, I did not initially consider these whilst making the advertisement.
Figure A. A highly visual advertisement with minimal linguistic modes present. Several call-to-actions exist (Casual Gaming Club.org website hyperlink and discord.gg/casualgamingclub group chat hyperlink) to redirect readers to more linguistic modes containing further information about the subject.
When I initially drafted this advertisement, I drew my intentions toward implementing a highly visual mode of communication. What this example particularly conveys, though, is that some modes may surface and showcase their visibility much more than others—in this case, the visual mode is highly present. Although one could argue that most comprehensive forms of texts contain multiple modes compromising with each other simultaneously to puzzle-piece together a panoramic message, often one mode will outweigh others in their presence throughout the text.
In this marketing material, we can scrutinise the elements found in each mode and their effects on readers. Examine the spatial mode of the content: we can easily absorb what takes up roughly three-quarters of the text, which is the visual imagery that also contains the gestural mode of the person’s pose. The entire imagery allows for multiple interpretations of the content, implemented originally to connect with the reader’s sense of humour and feelings.
‘What a peculiar poster. This individual is positioned in a strange manner. I wonder what this is advertising? Casual Gaming Club. This organisation looks informal, they seem to have a… unique sense of humour—it kind of looks like that they embody a ‘meme culture’? I kind of want to meet this guy, he looks peculiar. Is the rest of the organisation’s members like this too? Maybe I would enjoy seeing what the organisation is like and meeting some people. I could go to the website to see when the next event is, maybe it’ll be fun.’
Multimodal text functions in strange, yet complex ways, ultimately instilling readers to create a journey for themselves to embark on. With so many different literary ingredients infusing into a single concoction of meaning, the mixture of content challenges, and thus encourages readers to paint a clear definition themselves of the message that the abstract text conveys. Although not all planned journeys by the writer can go as planned once the text is shared, some deviations from the structured path allow for creative opportunities for the reader to imagine something beyond the scope of the writer’s originality. This allows for a collaborative moment for the reader to unknowingly work with the writer to complete the multimodal text with the reader’s own interpretations.
Part of this journey is also observing the intertwining interactions between modes that complement what the reader sees initially. For instance, after the imagery attracts the reader to focus on the advertisement, the reader will be in search of more information, scanning peripheral content in hopes of alleviating his/her desire for a comprehensible solution. The text reading ‘Casual Gaming Club.org’ is then the next piece of information the reader views to direct a conclusion to the reader’s information search. By fluidly connecting the visual text to point the reader toward the linguistic mode, this simultaneous interaction activates the reader’s ability to link the emotions and opinions extracted from the visual mode to the detailed call-to-action linguistic mode of the organisation’s name, and finally the website landing page shown below. In terms of spatial mode, the placement of the visual mode over the linguistic mode encourages the reader to understand the content top-to-bottom order as intended, leading them from the highly visual mode-based advertisement to the more informational, linguistic mode-based website.
Figure B. Website landing page from the advertisement. With more linguistic modes present on the website, interested students should be able to find more information beyond simply a vague image from the initial banner.
And, despite the lack of an intentional aural mode, the reader’s imagination can act as the catalyst to create their own modes. Maybe the reader recognises the backdrop of the first example’s imagery (which displays a local, Chinese tea café on campus) and then invents ideas of how that environment sounds or feels: bustling conversations, the buzzing of the AC and its cool, breezy breath on the reader’s arms whilst holding a cold, cup of tea. Perhaps the reader even knows the individual on the banner, and then imagines interacting with the individual, or perhaps even reading the linguistic mode vicariously through this person’s tone, pitch, and accent of voice.
What if one doesn’t know any information about the text? Everyone’s identities come into play when interpreting text. For instance, the reader might assume the person’s identity as a male gender and an East Asian ethnicity. All these assumptions may then lead up to biases or stereotypes that affect the reader’s interpretation and drive the text in different directions. Therefore, an important rule of thumb is to always take into account the different identities that any reader could hold in understanding the text.
Taking into account the importance of modes beyond linguistic representations, as I continue my writing experimentation for my papers, understanding the intrinsic value of all modes is highly important. Particularly when I am trying to connect and empathise with my readers, providing multiple access points for the reader to grasp the text is vital. With this in mind, I may consider supplementing my heavily linguistic text with more visuals.
Arola, Kristen L., et al. Writer/Designer: Guide to Making Multimodal Projects. Bedford/St. Martins, 2018.