The first chapter of Writer/Designer, a book by Kristin Arola, Jennifer Sheppard, and Cheryl E. Ball, talks about the different types of modes that we see in everyday life- whether that be with advertisements, articles, or just in the way that we talk to others.
There are five types of “modes” that this book mentions. They are as follows:
Linguistic: the use of language (written or spoken words) – a good example of the linguistic mode is a novel. In novels, the authors do not use pictures or spacial awareness in order to convey their ideas. Instead, they stick with words; this includes the “organization of writing or speech into phrases, sentences, paragraphs, etc.”
Visual: the choices involved with the color, layout, style, size and perspective of images that readers see – an example of the visual mode is a flyer or billboard (basically anything with visual information)
Aural: the sound that accompanies whatever we might be watching – i.e. music, sound effects, silence, tone of voice, volume
Spatial: the physical arrangement; the arrangement, the organization and the proximity between people or objects – an example of using the spatial mode would be how a website is organized or how a brochure might be folded
Gestural: the way that movement (i.e. body language) can make meaning; this includes facial expressions, hand gestures, body language and interaction with other people – when giving a speech, it’s extremely important to be aware of the gestural mode.
The book explains that most advertisements are “multimodal”, meaning they use two or more modes to help them get their point across. For instance, a billboard is not only visual, but it also uses text (linguistic) and the arrangement of the text and the images is well thought out, making it spatial as well.
Over the past few days, I’ve been on the lookout for multimodal examples. Here are a few that I came across:
- I’m in a graphic narrative class, so almost everything that we read is at least three modes – spatial, visual, and linguistic. I’m reading the book Persepolis for the first time and am amazed at how much thought went into each page design.
- My friend shared a video on Facebook from Refinery29’s page about Leslie Knope from Parks and Rec. The video involved the gestural mode, as the entire video was clips of the character Leslie Knope (so gestures, body language etc.). The video was also an example of aural mode, as there was background music playing in addition to Leslie talking. It was visual because the perspective and style of each clip had to be chosen and was also, for a similar reason, spatial– the creator of the video had to decide the order of the clips shown. Finally, the video is also linguistic, as Leslie Knope and other characters are talking the entire time. Therefore, I do think that this video (an other videos like it) cover all 5 modes.
- Snapchat has a feature where you can see various news articles above your friends’ stories. They have everything from CNN to Cosmo. When you open them, each one of these mini articles comes with sound (making them aural) and the background of the front page of them is usually a creative picture or comic (visual) accompanied with text (linguistic). The spatial mode is important to make the mini article easy/fun to read (the placement of the images vs. text). The only mode that not every single one of them uses is gestural, but because some of the articles have videos of people attached, gestural is often included as well.
After completing this exercise, I realized that almost everything that we see and read is multimodal – all the way from the cookbook on my shelf to the snapchats that I watch in the morning. We, as humans, have been using the multimodal technique for as long as we’ve been around. Even the cavemen had hieroglyphics in addition to drawings.
How the modes are used within the texts and how the artists choose to style each design differently does contribute to how different they all are. For example, the New York Times definitely would use different fonts and different layouts than a news source such as Refinery29.