Mastering Multimedia   

It’s not enough to post some text and then simply throw some video into the mix. To keep readers’ attention and enhance the audience’s understanding, it’s critical that each ingredient in a rich multimedia stew is placed precisely where it makes the most sense.

Mastering Multimedia It's not enough to post some text and then simply throw some video into the mix. To keep readers' attention and enhance the audience's understanding, it's critical that each ingredient in a rich multimedia stew is placed precisely where it makes the most sense.

By Ronald A. Yaros

It has been a while since video took the Web by storm. It has also been a long time since people started using the word "multimedia." Like journalism itself, such terms are rapidly redefined in a media landscape that constantly changes. That's why innovation in a newsroom isn't just learning how to shoot and embed video or using Twitter to cover a live event. Innovation must also include developing, testing and using new story techniques that keep audiences engaged.

For many years photos and graphics were largely an afterthought at newspapers. It wasn't uncommon for a reporter to approach the photo desk on the Friday before a big Sunday takeout was going to run and ask if maybe it couldn't find some way to illustrate the package.

That's hardly the case today, 27 years after USA Today revolutionized the look of broadsheets. But there's a similar phenomenon that takes place at online news sites.

Most use text and video as if they were as effective online as they are in print and on television. Sites assemble a text story, then often plunk down "related" graphics and video, which usually runs too long. Many still assume that "multimedia" refers only to those stand-alone, interactive Flash animations that focus on one topic rather than to a well-planned, well-produced Web page of interactive stories that strategically weaves multimedia features with text.

Taking a little more time, when possible, to properly organize an online story can pay big dividends. Typical visitors to a Web site want to scan for news of the hour as quickly as possible. But when they arrive, they too often are bombarded with a potpourri of options for video, text, feeds, tweets, polls, photos, maps and slideshows. Understandably, most users don't have the time or the desire to piece together all of the elements.

The challenge for journalists is to create new structures for online news. They must ask themselves, "How should the various forms of media be combined into the best possible story for rapidly changing and mobile communities?"

One of the keys to getting it right is contiguity, a fancy way to describe the process of adding multimedia elements and combining them with text at just the right place in a story. Put that video or timeline or reader comment or related link at the place in the text where it's most relevant — but also plan to provide some overlap between the video and text. That's because many readers will just read or watch one or the other.

Contiguity is to multimedia what coherence is to text. Achieving it requires more planning and a more complex structure than we currently use online — one that strategically combines multiple elements to present just one well-thought-out story. The user can choose different aspects of that story and still obtain some context.

The audience seeks, selects and processes information on the nonlinear Web much differently than it does in traditional media. At the same time, despite the nonlinear order of selections made online, users still mentally assemble information one piece at a time and in a linear sequence. The complexity in our new challenge as journalists is that readers progress linearly through a nonlinear environment of virtually unlimited choices. Many Web sites aren't effectively addressing that audience behavior.

Ten years of research with college students who grew up multitasking with video games and learning from multimedia shows that paying attention to contiguity can generate more interest in and comprehension of content on the part of younger readers, regardless of whether the news is produced by a professional journalist or your neighbor down the street. On the Web, it is time for every story to offer interactive and contiguous media in place of pages of related elements.

Since users jump word to word, word to graphic, word to video and back, the most effective multimedia story quickly provides key connections between text, video, polls and the like. Sure, a full-length video next to a page of text works for the intensely interested reader. But the general news audience, which is just trying to stay informed, benefits from small chunks of explanatory information. Research shows that readers will spend more time on a site when it includes text explaining how all of a story's elements relate to one another.

My research of how online readers approach and comprehend multimedia news also found that users who scan stories nonlinearly rate coherent, contiguous stories significantly more interesting than traditional stories. Users also learned significantly more from the contiguous stories.

Today's research results might help foretell the expectations of tomorrow's audiences as they engage in more texting, tweeting, game-playing and social networking to share information. People will continue to become more technically savvy, and journalists will need to become even more technically skilled. This is why it's no longer good enough to post just well-written text or embed a three-minute video in the text and assume the message stands out to the typical Web user.

Assembling an interactive multimedia story with contiguity from beginning to end is a new skill to teach and learn. It takes time, and is not easy for someone used to creating or editing a linear text or video. In a sense, the attention devoted to structuring multimedia replicates the attention journalists have always given to structuring a traditional print story for clarity.

This spring, CNN.com presented a large number of user-generated iReporter videos in which people commented on the large bonuses given to executives at AIG, the famously bailed-out insurance behemoth. Unfortunately, the videos were isolated from other stories on the topic. What the feature desperately needed was a chronological multimedia story with text to explain what had happened, accompanied by a strategically placed graphic outlining AIG's finances and equally well-placed reaction videos. Note the emphasis on placing the reactions at the appropriate place in the text as opposed to simply posting them "around" the story or on a different page.

In the case of the CNN graphic, for example, the site should have added text to guide the user: "Here, people across America sound off about the AIG bonuses" and "Here, you'll find expert opinions on executive compensation." Adding text that explains the relationships among the multiple features also helps readers to assemble their own digital version. This is a reader-centered structure, where all the components of a multimedia story are brief but explanatory nuggets of news that efficiently communicate.

Planning a reader-centered structure needs to start as soon as the story idea is conceived, not after the text is written. Ideally, blogs and audience-provided content that relate directly to the story appear next to relevant text in the story. The journalists craft the "discussion" from the community instead of just telling a story and waiting for reaction. Graphics explaining technical terms, trends or processes also significantly increase audience interest and understanding. All of the elements should blend in the eyes and minds of readers.

Our Lab For Communicating Complexity With Multimedia finds stories, restructures them and then tests an audience's level of interest and comprehension for the various versions of the story. When asked to analyze online stories, my students find plenty of examples of video and photos with text. However, they rarely find contiguity or coherence in the way the various elements are combined. In addition to learning the technology itself, students come to recognize that rethinking how content is assembled is the first step in producing more accessible news packages.

For example, an excellent piece on illiteracy in America at ABCNews.com contained all of the well-produced elements for a contiguous multimedia story. Personalization (based on audience input) and interactivity were combined with video and text. Some relatively minor revisions, such as shortening video clips, placing related elements closer together, reducing the amount of text and briefly explaining what each of the multimedia elements meant to the audience, significantly increased the story's effectiveness. More information was presented more efficiently, and online users appreciate that.

My students noted how the readers added an important dimension to the ABCNews.com story, but these reactions were even more effective when placed beside the text that describes how society responds to those who are considered illiterate. The story became a conversation.

While content management systems such as Drupal or Microsoft's SharePoint efficiently aggregate information, they miss the mark on contiguity that a human can provide when combining text and media. Page layouts and colors may be attractive, but it's ultimately the content and the way it is structured that engage the user for more than a few seconds.

Text with one or two specific photos plus at least one brief (i.e., 10-second) video clip, combined with appropriately placed audience input and an explanatory graphic or brief animation addressed in the text, is more effective at extending the engagement of the general news audience than a dense page of text or a page with four- or five-minute videos.

Perhaps news outlets could offer at least two versions of a story, one condensed and the other long-form, to accommodate all levels of interests. Of course, that takes even more time and resources. Short of that, the best thing to do is to provide overlap between the multimedia elements without being overly redundant. For example, posting a page of text and a long video from an interview — or even worse, the long video itself without text — won't engage most users. Producing a story so people easily navigate through text, understand what the video is about and find a graphic in the proper context may tempt them to click on more content. For stories about complex issues, such as the economy, science or health, embedding a brief but interactive animation where the text gets complex replicates a technique used in education that multimedia journalism should employ more often.

What happens when readers perceive that a multimedia story doesn't really hang together? They bail out. I call it a cognitive "kick-out," or the point at which users essentially say they are not interested.

Research shows that kick-outs include pop-up windows, unexpected surveys and audio that plays automatically. (For obvious reasons, most people checking out sites at work dislike pages that unexpectedly pump out music.) Forced registration before viewing selected content ranks almost as high on the kick-out list as slow download times. With the recent surge of discussion about the merits of erecting pay walls around news sites, there's debate as to what extent charging for content will kick out a Web site's audience.

Some journalists believe that academic research lags behind the "real world" and is not always the best source for answers to new questions. But applied research studies can help define and refine ways of using multimedia more effectively. That is particularly crucial in dealing with audiences that are getting more selective and technologically involved with each passing day.

Engaging audiences via new ways of assembling multiple media isn't easy, but it isn't impossible. Many online users can't describe in detail why they find certain content interesting, but they'll spend more time on coherent multimedia. Presenting text and countless link choices cafeteria style, with the hope that someone, somewhere, will click on something, risks losing an audience. That's a risk most news organizations can ill afford.

Ronald A. Yaros (ryaros@jmail.umd.edu) teaches at the Philip Merrill College of Journalism at the University of Maryland and is director of the Lab For Communicating Complexity With Multimedia. Details of his research can be viewed at: http://www.merrill.umd.edu/ronyaros/