Traffic Problems   

How the drive to attract massive numbers of visitors to their Web sites (and the advertisers that might follow them) is having a profound effect on news judgment at traditional news organizations.

On an ordinary weekday in August, the following stories and videos were prominently displayed on the Web sites of some of the nation's most respected news organizations:

–CBSNews.com's "recommended" videos included "Miss Transvestite South America," "Shark Swims Ashore Caught on Tape" [sic] and "Newlyweds Pay for Wedding by Recycling." Its "most popular" list featured "Diary of a Showgirl," "Smoking Baby is Real," "Fired for Being Too Sexy" and "Alligator Feeding Frenzy Caught on Tape."

–ABCNews.com had "How to Guide Your Dreams," "Sharks Scare East Coast Swimmers," "Lindsay Lohan Heads to Rehab" and " 'The View': Lady Gaga's Vagina Monologue."

–NBCNews.com's top-of-the-site display box included links to MSN.com stories such as "50 Stars from 50 States," "11 Telltale Signs He May be Having an Affair" and "6 Diet Trends You Should Never Try." Some of the site's own video news features were "Volunteers Drive into Russian Blaze" and "Falling Ice Kills Girl, 11."

–HuffingtonPost.com had such headlines as "Sex Tape Pics...," "Kardashian Visits Cowboys," "Killer Bat Fungus," "World's Worst Urinal" and "Naked Lady Gaga Talks Drugs and Celibacy."

It doesn't take a computer scientist to understand the whys and wherefores of this kind of editorial decision-making. High-minded headlines and stories about foreign wars, the federal deficit or environmental despoilage might have paid the bills in the age of Murrow and Cronkite, but they only go so far these days. Shark videos and "naked Lady Gaga" headlines get major play on "serious" news sites for an obvious and no longer terribly shocking reason: They draw traffic. And these days, traffic, the massive ebb and flow of clicks and hits, is the Internet equivalent of the Nielsen ratings, the currency that determines the course of billions of dollars in advertising.

Of the many changes that the Internet has delivered to the nation's newsrooms, the ability to measure traffic for a given story, blog or video may be among the most profound. Publishers, editors and advertisers have always tried to ascertain the public's preferences. But audience research and reader surveys were invariably slow and backward-looking. Until the Internet came along with its server logs and audience metrics, no system gave editors a near-instantaneous verdict on their editorial decisions.

For centuries, journalists divined what the public wanted to know essentially by guessing about it.

Now that journalists and advertisers do know, the question is, what's this knowledge doing to journalism? As the Lady Gaga headlines suggest, is the ability to discern what's "working" at any given moment a recipe for manipulation, titillation and pandering? Or is all that data really a godsend, providing the key to a better, more compelling and, yes, more financially sustainable type of news?

The advent of highly detailed metrics hasn't eliminated the need for creative editors, but it is guiding some of their calls. So-called content farms, such as Yahoo!'s Associated Content, AOL's Seed and Demand Media, have essentially automated the functions of the assignment desk. Demand, for example, uses sophisticated algorithms to sift through haystacks of Web searches and other data to identify popular search topics (say, "garden gnomes"). It then marries this intel with data on keywords that advertisers are paying the most to be associated with on search pages (say, "repairs"). Story topics ("how to repair garden gnomes") are then farmed out to armies of freelancers, who produce thousands of how-to articles and informational videos pegged to whatever the algorithms have detected.

The finished goods — featurettes on everything from travel tips to gardening to home repairs — then appear on popular Demand-owned Web sites like eHow.com and Livestrong.com, or on sites owned by mainstream media partners, such as USA Today, the Atlanta Journal-Constitution and the San Francisco Chronicle. Demand hardly makes its freelancers rich, paying as little as $15 for a few-hundred-word article or a short video, and none of what's produced will likely win an Oscar or Pulitzer. Nevertheless, these search-engine-friendly pieces can be highly profitable once they start climbing the search rankings and start capturing Google ads on the Web pages where they appear. "We believe that consumers tell you exactly what they need--if you'll just take the time to ask or listen," declares Demand's corporate "manifesto." And how: Although Demand hasn't turned a profit yet, it filed for an initial public stock offering this summer that could raise $125 million.

Demand doesn't do news content (it profits from articles with long shelf lives) but something like its robo-model is emerging at Yahoo!, the world's most heavily visited news site (see "Embracing Local Content," page 52). In early July, Yahoo! News unveiled The Upshot, a blog that could offer a glimpse of the future of news. The Upshot's stories about politics and government are determined primarily by mining Yahoo!'s massive database of search queries and by analyzing what's drawing traffic on Yahoo!'s homepage. Like Demand, the idea is not to fight the herd but to determine its direction and go along with it.

As Mark Walker, Yahoo!'s vice president of news and information, describes it, the portal got an early taste of the power of analytics-driven journalism in 2008. Staffers noticed that searchers were asking why Olympic divers took showers after climbing out of the pool (answer: a warm shower loosens the muscles after exposure to pool water). So it commissioned a story about the topic and posted it, drawing massive traffic. In mid-July, Walker says, Yahoo!'s analysts spotted rising interest in the search term "state fair"; The Upshot's staff quickly produced a piece detailing little-known facts about state fairs.

Walker insists there's still room for the human touch in Yahoo!'s brave new blog. "We're really seeking to balance the things that are great about traditional methods of journalism" with some new tools, he says. "We have a world-class network and robust insight into our audience's needs and behaviors, and we're marrying that with a team of world-class journalists. We can deliver great content to our audience in a way we've not done before. But we still need great reporting from people with insight into the world to make it work."

My own newsroom at the Washington Post certainly isn't as sophisticated in its use of analytical data as Yahoo!. But all of us have had to learn to adjust and adapt to a traffic-counting, click-centric age. At times, it has been an uncertain process, like entering a new land without maps or landmarks to guide us.

Every journalist at the Post is aware of the new metrics, and everyone has been schooled in such basics as search-engine optimization, some several times. Under Marcus Brauchli, who became the paper's executive editor in mid-2008, the paper's Web staff (then housed in a separate office in suburban Virginia) was merged into the paper's downtown newsroom. Many news and copy editors now report to a streamlined "universal news desk," which mobilizes reporters, photographers and graphic artists in response to breaking developments.

The Web's growing dominance has naturally affected whom the Post hires, and for what purpose. After a series of newsroom buyouts, the paper has begun to hire a few new reporters, mostly to beef up signature beats like national politics, national security and foreign coverage. But new-media production jobs have grown apace. Gene Weingarten, a double Pulitzer Prize-winner for feature reporting, parodied our direction in a recent magazine column: "Every few days at The Washington Post, staffers get a notice like this: 'Please welcome Dylan Feldman-Suarez, who will be joining the fact-integration team as a multiplatform idea triage specialist, reporting to the deputy director of word-flow management and video branding strategy. Dylan comes to us from the social media utilization division of Sikorsky Helicopters.'"

The things that made the Post a journalistic paragon, enterprise and investigative reporting, haven't gone away. "Top Secret America," our massive, three-part series about the nation's intelligence infrastructure, was the result of two years of reporting; it could very likely win a third Pulitzer for its coauthor, Dana Priest.

Still, there's no question that many of our daily editorial decisions are guided by what's "working," as defined by readers. Stories rise and fall on our homepage throughout the day, based in part on what the traffic data indicate about a story's "performance." Anyone in the newsroom who wants to know how a story, a blog, a video or an online chat is faring can easily find out. Our internal "Daily Trending Report" lays it all out, with green arrows indicating that a story is gaining traffic and red arrows signaling declines.

Understanding what readers want, and giving it to them, is hardly a bad thing, particularly in an era when print circulation is ticking ever downward. And the numbers sometimes offer powerful lessons. Jim Brady, who ran the Post's Web site until early last year, told me that the paper's editors gained a critical editorial insight from the huge traffic that followed in the wake of a story from the paper's Style section in 2005. The piece, by fashion writer Robin Givhan, was a sharp-edged critique of then-Vice President Dick Cheney's casual attire at an official ceremony commemorating the 60th anniversary of the liberation of the Auschwitz death camp. The enormous online reaction to the piece "identified the intersection of politics and fashion" as a topic of interest to the Post's readers, says Brady, who now runs TBD.com, a news site covering the Washington, D.C., area (see Drop Cap, Summer). Givhan produced a number of stories assessing the wardrobe choices of the powerful (including a memorable one about Secretary of State Condoleezza Rice's choice of "knee-high boots" on a visit to a U.S. military base), all of which received prominent play. (Givhan says she was just doing her job, and was never asked to produce a specific piece to stir up traffic.) Givhan won a Pulitzer for criticism the next year, becoming the first fashion writer so honored.

Daily life in the traffic lanes is a bit more mundane. Universal desk editors are under constant pressure to maintain the paper's traffic goals; several told me that they believed their job evaluations depended, at least in part, on how often they meet these goals (Brauchli says this isn't the case). Nevertheless, when the numbers fall below targets, the staff scrambles to goose the count. There's no real playbook for this drill, but there are some gimmicks. The Post often throws up celebrity photo galleries, sometimes with dubious or tenuous news value (a recent gallery featuring the British royal family was pegged to mere speculation about a forthcoming royal engagement). Another gambit: frivolous "user polls" (a recent one asked if readers in Montgomery County, Maryland, planned to flush their toilets in defiance of temporary water restrictions). Editors also monitor trending topics on Twitter and Google, and sometimes adjust their mix of stories to include something about a hot topic. Brauchli, however, says it's "somewhat misleading" to suggest that the paper chases pageviews when the numbers flag. Rather, he says, the effort is constant: "We should always be asking what we can do or should do, within the Washington Post's brand domain, to keep audience."

Some of the tactics sometimes inspire grumbling in the newsroom. The common complaint is that the Post is lowering its standards--"cheapening its brand" is the vogue-ish term. But Brauchli says a photo gallery on "a B-list celebrity" doesn't compromise the serious and substantial work the Post does. "We're not expending any real resources on this stuff," he says. Instead, galleries and the like help keep traffic and advertising flowing so that the paper can concentrate its real journalistic firepower on important things.

Of greater concern, perhaps, is how the Post plays the traffic game with breaking news. The paper's working assumption is that it cannot be left behind on any story of importance. Readers do want the latest, and they'll find it somewhere else if the Post doesn't give it to them. But this is where things can get really sticky. "Getting something up" on a breaking story can be risky. It usually means reporting fragments of the story. Worse, it increases the chances that a story will be inaccurate. Brauchli is adamant about this point: "We have no tolerance for getting things wrong.... Our market positioning, as it were, is being right. [We're] the place you can count on for factual reporting."

But even with the best of intentions, problems can arise given the post-it-now ethic of the Web. Like a number of news outlets, for instance, the Post's Web site in July was quick to link to a clip of Andrew Breitbart's video of a speech given by Shirley Sherrod, a Department of Agriculture official, at an NAACP event. The video, which Breitbart had first posted on his own site, was rapidly turning into a sensation, drawing TV coverage and widespread commentary. Breitbart's edited excerpts seemed to paint Sherrod as a racist; while employed by a nonprofit organization 24 years earlier, she seemed to say, she had denied assistance to a farmer simply because he was white.

As the Post's reporters learned more over the course of the day, they added details: the NAACP's reaction, the administration's response and, by the end of the day, the news that Sherrod had been asked to resign by her boss, Agriculture Secretary Tom Vilsack. Except, as the world knows now, the edited video posted by Breitbart gave a flawed impression. The full speech made clear that Sherrod helped the farmer, and they became friends. Sherrod urged her listeners to consider the story a cautionary tale about the evils of discrimination, more or less the opposite of what Breitbart's first posting seemed to show.

It wasn't until late that first night that the Post placed the Breitbart video in its proper context. The incident--essentially a case of linking first and finding out the facts later--began to look like its own cautionary tale of excessive speed. No less a media critic than Barack Obama said afterward that Vilsack "jumped the gun" on ousting Sherrod partly because "we now live in this media culture where something goes up on YouTube or a blog and everybody scrambles."

Brauchli says the Post covered the story properly, given its limitations. The rule, he says, is incrementalism and transparency: Tell the reader as much as you know, and tell him what's still being checked and verified. "We need to say [to the reader], 'Here are the limits of our knowledge,'" he says. "It's important to signal that this is breaking, and our knowledge may be limited."

Brauchli rejects the idea of waiting until the full picture has emerged before posting something online. The reason: Traffic. Or, as Brauchli puts it, "Dependability. You want readers to feel that they can come to you when something big is happening and learn everything that is available at that point, and know what is true and what to believe." He adds, "We can't be invisible on a [hot] subject. We don't have the luxury of waiting until the next day's paper to cover stories that are central to our readers."

You'll get a somewhat different perspective on this, of course, from many who came of age in the pre-Internet, 'if your mother says she loves you check it out' era.

"Journalism always put a premium on speed and scoops, but up until recently we never had to make the decision that speed trumps vetting or verification," observes my colleague Roxanne Roberts. "That dynamic is shifting because of the need for hits. It's a very slippery slope from an ethical standpoint."

Roberts, who co-writes the Post's Reliable Source column, says she and co-writer Amy Argetsinger report and verify all of the items about the famous and powerful that appear under their bylines. But celebrity news is such a potent traffic magnet that they are under constant pressure from the Post's online editors to post items that haven't been fully vetted. "Our Web folks will ask, 'Can't we post it and say we're checking it?' "

Actually, no, Roberts says she responds, because so much "news" about celebrities turns out not to be true upon checking. (Witness Tiger Woods, who continues to be the subject of dubious "scoops" --see "Lost in the Woods," March.) While there are consequences for being slow, there aren't many consequences for being wrong, Roberts says: "The feeling nowadays is, 'we don't make mistakes, we just make updates.'" By trying to grab traffic at all costs, "We've placed the premium not on being correct or thoughtful, but on being first. When you do that, everything is Balloon Boy."

What many editors may not realize is that a big spike in traffic may not mean much in terms of advertising. Traffic surges are like sudden jumps in TV ratings: They make everyone feel good, but they're almost always a non-event in terms of ad sales, since advertising is bought in advance. Shrewd advertisers know that sharp jumps in traffic are rarely maintained, since the people who flood a site one day may never return.

Another question is, where did a site's traffic come from? The Post is one of the few local newspapers in America with a national and international following. On a daily basis, the vast majority of its million-plus visitors come from outside the Washington area (unlike the printed Post, which circulates largely within greater Washington). This means that only a fraction of those visiting the paper's Web site are likely to be of interest to local advertisers.

Instead of simply counting traffic, "engagement" — how long visitors spend on a site, how many pages they view and how often they return — is a better measure of customer loyalty, says Randy Siegel, president of local digital strategy for Newhouse, publisher of Cleveland's Plain Dealer, Newark's Star-Ledger and other papers. "Ad agencies and [advertisers] are slowly figuring out that not all traffic is created equal," he says. "It's not how much but what kind you have. There's a lot of one-time traffic out there that you'll never see again."

The more valuable visitor is probably attracted to something more than a Lady Gaga headline or the summer's latest shark video. "It's not just about getting attention; it's about providing a rich experience," says Patricia Handschiegel, a media entrepreneur and Internet consultant. So create something of unique value, she urges, not just commodity headlines that get people to drive by a site with all the attention span of a motorist passing a billboard. "Newspapers still can report in a broad way," she says. "When was the last time a blogger went to [Iraq] and reported?"

TBD.com's Jim Brady says traffic data is like any tool; it all depends on how it's used. "You have to pick your niche and stay focused on it, or your mission will get muddled," he says. "If everyone followed the numbers, and only followed the numbers, we'd all end up with the same Web site in five years."