How can the mighty New York Times, which considers itself America's paper of record, be the paper of record in cyberspace when its articles barely show up on Google?
This has to be more than just a slight irritation to the Times, because search engines play a key role: They collate information, and on the Internet there's a whole lot of that, often too much. (Hence the term data smog.) In essence, they act as informational portals. So if you're trying to get the dope on your favorite author, hip-hop MC or representative, or learn more about an important issue dominating the news, your first stop may very well be Google.
But recently, when I googled the terms "Iraq torture prison Abu Ghraib" -- certainly one of the most intensively covered news stories of the year -- the first New York Times article was the 295th search result, trailing the New Yorker, Guardian, ABC and CBS News, New York Post, MSNBC, Slate, CNN, Sydney Morning Herald, Denver Post, USA Today, Bill O'Reilly on FoxNews and a host of others news sites.
What's more, tons of other non-traditional news sources came ahead of the Times, including a number of blogs and low-budget rabble-rousers like Antiwar.com, CounterPunch, truthout and Beliefnet (a site dedicated to spirituality). So did Al-Jazeera (twice). But the Times still ranked low, even after it plastered an Abu Ghraib story on its front page for 32 straight days between May and June. And Google isn't the only one to shun the Times: I got similar results from other search engines (AltaVista, Lycos, Yahoo).
Two years ago, Martin Nisenholtz, chief executive of New York Times Digital, bet $1,000 that nytimes.com would outrank all blogs on Google by 2007, based on a search of five keywords on a topical news issue. Unless Google and the Times work on their relationship -- Nisenholtz says they're talking, although they haven't come up with any answers yet -- there may be a day when The New York Times doesn't show up at all on the Net's most popular search engine. Ultimately, this could be a direct threat to the Times' legacy.
Of course, like many things about the business operations of a traditional publisher that has ventured online, the reasons are simple but the solutions complicated. The New York Times requires that its users register, which makes it difficult for search engines to spider its content. Perhaps an even more impenetrable barrier is the Times' paid archive. Because it stows material more than a week old behind an archive wall, you have to cough up $3 per article. Since few are willing to pay for content they can get free elsewhere, search engines, which often base results on relevancy (read: popularity), will continue to dis the Times -- as well as other media sites that make you register or pay for old news (The Washington Post, The Wall Street Journal).
It's not like the Times reaps a whole lot from its Web archive. The archive accounts for only 2 to 3 percent of the profit for its digital division (which includes both the Times and Boston Globe websites, as well as all its electronic database lease deals). In fact, New York Times Digital earns most of its money from a pre-existing agreement with Lexis-Nexis, which brings in more than $20 million a year. This year, NYTD will report a profit of about $25 million. Without this Lexis-Nexis bounty, however, the color of the ink on its digital balance sheet would run decidedly more gray than black.
So it's no surprise that Times management has no plans to completely open up its archive. "There isn't a compelling business argument today that would suggest that giving away our content is a good idea," Nisenholtz said. Even though the Lexis-Nexis deal is an all-you-can-eat model -- not based on usage -- the Times can ill afford to undermine its relationship with such an important customer. It simply can't charge Lexis-Nexis tens of millions of dollars while giving away the same content free over the Web.
But the Times already gives away content on its website, which is updated between 20 and 30 times a day, offering the day's paper, plus other articles in advance of publication, pictorial slide shows and reader forums. When you think about it, the Times may have it backward. It charges $1 for the latest news in print, and offers it free over the Web, but for old material demands $3, which is three times the price of an entire newspaper.
"The Web isn't just a sales outlet, it's much more akin to a vast public library," said Aaron Swartz, who coded the Times' link generator, which enables bloggers to get around the Times' paid archive by transforming temporary article links into permanent ones, so that its articles don't expire in a week. "A far more sensible position for the Times would be to charge for new news, not old news. Can you imagine the possibilities if it opened up its archive?"
Swartz has a point. Behind its firewall, the Times houses a newspaper archive dating back to 1996, complete with photos and graphics, and occasional multimedia features with movies and animation. If it were made available, "weblogs could thoroughly compare modern political coverage with coverage from 1996 or 2000," Swartz said. They could "see which Times predictions were right and wrong, and reread with hindsight the coverage of the dot-com boom and bust." The permanence of its reporting and extension of its brand name in cyberspace would more than compensate for the paltry amount of money it would lose from its paid archive.
Media consultant Vin Crosbie of Digital Deliverance said he thinks the archive issue points to a greater deficiency in the Times' long-term strategy. "The Gray Lady is a beautiful clipper ship, but it's losing steam and not taking hold of the potential of the Internet," he said.
Crosbie believes the Times should customize its content so that readers could pick and choose which stories they want based on their own particular interests, rather than having to wade through the site's table of contents. Indeed, only half of the Times Web users enter through the homepage. The rest come via links provided by mass e-mails, blogs and other publications (like Wired News). On average, visitors spend about 43 minutes a month on the website, according to Nielsen/NetRatings, which gives them just enough time to sample a story or two that interests them, then leave for more topical environs. That's a fraction of the 28.2 minutes per day (3.4 times per week) a typical reader spends on the morning newspaper.
This a la carte mentality has already permeated another digital medium: music. Downloaders are skipping albums in favor of singles because people refuse to pay $17 for a compact disc when all they want is one or two songs -- which has led some soothsayers to forecast the death of the album. If so, will the death of the newspaper as we know it follow? Because if that's the case, the Times could be in trouble.
The economics of digital media are certainly working against it -- even though Nielsen ranks The New York Times on the Web as the No. 1 newspaper site on the Internet. (And contrast the Times with the Tribune, which spent $600 million to develop an online presence, without much to show for it.) The Times attracts 9 million unique visitors a month, while only about 1 million read the daily paper. But the dot-com makes a scant $11 per user, while the printed paper earns the Times a whopping $900 per reader (in subscription fees and advertising).
As Crosbie said, "as fewer and fewer people read print publications and (more) make the switch to online, the Times" -- and its competitors -- are "going to have to figure out a way to make more money on the Web."
Haven't people been saying that since the boom and bust days of the Internet?
I consulted Google for some answers, but when I plugged in "future viability Internet news," no New York Times article was anywhere to be found.
Adam L. Penenberg is an assistant professor at New York University and the assistant director of the Business and Economic Reporting program in the department of journalism.
