Overview
New Media, Old Media is based off the data collected from PEJ’s New Media Index and News Coverage Index. The NMI is a weekly report that captures the leading commentary of blogs and social media sites focused on news and compares those subjects to coverage in the mainstream press.
This study aggregates the weekly data collected for the News Media Index from January 19, 2009, through January 15, 2010. It then compares that to data collected for the NCI.
Universe
To study new and social media, PEJ wanted to be able to include as wide a range of outlets as possible. For unlike the traditional press, blogs and social media pages reach into the millions and change daily as new ones emerge and other dissolve. In exploring various options, we saw value combining the work of some sites that specialize in tracking these outlets continuously with our own coding scheme and analytics.
Two prominent Web tracking sites, Technorati and Icerocket, monitor millions of blogs and pieces of social media, using the links to articles embedded on these sites as a proxy for determining what these subjects are. The website Tweetmeme uses a similar method to monitor the popular links on the social networking site Twitter.
Each of these sites offers lists of the most linked-to news stories based on the number of blogs, tweets, or other pages that link to them. PEJ does not determine what constitutes a “news” story (as opposed to some other topic), but rather relies on the classifications used by each of the tracking sites.
A PEJ staff member manually captured the lists from each site every weekday between 9 and 10 am ET. From those lists, the top five linked to articles were captured for further analysis by PEJ staff (SEE BELOW).
Through July 3, 2009, PEJ captured information about blogs from both Technorati and Icerocket. However, the relevant component of Technorati’s site stopped working in early July and has been down ever since. Therefore, the 26 NMI reports beginning the week of July 6-10 only included blog data from Icerocket.
Time Period
From January 19, 2009 through January 15, 2010 PEJ produced 49 weekly reports. Blogs and YouTube were included in all 49 reports.
Twitter was added beginning the week of June 15-19, and therefore there were 29 weeks that included both blogs and Twitter.
There were three weeks in 2009 when no report was issued (March 2-6, November 16-20, and December 14-18).
Coding Procedures
Once the lists of articles were compiled, PEJ staff conducted a content analysis of the subject matter of these linked-to news articles in a similar manner to the News Coverage Index.
Almost all of the codes and rules are the same as with the NCI. The variables coded in both projects include story date, source, story word count, story format, story describer, big story and broad story topic.
The source variable includes the tracking websites we code. The variable for story word count designates the word count of each individual news story. Story format measures the type and origin of the stories, which designates, at a basic level, whether the news story is a product of original reporting, or drawn from another news source. Story describer is a short description of the content of each story. Big stories are particular topics that occurred often in news media during the time period under study. The variable for the broad story topic identifies which of the type of broad topic categories is addressed by a story.
In order to meet high standards of reliability, these variables are all included as part of PEJ’s continuing intercoder testing involving 15 coders and reached levels of agreement above 80%.
For more details about PEJ’s intercoder testing procedures for these codes, refer to the detailed methodology about the News Coverage Index.
The only additional variables used in the NMI were identifying the original outlet of the news story and tracking the number of links aimed at each story included in the sample. Technorati, Icerocket, and Tweetmeme each provided the number of links within their lists.
Calculations
The priorities of the bloggers are measured in terms of percentage of links. Each time a blog or social media page adds a link to its site directing its readers to a news story it suggests that the author places at least some importance on the content of that article. The user may or may not agree with the contents of the article, but they feel it is important enough to draw the reader’s attention to it.
The calculations for the NCI have a different base. That Index measures the time (in seconds) or space (in words) of each story. That is then used to calculate the percent of newshole devoted to each topic.
The reason that the New Media Index uses a different measure, links rather than newshole, is because the nature of online media is different from other traditional forms of media. First, there is no limit to the amount of space that can be devoted to a specific story. In a newspaper, there is a limited amount of space on a front page, for example, and a television newscast is limited by its allotted amount of time. Web sites have no such limits.
Second, PEJ determined that in this procedure, the number of blogs that link to a news article are a far greater measure of the significance of that article online than the length of the story. A particular article might be quite long in terms of number of words, but if only a few blogs link to it, that article would have only a small influence in the new media environment. A short story that gets linked to many times has a far greater influence.
The percent of links for each big story is determined by taking the total number of links in the sample and then dividing that number by the number of links devoted to each specific big story. The percentages are then ranked in order to discover the five storylines that were most present in online commentary.
A Note about the Data
Most of this analysis for this report is based on the top five stories within each particular week. In other words, PEJ compiled the specific big stories that made up the top five within each of the 49 weekly reports in order to arrive at most of the topic data.
However, in a few sections, the data discussed refers to all of the stories and links collected each week, regardless of whether that subject made the list of top stories in the given week. The reason for this difference is because during many weeks, the top big stories were made up of several news articles from different sources. Each of those articles has their own unique qualities, such as format and origination. In order to accurately reflect the format and origination of the all stories that users are linking to, the data must measure all story links and not just the top weekly subjects.
The sections that use this different form of data are acknowledged with footnotes and include the section on sources of blog links and the section on sources of Twitter links.
Differences from the NCI
In addition to the base calculation, there are three differences between the NMI and the NCI to note:
1. The capture times for the Web sites included in the News Coverage Index rotate each day. In the New Media Index the times are the same each day. Since these lists compile the number of links to stories over a 48-hour window, rotating the time of capture would result in different increments of times between each capture. Through testing, PEJ has discovered that the stories on the lists change significantly more over a 24-hour period than they do over a 12 or 16-hour period. Thus it is more methodologically sound to capture at the same time each day.
2. The News Coverage Index is comprised of primarily U.S.-based media outlets, but the aggregators of blogs and social media include both U.S. and non-U.S. blogs. In addition, stories that are linked-to can be from non-U.S. sources.
3. PEJ’s weekly News Coverage Index includes Sunday newspapers while the New Media Index is Monday through Friday.
YouTube Videos
The New Media Index also includes a section of the most popular news videos on YouTube each week.
Each Friday at noon ET, a PEJ staff member captured the list of most viewed news and politics videos on YouTube over the previous week. These videos are categorized as such on the YouTube site and are often a mix of mainstream news reports, raw footage relating to breaking events, or other types of public affairs clips. PEJ determined the top five most viewed videos as they are listed on YouTube’s page at the time of capture.
Note: After consulting various reference guides and outside consultants on usage, the Project has chosen to refer to its several weekly content analysis reports as “indexes”—the version largely accepted in journalism—instead of “indices”—a term used more frequently in scientific or academic writing.
