Project for Excellence in Journalism (PEJ) staff and researchers at The George Washington University’s School of Media and Public Affairs (SMPA) conducted this study and contributed to the production of this report, "How Mainstream Outlets Use Twitter."
The lead researchers on this project were PEJ Research Associate Jesse Holcomb, Research Analyst Laura Santhanam and Deputy Director Amy Mitchell, in collaboration with The George Washington University’s Dr. Kimberly Gross and Research Assistant Rachel Weisel. PEJ Director Tom Rosenstiel and GWU’s Dr. Robert Entman and Research Assistant Lauren Martens also participated. Initial coding of the tweets was done by undergraduate political communication students in senior seminar classes at The George Washington University’s School of Media and Public Affairs.[1]
Sample
"How Mainstream Outlets Use Twitter" analyzed one week of tweets on the main Twitter feed of 13 different news organizations-including national and local newspapers, broadcast, radio and online media. To allow the comparison of the news agenda and nature of coverage on Twitter with legacy media, outlets were selected from the list that PEJ regularly monitors for its weekly News Coverage Index.[2] See the table below for the outlets and their Twitter handles that were included in the study.
The abridged NCI sample included six newspapers-three "tier 1" national newspapers with the highest print circulation, and three high-circulation "tier 2" major metropolitan newspapers that were a part of PEJ’s NCI coding rotation.
Each of the three domestic cable outlets entirely devoted to news were included.
In addition, one network news division was included. In order not to over-sample NBC News, given the presence of MSNBC in the sample, researchers selected the news division that, next to NBC, had the largest viewing audience, which is ABC News.
Finally, the web-only outlets in the study included The Huffington Post and The Daily Caller, two online-only outlets that have carved distinct identities with a political point of view. The Daily Caller, which is not a part of PEJ’s NCI sample, was not included in the comparisons between legacy media and Twitter for reasons of consistency. (The Daily Caller was included in the study to provide a leading conservative counterpoint to The Huffington Post.)
Outlets Included in Study with Twitter Handles
|
Newspapers |
Broadcast |
Radio |
Online |
|
The New York Times |
ABC |
NPR |
The Huffington Post |
|
The Washington Post @washingtonpost |
CNN |
The Daily Caller |
|
|
USA Today |
MSNBC |
||
|
The Wall Street Journal |
Fox News @foxnews |
||
|
The Arizona Republic |
|||
|
The Blade (Toledo) |
Researchers captured every tweet during the week of February 14-20, 2011, on the main Twitter account from each outlet, 2,969 tweets in all. The week was chosen because it resembled a typical news week. It is conceivable that in another week, one in which a major breaking news event occurred, the Twitter universe among mainstream media outlets would look different. The goal of this study, however, is to assess the nature of Twitter use during a typical week, not an extraordinary one.
In addition, researchers captured the tweets for this same week from two sets of reporters working for these outlets. These included the reporter from each outlet who had the greatest number of followers on Twitter. To determine the reporters at each organization on Twitter, researchers used a number of strategies. They began by identifying reporters by searching for the outlet on MuckRack.com, a website that lists journalists on Twitter by media outlet and links to their Twitter feeds. In addition, researchers found reporter Twitter accounts by searching for the outlet in Twitter and viewing all of the "people" results. Reporters were added to the list if the Twitter biography identified the person as a reporter for the outlet. Researchers also searched for the outlets in Twello, a website that allows users to search Twitter profiles for key terms such as news organization names. Once again, researchers added any feeds that identified the reporter as working for the outlet in question. Finally, researchers also looked at staff lists (if one was available) on the news organizations’ websites and then searched for the names of staff on Twitter. Here researchers included Twitter feeds for those on staff lists regardless of whether their Twitter profile mentioned the news organization. This combination of searches allowed researchers to build as comprehensive a list as possible of reporters on Twitter from the media organizations of interest in late January/early February 2011. Once researchers had identified all reporters on Twitter, they selected the reporter who had the greatest number of followers as of the end of January for inclusion in the study.
In addition, researchers examined the Twitter feed from a health reporter at each organization in order to examine how reporters on a specialty beat use Twitter (researchers were unable to identify distinct health specialists at the Toledo Blade or The Daily Caller). In the event that an outlet had more than one health reporter, researchers selected for analysis the reporter who had the greater number of followers.
The list of specific reporters included in the study along with their Twitter handles can be found in the table below. There were 677 reporter tweets studied in all.
List of Reporters with Twitter Handles That Were Included in Study
|
News Organization |
Most Followed Reporter |
Health Reporter |
|
The Washington Post |
Steven Goff (Sports reporter focused on soccer) @soccerinsider |
Jennifer LaRue Huget |
|
The New York Times |
David Pogue (Technology columnist) @Pogue |
Tara Parker-Pope |
|
USA Today |
Whitney Matheson (Pop culture blogger) @popcandy |
Liz Szabo |
|
The Wall Street Journal |
Kara Swisher (Columnist for All Things Digital) @karaswisher |
Katherine Hobson |
|
The Arizona Republic |
Paul Coro (Sports reporter) |
Ken Alltucker |
|
Toledo Blade |
Ryan Autullo (Sports reporter) @RyanAutullo |
|
|
CNN |
Jack Gray |
Dr. Sanjay Gupta @sanjayguptaCNN |
|
MSNBC |
Rachel Maddow |
Melissa Dahl |
|
Fox News |
Megyn Kelly |
Dr. Manny Alvarez |
|
ABC |
George Stephanopoulos @gstephanopoulos |
Dr. Richard Besser |
|
NPR |
Scott Simon |
Scot Hensley |
|
The Huffington Post |
Arianna Huffington |
Dr. Patricia Fitzgerald @DrPatriciaFitz |
|
The Daily Caller |
Tucker Carlson @TuckerCarlson |
Audit
In order to understand the broader presence on Twitter, beyond the main news feed of the news outlets in the study, researchers conducted an audit in October 2011.
First, researchers tallied the number of Twitter feeds associated with each outlet. Defunct feeds were not included, nor were reporter feeds. (However, television programs that are branded around an individual were included, for instance, @hannityshow, but not @rachelmaddow.)
Because some outlets did not provide a Twitter directory, slightly varying methods were used to tally the number of feeds.
For the following outlets, researchers relied on an official directory posted on their site:
- The New York Times
- The Wall Street Journal
- The Arizona Republic
- ABC News
- USA Today
For the following outlets, researchers relied on a combination of Twitter lists curated by the organization itself, searches on Twitter, and third-party lists such as those provided by Listorious:
- NPR
- The Daily Caller
- The Blade (Toledo)
- CNN
- Fox News Channel
- MSNBC
- The Washington Post
For the Huffington Post, a careful search of each section on the website yielded its universe of Twitter feeds.
The audit yielded a total of 527 feeds.
Next, a snapshot of Twitter activity was taken by analyzing each of those 527 Tweets during the space of a single day. On October 6, 2011, researchers tallied the number of followers listed, and then tallied the number of tweets that were posted on October 4. October 4 was a relatively typical news day.
Researchers had originally recorded the number of followers for each of the outlets’ main Twitter news feed on February 21, 2011 (except for The Arizona Republic, whose number of followers was recorded on February 25). This number was used to calculate the percentage growth in followers between February and October 2011.
Coding
In developing coding categories, researchers drew on the methodology used by PEJ for its weekly News Coverage Index coding, adapting and expanding it for an analysis of Twitter.[3] For each tweet, coders recorded the following, based only on looking at the tweet itself: Tweet date, source, date coded, number of people who retweeted the tweet, whether the story focus of the tweet was one of the big stories in the NCI for that week, the geographic focus of the story, and the broad story topic.
In order to better understand how Twitter is used by legacy news organizations, tweets were also coded for whether they sought information, were retweets, contained links and, in the case of individual reporters, contained personal anecdotes. Coders noted if the tweet was a retweet and, if so, whether the source of the original tweet was a reporter or another type of Twitter feed at the same news organization, at another news organization, or some other type of source. To understand how much Twitter is used for promotion and driving site traffic, coders noted if the tweet contained a link and characterized the content the link took a reader to. Specifically, researchers recorded whether the link was to news, editorial or commentary, or some other type of content and whether that content was on the news organization’s own website or came from outside the news organization being coded. Every tweet was also coded for whether or not it explicitly sought information from followers. This would include seeking quotes or sources for a story, seeking opinion from followers or asking followers to provide some kind of response or feedback. Finally, in the case of tweets from individual reporters, researchers also coded for whether the tweet contained a personal anecdote. The question here was whether the reporter was relating a personal anecdote about their own life such as discussing what they were doing or commenting on their life.
Every tweet was independently coded by two separate trained undergraduate coders from the School of Media and Public Affairs at The George Washington University. Coders were assigned a set of tweets from one or more news organizations and reporters. Coding was done over one month in late March and April of 2011. If intercoder reliability for a pair of coders on a given variable was 90% or above, the study principals randomly determined which coder’s results to use for the final dataset. If intercoder reliability was less than 90% for a given pair on a given variable, a PEJ or SMPA researcher coded the tweets to reconcile differences between coders and make a final coding decision. Intercoder reliability prior to resolution by the third coder ranged from 77% to 100%.
|
Variable |
Average Intercoder Reliability* |
|
Number people who retweeted (range) |
97.0% |
|
Big Story |
89.0% |
|
Geographic Focus |
79.2% |
|
Personal Anecdote |
93.1% |
|
Information Gathering |
96.9% |
|
Retweet |
98.4% |
|
Retweet Source |
97.2% |
|
Tweet Contain Link |
99.7% |
|
Driving Site Traffic (characterize link) |
77.2% |
*Pairwise percent agreement across 15 pairs of coders
In resolving discrepancies, PEJ and SMPA researchers recoded the first 30 tweets where there was disagreement between coders. If, for a given variable, the third coder agreed with one student in the pair 90% or more of the time, that student coder’s data was used. If the third coder did not agree with one student 90% of the time on the first 30 discrepancies, the coder recoded all tweets where the initial pair disagreed to resolve discrepancies.
[1] The students who participate in the data collection were: Colby Anderson, Molly Anixt, Jennifer Avallon, Christopher Borchert, Andrew Clark, Jamarie Copestick, Tyler J Ducklo, Jonathan Ewing, Andrew Feldman, James Greene, Zachary Hanover, Sarah Hoberman, Owen Hooks Davis, Matthew Ingoglia, Alec Jacobs, Bradley Komar, Thomas Lawrence, Molly Lukash, Sarah Mersky, Evan Miller, Jennifer Nason, Charles Rybak, Carly Schildhaus, Samantha Schneider, Azim Shivji, Emily Smith, Eric Thibault, Brittany Tibbetts, Madeline Twomey and Maria Zisa.
[3] PEJ’s weekly News Coverage Index examines the news agenda of 52 different outlets from five sectors of the media: print, online, network TV, cable and radio. The weekly study, which includes some 1,000 stories, is designed to provide news consumers, journalists and researchers with hard data about what stories and topics the media are covering, the trajectories of that media narrative and differences among news platforms. The percentages are based on "newshole," or the space devoted to each subject in print and online and time on radio and TV. (See complete methodology.)
