Other Analyses of the Twitter Conversation
State of the Union 2013 and Twitter
Winning the Media Campaign 2012
Social Media Debate Sentiment Less Critical of Obama than Polls and Press Are
How Social and Traditional Media Differ in Treatment of the Conventions and Beyond
In Social Media, Support for Same-sex Marriage
Methodology
Data regarding the tone of conversation on Twitter were derived by the Pew Research Center’s Project for Excellence in Journalism from a combination of traditional media research methods, based on long-standing rules regarding content analysis, along with computer coding software developed by Crimson Hexagon. That software is able to analyze the textual content from millions of posts on social media platforms. Crimson Hexagon (CH) classifies online content by identifying statistical patterns in words. The parallel opinion surveys were conducted by the Pew Research Center for the People & the Press during the same general time period as the Twitter data were aggregated.
The data on Twitter comes from an analysis of all publicly available Tweets. The time period for each event varied, but none included more than three days worth of reaction. For each subject, multiple search terms were used to identify appropriate tweets. For example, to find messages commenting on President Obama’s 2013 State of the Union Speech, Tweets were included if they appeared in the four hours following the start of his speech and used the words “state” and “union,” or “Obama,” or “SOTU.”
Unlike most human coding, CH does not measure each post as a unit, but examines the entire discussion in the aggregate. To do that, the algorithm breaks up all relevant texts into subsections. Rather than dividing each Tweet, paragraph, sentence or word, CH treats the “assertion” as the unit of measurement. If 40% of a story fits into one category, and 60% fits into another, the software will divide the text accordingly. Consequently, the results are not expressed in percent of Tweets, but rather the percent of assertions out of the entire body of stories identified by the original Boolean search terms.
Extensive testing by Crimson Hexagon has demonstrated that the tool is more than 90% reliable, that is, in more than 90% of cases analyzed, the technology’s coding has been shown to match human coding. Pew Research spent more than 12 months testing CH and its own tests comparing coding by humans and the software came up with similar results.
In addition to validity tests of the platform itself, PEJ conducted separate examinations of human intercoder reliability to show that the training process for complex concepts is replicable.
In some instances, the Twitter reaction was more pro-Democratic or liberal than the balance of public opinion. For instance, when a federal court ruled last February that a California law banning same-sex marriage was unconstitutional – a case that is now coming before the Supreme Court – the reaction on Twitter was quite positive. Twitter conversations about the ruling were much more positive than negative (46% vs. 8%). But public opinion, as measured in 
This tilt to the Twitter conversation was evident throughout the fall campaign. In nearly every week from early September through the first week of November, the Twitter conversation about Romney was substantially more negative than the conversation about Obama.
Still, the overall negativity on Twitter over the course of the campaign stood out. For both candidates, negative comments exceeded positive comments by a wide margin throughout the fall campaign season. But from September through November, Romney was consistently the target of more negative reactions than was Obama.
The contrast was particularly striking in assessments of last year’s State of the Union. The president’s speech was generally 
Of the eight events that the Pew Research Center tracked since the beginning of last year, there were two – Mitt Romney’s selection of Paul Ryan as his vice presidential running mate and the Supreme Court’s ruling on the 2010 Affordable Care Act – when the reaction on Twitter paralleled public opinion.
When Mitt Romney tapped Ryan as his running mate, it received a more negative than positive reaction both from the 
This is clear when we look at the volume of Twitter discussion on each of the events studied. In the two days following Obama’s re-election on Nov. 6, there were nearly 14 million Tweets from people expressing their reaction. And more than five million expressed their reactions to the first presidential debate. But other events, particularly the federal court ruling on same sex marriage in California last February and Obama’s nomination of John Kerry in December, drew a much lower volume of tweets.
Overall, the reaction to political events on Twitter reflects a combination of the unique profile of active Twitter users and the extent to which events engage different communities and draw the comments of active users. While this provides an interesting look into how communities of interest respond to different circumstances, it does not reliably correlate with the overall reaction of adults nationwide.
