How the Presidential Candidates Use the Web and Social Media

Methodology

By Pew Research Center: Journalism & Media staff

About This Study

The primary PEJ staff members conducting the research, analysis and writing for this report included: Research Associate Tricia Sartor, Research Analyst Katerina-Eva Matsa, researchers Nancy Vogt and Steve Adams, Deputy Director Amy Mitchell and Director Tom Rosenstiel.

Other staff members who made contributions were Senior Researcher Paul Hitlin and Communications and Creative Design Manager Dana Page. Copy editing was done by Molly Rohal, communications associate for the Pew Research Center.

Methodology

This study, The 2012 Digital Campaign, had two main research components: a detailed content analysis of the daily content posted on the two presidential candidates’ social media accounts and an audit of the design and more static content offered on their respective websites and social media platforms.

For the study of new posts, PEJ analysts examined all the new content published by the presidential candidates on their respective Facebook, Twitter and YouTube accounts, and the posts published on their website blogs for 14 days in June, and coded them according to 39 variables. This report studied the accounts on Facebook, Twitter and YouTube, which the two candidates’ websites listed and linked to. (All new posts were examined, including status updates, shares, tweets, retweets, videos, blog posts and new homepage content.)

The audit of websites examined each platform for 46 variables and was performed twice, in June and again in late July, to look for changes. The audit of social media platforms examined each for Facebook 14 variables, for Twitter 4 variables and for YouTube 7 variables

The Universe

The 2012 Digital Campaign analyzed all the posts, updates, tweets and videos that each candidate published on their main Facebook, Twitter and YouTube accounts, and their websites for a period of 14 days, from June 4, 2012 through June 17. That totaled 782 posts or pieces of content during the period studied. In addition, PEJ researchers coded the top four pieces of content on the candidates’ homepages during the same period along with all the blog posts entries published during that time.

Capture and Coding

The initial phase of coding involved the training of four coders and five different packages of mixed data, and continued until intercoder testing revealed acceptable levels of agreement on all variables. The rates of intercoder agreement for the "housekeeping" variables such as the kind of tweet, the number of likes and the personal traits of speakers were all at 95% or above. The rates of agreement for the additional key variables, such as topic, focus of post and call to action were 82% or higher.

Once training was completed, researchers began coding the content for the study. Each post was captured at 9 a.m. EST two days after it was posted.

Each post captured was coded for the following variables:

Attention: Determines which posts generate the most attention (through shares, likes, comments, retweets, favorites) in the two days after it was posted.

Technological Format: Determines the type of the post. It designates the technological format of the post, whether, for example, the post was text, video, audio, etc.

Link to Destination: Designates whether the post linked to an additional source, either external or internal. This variable characterized where the link takes the user.

Trigger: Determines what prompted the post, whether it was published in response to an outside stimulus or was initiated by the campaign not based on any external stimuli.

Main Speaker (in video): Determines the main character or speaker in any video posted. This variable followed the 50% rule, i.e. the person had to speak in at least 50% of the post to be considered a main speaker. In the cases in which multiple speakers were seen in at least half of a video, the person speaking more was chosen. (If the persons appeared the same amount of time, the one who appeared first was chosen).

Personal traits and affiliations: Determines the personal traits and characteristics of the video’s main speaker/speakers. For example, traits as gender, race, religion, age etc.

Focus of Post: Determines which candidate or politician(s) is the focus of the post.

Intent of Post: Determines the tone of the post towards the politician focused on. This variable reflects whether the post is attacking, praising, contrasting that person with another or is neutral towards the political figure. An attacking post contains only content critical of an opponent. A contrasting post contains information about both candidates.

Sub-group or Vehicle of Focus: Determines the group or person the candidate is using to make his point or send a message. For example, a candidate might use his spouse, another family member, staff, an outside group or media personality to make a point.

Topic: Determines the subject or topic of the post. This variable designates which issue or event is being discussed in the post, i.e. the economy, domestic politics, fundraising, foreign policy, etc.

Call to Action: Determines whether the post invites the reader to get involved or act in some way. This would include seeking quotes or feedback for an issue, posts that seek opinion from readers, ask what readers think, ask readers to perform an action, such as volunteer or donate, or ask readers to provide some kind of response or feedback.

Website and Social Media Audit

The second component of the analysis was an audit of design and more static content of the candidates’ websites and their respective Facebook, Twitter pages and YouTube channels (as opposed to each new post added to those platforms). The website auditing aimed to reveal the differences between the two candidates’ websites but also the differences with the 2008 Presidential candidates’ websites.

A preliminary test audit was conducted on April 12, 2012. A first formal audit was conducted on June 6, 2012. A second audit was conducted on July 31, 2012, to look for any changes, updates or redesigns to the sites and pages.

The websites were captured in their entirety and coded for the following variables:

Information delivery options: This variable reflects the ways a visitor/supporter gets information. It examined sites for the existence of RSS Feeds, Podcasts, search tools, email and mobile alerts, and a mobile version of the website.

Grass roots involvement: This variable reflects the way a visitor/supporter may become involved with the campaign. It examined sites for whether they contained a calendar of events, a fundraising page, an option to make calls for a candidate, a sample script for making such calls, options to pick a state or issue you would make calls about, an option to send tweets for a candidate, the ability to host an event, register to vote, work on voter registration, shop, donate, contribute to a citizen blog, or comment on the candidate’s content.

Social Networking: This variable reflects the way a visitor/supporter may become involved with the candidate’s social media. It looked for the existence of a social media feed. It also designated how many and which social media sites the candidate was involved in.

News Room: This variable reflects the ways the campaign delivers news and engages with the news media. It checked for whether the site offers press releases, news articles, blog posts, and videos.

Targeting By State: This variable reflects the way a visitor/supporter may join different state groups and get tailored information based on the state chosen. PEJ analyzed 15 state pages to see the level of customization depending on a user’s state of choice. States included: Hawaii, Illinois, Michigan, Massachusetts, Nevada, Colorado, Iowa, Ohio, Pennsylvania, Virginia, North Carolina, Florida, New Hampshire, California, Texas (home states, states where they were in office, swing states, and two big states for each candidate.)

The social media pages audit followed the same schedule as the website audit; social media pages were audited on June 6, 2012 and on July 31, 2012.

The social media pages were captured and coded for the following variables:

For Facebook, the variables examined included the number of likes of the candidate’s Facebook page and what kind of information the candidates mentioned in their "About" sections. Also, PEJ researchers examined various Facebook page features related to the different activities that a candidate engages in on this platform. These features included information about the candidates’ education, activities and interests, favorite books, music and TV shows, political views, relationship status, and religion. Researchers also examined how many photo albums and videos the candidate had uploaded.

The variables examined for Twitter included the number of followers for individual feeds, the number of other accounts that Twitter accounts studied followed, the number of total tweets and what kind of information the candidates included in their "About" sections.

For YouTube, the variables examined included the number of subscribers, the number of uploaded videos, the date the candidate joined YouTube, the total video views, what kind of information the candidates mentioned in their "About" sections and how many playlists their channels had.