Study Design & Documentation

Introduction

The Pew Research Center’s Teen Relationship Study was funded, designed and analyzed by Pew Research staff. Quantitative fieldwork was conducted by the GfK Group (GfK, formerly Knowledge Networks). Specifically, the survey examined the attitudes of teens ages 13 to 17 years old, as well as those of their parents, toward technology. The survey examined friendships and romantic relationships within the context of technology use. The survey was conducted using sample from KnowledgePanel®.

The study also conducted 12 in-person focus groups and four online focus groups. The in-person groups interviewed a total of 70 teens ages 13 to 17 years old in three cities in the United States in November 2014. In-person focus groups ranged in size from four to eight participants, and were separated by gender and divided by middle schoolers and high schoolers. Additionally, participants in four of the high school groups (two each with boys and girls) were required to have had some romantic relationship experience, either currently or in the past, to be in the group. Participants were paid a $50 incentive as a thank you for their participation in the research. Participants were recruited with the help of Resolution Research. The online focus groups, conducted in April 2014, ranged in size from seven to nine teens from around the United States in each group. The online groups interviewed a total of 32 teens ages 13 to 17. Each group was gender and age segregated (boys/girls and middle school/high school.) Each participant received a $60 incentive for their participation. The online focus groups were recruited and hosted by 20|20 Research and moderated by the lead author.

The rest of the Methods section describes details about the recruitment, interviewing and weighing of the quantitative survey.

Quantitative Sample Definition

The target population consists of the following: parents of teens ages 13 to 17 and teens 13 to 17 years old residing in the United States. To sample the population, GfK sampled households from its KnowledgePanel, a probability-based web panel designed to be representative of the United States. The survey consisted of three stages: initial screening for the parents of teens ages 13 to 17, the parent survey, and the teen survey.

The main data collection field periods were from September 25, 2014, through October 9, 2014, and from February 10, 2015, to March 16, 2015. The second data collection was targeted toward black parents and teenagers, with the intent of increasing to reportable levels the number of black teens in the sample. Parents completed the parent section of the survey in six minutes (median). Teens completed the teen section of the survey in 16 minutes (median). The survey was conducted in English and Spanish. Parents and teens could select different languages for the survey.

Survey Completion and Sample Sizes

The number of respondents sampled and participating in the survey, the survey completion rates for the screener and main interview, and the incidence/eligibility rate are presented below.

Key Survey Response Statistics: In-Field Screening

  • N Sampled for Screener: 4111
  • N Complete Screener: 1637
  • Screener Survey Completion Rate: 39.8%
  • Qualified for Main Survey: 1060
  • Incidence Rate: 64.7%

Margins of ErrorWhile 1,084 parents completed the parent section of the main survey, 1,060 teens completed the teen section of the main survey; the 24 unpaired parents were not included in the final counts. The margin of error for the full sample of teens (n=1060) or parents (n=1060) is plus or minus 3.7 percentage points. This report is focused on teens with any romantic relationship experience – who replied yes to the question “Have you ever dated, hooked up with or otherwise had a romantic relationship with another person?” 35% of teens responded yes, and for “teen daters” n=361, Please see the adjacent chart for the margin of error for this and other subsamples in this study.

Survey Cooperation Enhancements

As a standard, email reminders to non-responders were sent on day three of the field period. Beyond the standard email reminder on day three of the field period, the following steps were also taken:

  • Additional email reminders to non-responders were sent on day 7 of the field period;
  • Teens received a cash-equivalent of $5 for their participation;

Documentation regarding KnowledgePanel sampling, data collection procedures, weighting and IRB-bearing issues are available at the below online resources.

KnowledgePanel Methods Information

Complete and current information about KnowledgePanel sampling and recruitment methodology and design is available at:
http://www.gfk.com/Documents/GfK-KnowledgePanel-Design-Summary.pdf

KnowledgePanel’s recruitment process uses an Address Based Sampling (ABS) methodology for selecting panel members. This probability-based sampling methodology improves population coverage, and provides a more effective sampling infrastructure for recruitment of hard-to-reach individuals, such as young adults and those from various minority groups. It should be noted that under the ABS recruitment households without Internet connection are provided with a web-enabled device and free Internet service.

After initially accepting the invitation to join the panel, participants are asked to complete a short demographic survey (the initial profile survey); answers to which allow efficient panel sampling and weighting for future surveys. Completion of the profile survey allows participants to become panel members, and all respondents are provided the same privacy terms and confidentiality protections.

ABS Recruitment

The ABS recruitment protocol relies on probability-based sampling of addresses from the United States Postal Service’s Delivery Sequence File (DSF). The key advantage of the ABS methodology is that it allows sampling of almost all United States households. Regardless of household telephone status, all households can be reached and contacted through postal mail. Pre-identified ancillary information about addresses was used to construct and target households in the following four sampling strata:

  • Hispanic ages 18-29
  • Non-Hispanic ages 18-29
  • Hispanic ages 30+
  • Non-Hispanic ages 30+

As detailed below, specific adjustments are applied to compensate for any oversampling that is carried out to improve the demographic composition of the panel.

Randomly sampled addresses from the DSF are invited to join KnowledgePanel through a series of mailings, including an initial invitation letter, a reminder postcard, and a subsequent follow-up letter. Given that approximately 45% of the physical addresses can be matched to a corresponding landline telephone number, about 5 weeks after the initial mailing, telephone refusal-conversion calls are made to households for whom a telephone number was matched to the sampled address. Invited households can join the panel by:

  • Completing and mailing back a paper form in a postage-paid envelope
  • Calling a toll-free hotline phone number maintained by GfK
  • Going to a designated GfK website and completing the recruitment form at the website

Household Member Recruitment

For all recruitment efforts, during the initial recruitment survey, all household members are enumerated. Following enumeration, attempts are made to recruit every household member who is at least 13 years old to participate in KnowledgePanel surveys. For household members aged 13 to 17, consent is collected from the parents or the legal guardian during the initial recruitment interview. If no consent is given, no further direct communication with the teenagers is attempted.

Survey Sampling from KnowledgePanel

For this survey, a nationally representative sample of U.S. parents of teens ages 13 to 17 was selected. The general sampling rule is to assign no more than one survey per week to individual members. Allowing for rare exceptions during some weeks, this limits a member’s total assignments per month to four or six surveys.

Survey Administration

Once assigned to a survey, members receive a notification email letting them know there is a new survey available for them to take. This email notification contains a link that sends them to the survey questionnaire.

After three days, automatic email reminders are sent to all non-responding panel members in the sample. If email reminders do not generate a sufficient response, an automated telephone reminder call can be initiated. The usual protocol is to wait at least three to four days after the email reminder before calling. To assist panel members with their survey taking, each individual has a personalized “home page” that lists all the surveys that were assigned to that member and have yet to be completed.

GfK also operates an ongoing modest incentive program to encourage participation and create member loyalty. Members can enter special raffles or can be entered into special sweepstakes with both cash rewards and other prizes to be won.

The typical survey commitment for panel members is one survey per week or four per month with duration of 10 to 15 minutes per survey. In the case of longer surveys, an additional incentive is typically provided.

Sample Weighting

For selection of general population samples from the Knowledge Panel (KP), however, a patented methodology has been developed that ensures the resulting samples behave as EPSEM (Equal Probability of Selection Method) samples. Briefly, this methodology starts by weighting the entire KP to the benchmarks secured from the latest March supplement of the Current Population Survey (CPS) along several dimensions. This way, the weighted distribution of the Knowledge Panel matches that of the US adults – even with respect to the few dimensions where minor misalignments may result from differential attrition rates.

Study-Specific Post-Stratification Weights

Once the study sample has been selected and fielded, and all the survey data are edited and made final, design weights are adjusted for any survey nonresponse as well as any under- or over-coverage imposed by the study-specific sample design. Depending on the specific target population for a given study, geo-demographic distributions for the corresponding population are obtained from the CPS, the American Community Survey (ACS) or in certain instances from the weighted KP profile data. For this purpose an iterative proportional fitting (raking) procedure is used to produce final weights that will be aligned with respect to all study benchmark distributions simultaneously. In the final step, calculated weights are examined to identify and, if necessary, trim outliers at the extreme upper and lower tails of the weight distribution. The resulting weights are then scaled to the sum of the total sample size of all eligible respondents.

For this study, the following benchmark distributions of parents with teens ages 13 to 17 from the 2010-2012 American Community Survey (ACS) were used for the raking adjustment of weights for parents (par_weight):

  • Gender (Male/Female) by Age (18–39, 40–49, and 50+)
  • Race/Hispanic ethnicity (White/Non-Hispanic, Black/Non-Hispanic, Other/Non-Hispanic, 2+ Races/Non-Hispanic, Hispanic)
  • Metropolitan Area (Yes, No) by Census Region (Northeast, Midwest, South, West)
  • Education (Less than High School, High School, Some College, Bachelor and beyond)
  • Household income (under $25k, $25K to <$50k, $50K to <$75k, $75K to <$100k, $100K+)
  • Primary Language (English-dominant, Bilingual, Spanish-dominant, Non-Hispanic)
  • Age (18–39, 40–49, and 50+) by Race/Hispanic ethnicity (White/Non-Hispanic, Black/Non-Hispanic, Other/Non-Hispanic, 2+ Races/Non-Hispanic, Hispanic)
  • Gender (Male/Female) By Race/Hispanic ethnicity (White/Non-Hispanic, Black/Non-Hispanic, Other/Non-Hispanic, 2+ Races/Non-Hispanic, Hispanic) (collapsed metro status together for Others/2+ Races because of not enough cases))
  • Census Region (Northeast, Midwest, South, West) by Race/Hispanic ethnicity (White/Non-Hispanic, Black/Non-Hispanic, Other/Non-Hispanic, 2+ Races/Non-Hispanic, Hispanic)
  • Education (Less than High School, High School, Some College, Bachelor and beyond) by Race/Hispanic ethnicity (White/Non-Hispanic, Black/Non-Hispanic, Other/Non-Hispanic, 2+ Races/Non-Hispanic, Hispanic) (collapsed HS/LHS for AA and HS/LHS for Others/2+ Races)
  • Household income (under $25k, $25K to <$50k, $50K to <$75k, $75K to <$100k, $100K+) by Race/Hispanic ethnicity (White/Non-Hispanic, Black/Non-Hispanic, Other/Non-Hispanic, 2+ Races/Non-Hispanic, Hispanic) (collapsed income into two categories for Others/2+ Races — (under $50K, $50K+))
  • Metropolitan Area (Yes, No) by Race/Hispanic ethnicity (White/Non-Hispanic, Black/Non-Hispanic, Other/Non-Hispanic, 2+ Races/Non-Hispanic, Hispanic) (collapsed metro status together for Others/2+ Races because of not enough cases)

The following benchmark distributions of children ages 13 to 17 from the 2014 March Supplement of the Current Population Survey (CPS) were used for the raking adjustment of weights for teens (teen_weight):

  • Gender (Male/Female) by Age (13, 14, 15, 16, 17)
  • Teen Race/Hispanic ethnicity (White/Non-Hispanic, Black/Non-Hispanic, Other/Non-Hispanic, 2+ Races/Non-Hispanic, Hispanic)
  • Metropolitan Area (Yes, No) by Census Region (Northeast, Midwest, South, West)
  • Age (13, 14, 15, 16, 17) by Teen Race/Hispanic ethnicity (White/Non-Hispanic, Black/Non-Hispanic, Other/Non-Hispanic, 2+ Races/Non-Hispanic, Hispanic)
  • Gender (Male/Female) by Teen Race/Hispanic ethnicity (White/Non-Hispanic, Black/Non-Hispanic, Other/Non-Hispanic, 2+ Races/Non-Hispanic, Hispanic)
  • Census Region (Northeast, Midwest, South, West) by Teen Race/Hispanic ethnicity (White/Non-Hispanic, Black/Non-Hispanic, Other/Non-Hispanic, 2+ Races/Non-Hispanic, Hispanic)
  • Metropolitan Area (Yes, No) by Teen Race/Hispanic ethnicity (White/Non-Hispanic, Black/Non-Hispanic, Other/Non-Hispanic, 2+ Races/Non-Hispanic, Hispanic)
  • Parental Education (Less than High School, High School, Some College, Bachelor and beyond) by Parental Race/Hispanic ethnicity (White/Non-Hispanic, Black/Non-Hispanic, Other/Non-Hispanic, 2+ Races/Non-Hispanic, Hispanic) (collapsed HS/LHS for AA and HS/LHS for Others/2+ Races)

The starting weight for teens is the final parent weight multiplied by the number of children ages 13 to 17 years old in the household (1, 2+).

Detailed information on the demographic distributions of the benchmarks is available upon request. Please contact Kyley McGeeney at KMcGeeney@PewResearch.org for more information about the study methodology.

Pew Research Center is a nonprofit, tax-exempt 501(c)3 organization and a subsidiary of The Pew Charitable Trusts, its primary funder.