Ask a survey methodologist to explain random sampling, or how non-probability polls work, and they’ll probably give you a detailed answer. But explaining these topics to a general audience calls for a more concise approach. Over the past three years, Pew Research Center has created a series of short videos to make survey research topics and techniques like these more accessible to non-specialists. We call the series Methods 101. Here’s a closer look at how we distilled the Center’s experience and expertise into this series of four-minute videos.
Background
Transparency is a key part of the Center’s mission. That includes making sure our readers have access to our methodology and a window into our thinking. More broadly, it means doing whatever we can to help our readers understand an often dense and nuanced process.
Amid a rise in social media use, we recognized the need to reach a broader audience. Video is an especially useful promotional tool on platforms such as Facebook and Twitter, and in email newsletters or blog posts — media frequented by our more casual followers. For this reason, we decided in 2017 to invest in video as a new distribution tool, with the intent to further democratize access to our research. Taking a cue from publications such as Vox and The Washington Post, we began exploring ways to use scripted footage, B-roll and motion graphics to break down some of the big, overarching themes of survey methodology. Thus, the Methods 101 series was born.
How we did it
Planning a video series takes big-picture planning and dedicated staff time. Before diving too deeply into the work, we made a few initial decisions, including:
- The target audience
- The overarching focus
- The digital products and platforms that would house the series
- A team of stakeholders who would be part of the feedback process
- A series title
With the big picture in mind, a team of methodologists and communications and digital staff came up with 10 to 15 potential videos for the series. From that initial list of ideas, we further narrowed it down to ensure the topics stayed true to the “101” level. (We wanted to focus on our target audience without adding too many ideas or diving too deeply into the weeds.)
After the initial brainstorming sessions, we decided that our first video would tackle random sampling — a concept that lies at the heart of all probability-based survey research.
Scripting
We quickly realized it would be a challenge to find a comfortable balance between brevity and clarity while scripting. To give a sense of how much careful editing was required, the final “Methods 101” video script was less than 400 words — about a one-page Word document — and took two minutes and 25 seconds to read. That involved a lot of whittling down of the initial draft, which was about twice as long.
At an organization that values thoroughness and collaboration, it took a while to get comfortable with the idea that we wouldn’t be able to explain every caveat in a video intended to be brief and accessible. The process started with Courtney Kennedy, our director of survey research, writing a thorough answer to the topic question. Our communications team then made sure we anticipated any common questions or pushback from our audiences, and our digital and social media teams made sure to keep the writing conversational and snappy.
Still, every sentence added or subtracted and every idea introduced was an opportunity to bring up a new “if,” “and” or “but,” and a hole patched in one place seemed to open a new hole somewhere else. The good news is that script writing became easier with every new video in the series. The team adjusted from writing multi-page reports to writing conversational video scripts. We even allowed ourselves to start sentences with words like “but” or “so” — which isn’t always the case in more formal publications by the Center.
Shooting
While script-writing for the first video was underway, the digital team identified a few key visual components for shooting. We scouted locations around the Center’s office, including our in-house TV studio, and decided to do most of our shooting in one of our largest conference rooms, which is often filled with daylight and has a view of downtown Washington, D.C. We also created a “classroom” set, complete with a desk, textbooks and a whiteboard. And we decided that Courtney would be the host of the video.
The classroom set and our mighty host would become central to all of the Methods 101 videos. Depending on the video topic, we recreated the same set with minor variations in desk props. This consistency contributed to the series’ brand. If there’s one major lesson to share about visuals in a video series, it’s that when certain elements remain uniform from one video to the next, your videos will actually look like a package.
Even though Courtney was our host, we tapped other researchers to be part of the series to further humanize the work we do. And when it came to filming, our researchers were troupers. Not everyone at the Center has on-camera or radio experience, so at times, some coaching was necessary. Sometimes we had sessions with three takes, sometimes 10. That said, everyone is capable of delivering a message with energy. Everyone is capable of inflection and emphasis in their everyday conversations. But it’s the responsibility of the video producer to help unlock these abilities so they translate onto the screen.
No two film sessions were alike because of all these variables. However, one trick we tested and continued to use was telling our on-air researchers a little white lie. After a few good takes, we still wanted to see if we could unlock an even more energetic version. So we asked the researchers to give us their highest level of energy — something they might see as over the top — and told them it would end up on the cutting room floor. Much to their surprise, these takes often looked and sounded great on camera. Internet videos are a competitive space, so some added energy can go a long way to boost engagement.
Stylistically, the Methods 101 videos are a departure from more standard, straight-ahead Center publications. We allowed ourselves to have some fun with the B-roll footage along the way — whether it was a simmering alphabet soup as a metaphor for random sampling, or using gumballs to illustrate a nonprobability poll. We even cut in some vintage video clips and filmed members of our own staff as extras.
Animation and graphics
Aside from shooting, we storyboarded and created motion graphics for our videos as well. Once a video was shot, the video producer worked with the digital producer and design director to break down the methodology concepts into doodles that would eventually transition from one to another. We used the Boords platform to collaborate on the storyboard as a digital team, but a simple Google Doc with two columns and an infinite amount of rows would also get the job done.
The most labor-intensive part of making our videos turned out to be design and animation, partly because our video team consists of one person who animates alone. But we also tapped a couple of other designers at the Center to build charts and other graphics, which sped the design process up. We typically used Adobe Illustrator to make these graphics.
After designing, we cleaned up the Illustrator file and separated each layer before importing it into an animation program known as After Effects. (We did this because After Effects treats a layer as one flat, joint object. To animate, each object must be animated separately, creating a need for many independent layers.)
In After Effects, the graphics were ready to come to life. Using the narration and shot footage as a tempo baseline, we added keyframes and effects to the layered graphics so they could move along with what our on-air researcher was saying, at the same pace. Within the program, the “easy-ease” feature was our friend, as it softened transitions that could otherwise appear jolty. The “pre-comp” feature was key, too, as it allowed us to nest one animation sequence inside another and another. (Think “Inception.”) This helped us build more complicated scenes. Another tip worth mentioning is that because some aspects of the videos were templatized — including the intro title, lower thirds and end credits — we created a “master” After Effects file with these compositions, which was duplicated and used for subsequent videos.
Conclusion
A lot went into developing our survey methodology video series. However, we made sure to allow room for trial and error, experimentation and refinement along the way. For example, lighting was an evolutionary process for us, as halfway through the series we purchased an LED light kit to improve the video quality and color balance. All this to say, it’s OK to grow, and you don’t have to have it all figured out to get started.
The Methods 101 video series has continued to be a useful outreach tool long after the first video went live. To this day, our communications staff references the entire series in emails to reporters, academics, students and members of the public who want to learn more about polling, while our social media team posts the videos on multiple online platforms. We also appreciate the comments we receive about them and read each one. Many have been from students and faculty all over the world, telling us they plan to share the videos in their classes.
You’ll find our Methods 101 series, as well as the rest of our video products, on our YouTube channel. We hope they help untangle some of the big questions in survey methodology and polling.
