On Jan. 21, 2010, the Pew Research Center hosted a conversation about the 2010 Decennial Census with Director of the U.S. Bureau of the Census Robert Groves and a panel of experts. Groves discussed the operational flow of the 2010 Census, design features intended to increase participation, the department’s communications campaign, real-time monitoring/management, and evaluation of the quality of the census. The session was jointly sponsored by the center, the Washington chapter of the American Association of Public Opinion Research and the Washington Statistical Society.
Speakers:
• Robert Groves, U.S. Census Bureau director, formerly director of the University of Michigan Survey Research Center.
• Constance F. Citro, Director, Committee on National Statistics
• Jeffrey Passel, Senior Demographer, Pew Research Center
• Joseph Salvo, Director, Population Division, New York City Department of City Planning
Moderator: Scott Keeter, Director of Survey Research, Pew Research Center
In the following edited excerpt, ellipses have been omitted to facilitate reading. Listen to the 50-minute audio of Groves’ presentation, including an introduction and presentation of survey findings by Pew Research Center President Andrew Kohut.
ROBERT GROVES: When I saw that this was sponsored by AAPOR and WSS and I see tons of my survey researcher friends out there, I thought you’d be more interested in one particular challenge of the 2010 Census, and that is to get participation at the levels we need to have a good census. So I concentrate my remarks on that.
I’ll tell you just a little about the operational flow of this massive survey, and I’ll focus in on design features that were in place way before I got there in July. We made a few changes on the edges [after I came to the Census Bureau], but basically this is a design that’s been in place for some time.
A big part of the design to get greater participation is a communications campaign that’s rolling out right now. The first national ad was on Sunday on the Golden Globe Awards. The 2010 design is an empirically oriented design in that there’s a wonderful data set that you can actually download that has tract-level data on 2000 participation rates and a lot of other variables.
That data set was supplemented with other survey data to guide the construction of a communications campaign and outreach campaign. My colleagues studied the correlates — the influences, if you will — on census participation, and through those influences identified levers that could be pulled in an advertising campaign to try to change the attitudes and behavior of the population.
[These] are the key steps. [First], the census must be geographically based. We just don’t count people. We have to count them and place them in space in order that the reapportionment — redistricting process can be done. So we have a lot of geographers who worry about that spatial integrity.So what we do in this step is actually build a massive address file that is spatially linked — every address in that file is linked to a block-like structure. And those of you who are real Census groupies know that this decade, in summer of 2009, a bunch of people, about 150,000 people, walked every street in the country with little handheld machines that had GPS capabilities in them. And so we have GPS spots now on about 98 percent of the addresses in that [file].
That forms the basis of a massive mailing — about 90 percent of the [questionnaires] will be mailed in early March. I know everyone in this audience, when you get your questionnaire you will fill it out as soon as you can, certainly by April 1. But we don’t stop there. After April 1, if you don’t fill it out, then we hire a large number of people – almost a million people – to start knocking on the doors of those who didn’t return it. And we take the data in a face-to-face interview.
[T]hen we have a hard deadline at the back end. That is December 31st, where we deliver to the president the counts at a state level that permit the reapportionment process. We do the statistical analysis — it’s actually arithmetic — to produce the allocation of the House of Representatives. And then in late March 2011, the redistricting data are supplied and the states begin their work. So that’s the basic schedule. [This process] is like a moon shot in its planning horizon. We have a group working right now on the 2020 design, believe it or not. [T]he logistical components of this are complex, numerous, they have to interact; and as technology changes throughout the decade, they have to be somewhat adaptive. But this talk is really about the very near future. And so I want to talk about things in the design that address the burden of response, of language impediments, the nature of varying community resources to encourage participation, a little of the benefits of participation, and then some safety [measures].Reducing the burden of response
This is the shortest census questionnaire in our lifetimes. [see the form at 2010.census.gov ] It is close [in length] to the 1790 Census. [In planning for] the 1790 Census, James Madison was arguing that occupation should be asked on the census form. There was a huge debate in Congress. Madison didn’t win the debate because the same arguments were being made then: “You’ve turned this census, this simple enumeration for reapportionment, into some statistical study. We don’t need this. We just need to reapportion the House.”
You’ll get an advance letter from me the first week of March. You’ll get the questionnaire the second week of March. [Next] we send a reminder card. And then, for the first time, after looking at 40 years of research [showing] that replacement questionnaires help response, we will have a replacement questionnaire in areas that historically have lower response rates in early April. [W]hy aren’t we doing replacement questionnaires everywhere? It turns out that the country doesn’t have the printing capacity to do that. We have almost blown out the entire printing capacity of the country in just getting the questionnaires done.
Now, let me give you a sense of this questionnaire. If you have a one-person household, this is the questionnaire. [The] “own” and “rent” [choice] is the only housing question. We ask telephone number for follow-up [when we find discrepancies]. So this is a short questionnaire.
Dealing with Language Impediments
A cool thing is for the first time, about 13 million households will be mailed a bilingual form. Those households, guided by this planning database, are in tracts that are disproportionately high in Spanish-only speakers. This is an innovation, too. There are six languages used in the questionnaire. There are [also] 59 different language assistance guides. You can download [them] on our Web site.
Now, what is a language assistance guide? If you look at [the example on the right], this looks like the questionnaire, doesn’t it? The idea behind a language assistance guide is to put it next to the English form that came in the mail. If you read only Somali, the format is exactly the same. You can read this question and then choose the correct box in the same order on the English form.
How do you figure out what languages to use? Well, with ACS [American Community Survey] and other data you can get estimates of what groups speak different languages. [T]hese six languages — English, Chinese, Simplified Korean, Russian, Spanish and Vietnamese — cover an estimated 97.8 percent of the population.
In our advertising campaign, we cover 28 different languages. That gets you up to 99.4 percent. The language assistance guides go up to 59 languages; you’re at 99.7. To get to 99.8 – you can see the falloff – you’d have to go to 101 languages. On our staff throughout the country now, we have people who speak 124 different languages. In the L.A. school system the kids speak 127 languages. And what is New York City, Joe?
JOSEPH SALVO: We’ve maxed out the list. We use 175 now.
GROVES: A survey researcher [can make] cost-error tradeoffs and say, if we can get 90 or 95 [percent], it’s good enough for us. As a census, you can’t stop there. In fact, if you study the response characteristics of these language groups, they tend to respond at lower levels.
Why? Well, some of them can’t read the English form that we mail them, so it makes all sorts of rational sense to take language as a big design feature and attack that through reducing the burden of different languages. I’m quite hopeful that our advertising in multiple languages and our supply of different language assistance guides may make a difference in this.
To give you a sense, this is one of our warehouses. Do you remember the last scene in “Raiders of the Lost Ark”? That’s nothing. These are pallets, seven high. Every one of these boxes has been bar coded on the box to know what local census office or post office we have to deliver it to. Inside these things are sometimes questionnaires, sometimes things that we call kits that are given to field workers to do different things. Sometimes the kits have questionnaires in areas where we actually drop off the questionnaire; leave it on a hanger on your door. But this is one aisle of, I don’t know, 50 aisles. The whole point of these two pictures is to remind us that you can’t just sort of decide to change something at the last minute. The logistics of this operation are such you have to frontload warehouses with materials because you don’t have much time to get everything out. It’s a little harder than fighting a war because you have to fight it in just a matter of months. You win or lose in a matter of months. And there are a lot of places that look like this. This’ll give you [an idea] of why we were close to burning out the printing capacity of the country for several months.
Tract-based Outreach and Partnerships
Now, a cool thing that happened, in my personal opinion, is that the planning database, this tract-level database, was actually used for part of the outreach. We have people in all of our regions called partnership specialists, and one effect of the stimulus money was to quintuple the size of this staff. These people sat down with our operation staff in every census office, and for all of the tracts that were labeled “hard to count,” they developed a plan.
They drove every street of the tract. They marked where the churches were, where the community centers were; if people tend to hang out at the barber shop or the convenience store. Was there a residents association in the place? Was it a strong faith-based organized community? Were there important businesses that gave back to the community? Then they sat down and said, how can we use these resources on this tract to increase the response rates in that tract? A tract is about 4,000 people — a geographically clustered area. We have a database now, so for every one of these tracts we have a set of checkmarks on what is going to happen in that tract.
Let me say a bit about these partners. A partnership is a voluntary agreement with another organization. No money changes hands. It’s essentially an organization saying, okay, we get it, we get your message. We think our group will be benefited by full census participation. We’ll help you get the word out.
Some of these groups are as small as a little residents association. Some of them are national organizations like Target and Best Buy. So pretty soon, when you walk into your local Best Buy, you know that panel of plasma screens at the back? It will have the census logo. And pretty soon, on the Sunday supplement for Target, there’s going to be a page that reminds you that the census form is coming. These are groups that believe it’s useful for their mission to have strong census participation, and we depend on them. We have over 160,000 now.
Integrated Communications Campaign
The partnership program is one of five pieces of a big communications campaign. Let me go across the fingers [shown in the graphic].
The Census in Schools is really cool. You can go to Scholastic.com [and] download the census materials. This is a K-12 curriculum. We’re asking schools around the country — 118,000 schools — to have a Census in Schools week as a once-in-a-10-year opportunity to teach kids what the census is, what it’s about. We’re assisted in this by the Sesame Street characters, Count Von Count, who’s my senior technical adviser, and Rosita, who — I didn’t know about Rosita, but she’s cool, too.
I was in a Census in Schools in Delaware and the characters under contract don’t talk. They just sort of walk around. And they’re like six-and-a-half feet tall — really big. So we enter this fifth-grade room and the kids are all in their census t-shirts. Rosita starts lumbering down one of the aisles and there was a little girl working on her census form and she looked up suddenly and Rosita was right in her face. The girl screams. So they had to kind of escort Rosita back. It was a little too much Rosita.
Now, why are we doing this? We have learned over and over and over again that the sons and daughters of new immigrant groups, through school education, are a wonderful conduit of both English language skills and the communication of the new society to the whole family. When they understand the census, understand safety guidelines of the census, it helps family participation.
There’ll be digital media. The biggest thing we’re doing is a road tour. We have 13 vehicles that are all over the country now stopping at local fairs, big sporting events. [They]’ll be at Chinese New Year in San Francisco, at the Super Bowl, and basically wherever a whole lot of people gather. These are vans that are pretty interactive and have a lot of technology in them and are meant to educate people about how simple this form is.
I’ll talk more about paid advertising. This is a fairly big campaign. For advertisers, if you’re in the advertising business, it’s a weird campaign because if you ask the question, “What’s your target market?” It’s everybody. So it’s an odd campaign in that you’re not going after a particular slice. Because we’re trying to reach everyone, we have disproportionately spent locally rather than nationally. We are disproportionately spending in language groups. We are trying to get the message out to those that, in past, cooperated and participated at lower levels.
It is focused on three sorts of things: increasing the mail response rate — for every 1 percentage point that we raise the mail response rate through this advertising campaign, we will reduce the total cost of the census by about $85 million. All we have to do is reduce this thing by 1 percentage point and it pays for itself almost. We’re really close to getting those kind of returns.
We go through three phases. Right now you’ll see the awareness phase. Basically the only message is the census is coming and it’s a good thing. The next phase is the motivation phase at the end of March/beginning of April. You have the form; please turn it in. Then May through July, the message changes. It’s: “A census person may come to your door if you haven’t returned [the form]. Be nice to them. Answer their questions.”
[In] addition to [going] deeper in the language set, [t]here are a bunch of tailored messages. If you go to our Web site you can now see these videos yourself. The messages are slightly different for every group. Why? In focus groups and other early research, it became clear that the concerns of different subgroups in the population about the census vary greatly. The communications [program] is trying to tailor its message to those concerns.
[Commercials have been designed for each subgroup.] The details in the background are appropriate to the subgroup. So in the Filipino version of one ad, there’s a Filipino rice cooker on the shelf. In the Chinese version of the ad, it’s a Chinese rice cooker. Every detail has been tailored to the population as much as possible.
Let me turn to [our] real-time evaluation. We now have a daily tracking survey going on, and we are grateful to the Pew Center for doing the study that was released yesterday. Anything we can know right now about the American public’s attitude will help us. We are analyzing these data in a variety of ways. The data set contains key predictors from prior research of returning the questionnaire and then a behavioral intention question — the likelihood of their participating in the 2010 Census.
So every day we can actually build a little predictive model. We can watch how the predictors are changing. We can see whether awareness is rising in all the subgroups of the population. And if we get really clever about this, we can tweak the advertising if one subgroup is lagging in getting the message that we’re trying to deliver.
I better stop right now and say this is subject to great misinterpretation. These data are from an RDD [random digit dialing] survey. The proportion of all people sampled that became respondents is much, much, much, much, much, much lower than we’ll ever get in the census. I think the usefulness of this is to watch it over time, to see if things are moving, to see if major subgroups are moving the way you want.
We’re running multinomial logistic regression models every day. The dependent variable is the behavioral intention to complete the form. The independent variables are these knowledge and attitude questions. And then we have three models: just straight knowledge/attitude; straight sociodemographic; then we put them together. We’re beginning to look at interaction models. We’re asking questions like, “Are the different attitudes more or less important drivers for different subgroups? Are those attitudes moving in the right direction, beneficial to census participation?” If we’re smart, we’ll learn how to tweak the advertising campaign to make real-time adjustments on it.
Tracking Participation
We’re also going to do something that you yourself can see [on the Census Bureau Web site]. Starting at the end of March, you can watch the participation rate — the ratio of returned completed questionnaires to mailed-out questionnaires minus those that were [returned because delivered to the wrong address].
We have also engaged [what we call] complete count committees. These are local committees that are trying to help us get the word out — elected officials and so on. And we’re going to give these groups and these data over time. And you can drill down [in the data on the Web site]. You can enter your ZIP code if you want and you’ll see county data. If you want to drill down to your tract, you can do that. You can also have little races. We’re doing this every day. It’ll be updated daily.
Now, something that you need to know is that at the end of the process, about 67 percent of the questionnaires were returned of those mailed out in 2000. By April 1, that 67 number was 57 [percent]. Most of the action happens before April 1. So by April 1, we are going to know what are the tracts that aren’t performing as well as we thought. Some of them are going to be the hard-to-count tracts from last time. But there’ll be surprises. So every day, we will have a deviant score model; the dependent variable will be the change in participation rates between 2010 and 2000.
Now, what are the questions [we’ll be] asking? Did the tract action plans that I mentioned – were they working the way they wanted? Are there some ideas for certain kinds of tracts that were just killer ideas that really worked well? And were those killer ideas used or not used in these newly identified outlier tracts? If they weren’t used, then we have April 1 through April 22nd to intervene.
And we have our partners ready to go into tracts that are unusually low performers, surprise-low performers, and do things for a three-week period to get the forms back. This is new. Whether it will work, since it’s never been done before, it won’t work perfectly; it’ll be ugly in certain areas. I think it makes sense to try it. And we’re calling this the “April Blitz.”
The big thing that we have this decade that we had last decade also is trying to get a handle on the undocumented immigrant population. This is a component of demographic analysis that’s really hard to estimate. What we are planning to do actually, in an honest portrayal of demographic analysis this decade, is to present several different estimates of counts for these subgroups based on different assumptions; different logical, plausible options for the value of key components. It will be an honest portrayal — what we know and don’t know — and hopefully be more useful and less susceptible to misunderstandings.
Post-Census Evaluation
We also do a very large post-enumeration survey. For some reason, the name changes every decade. This decade, it’s called “Census Coverage Measurement.” But it is, in essence, a very large post-censal sample survey independently done of the census. So if you go into our local census offices now, there’s always a locked subset of the census office. It’s a room filled with people who are preparing to do this. They are like the Secret Service of the census.
As you know, the 2010 Census will not have a statistical adjustment. This is something that became crystal clear from a Supreme Court case at the end of the ’90s. We are using this post-enumeration survey to evaluate. What’s new about this is that we’re going to be able to track components of census coverage at certain levels.
We’re also going to be able to break down the value of a census operation. For example, the mail-out areas versus the areas where we drop off a questionnaire, nonresponse follow-up as an operation, to assess its value in affecting differential undercount. That’s something new; we’ll see how well it goes.
The summary of this is — if you didn’t get it yet — this is a massive undertaking. I thought I did big sample surveys in my life before [but] nothing relative to this. It is not unlike going to war. You need field generals, you need tacticians, you need strategists, and you need foot soldiers.
It is not a pretty process. It is a lot like all sample surveys that I’ve done: The data set often looks a lot better than the interviewing and all the horrible things that happen when people try to get information from people. It’s an ugly process. There are going to be things that will happen over the next few weeks that I can’t predict, you can’t predict; they’re going to happen, though. And they’re mainly bad things, it seems.
For more on the census, check out “All Things Census,” a gathering place for frequent postings about census methodology, findings and resources at census.pewresearch.org/social-trends
