The Census Bureau plans to take a big step into the world of digital data collection starting in January, offering more than 3 million households that receive the American Community Survey each year the option to respond online for the first time.
The agency has proposed that all households sent the American Community Survey (ACS) be asked to respond online, and that they only receive a paper questionnaire if they do not respond within about two weeks. The Census Bureau’s proposal requires approval of the Office of Management and Budget, which already authorized several rounds of testing.
The ACS, which has produced national-level demographic, social, economic and housing data since 2005, has mainly been a mail-back survey, with follow-up interviews of non-responding households conducted over the phone or in person. The Census Bureau has been urged for years by Congress and outside organizations to offer an online option for its major surveys and the decennial census.
Plans for 2020 Census
An online response option was available for the 2000 Census, but not publicized, and only about 63,000 households (out of more than 100 million) successfully filed information over the internet. The Census Bureau did not offer an internet option in the 2010 Census, citing the potential risk that personal data could be hacked, as well as logistical concerns and lack of proof that this option would save money or improve the response rate.
However, the climate has changed. Census Director Robert Groves has announced that the 2020 Census will include an online response option, and that the bureau will be testing internet data collection within the 2020 Census Research and Testing Program, as well as ACS, in preparation for the census.
In announcing the ACS online option in testimony prepared for a congressional hearing, Groves said that it could save money in costs of printing, mailing and capturing data, depending on how widely it is used.
Census officials say they do not expect everyone to respond online. Some groups are more likely than others to want to respond online, and not everybody has internet access. At the bureau, internet data collection is seen as one option in a “mixed mode design” that is intended to improve or maintain response rates in an era when many Americans are reluctant to answer surveys.
The Census Bureau conducted a test of data collection over the internet in 2010 as a reinterview survey known as the 2010 Census Quality Survey. Internet data collection testing for the American Community Survey was conducted in April and November 2011. According to 2010 Census Quality Survey results and recently published April test results, Census Bureau researchers were pleased with response rates of households that had been pushed to respond online.
For the ACS, detailed results of the November test have not been released, but based on that second test, the Census Bureau proposed to mail a nudging postcard to households that received a paper questionnaire after they did not respond online. The postcard would be sent a few days after the paper questionnaire.
“Based on our test results, we expect that using this method (if approved) will result in a little over half of the households that currently respond by mail in our first month of data collection to shift to internet,” said Jennifer G. Tancreto, chief of the ACS Data Collection Methods Staff.
How the Internet Test Worked
In the April test, the Census Bureau experimented with how hard it should push people into responding by internet. One group of households was mailed an ACS paper questionnaire that either prominently or inconspicuously advertised an internet-response alternative to mailing back the survey form. Another group of sampled households was told to respond over the internet and received a paper questionnaire (either two or three weeks later) only if they did not respond online–the so-called “push internet methodology” the Census Bureau would like to adopt for ACS use next year.
For security purposes, households that responded by internet had to use a randomly generated 10-digit user ID to enter the survey site, and a four-digit PIN if they wanted to come back to the survey later to complete it.
As expected, the households told to use the internet (and not initially provided a mail questionnaire) were more likely to respond online than other households. Internet responses also came in faster than mail responses.
One important metric was the “self-administered response rate”—the share of households that responded to the survey online or by mail without the need for a follow-up interview that adds to the costs and time of data collection.
According to results from the April test, the self-administered response rate for households offered the choice between online and mail was as good as those offered mail only. This was considered a win, because a widely cited ACS test in 2000 had resulted in lower response rates for households offered a choice of response modes.
What surprised bureau researchers, however, was that households pushed to respond online (with a mail questionnaire follow-up about two weeks later) performed well in relation to those that received only a paper questionnaire or were offered a choice. This finding held across two conglomerate groups, one representing geographic areas with high shares of people in groups that are heavy internet users (young and college-educated, for example) and one without a lot of those people.
Most respondents urged to use the internet did answer online. However, not all did. Some households that were told to use the internet in the April test but ended up responding by postal mail said they didn’t have internet access or had computer problems.
Demographically, respondents who responded online “were more likely to be younger, Asian, non-black, ‘other’ race, with higher education and living in larger households than mail respondents,” Census Bureau researchers reported. “They were also more likely to speak a language other than English at home.”
Response Quality and Break-Off
In addition to looking at response rates, Census Bureau researchers also looked at differences in the quality of responses.
For example, the online experiments turned up problems related to “break-off”–people who started to answer online and didn’t finish. A small share of those who did not finish online ended up mailing back the paper questionnaire.
Break-off rates were higher in households pushed to use the internet and not initially sent a paper questionnaire, compared with those offered a choice between internet or paper. This may be because respondents who did not see the questionnaire before they began “may not have expected the length or content of the survey when attempting to respond,” according to the bureau’s report on the April test results.
The ACS has four dozen questions for each person in the household, and nearly another two dozen housing questions. Some of them require detailed responses—for example, heating fuel costs over the past year or annual payment for fire, flood or hazard insurance on the property.
In fact, bureau researchers reported that some questions in the later part of the survey form were almost twice as likely to be left blank or incomplete by households responding online compared with households responding by mail. Break-offs were a big reason for that. However, item-response rates for questions earlier in the survey were better among those who responded online than among those who responded by mail.
The issues around break-offs required further study, bureau researchers said, and they examined them in the second internet test in November. The main focus of the November test was to improve the bureau’s strategy for achieving high response rates. Those results are expected this spring. Census Bureau officials are scheduled to present their research on internet data collection for the ACS and 2020 Census at the annual meeting in May of the American Association for Public Opinion Research.
