What would happen if Americans were not required by law to respond to census surveys? This is of interest because Canadian government officials have decided to drop the mandatory long form in next year’s census and replace it with a voluntary household survey. All Canadian households will still be required to answer basic questions on the census short form.
The Census Bureau tested this idea a few years ago when it researched how response rates to the American Community Survey would change if survey response was not required. The American Community Survey, a monthly survey that has replaced the census long form, includes questions about topics that include educational attainment, commuting, citizenship, income and housing costs. The bottom line: There is a “significant drop in mail cooperation” when respondents are not required to respond, Census Bureau researchers reported.
Experiment Design and Results
Researchers came to this conclusion after testing four different types of messages as part of American Community Survey mailings in March and April 2003 to about 138,000 addresses. Three-quarters of the sample was evenly split between two “voluntary mail treatments,” and the remaining 25% was split evenly between two “mandatory mail treatments.” Results were calculated not only for the national sample, but also for areas that had had high and low response rates to the 2000 Census.
Among households that were told survey response was mandatory, the “mail cooperation rates” (mail-back rates from contacted, occupied households) were 59.2% and 57.3%, depending on how the message was framed. Among those sent a voluntary message, response rates were 38.8% and 34.8%, depending on the wording of the letter accompanying the survey form; the lower cooperation rate came from households that received a more direct message stating in the first paragraph that participation was voluntary. Mail cooperation rates dropped the most in areas that had high cooperation rates during the 2000 Census, compared with low-cooperation areas.
If households do not return their forms after being reminded in a follow-up postcard, the Census Bureau tries to reach them for a telephone interview and then contacts a sample of non-responding households for face-to-face interviews, which cost more than processing a mailed-back form. Researchers said they found more resistance to those interviews from households mailed the voluntary form than from those mailed the mandatory form.
For the mandatory survey, the combined weighted response rate from mailed-back forms and the sample of follow-up interviews was about 98%, but was only about 93% for the voluntary survey. Researchers say the voluntary survey also could result in lower data quality — unless sample size were increased — because its greater reliance on personal interviews contributes to an increase in sampling error as the result of weighting issues.
Higher Costs of Voluntary Survey
If the American Community Survey were made voluntary, the Census Bureau concluded that it would have to mail survey forms to a larger number of households and conduct a larger number of follow-up phone or personal interviews in order to assure the same quality of results that it promises from the mandatory survey. Thus, a voluntary survey would cost at least $59.2 million more, in 2005 fiscal-year dollars, than a mandatory survey, the bureau concluded.
Interestingly, survey respondents who mailed back their forms were about equally likely to fill out all items on the form whether response was mandatory or voluntary. Among those interviewed by telephone or in person, researchers reported an increase in missing data from households that received the voluntary survey, compared with those that received the mandatory survey, but the difference was small.
The experiment, conducted at the request of Congress, may have reduced any pressure for the bureau to consider a voluntary American Community Survey. But it did result in one operational change. In comparing the two mandatory-survey messages it tested, the Census Bureau concluded that the one it had been using produced a slightly lower response rate than a more user-friendly version, in which some explanatory information was moved from a letter to a brochure. After reporting the results, the bureau implemented the more user-friendly mandatory mailing package.
