Are things really as bad as we thought?

The findings of the 2004 GSS rely on a key question that asks survey participants to list by name, those people “with whom you discussed matters that are important to you.” The authors of the original study point to a number of possible “technical problems” with this question that may have created or inflated the trend that they observed [13, 52, 53]. Claude Fischer, the author of a number of seminal works on social networks [14, 54], has also emphasized that the 2004 GSS contradicts other relevant data on social isolation, and suggest that the data contain serious anomalies [55]. Although there is no “smoking gun” that clearly demonstrates a technical problem with the GSS data, these authors suggest the following:

  • Problems with the survey instrument. Surveys can introduce unexpected bias into how participants respond to questions. Context effects, as a result of having placed particularly onerous questions ahead of the GSS module on core discussion networks or questions that trained respondents to answer with fewer names (knowing that more names would lead to even more questions) may have introduced an unknown bias.
  • A random technical error. The unexpected increase in the number of Americans who said that they have no one with whom they discuss important matters may be a result of an unknown artifact in how the survey data were coded. It would be unusual for a survey as large and reputable as the GSS to have such a problem. However, in September 2008, the National Opinion Research Center, the organization that runs the GSS, discovered that forty-one of those who declined to answer the question on discussion partners were misclassified in a way that lumped them in with those that said that they do not have anyone with whom they discuss important matters [11]. These cases should have been excluded from the analysis. Other errors may exist that cannot be detected.
  • Problem with the question wording.  There may have been a change since 1985 in how some people interpreted the meaning of the word “discuss.” They may have interpreted the word in a way that excludes important conversation that does not take place in person. One possible reason for such a change between 1985 and 2004 is that communication increasingly occurs online, on the internet and through other communication devices. If people were not considering those conversations in their answer to the question “with whom do you discuss important matters?” then a potentially significant amount of communication was excluded from the analysis of what is happening to Americans “discussion networks.”