I scan menus for keywords (fig, parsnips, salmon…) and it turns out I scan Twitter the same way, looking for anyone who is talking about my favorite topics (data, consumers, information quality…)
So when I saw Jonathan Richman’s tweet the other night, I couldn’t resist it:
Anyone ever seen data on the overall accuracy of medical information found online? Need help for some final stats for #fdasm
Short answer:
No.
Long answer:
The Pew Research Center’s Internet & American Life Project has been reporting on the social impact of the internet since 2000, when “information quality” on health websites was a big part of the conversation. It was the era of wagging fingers, scolding patients for straying too far outside their boundaries, and Pew Internet data was ammunition.
We released our first report about the internet’s impact on health & health care in November 2000. The Medical Library Association (MLA) contacted us, asking for research looking at how consumers decide which sites/sources to trust. With their help we created a set of questions asking first if respondents went online for health info, then asking if they look for the source and date of the info they find (the two key quality indicators according to the MLA).
Our 2002 report, Vital Decisions, reported the findings:
Only about one quarter of health seekers follow the recommended protocol on thoroughly checking the source and timeliness of information and are vigilant about verifying a site’s information every time they search for health information. Another quarter of health seekers check a site’s information “most of the time.” Half of all health seekers search for medical advice and “only sometimes,” “hardly ever,” or “never” check the source or date of the information they read online.
We repeated the source/date questions in August 2006 and reported on the “demand side” of health information in our report, Online Health Search 2006:
Three-quarters of health seekers do not consistently check the source and date of the health information they find online.
Unbeknownst to us, the U.S. Department of Health and Human Services was in the field with their study about the “supply side” of health info: Estimating the Proportion of Health Related Websites Disclosing Information That Can Be Used to Assess Their Quality.
I summarized their findings in our report, hoping to soften the impact of the ammunition we were handing out:
Health seekers might be forgiven if they give up what at times is a search for a needle in a haystack. A recent study commissioned by the U.S. Department of Health and Human Services (HHS) finds that a tiny percentage of health sites display the source and date of the information on their pages…
HHS’s Office of Disease Prevention and Health Promotion, working with industry experts, identified six types of information that should be publicly disclosed to health seekers: the identity of the site’s sponsors, the site’s purpose, the source of the information provided, privacy policies to protect users’ personal information, how users can provide feedback, and how the content is updated. Of the 102 websites reviewed for the report, none met all six of the disclosure criteria and only six complied with more than three criteria. Just 4% of “frequently visited” health websites disclosed the source of the information on their pages and 2% disclosed how the content is updated. Less-popular health sites fared even worse: 0.3% of these sites listed their content’s source and only 0.1% disclosed how the content is updated.
After reading the HHS study, looking at our data, and scanning the changing landscape (ie, consumers diversifying their online research to include user-generated content), I decided that trying to measure the quality of health info had run its course as a research question.
Pew Internet was also gathering evidence that consumers were not being harmed by inaccurate information. In a 2008 survey we conducted in partnership with the California HealthCare Foundation, just 3% of e-patients said they or someone they know has been harmed by following medical advice or health information found on the internet, a number that has remained stable since 2006. Meantime, 60% of e-patients (or 42% of all adults) said they or someone they know has been helped by following medical advice or health information found on the internet. That’s an increase from 2006 when 31% of e-patients (25% of all adults) said that.
After exchanging a few tweets and then sending a long email, I was pleased to see our data used in Jonathan’s upcoming presentation to the U.S. Food and Drug Administration’s hearing on social media. But I worry that only the “three-quarters don’t check” data made it in to his slides, not the needle-in-a-haystack caveat provided by the HHS study.
Are we really going back to the era of scolding patients for not checking the source and date of health information they find online? If so, it’s my duty as the purveyor of that ammunition data point to speak up.
To everyone who is providing insights to the FDA: Tell the whole story. Help the FDA see not only the reality of today’s information marketplace, but tomorrow’s, which is increasingly mobile and social.
Join the conversation at e-patients.net.
