Dr. Elmer Bernstam of the University of Texas Health Science Center at Houston and Dr. Funda Meric-Bernstam of MD Anderson Cancer Center are co-Principal Investigators on a Methodology and Design project entitled Validation of Quality Criteria for Health Information on the World Wide Web.

Q: What is unique and/or innovative about your study?
Our goal is to empower health care consumers to evaluate online health information. Although some Web sites contain high quality, accurate information there are many other sites whose information is inaccurate and may result in harm to consumers. In previous studies, we found that poor quality sites were just as popular as high quality sites; implying that health care consumers do not discriminate on the basis of quality. However, in spite of multiple proposed quality criteria such as “references provided” or “author identified,” no study has systematically evaluated quality criteria to determine if they can be reliably assessed or whether they correlate with health information accuracy. In this project, we attempt to find objective and assessable domain-independent quality criteria that correlate with health information accuracy as judged by an expert. If we can find such criteria, then consumers would have the tools to independently evaluate online health information.

Q: How is your project progressing so far?
Our first task was to find quality criteria that can be reliably assessed. In other words, if two raters were to evaluate the same Web site, they should obtain the same results. Surprisingly, we found that subjects could not reliably assess seemingly objective criteria such as “disclosure of ownership” without very precise operational definitions. We therefore developed strict operational definitions for the most commonly used quality criteria. Subsequent experiments yielded more reliable results.

Our second task was to evaluate a large sample of sites containing information about complementary and alternative medicine. We recorded quality and “potential harm” information for each site. Potential harm refers to the judgment (by clinicians) that harm could result from the information provided on the Web site. We are currently in the process of analyzing these data. The next task will be to repeat the process on a sample of breast cancer Web sites.

Q: What prompted you to explore this research?
As clinicians, we frequently encounter patients with stacks of laser printed pages of information from the Internet. We also recognize that health care consumers need more information than we can provide during a brief encounter in the office or hospital. As clinicians are forced to see more patients in less time, the need for additional sources of information will increase.

Q: How would a typical end-user utilize the final product/results of your research?
If we are able to find quality criteria which help consumers screen out potentially harmful or inaccurate information, then these could benefit users searching for online health information.

Q: What are the greatest challenges in eHealth and more specifically, your project?
eHealth is a broad area with many challenges. In this project, we focus on the quality and accuracy of online health information. Since the Web is essentially a record of human experience it covers an unimaginable range of topics. Therefore, criteria which apply to one subject area may not be relevant to another. For example, what does it mean to determine the accuracy of information about complementary and alternative medicine? Further, there is no “gold standard” of accuracy. Specifically, qualified observers may legitimately disagree regarding whether given statement is true; and frequently they do disagree. Therefore, it is very difficult to “prove” anything.

Q: In what ways would you like to see eHealth evolve?
We would like to see methods develop which allow users to determine for themselves whether or not particular online information is reliable. Clearly, because online information is independent of physical location, this cannot occur through government intervention; there is no global government. Therefore, we must study the Web to find ways to screen out “bad” information.

Q: How do you stay informed of advances and innovations in eHealth?
Although traditional print media such as biomedical journals are still important, eHealth moves too fast for print media to keep up. Therefore, online journals such as the Journal of Medical Internet Research (JMIR; http://www.jmir.org) and Web sites such as the one that you are currently reading are very important to disseminate information about ongoing efforts.

Many thanks to Elmer and Funda!

Coming in May - we’ll hear from Dr. Susan Palsbo of MedStar.