Finding Trustworthy Information

What Can You Trust?

Search the Internet for “cancer therapy” and you will find tens of millions of web pages. How can you sort through to find the few pages that will be most valuable for you? How can you possibly know which websites or information to trust? How do you analyze and find clarity when “experts” give opposite recommendations? How can you tell the “snake oil” from truly valuable therapies?

First, please understand that LHC does not claim to have all the answers. As we pull together our information and summaries, we struggle with these same issues: Is this source reliable? Is this research valid? Is this expert qualified to speak about this therapy?

However, even as we are always learning ourselves, we share with you some guidelines for determining whether a website or article rates higher or lower on a trust scale.
Nancy Hepp

Type and Strength of Evidence

Evidence Trade-offs

Evidence that a therapy “works” runs a whole range from completely unreliable to trustworthy. We present an overview of some of the issues in determining whether evidence is reliable and appropriate, first from a researcher’s viewpoint, and then from a clinician’s frame.

The Researcher’s View: Hierarchy of Evidence

We present 12 levels in a research hierarchy of strength of evidence, starting with unreliable evidence and working down through increasingly credible sources of evidence (all examples are fictional). The evidence toward the bottom of this list (larger numbers) is generally regarded as stronger and more reliable than evidence toward the top of the list.

1. Nonspecific testimonials: People without training or credentials provide their overall impression of a therapy. Examples: “This stuff is great!” or “This therapy has improved my life!” or “This therapy doesn’t work.”
2. Specific testimonials: People without training or credentials provide specific information about how a therapy worked for them. Examples: “My pain disappeared in two days” or “I was finally able to sleep through the night.” Side effects are not often reported.
3. Clinical or expert testimonial: A medical provider provides information from their personal experience treating patients. Example: “During 32 years in medical practice, I’ve seen this treatment help hundreds of patients with hot flashes.”
4. Clinical observations, also called case studies: One patient or a small group is treated and observed over time. Case studies can be published in medical journals. Examples: “Doctors report that six patients given this treatment experienced less nausea and vomiting than before treatment” or “Ms. X was followed for seven months on this treatment, with these results.” Both benefits and burdens such as side effects are reported.
5. Retrospective observational studies: Patients are asked to remember what therapies they used in the past, and researchers look for patterns and compare them to current health status. Patients are not always accurate in remembering or reporting past practices, which is a serious problem in these studies. Example: “Five thousand breast cancer patients were asked about their diets for the last 20 years.”
6. Prospective observational studies: Patients’ treatments are observed and recorded by researchers, or patients are asked to record therapies they use in the present, and researchers look for differences in outcomes based on differences in therapies. Example: “Five thousand breast cancer patients were asked to keep food journals for six months following their diagnosis.” Observational studies show a relationship between a therapy and an outcome, but they do not show that the therapy caused the outcome.
7. Experimental studies with animals: Similar animals are divided into two or more groups, with one group receiving the treatment and others given no treatment (or a different treatment).
8. Small prospective, experimental clinical studies, sometimes called “pilot studies”: Two or more small groups of patients (typically less than 100) are made as similar as possible or are randomly divided into groups. Groups receive different treatments, or different levels of a treatment, and typically one group receives standard care or no treatment (perhaps in the form of a placebo). The health outcomes of the various groups are compared. Sometimes the groups switch treatments after a period to further determine whether the health outcome is due to the treatment or to differences in the patients. Patients and even their healthcare providers may not know which treatment they are receiving (a “blinded” study).
9. Large prospective, experimental clinical studies: A study starts with several hundred patients, with even larger numbers considered stronger. This group is divided into two or more groups, with random assignment considered a stronger study design. As in smaller studies, placebos or blinding may be used. The health outcomes of the various groups are compared. Note that sometimes study effects (the effects of the treatment) can seem modest for each patient, but a statistically significant effect is found for the whole group. Sometimes only a minority of patients show any effect of the treatment, but the treatment is considered a success for those patients.
10. Review of groups of studies: Researchers analyze and synthesize the results of all  studies to date involving a treatment to find patterns, similarities and differences in study results.
11. Meta-analysis of several studies: Researchers conduct a review and also combine the results of many studies. This approach can often find more subtle effects that may have been overlooked or dismissed in the individual studies. A meta-analysis may be able to find reasons that smaller studies found opposite outcomes.
12. Clinical practice guidelines: A panel of medical researchers reviews all the evidence to date and concludes that a therapy fits into categories of recommendation for specific medical conditions. The following examples of categories are based on clinical practice guidelines from the Society for Integrative Oncology:

• • Strong recommendation in favor of use:
    • High- or moderate-quality evidence shows that benefits clearly outweigh risk and burdens.
  • Weak recommendation in favor of use:
    • High- or moderate-quality evidence shows that benefits are closely balanced with risks and burden.
    • Weak (inconclusive or conflicting) evidence leaves uncertainty in estimates of benefits, risks and burden; no clear advantage is shown over other options.
  • Weak recommendation against use:
    • High- or moderate-quality evidence shows that risks and burden are probably greater than benefits.
    • Evidence shows no advantage compared to other options, while risks and burdens may be greater.
    • Weak (inconclusive or conflicting) evidence leaves uncertainty in estimates of benefits, while risks and burden are established.
  • Strong recommendation against use:
    • High- or moderate-quality evidence shows that risks and burdens clearly outweigh benefits.

Our Approach to Information

When we evaluate claims regarding therapies and treatments, LHC strives to consider both experts’ financial interests and where they draw evidence from. We indicate in our footnotes where our information comes from and provide a link if possible so that you can check the source yourself. We look for the most credible sources available. Most of our information is referenced from Beyond Conventional Cancer Therapies which we trust as a credible source of information.

Clues to be skeptical and to verify any information through more reliable sources –

• Is this person or website encouraging you to buy something?
• Does this person or site make only vague statements about effectiveness without any evidence?
• Do you have to pay for a product or therapy before you can receive specific information about it?

Sources We Trust

Our Resources collection includes many books, websites, videos and other resources that we have found to be trustworthy. However, we are open to critiques from our users of these resources. We are deeply grateful to our users for alerting us about resources that should be reconsidered or removed.

Written by Nancy Hepp, MS and reviewed by Laura Pole, RN, MSN, OCNS, and Michael Lerner, PhD; most recent update on October 23, 2018.

Article Credits – Beyond Conventional Cancer Therapies