Skip to main content

The confusing morass of medical evidence

Practitioners of evidence-based medicine use published evidence from the medical literature to guide them as they try to provide the best care for each patient. But sometimes the medical literature just feels like a big morass.

The difficulty applying the medical literature to patient care is illustrated by two studies published in the past few months in the very best medical journals. The studies give precisely opposite answers to an important clinical question.

The studies focus on the management of COPD (sometimes called emphysema or chronic bronchitis). COPD, which causes very distressing symptoms such as shortness of breath, is very common, especially in Geriatric patients. It is one of the most common causes of hospitalization, and costs Medicare billions of dollars. One of the studies, published in the NEJM, was recently reviewed on GeriPal. The other study was published in the Annals of Internal Medicine this month.

Patients with COPD are often treated with long acting inhaler medicines to control symptoms, and hopefully prevent exacerbations that lead to hospitalization. However, there are two types of long acting medicines that are available: Beta agonists (such as salmererol) and anticholinergics (such as tiotropium). It is has been a guess as to which of these medicines are better, and there has been little to guide clinicians as to which medicine to use.

Well, after these two studies, it is still a guess. It is worth briefly reviewing each of these studies, since they provide a useful allegory about the limitations of evidence-based medicine. One could teach a entire class on evidence based medicine or epidemiology focused on these two papers. And if you have a good understanding of the strengths and limitations of both studies, you can have a feeling of confidence in your skill critically evaluating the medical literature.

The more recent study published in the Annals of Internal Medicine leveraged an administrative database in Ontario that had information on nearly all patients in the Province over the age of 65 with diagnoses of COPD. They used billing codes from patient encounters to identify patients with COPD. Then they determined whether these patients were first prescribed a Beta agonist or an anticholinergic. Since Ontario provided these patient prescription coverage, use of these drugs was recorded in the database. The study then compared rates of death and hospitalization in elders prescribed Beta agonists and anticholinergics. The study suggested patients given B-agonists did better. Patients given anticholinergics had 14% higher risk of death and a 13% higher risk of hospitalization for COPD than patients given Beta Agonists.

But, this study does not prove B-agonists are better. This study was not randomized. It compared patients who happened to be given a B-agonist against those who happened to be given an anticholinergic. It is possible patients given anticholinergics were sicker than patients given Beta agonists and that higher levels of illness severity, not drug choice explain outcome differences. The authors did a lot of analyses to address this possibility that seem to suggest that differences in illness severity do not explain their results. However, with this type of study design, there is simply no way to know for certain are due to the drug choice rather than other patient characteristics.

The other study, published in NEJM was recently discussed in detail on GeriPal, so we will only discuss it briefly here. This study was a randomized trial comparing a B-agonist and anticholinergic. It showed that patients who were treated with the anticholinergic did better—they were less likely to be hospitalized for COPD.

Those who know guidelines for medical evidence might instinctively say “duh—what’s the problem?? Randomized studies are the gold standard, so I am going to believe the NEJM randomized study. Bring out the anticholinergics.” After all, randomization (when successful) assures the groups are virtually identical except for the choice of drug. As a result, outcome differences can be attributed to the drug.

Unfortunately, it is not so simple. As we discussed, the subjects in the NEJM study do not look like most patients with COPD. They were much younger than the typical patient. (Average age of 62). Also, there was an absurd set of enrollment restrictions. Most patients with COPD would probably not qualify for the study. Even though the vast majority of patients with COPD have several other medical problems, the study excluded patients with many co-existing illnesses. So, while the ideal study design of randomization was used, the nature of the patients in the study markedly limit the ability to generalize the findings to real world COPD patients. (In contrast, the patients in the Annals study closely resemble real world patients with COPD).

So, what should one do? If you have a patient with COPD, should you use a long acting Beta-Agonist or an anticholinergic?

My first answer would be that I have no idea.

But of course, we have real patients to treat, and we have to make a decision. So my vote would be for the B-agonist, especially for the older patients I see. I slightly favor the results of the Annals study because the patients were much more like the patients I see with COPD and outcome of mortality examined in this study is more important than the hospitalization outcome assessed in the NEJM study. The stronger study design of the NEJM study does not make up for focus on patients that bear limited resemble to the typical patient.

But, in the end the choice of drug remains an educated guess. As is often the case, the application of evidence based medicine to a common clinical question yields as much confusion as clarification.


by: Ken Covinsky

Comments

Very perceptive analyses from EBM perspectives. Unfortunately, we may be seeing increasing numbers of 'data dredging' studies like the first one, as electronic medical records become more widespread. If anything, the two studies exemplify the "all that glitters is not gold" phenomenon -- even the most illustrious journals need to fill their pages with "something."

Popular posts from this blog

Practical Advice for the End of Life: A Podcast with BJ Miller

This week we talk with BJ Miller, hospice and palliative care physician, public speaker, and now author with Shoshana Berger of the book "A Beginner's Guide to the End."

As we note on the podcast, BJ is about as close as we get to a celebrity in Hospice and Palliative Care.  His TED Talk "What Really Matters at the End of Life" has been viewed more than 9 million times.  As we discuss on the Podcast, this has changed BJ's life, and he spends most of his working time engaged in public speaking, being the public "face" of the hospice and palliative care movement.

The book he and Berger wrote is filled to the brim with practical advice.  I mean, nuts and bolts practical advice.  Things like:
How to clean out not only your emotional house but your physical house (turns out there are services for that!)Posting about your illness on social media (should you post to Facebook)What is the difference between a funeral home and mortuaryCan I afford to die?  …

Improving Advance Care Planning for Latinos with Cancer: A Podcast with Fischer and Fink

In this week's GeriPal podcast we talk with Stacy Fischer, MD and Regina Fink, RN, PhD, both from the University of Colorado, about a lay health navigator intervention to improve advance care planning with Latinos with advanced cancer.  The issue of lay health navigators raises several issues that we discuss, including:
What is a lay health navigator?What do they do?  How are they trained?What do lay health navigators offer that specialized palliative care doesn't?  Are they replacing us?What makes the health navigator intervention particularly appropriate for Latinos and rural individuals?  For advance care planning? Eric and I had fun singing in French (yes French, not Spanish, listen to the podcast to learn why).
Enjoy! -@AlexSmithMD




You can also find us onYoutube!



Listen to GeriPal Podcasts on:
iTunes Google Play MusicSoundcloudStitcher

Transcript

Eric: Welcome to the GeriPal podcast. This is Eric Widera.

Alex: This is Alex Smith.

Eric: And Alex, I'm really excited about toda…

The Dangers of Fleet Enemas

The dangers of oral sodium phosphate preparations are fairly well known in the medical community. In 2006 the FDA issued it’s first warning that patients taking oral sodium phosphate preparations are at risk for potential for acute kidney injury. Two years later, over-the-counter preparations of these drugs were voluntarily withdrawn by the manufacturers.  Those agents still available by prescription were given black box warnings mainly due to acute phosphate nephropathy that can result in renal failure, especially in older adults. Despite all this talk of oral preparations, little was mentioned about a sodium phosphate preparation that is still available over-the-counter – the Fleet enema.

Why Oral Sodium Phosphate Preparations Are Dangerous 

Before we go into the risks of Fleet enemas, lets spend just a couple sentences on why oral sodium phosphate preparations carry significant risks. First, oral sodium phosphate preparations can cause significant fluid shifts within the colon …