Skip to main content

The confusing morass of medical evidence

Practitioners of evidence-based medicine use published evidence from the medical literature to guide them as they try to provide the best care for each patient. But sometimes the medical literature just feels like a big morass.

The difficulty applying the medical literature to patient care is illustrated by two studies published in the past few months in the very best medical journals. The studies give precisely opposite answers to an important clinical question.

The studies focus on the management of COPD (sometimes called emphysema or chronic bronchitis). COPD, which causes very distressing symptoms such as shortness of breath, is very common, especially in Geriatric patients. It is one of the most common causes of hospitalization, and costs Medicare billions of dollars. One of the studies, published in the NEJM, was recently reviewed on GeriPal. The other study was published in the Annals of Internal Medicine this month.

Patients with COPD are often treated with long acting inhaler medicines to control symptoms, and hopefully prevent exacerbations that lead to hospitalization. However, there are two types of long acting medicines that are available: Beta agonists (such as salmererol) and anticholinergics (such as tiotropium). It is has been a guess as to which of these medicines are better, and there has been little to guide clinicians as to which medicine to use.

Well, after these two studies, it is still a guess. It is worth briefly reviewing each of these studies, since they provide a useful allegory about the limitations of evidence-based medicine. One could teach a entire class on evidence based medicine or epidemiology focused on these two papers. And if you have a good understanding of the strengths and limitations of both studies, you can have a feeling of confidence in your skill critically evaluating the medical literature.

The more recent study published in the Annals of Internal Medicine leveraged an administrative database in Ontario that had information on nearly all patients in the Province over the age of 65 with diagnoses of COPD. They used billing codes from patient encounters to identify patients with COPD. Then they determined whether these patients were first prescribed a Beta agonist or an anticholinergic. Since Ontario provided these patient prescription coverage, use of these drugs was recorded in the database. The study then compared rates of death and hospitalization in elders prescribed Beta agonists and anticholinergics. The study suggested patients given B-agonists did better. Patients given anticholinergics had 14% higher risk of death and a 13% higher risk of hospitalization for COPD than patients given Beta Agonists.

But, this study does not prove B-agonists are better. This study was not randomized. It compared patients who happened to be given a B-agonist against those who happened to be given an anticholinergic. It is possible patients given anticholinergics were sicker than patients given Beta agonists and that higher levels of illness severity, not drug choice explain outcome differences. The authors did a lot of analyses to address this possibility that seem to suggest that differences in illness severity do not explain their results. However, with this type of study design, there is simply no way to know for certain are due to the drug choice rather than other patient characteristics.

The other study, published in NEJM was recently discussed in detail on GeriPal, so we will only discuss it briefly here. This study was a randomized trial comparing a B-agonist and anticholinergic. It showed that patients who were treated with the anticholinergic did better—they were less likely to be hospitalized for COPD.

Those who know guidelines for medical evidence might instinctively say “duh—what’s the problem?? Randomized studies are the gold standard, so I am going to believe the NEJM randomized study. Bring out the anticholinergics.” After all, randomization (when successful) assures the groups are virtually identical except for the choice of drug. As a result, outcome differences can be attributed to the drug.

Unfortunately, it is not so simple. As we discussed, the subjects in the NEJM study do not look like most patients with COPD. They were much younger than the typical patient. (Average age of 62). Also, there was an absurd set of enrollment restrictions. Most patients with COPD would probably not qualify for the study. Even though the vast majority of patients with COPD have several other medical problems, the study excluded patients with many co-existing illnesses. So, while the ideal study design of randomization was used, the nature of the patients in the study markedly limit the ability to generalize the findings to real world COPD patients. (In contrast, the patients in the Annals study closely resemble real world patients with COPD).

So, what should one do? If you have a patient with COPD, should you use a long acting Beta-Agonist or an anticholinergic?

My first answer would be that I have no idea.

But of course, we have real patients to treat, and we have to make a decision. So my vote would be for the B-agonist, especially for the older patients I see. I slightly favor the results of the Annals study because the patients were much more like the patients I see with COPD and outcome of mortality examined in this study is more important than the hospitalization outcome assessed in the NEJM study. The stronger study design of the NEJM study does not make up for focus on patients that bear limited resemble to the typical patient.

But, in the end the choice of drug remains an educated guess. As is often the case, the application of evidence based medicine to a common clinical question yields as much confusion as clarification.


by: Ken Covinsky

Comments

Very perceptive analyses from EBM perspectives. Unfortunately, we may be seeing increasing numbers of 'data dredging' studies like the first one, as electronic medical records become more widespread. If anything, the two studies exemplify the "all that glitters is not gold" phenomenon -- even the most illustrious journals need to fill their pages with "something."

Popular posts from this blog

Geroscience and it's Impact on the Human Healthspan: A podcast with John Newman

Ok, I'll admit it. When I hear the phrase "the biology of aging" I'm mentally preparing myself to only understand about 5% of what the presenter is going to talk about (that's on a good day).  While I have words like telomeres, sirtuins, or senolytics memorized for the boards, I've never been able to see how this applies to my clinical practice as it always feels so theoretical.  Well, today that changed for me thanks to our podcast interview with John Newman, a "geroscientist" and geriatrician here at UCSF and at the Buck Institute for Research on Aging.

In this podcast, John breaks down what geroscience is and how it impacts how we think about many age-related conditions and diseases. For example, rather than thinking about multimorbidity as the random collection of multiple different clinical problems, we can see it as an expression of the fundamental mechanisms of aging. This means, that rather than treating individuals diseases, targeting …

The Dangers of Fleet Enemas

The dangers of oral sodium phosphate preparations are fairly well known in the medical community. In 2006 the FDA issued it’s first warning that patients taking oral sodium phosphate preparations are at risk for potential for acute kidney injury. Two years later, over-the-counter preparations of these drugs were voluntarily withdrawn by the manufacturers.  Those agents still available by prescription were given black box warnings mainly due to acute phosphate nephropathy that can result in renal failure, especially in older adults. Despite all this talk of oral preparations, little was mentioned about a sodium phosphate preparation that is still available over-the-counter – the Fleet enema.

Why Oral Sodium Phosphate Preparations Are Dangerous 

Before we go into the risks of Fleet enemas, lets spend just a couple sentences on why oral sodium phosphate preparations carry significant risks. First, oral sodium phosphate preparations can cause significant fluid shifts within the colon …

Length of Stay in Nursing Homes at the End of Life

One out of every four of us will die while residing in a nursing home. For most of us, that stay in a nursing home will be brief, although this may depend upon social and demographic variables like our gender, net worth, and marital status. These are the conclusions of an important new study published in JAGS by Kelly and colleagues (many of whom are geripal contributors, including Alex Smith and Ken Covinsky).

The study authors used data from the Health and Retirement Study (HRS) to describe the lengths of stay of older adults who resided in nursing homes at the end of life. What they found was that out of the 8,433 study participants who died between 1992 and 2006, 27.3% of resided in a nursing home prior to their death. Most of these patients (70%) actually died in the nursing home without being transferred to another setting like a hospital.

 The length of stay data were striking:

the median length of stay in a nursing home before death was 5 months the average length of stay was l…