Skip to main content

The confusing morass of medical evidence

Practitioners of evidence-based medicine use published evidence from the medical literature to guide them as they try to provide the best care for each patient. But sometimes the medical literature just feels like a big morass.

The difficulty applying the medical literature to patient care is illustrated by two studies published in the past few months in the very best medical journals. The studies give precisely opposite answers to an important clinical question.

The studies focus on the management of COPD (sometimes called emphysema or chronic bronchitis). COPD, which causes very distressing symptoms such as shortness of breath, is very common, especially in Geriatric patients. It is one of the most common causes of hospitalization, and costs Medicare billions of dollars. One of the studies, published in the NEJM, was recently reviewed on GeriPal. The other study was published in the Annals of Internal Medicine this month.

Patients with COPD are often treated with long acting inhaler medicines to control symptoms, and hopefully prevent exacerbations that lead to hospitalization. However, there are two types of long acting medicines that are available: Beta agonists (such as salmererol) and anticholinergics (such as tiotropium). It is has been a guess as to which of these medicines are better, and there has been little to guide clinicians as to which medicine to use.

Well, after these two studies, it is still a guess. It is worth briefly reviewing each of these studies, since they provide a useful allegory about the limitations of evidence-based medicine. One could teach a entire class on evidence based medicine or epidemiology focused on these two papers. And if you have a good understanding of the strengths and limitations of both studies, you can have a feeling of confidence in your skill critically evaluating the medical literature.

The more recent study published in the Annals of Internal Medicine leveraged an administrative database in Ontario that had information on nearly all patients in the Province over the age of 65 with diagnoses of COPD. They used billing codes from patient encounters to identify patients with COPD. Then they determined whether these patients were first prescribed a Beta agonist or an anticholinergic. Since Ontario provided these patient prescription coverage, use of these drugs was recorded in the database. The study then compared rates of death and hospitalization in elders prescribed Beta agonists and anticholinergics. The study suggested patients given B-agonists did better. Patients given anticholinergics had 14% higher risk of death and a 13% higher risk of hospitalization for COPD than patients given Beta Agonists.

But, this study does not prove B-agonists are better. This study was not randomized. It compared patients who happened to be given a B-agonist against those who happened to be given an anticholinergic. It is possible patients given anticholinergics were sicker than patients given Beta agonists and that higher levels of illness severity, not drug choice explain outcome differences. The authors did a lot of analyses to address this possibility that seem to suggest that differences in illness severity do not explain their results. However, with this type of study design, there is simply no way to know for certain are due to the drug choice rather than other patient characteristics.

The other study, published in NEJM was recently discussed in detail on GeriPal, so we will only discuss it briefly here. This study was a randomized trial comparing a B-agonist and anticholinergic. It showed that patients who were treated with the anticholinergic did better—they were less likely to be hospitalized for COPD.

Those who know guidelines for medical evidence might instinctively say “duh—what’s the problem?? Randomized studies are the gold standard, so I am going to believe the NEJM randomized study. Bring out the anticholinergics.” After all, randomization (when successful) assures the groups are virtually identical except for the choice of drug. As a result, outcome differences can be attributed to the drug.

Unfortunately, it is not so simple. As we discussed, the subjects in the NEJM study do not look like most patients with COPD. They were much younger than the typical patient. (Average age of 62). Also, there was an absurd set of enrollment restrictions. Most patients with COPD would probably not qualify for the study. Even though the vast majority of patients with COPD have several other medical problems, the study excluded patients with many co-existing illnesses. So, while the ideal study design of randomization was used, the nature of the patients in the study markedly limit the ability to generalize the findings to real world COPD patients. (In contrast, the patients in the Annals study closely resemble real world patients with COPD).

So, what should one do? If you have a patient with COPD, should you use a long acting Beta-Agonist or an anticholinergic?

My first answer would be that I have no idea.

But of course, we have real patients to treat, and we have to make a decision. So my vote would be for the B-agonist, especially for the older patients I see. I slightly favor the results of the Annals study because the patients were much more like the patients I see with COPD and outcome of mortality examined in this study is more important than the hospitalization outcome assessed in the NEJM study. The stronger study design of the NEJM study does not make up for focus on patients that bear limited resemble to the typical patient.

But, in the end the choice of drug remains an educated guess. As is often the case, the application of evidence based medicine to a common clinical question yields as much confusion as clarification.


by: Ken Covinsky

Comments

Very perceptive analyses from EBM perspectives. Unfortunately, we may be seeing increasing numbers of 'data dredging' studies like the first one, as electronic medical records become more widespread. If anything, the two studies exemplify the "all that glitters is not gold" phenomenon -- even the most illustrious journals need to fill their pages with "something."

Popular posts from this blog

The Dangers of Fleet Enemas

The dangers of oral sodium phosphate preparations are fairly well known in the medical community. In 2006 the FDA issued it’s first warning that patients taking oral sodium phosphate preparations are at risk for potential for acute kidney injury. Two years later, over-the-counter preparations of these drugs were voluntarily withdrawn by the manufacturers.  Those agents still available by prescription were given black box warnings mainly due to acute phosphate nephropathy that can result in renal failure, especially in older adults. Despite all this talk of oral preparations, little was mentioned about a sodium phosphate preparation that is still available over-the-counter – the Fleet enema.

Why Oral Sodium Phosphate Preparations Are Dangerous 

Before we go into the risks of Fleet enemas, lets spend just a couple sentences on why oral sodium phosphate preparations carry significant risks. First, oral sodium phosphate preparations can cause significant fluid shifts within the colon …

Dying without Dialysis

There is a terrific article in this weeks Journal of Pain and Symptom Management by Fliss Murtagh of King's College in London about the epidemiology of symptoms for patients with advanced renal failure who die without dialysis.  This study is important because while we know that patients with advanced renal failure have a limited life expectancy and the average age of initiation of hemodialysis is increasing, we know little about the alternatives to hemodialysis.  Specifically, we know nothing about symptoms affecting quality of life among patients who elect not to start dialysis (so called "conservative management" - is this the best label?).  This article provides a terrific counterpoint to the article in last years NEJM showing that nursing home residents who initiated hemodialysis tended to die and decline in function (see GeriPal write up here). 

The study authors followed patients with the most advanced form of chronic kidney disease (the new name for renal failu…

Survival from severe sepsis: The infection is cured but all is not well

Severe sepsis is a syndrome marked by a severe infection that results in the failure of at least one major organ system: For example, pneumonia complicated by kidney failure. It is the most common non-cardiac cause of critical illness and is associated with a high mortality rate.

But what happens to those who survive their hospitalization for severe sepsis? An important study published in JAMA from Iwashyna and colleagues provides answers and tells us all is not well. When the patient leaves the hospital, the infection may be cured, but the patient and family will need to contend with a host of major new functional and cognitive deficits.

Iwashyna examined disability and cognitive outcomes among 516 survivors of severe sepsis. These subjects were Medicare enrollees who were participants in the Health and Retirement Study. The average age of patients was 77 years.

When interviewed after discharge, most survivors were left with major new deficits in their ability to live independently. …