Tuesday, March 15, 2011

The Past is Hard to Forget when Evaluating New Dementia Screening Tests

If we care about primary care physicians actually using the screening tests we develop, then we should care about their accessibility to those clinicians. To put it simply, the more barriers we put in place, the less likely anyone will use them to assess cognitive status in the elderly.

The prior gold standard for cognitive screening was the mini-mental status exam (MMSE). This test used to be freely available online, in books, and on pocket cards that were distributed to medical students and residents throughout the country. This all changed in March of 2001 when MiniMental, LLC (the current owners of the MMSE copyright) granted Psychological Assessment Resources (PAR) the exclusive rights to publish, license, and manage all intellectual property rights to the MMSE. Suddenly, after decades of neglect, PAR began enforcing the copyright on the MMSE (see "stealth patents"). Now physicians would have to pay about $1 per test, and importantly, another barrier to cognitive screening was erected.

In the wake of the MMSE copyright enforcement, several new and improved cognitive screens began hitting the geriatrics store shelves. One excellent example is the Montreal Cognitive Assessment (MoCA). This is a free, brief, and validated screening tool with high sensitivity and specificity for detecting MCI and dementia (http://www.mocatest.org). It’s easy to use, but does require one to print out the actual test in order to administer it to patients.

What about a test that requires no props and no special forms? This months Archives of Internal Medicine released a study on the Sweet 16. The article describes the creation of this new brief cognitive assessment tool, and its comparison to the MMSE in a cohort of patients. A long story made short: the Sweet 16 was found to be at least equivalent to, and possibly superior to that of the MMSE (at least in this cohort of patients). Some of the results included:
  • The average time to complete the test was 1.4 to 2.9 minutes
  • When compared to the IQCODE, a Sweet 16 score of less than 14 demonstrated a sensitivity of 80% and a specificity of 70%, whereas an MMSE score of less than 24 showed a sensitivity of 64% and a specificity of 86%.
  • When compared with clinical diagnosis, a Sweet 16 score of less than 14 showed a sensitivity of 99% and a specificity of 72% in contrast to an MMSE score with a sensitivity of 87% and a specificity of 89%.

The authors, though, sent mixed messages on how accessible this test will be to primary care physicians. Initially they note a key problem in the routine measurement of cognitive status is that “a number of cognitive assessment instruments, including the MMSE, are copyrighted and now have restrictions or fees associated with their use”, and later state that the Sweet 16 solves this as it “is open access, whereas the MMSE and the MMSE-2 are restricted by copyright.” However, the website for the Sweet 16, as well as the Sweet 16 training manual, give a very different impression than a true open access instrument, as noted in the following statement regarding use of the tool:

“The Sweet 16 is a copyrighted instrument. It can be used free of charge only by nonprofit organizations and educational institutions (such as universities).” As well as: “All for-profit organizations or entities must contact the Aging Brain Center at: 617-971-5390 or email: AgingBrainCenter@hsl.harvard.edu for permission to use the Sweet 16.”

So, what does this mean for the private practice primary care physician who doesn't happen to work for a non-profit entity? Is it open access or is it pay-per-view? After being burned by the MMSE, I now want some confirmation that it is indeed ‘open access’ – meaning that it can be found online, free of charge, and free of most copyright and licensing restrictions.

In the meantime, I’m going to stick with using and teaching the MoCA. The authors of the MoCA have alleviated my fears with a website detailing permissions for use including that Universities, Foundations, Health Professionals, Hospitals, Clinics, and Public Health Institutes can use the MoCA in clinical, educational, and research settings free of charge, and may use, reproduce, and distribute it WITHOUT permission. Now that is what I call open access.


by: Eric Widera

8 comments:

Christian Sinclair, MD said...

It is like they are trying to make diagnostic tests that use cogintive skills to be applied as profitable as radiology or lab testing. On one hand I could see the logic, but in practice this just stifles implementation and usability. If you made a screening test to help find disease and illness wouldn't you want everyone to use it? Reduce the barriers and it will be used.

Open access, profitability (or some may say sustainability) and the like are getting me really frustrated about medical education and research.

ken covinsky said...

The removal of the MMSE out of the public domain was a great disservice to patients. Removing this test from the public domain was inexcusable.

The MOCA and Sweet 16 seem like great alternatives. I hope and trust the developers of the Sweet 16 will make good on the committment they made in their Archives paper to make the test open access and easily available.

John Newman said...

If only there were a movement, parallel to the open-access movement for publishing, to require government-funded researchers to release assessment scales and tools under the GNU Public License or similar "open source" copyright. This is a huge issue in the scientific community as well, where it is very unusual to release the source code of, for example, a promising new data mining program. All unintended consequences of the Bayh-Dole Act, I wonder if the benefit of universities having more independent funding outweighs the dampening effect on research and clinical use.

Bruce Scott, MD said...

Removal of the Folstein from the public domain actually has the potential to be a great SERVICE to patients, I think.

The test was cumbersome and inconsistently used. Patients didn't like it. People can get a 27 or 28 on the MMSE and still be demented, especially with high education levels. Too many docs (hopefully not too many geriatricians) would stop evaluation after the patient scores their 28, even though they missed 2/3 on the recall.

The most useful part of it for early diagnosis (the three item recall) is incorporated into other tools.

Depending on the circumstances, I either use the Mini-Cog (three item recall plus clock-drawing), described by Soo Borson out of Washington or a modified SLUMS (Saint Louis University Mental Status Examination). The Mini-Cog pretty much does most of the work of the MMSE anyway, and takes much less time. In fact, you can throw in a one minute animal naming and/or a Geriatric Depression Screen (4 or 5 item) and still keep it under 5 minutes.

Now I suppose that I'll have to go check to see if either of these are considered copyrighted and require payment.

Daniel Habtemariam said...

Great comments all around.

For what it's worth, I'm an analyst on the development team for the Sweet 16 and can field any specific questions people have.

To address ken covinsky's last statement, the Sweet 16 is and will continue to be open-access for as long as we have any say over it.

Daniel Habtemariam

Eric Widera said...

Thank you Daniel for the comment. I guess I remain confused at the permissions for use for the Sweet 16. It looks like only nonprofits can use the sweet 16 without permission from The Aging Brain Center. Do you know if this has changed? Does a physician with a private practice need to get approval from you prior to use? Does a for profit hospital also need permission to use it hospital or clinic wide?

Thanks

Eric

Tony Witte said...

Access to the Sweet 16 has been withdrawn at the HELP Center, according to the website. What gives? As a family physician with a large geriatric panel I was eager to use this promising tool.

Eric Widera said...

Oh no. The website now reads "In response to requests from Psychological Assessment Resources (PAR), Inc., we are removing the Sweet 16 from our website. We hope that it will be available again soon, and apologize for any inconvenience." That is never a good sign. I can only guess that PAR's lawyers are now involved over some infringement issue.