You’ve probably heard about them from your patients: computerized “tests” for ADHD. Do they work? Are they helpful? Or are they money-making scams?
There are two particularly popular tests: the T.O.V. A. (Test of Variables of Attention) ($375 plus $15/use) http://www.tovatest.com, and the Connors CPT (Connors Continuous Performance Test) http://www.devdis.com/ conners2.html (version 5.1 for windows, $645, unlimited uses).
Both tests work in similar ways, by presenting patients with a boring computer “game” requiring vigilance. In the T.O.V. A., a little box appears within a bigger box. When the little box is at the top, you are supposed to click your mouse; when it is at the bottom, you do not click. The Connors CPT flashes letters randomly on the screen and the task is to tap the spacebar for every letter except for the “X.” Both tests score participants on errors of commission (clicking when you’re not supposed to–theoretically a measure of impulsivity) and errors of omission (not clicking when you should–theoretically a measure of inattention). Both companies have large databases of test results from both clinical samples (primarily ADHD) and non-clinical samples. Patients’ scores are compared with these “norms” and reports are automatically generated indicating how likely patients are to fit the ADHD profile. The T.O.V. A. takes 22 minutes to complete, while the Connors CPT takes 14 minutes. They can be easily administered with a laptop computer in the office.
In order to decide whether there is evidence for CPT’s utility (note: I use CPT to refer to all continuous performance tests, including the T.O.V. A.), we need to first define exactly how we would want to use it clinically. Sometimes the diagnosis of ADHD is easy to make on clinical grounds, but the diagnosis is often difficult, because the patient may have other underlying disorders leading to ADHD symptoms. Symptoms of distractibility and impulsivity can be caused by conditions such as bipolar disorder, depression, anxiety disorders, oppositional defiant disorder, conduct disorder, and learning disorders–to name a few (McGough JJ, et al., Am J Psychiatry 2005;162:1621-1627.) We would welcome a computer test to help us differentiate among these conditions.
Another major issue is treatment guidance. Once we diagnose ADHD, we are faced with dozens of different medication and behavioral treatment choices; furthermore, it’s not always clear whether a treatment is actually working. Thus, a test that would help us to select treatment or to monitor treatment progress would be quite welcome.
The manufacturers of both devices claim on their websites that CPT is helpful for both of these clinical issues. Do the published data back up these claims? I located two comprehensive reviews (Nichols SL and Waschbusch DA, Child Psychiat Hum Dev 2004;34:297-315; ECRI, Full Health Care Technology Assessment (CLIN 0001), Department of Defense, 2000, accessed online at http://ablechild.org/right%20to%20refuse/continuous_performance_ tests.htm).
It is clear from reading these reviews that dozens of studies have been done evaluating these systems, but that unfortunately the most commonly used research design does little to speak to relevant clinical needs. For example, many studies have shown that CPTs are pretty good at distinguishing children with ADHD from carefully selected normal children with no psychiatric diagnoses. But such studies are not really useful for clinicians, since completely normal people rarely seek our services. People who come into our offices have psychiatric problems, and for a diagnostic test to be useful, it must help with the notoriously difficult differential diagnoses in psychiatry.
The few studies that have used CPTs to distinguish ADHD patients from patients who have a range of other psychiatric disorders have yielded mixed results. The Positive Predictive Value in these studies ranges from a low as 9% (meaning that 91 out of 100 patients would be incorrectly diagnosed with ADHD) to a high of 100%. While this 100% PPV outcome sounds good (no false positives), it comes with a low Negative Predictive Value of 22%. What does this mean? It means that while 100% of patients diagnosed with ADHD actually had ADHD, 78% of kids who were labeled “normal” actually had ADHD. Because of such problems, the authors of both reviews concluded that CPT is of unproven utility for diagnosing ADHD.
What about using CPT to predict or to monitor response to treatment? While the authors cited studies showing that computer scores improve when patients are on medication, it is not clear what this means, because it is not clear that improvement on CPT correlates meaningfully with clinical improvement in settings such as school and home. In other words, you may be able to show that stimulants make ADHD kids more efficient at tapping a spacebar for 15 minutes in front of a computer, but how well does this translate to remembering to bring their assignments home or not blurting things out in class? In fact, the authors could not find a single medication follow-up study comparing CPT scores with the current diagnostic gold standard, which is a thorough clinical evaluation.
The bottom line is that there is little evidence that CPTs can be used for diagnosing ADHD or for monitoring treatment response. But it may not be completely useless. As a non-specific test of the attention, it may have some value. For example, Karen Postal, a neuropsychologist in Andover, Mass., and president of the Massachusetts Psychological Society, finds the Connors CPT helpful in evaluating patients in their 50s who come to her worried about dementia because their memory seems poor. “I often discover that these patients have a normal memory, but when they do the Connors CPT they may have a significant attention deficit as compared to age-matched norms.” She finds the test helpful in convincingly demonstrating to these patients that the real problem is not memory, but sustained attention, and the culprit is often a treatable condition such as chronic insomnia or depression.
Before sending this article to press, I corresponded with Dr. Lawrence Greenberg, the developer of the T.O.V.A. Sounding somewhat less enthusiastic about the T.O.V.A. than the company’s website, Dr. Greenberg said, “we are very clear that [the T.O.V.A.’s ADHD score] is not a diagnostic statement. Rather, this score is helpful in confirming a diagnosis of ADHD based on the appropriate DSM diagnostic criteria.” Fair enough, but without studies actually demonstrating utility above and beyond clinical diagnosis, it seems a tough sell to get psychiatrists to use this test.
TCPR VERDICT: Computerized ADHD testing adds little of value