Cognitive Assessment

Attention Tests & ADHD: A Modest Proposal

Some people did not like my post suggesting that attention tests are really important for understanding a person but that they are not awesome in every way that a test can be awesome.

I’ve found their arguments to be persuasive and now have an updated position:

It Isn’t “ADHD” Unless You Fail My Tests

ADHD is a problem. Is it even real?

If it is real, who has it? For really real, I mean.

What doesn’t work:

  1. We cannot trust self-reports, for sure. Kids don’t know anything and adults—well, Ritalin junkies will say or do anything.
  2. Parent and teacher rating scales? Don’t get me started.
  3. Interviews? Unstandardized rating scales in conversation form. A lotta yada yada for a lotta nada nada.
  4. Comprehensive reviews of school records? Are there any studies with real data proving that this works?
  5. Direct observation? That’s for grad students and research studies! Who has the time? I’ll “observe” during testing.
  6. A comprehensive consideration of all the available evidence, including attention tests? Well, this is the obvious answer people give. It does seem like a reasonable answer but in reality it is not. Too expensive, too much work, and too much “peace, love, & rainbows” if you know what I mean.

What does work:

We need objectivity and we need it quick! Like brain scans or something—but less expensive. Like tests, especially computerized ones with T-scores rounded to 2 decimals of precision.

The good news is that the tests we already have are great. And there are so many of them! If a person performs well on one test, there is always another one you can give. There is a low score in that profile somewhere…

When you find a low score, give the test again for the sake of reliability. Or an alternate form. If the score isn’t low the second time around, keep looking. Usually you won’t have to look long—it’s a rare person who does well on everything. If the person is really smart, count an average score as a deficit.

Whatever the low score is, as long as seems like it measures attention, it is probably the explanation you are looking for. Consult the research literature….Oh good! There was a study once that showed that this test was correlated with…problems (p < 0.05).

Don’t listen to people who claim that our knowledge about ADHD is limited. Ever gone to a library? Those things are freaking huge! And there are lots of them. In universities, even. In one of those books or in one of those journals, somebody with a Ph.D. said something about something (more p-values) that totally agrees with you. Find it. Cite it. There might be more than one study. The Internet can help. Don’t tell me that knowledge is limited until you’ve read everything.

Talking with Evaluees

Now that the explanation has been found, explain to the person that his or her attention problems are really a manifestation of something deeper: executive function deficits. Explain that executive functions are like attention but fancier. Because working memory. Point to a picture of the frontal lobes. There are lots of studies about the brain (p < 0.05).

The goal here is to be right. And you can always be right. You sacrificed your 20s for that degree…you earned the privilege never to be questioned.

Speaking of being questioned, what’s with these people who claim to have attention problems and then reject my non-diagnosis of ADHD because of good test performance? Liars all! Or whiners. Or both. Lying whiners are the worst! I tell them:

“Look, you are completely normal. I’ve seen people with problems way worse than yours. They failed my tests. They have ADHD. You don’t. If you did, you’d’ve have failed my tests. I know, I know—your life is a mess, your work history is spotty, and that you have a long history of failed relationships. There’s no need to call this ADHD. Maybe you should try harder to pay attention to things and do them right—like you did on my tests. Ever think of that? It’s a shame because you’re a bright guy. I’m not just saying that to be nice—you did really well on my tests. You should’ve finished your college degree—nevermind…water under the bridge! I see you’ve been in a number of car accidents. Be safe! Also, return your library books on time—you’re losing a lot of money that way. Get more sleep. Don’t procrastinate any more either—that’s half your problem right there. The amount of time that you play video games is not healthy. All things in moderation as they say! I don’t mean to be rude but I’d like to give you some constructive criticism: When I am trying to say something, you often interrupt with things you are thinking about. Sometimes it isn’t even relevant to what I was saying. I do want to hear you but I did find it a little annoying. I bet other people don’t appreciate it either. A little courtesy goes a long way…I’m just being honest. One more thing, when your wife was here, she said that you mean well but often fail to follow though with things and that she can’t trust you with anything time-sensitive. She seems real nice but if you keep letting her down like that…you are going to lose her. I thought you should know. Keep your promises and things should get a lot better. Okay, now…on your way. Good luck!”

Standard
Cognitive Assessment

What role should cognitive tests of attention play in ADHD diagnosis?

Short Answer: Practically none—but attention tests have other uses.

Longer Answer: Attention tests are very important for discovering what a person can and cannot do. However, for purely diagnostic purposes, cognitive tests of attention should at most play a very limited role, perhaps nudging a diagnostic decision one way or the other in cases that are on the threshold of diagnosis.

Graphical Abstract:

Image

Extended Answer:

There are good reasons to assess cognitive abilities but ADHD diagnosis is not one of them. It is true that people with ADHD are a bit more likely to have certain cognitive profiles than are people without ADHD. For example, people with ADHD, on average, tend to score slightly lower on tests of working memory, processing speed, and verbal fluency than on tests of visual–spatial reasoning, fluid reasoning, and crystallized intelligence (Schwean & Saklofske, 2005). However, most people with ADHD do not have this particular profile and most people with this profile do not have ADHD. In the end, a diagnosis of ADHD cannot be ruled in by any particular cognitive profile, nor can any particular cognitive profile rule it out.

It would be nice if there were a well validated set of attention tests that could identify who has ADHD and who does not. What we need is a set of objectively scored tests of attentional ability on which most people with ADHD score very poorly and most people without ADHD do not score poorly. It would be even better if people with other disorders that affect attention did well on such tests so that the attention deficits measured by the tests would be specific to ADHD.

Sadly, no such battery of tests exists and probably never will. It is not for lack of effort that such a test battery has not been found. Indeed, there is no shortage of tests of attention and there are many studies showing that, on average, people diagnosed with ADHD score somewhat worse on such tests than do people who do not have ADHD. Tests that discriminate between people with ADHD and same-age peers from the population include measures of sustained and divided attention, working memory, verbal fluency, complex information processing speed, response inhibition, and verbal list learning (Schwean & Saklofske, 2005; Woods, Lovejoy, and Ball, 2002). Unfortunately, when we compare people with and without ADHD, the mean performance difference on such tests is only about 0.6 standard deviations (Frazier, Demaree, & Youngstrom, 2004), which means that the distributions of scores in the two groups are mostly overlapping.

Although it is possible for researchers to learn quite a bit about ADHD from small-to-modest mean differences in test performance, attention tests do not improve individual diagnostic accuracy very much (Goldstein & Kennemer, 2009). There are useful formulas (e.g., Elwood, 1993) that can be used with attention test scores such that they can assist clinicians in making ADHD diagnoses.  In general, when the base rates of ADHD are high, tests of attention tend to have modest positive predictive power but poor negative predictive power (e.g., Doyle Biederman, Seidman, Weber, & Faraone, 2000). In some settings it might be worth the effort to calculate the positive predictive power and negative predictive power of attention tests, but for most purposes, the following guidelines are suggested:

  • If you are already confident (in either direction) about whether ADHD is the correct diagnosis, there are no cognitive test results in the world that should undermine your confidence. However, attention tests might be extremely useful for other purposes.
  • If you are leaning toward a certain diagnostic decision and the person’s performance on cognitive tests of attention are consistent with your emerging case conceptualization, your diagnostic decision can be made with more confidence.
  • If you are dealing with a complex case and are quite unsure about whether ADHD is the correct diagnosis, administering a battery of attention tests is unlikely to help you make up your mind. However, in the process, you might discover really important information about the person.

References

Doyle, A.E., Biederman, J., Seidman, L.J.,Weber,W., & Faraone, S.V. (2000). Diagnostic efficiency of neuropsychological test scores for discriminating boys with and without ADHD. Journal of Consulting and Clinical Psychology, 68, 477–488.

Elwood, R. W. (1993). Clinical discriminations and neuropsychological tests: An appeal to Bayes’ theorem. The Clinical Neuropsychologist, 7(2), 224−233.

Frazier, T. W., Demaree, H. A., & Youngstrom, E. (2004). Meta-analysis of intellectual and neuropsychological test performance in Attention Deficit Hyperactivity Disorder. Neuropsychology, 18, 543−555.

Goldstein, S., & Kennemer, K. (2009). Neuropsychological aspects of attention-deficit hyperactivity disorder. In C. R. Reynolds & E. Fletcher-Janzen (Eds.), Handbook of Clinical Child Neuropsychology (3rd ed.; pp. 617−633). New York, NY: Springer.

Grodzinsky, G. M., & Barkley, R. A. (1999). Predictive power of frontal lobe tests in the diagnosis of attention deficit hyperactivity disorder. The Clinical Neuropsychologist, 13(1), 12−21.

Schwean, V. L., & Saklofske, D. H. (2005). Assessment of Attention Deficit Hyperactivity Disorder with the WISC-IV. In A. Prifitera, D. H. Saklofske, & L. G. Weiss (eds.), WISC-IV clinical use and interpretation: Scientist-practitioner perspectives (pp. 235–280). San Diego, CA: Elsevier Academic Press.

Woods, S. P., Lovejoy, D. W. & Ball, J. D. (2002). Neuropsychological characteristics of adults with ADHD: A comprehensive review of initial studies. The Clinical Neuropsychologist, 16(1), 12−34.

Standard