Cognitive Assessment, Research Link

No, the WISC-IV doesn’t underestimate the intelligence of children with autism.

The title of a new study asks “Does WISC-IV underestimate the intelligence of autistic children?” The authors’ answer is that it probably does. I believe that the reasoning behind this conclusion is faulty.

This study gives the unwarranted impression that it is a disservice to children with autism to use the WISC-IV. Let me be clear—I want to be helpful to children with autism. I certainly do not wish to do anything that hurts anyone. A naive reading of this article leads us to believe that there is an easy way to avoid causing harm (i.e., use the Raven’s Progressive Matrices test instead of the WISC-IV). In my opinion, acting on this advice does no favors to children with autism and may even result in harm.

Based on the evidence presented in the study, the average score differences between children with and without autism is smaller on Raven’s Progressive Matrices (RPM) and larger on the WISC-IV. The rhetoric of the introduction leaves the reader with the impression that the RPM is a better test of intelligence than the WISC-IV. Once we accept this, it is easy to discount the results of the WISC-IV and focus primarily on the RPM.

There is a seductive undercurrent to the argument: If you advocate for children with autism, don’t you want to show that they are more intelligent rather than less intelligent? Yes, of course! Doesn’t it seem harmful to give a test that will show that children with autism are less intelligent? It certainly seems so!

Such rhetoric reveals a fundamental misunderstanding of what individual intelligence tests like the WISC-IV are designed to do. In the vast majority of settings, they are not for certifying how intelligent a person is (whatever that means!). Their primary purpose is to help psychologists understand what a person can and cannot do. They are designed to help explain what is easy and what is difficult for a person so that appropriate interventions can be selected.

The WISC-IV provides a Full Scale IQ, which gives an overall summary of cognitive functions. However, it also gives more detailed information about various aspects of ability. Here is a graph I constructed from Figure 1 in the paper. In my graph, I converted percentiles to index scores and rearranged the order of the scores to facilitate interpretation.

asdf

Average Raven’s Progressive Matrices (RPM) and WISC-IV scores for children with and without autism

It is clear that the difference between the two groups of children is small for the RPM. It is also clear that the difference is also small for the WISC-IV Perceptual Reasoning Index (PRI). Why is this? The RPM and the PRI are both nonverbal measures of logical reasoning (AKA fluid intelligence). Both the WISC-IV and the RPM tell us that, on average, children with autism perform relatively well in this domain. The RPM is a great test, but it has no more to tell us. In contrast, the WISC-IV not only tells us what children with autism, on average, do relatively well, but also what they typically have difficulty with.

It is no surprise that the largest difference is in the Verbal Comprehension Index (VCI), a measure of verbal knowledge and language comprehension. Communication problems are a major component of the definition of autism. If children with autism had performed equally well on the VCI, we would wonder whether the VCI was really measuring what it was supposed to measure. Note that I am not saying that a low score on VCI is a requirement for the diagnosis of autism or that the VCI is the best measure of the kinds of language problems that are characteristic of autism. Rather, I am saying that children with autism, on average, have difficulties with language comprehension and that this difference is manifest to some degree in the WISC-IV scores.

The WISC-IV scores also suggest that, on average, children with autism not only have lower scores in verbal knowledge and comprehension, they are more likely to have other cognitive deficits, including in verbal working memory (as measured by the WMI) and information processing speed (as measured by the PSI).

Thus, as a clinical instrument, the WISC-IV performs its purpose reasonably well. Compared to the RPM, it gives a more complete picture of the kinds of cognitive strengths and weaknesses that are common in children with autism.

If the researchers wish to demonstrate that the WISC-IV truly underestimates the intelligence of children with autism, they would need to show that it underpredicts important life outcomes among this population. For example, suppose we compare children with and without autism who score similarly low on the WISC-IV. If the WISC-IV underestimated the intelligence of children with autism, they would be expected to do better in school than the low-scoring children without autism. Obviously, a sophisticated analysis of this matter would involve a more complex research design, but in principle this is the kind of result that would be needed to show that the WISC-IV is a poor measure of cognitive abilities for children with autism.

Standard

8 thoughts on “No, the WISC-IV doesn’t underestimate the intelligence of children with autism.

  1. concerned parent says:

    Mr. Schneider – I read you article and I’m not a psychologist just an interested parent looking for information about testing for my child. How would you account for average scores on the WISC and superior academic scores for a child with mild autism?

    • Good question.
      There are lots of possible reasons for this:
      1. Measurement error in either the WISC or the academic scores.
      2. A child who is working really hard in school.
      3. A parent who is working really hard to help the child in school.
      4. A really fantastic school.
      and many, many others.

  2. Shawn says:

    I’m not real convinced by your analysis here. You assert people with autism “are more likely to have other cognitive deficits”. Simply because on average autistics have lower scores in working memory for example doesn’t make it a better test.

    I guess what you are asserting is that Raven doesn’t capture working memory and the WISC-IV does.

    I believe your assertation to be demonstrably false and evidenced in the literature … there is a “typically strong correlation between measures of WMC and overall performance on Raven” https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3968765/

    So if Raven scores are higher and that reflects working memory, then why does WISC-IV

  3. Interesting article, but I fear that in doesn’t speak to the main problem of that gap, nor acknowledge the proven power of RPM. As you concentrate on, WISC-IV is still a better test of where a child is now, but RPM remains the supreme test of where they could be. This is why I find the gap between RPM and *all* WISC-IV subtests troubling, and any diminution of that problem a worry – as it should otherwise be a Clarion call for research in to radical new teaching strategies.

  4. Justin Digney says:

    Thanks Joel,

    I came across your article after reading a paper that convincingly demonstrated that the RPM does better illustrate the strengths of Autism/Aspergers. Of course that was the purpose of the research, to show Autism Spectrum Disorder does not necessarily have a lower IQ but rather a different IQ.

    While originally highly skeptical of your article it is clear that you are correct. From an “employability” perspective, the RPM over estimates an employees Overall skills.

    However the summation process of General IQ is highly flawed in this regard. For example my Spacial skills IQ (assessed at age 6) was 130, and my general IQ was 120. My high spacial skills along with other strengths ‘clearly’ significantly skewed my overall IQ results.

    From a social interaction point of view the summation process to form a general IQ result does ASD a total disjustice. Indeed, looking at the Quality of Life (QoL) findings for Adults on the spectrum, and government policies, (particularly here in Australia), Higher general IQ is strongly correlates with low QoL.

    I don’t know if an IQ test for social awkwardness exists. I know from my own observation that my inference (as related to literacy) skills are significantly atypical. Autism appears to focus around fundamental ‘use’ of language. Written, verbal and non-verbal language (facial expression, body language, tone, volume).

    The impairment of having atypical language skills is SIGNIFICANTLY underplayed in public policy around the management/support afforded to Autism.

    In QoL terms ASD outcomes worsen with increased general IQ, yet support is non-existent. While it is the expectation of general society for ASD (and all individuals) to conform to social expectations, ASD appears to draw its social expectations via alternative means (I am not sure if this has even been researched), reflecting societies mantras and ideals as opposed to its realities.

    For Psychology to do service to ASD it must find a way to highlight the ‘very’ real disadvantage atypical IQ (or specific/related peak IQ) has on adult function in the contemporary workplace.

    I have no doubt (zero) that general IQ is a relevant measure in its current form. The issue relates 100% to social expectations. The broader public expect people with a given level of intelligence to have a typical IQ distribution (form of standard deviation if you like). Individuals with an atypical ‘standard deviation’ are impaired demonstrably.

    Infact, the relatively low nuanced the understanding of General IQ (by the general public), that general IQ could/should be reported differently. Without knowing anything of the calculation of General IQ, I proposed to report the highest IQ, and the difference between the highest and the lowest, therefore IQ could/should be reported as 100/30 (ie lowest facit 100, highest 130). Clearly for Neuro-Typical this would be expected to be something in the order of (100/5).

    This cleary has more value if we can accurately measure an individual’s Emotional Intelligence (EQ) and incorporate this into the general IQ score. Alternativley an accurate measure of EQ to be used for disability support funding? I am not aware of research being done into the relationship between EQ/IQ and QoL. I speculate EQ=IQ has a correlation with IQ/QoL. Whereas IQ>EQ has a correlation with low QoL, and EQ>IQ has a correlation with high QoL.

    • Interesting response/ideas, much/most of which I agree with!

      General IQ gives a predictor of ability/potential only for those with an even cognitive profile, where all scores – lowest to highest – fall within a parameter of two standard deviations (SD). As I understand it, one standard deviation in IQ comes in at 15 percentage points: with an average IQ of 100, this would allow for a highest sub-test score of 115 and a lowest one of 85. It’s worth noting here that your 30% spread actually falls – just – within the norm(!). However, from what I’ve read, the spread of test scores is usually more closely spread than the maximum two SDs.

      Although those tested for IQ (certainly those using the WASI: Weschler Abbreviated Scale of Intelligence) do so in only 10 to 12 different areas of cognition, one can usually extrapolate from these to indicate a general level of intelligence/potential across other areas. That is certainly the case for those with an even cognitive profile. So the Full IQ (FIQ) is a useful indicator here.

      For those of us with a spiky profile (scores being separated by two or more SDs, eg a spread of 30+ points), neither a FIQ nor extrapolating from it works. Instead, the FIQ averages out our scores, so undervaluing our areas of strength while covering up those areas we struggle in. Worse, for those of us whose strengths might be apparent (eg having good verbal ability), others subconsciously expect more from us than we may be able to deliver (they simply see we are ‘able’), and demonising us for what they see as us being unprofessional. Conversely, for those whose weaker areas are apparent (eg having poor verbal ability), others underestimate their abilities (they only see lack of ability) so have reduced expectations, reducing chances of engagement or promotion for the individual. Either way, we spiky profilers lose out.

      Me? Highest scores (Block Design and Matrix Reasoning): both 130-132, lowest score (Coding): 90. So that would give me (in your revised way of indicating IQ) an IQ of 90/130. (Weirdly, my actual FIQ came in at 134, with a confidence interval (95%) of 128-137 FIQ. Don’t know how they worked that one out! Best left for another day…)

      One flaw that I can see in your revision: the lowest/highest IQs given still only show strengths and weaknesses *in those areas tested*. Whereas this would work for the general population (because extrapolating from FIQ of those with an even cognitive profile works), this still leaves those of us with an uneven cognitive profile wanting. Why? Because it fails to show possibly higher or lower levels of ability. We can’t assume that, just because my top scores were 130-132, that is my ceiling on ability: it simply shows the highest score on the areas I was tested in. (So I may be much brighter than even my best score shows.) Similarly, my lowest score (90) only shows my least able area in those areas tested. But my struggles can be quite profound: I have (until relatively recently) been able to cover these up by simply working harder, longer than my peers in order to just get by (thank you, ADHD uber energy!). So

      Further, the score of 90 doesn’t look that bad, given that the average IQ is given as 85-115. Except the score actually equates to a percentile rating of 25, putting me in the bottom quarter of the population for this area (Coding), intelligence-wise. So I’m seen as bright – yet three-quarters of the population do better than me in a significant area of brain processing. You wouldn’t pick that up simply from the IQ score of 90.

      One more thing that may or not be relevant (I’ve no idea how common this is or whether I just got a bum deal!): the Educational Psychologist who ran the tests on me decided I didn’t need to be tested in maths as – to quote my report – ‘Not assessed as Gilly achieved well at GCSE’. In fact I didn’t; I scraped a C (A-C is Pass, D-E is fail). Regardless, this means my cognitive profile does not include a score (good or bad) for one very significant area. Suffice to say, my maths *is* excellent, much more than my C grade at school might indicate (just the typical underachievement of an individual whose neurodivergent mind hasn’t yet been recognised).

      Sorry; I’m sure my thoughts must come across to you as negative – but you couldn’t be more wrong! Your reworked way of showing IQ, if used, would be a massive step in the right direction for what is a hugely unhelpful way of indicating our strengths and challenges. Thanks for that!

      • Justin Digney says:

        Dear GitllyWilly,

        Forgive me, I have not considered this topic in some time, and am not current in any way shape or form.

        From the amazing detail you have been able to provide, it is clear FIQ is calculated mathematically based purely on some statistucal mean, which again has less validity for neuro-divergent.

        Futher FIQ and any/all sub meanures is declared to be relatively stable throughtout a lifetime. However in the case of ASD and presumeably other neuro-divergent brains pronounced late synapsis trimming has been well documented, however the documented, day to day and lived experience of this process is unknown to me and significant work in their area may have been undertaken recently.

        By the magic of intuitive projection, I would suggest such transitions in the physical brain structure would/must be associated with significant changes is psychy, perhaps a result of deep and/or prolonged mid life crisis, existential crisis, depression or other identified (or un-identified) transition period(s).

        It also appears other non-scholarly (and perhaps purely sectarian) sources identify with this second life pethaps in those divoted to wisdom and matters of the soul, (the essential question of an existential crissis is realted to the Soul, and the underlying ‘knowing’ that there “IS” more to life than our physical being.

        But is this reflected in brain images? Does our brain physically reconfigure to actively step back out of the logical (pre-frontal cortex), meet our physical needs world and transition to the metaphysical, soul and spiritual world?

        And if so is this shift more pronoumced in Certain groups of the neuro-divergent? Indeed without looking at brain scans and from the limited understanding of brain anatomy, I could assert ASD is a neurological condition of extreme dominence of the pre-frontal cortex – the most biologically ‘new’ and specifically ‘human’ physiological adaptation. Does this late synapsis trimming simply make the ASD brain “more efficient” as the accepted explanation of the typical synapsis trimming in adolecence? Or are these structural changes of the brain inherently different and related to a second comming (so to speak) of a spiritual (or other) nature?

        Is our socities un-precidented move away from spirituality and mysticism the cause or at least a significant factor in suicide? Indeed the anecdotal and scientific evidence (curtesy of positive psychology) suggests it is.

        Is it then any co-incidence that those in society who “may” be undergoing the largest transition between the dominant pre-frontal state and a second more “spiritual” state (for a want of more clearly defined concept), also account for a significantly larger invidence of both youth and midlife suicide?

        Anicdotally are not the lives of those on the spectrim (ASD) reflective of those of all the great profits? Perhaps in times gone by the “special interest” of individuals on the spectrim uncluded more religious content.

        ASD and other neuro-divergent sets itself well to become an expert in the contemporary spiritual teachings of the day, but maintain a totally divergent understanding?

        In as much as I can affirm the benefit of divergent thought as it is ‘THE’ super power of humanity, and as much as humanity accepts the gifts of many divergent thinkers “eventually”, it isn’t without persecution of the messanger.

        Can man believe 100% in science and 100% in God? Not only can he, but if the truth be in him, he must.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s