CHC Theory, Cognitive Assessment

Exploratory model of cognitive predictors of academic skills that I presented at APA 2014

I have many reservations about this model of cognitive predictors of academic abilities that I presented at APA today (along with co-presenters Lee Affrunti, Renée Tobin, and Kimberley Collins) but I think that it illustrates an important point: prediction and explanation of cognitive and academic abilities is so complex that it is impossible to do in one’s head. Eyeballing scores and making pronouncements is not likely to be accurate and will result in misinterpretations. We need good software that can manage the complex calculations for us. We can still think creatively in the diagnostic process but the creativity must be grounded in realistic probabilities.

The images from the poster are from a single exploratory model based on a clinical sample of 865 college students. The model was so big and complex I had to split the path diagram into two images:

Exploratory Model of WAIS and WJ III cognitive subtests

Exploratory Model of WAIS and WJ III cognitive subtests. Gc = Comprehension/Knowledge, Ga = Auditory processing, Gv = Visual processing, Gl = Long-term memory: Learning, Gr = Long-term memory: Retrieval speed, Gs = Processing speed, MS = Memory span, Gwm = Working memory capacity, g = Anyone’s guess

Exploratory model of cognitive predictors of WJ III academic subtests

Exploratory model of cognitive predictors of WJ III academic subtests. Percentages in error terms represent unexplained variance.

Cognitive Assessment

Cognitive profiles are rarely flat.

Because cognitive abilities are positively correlated, there is an assumption that cognitive abilities should be evenly developed. When psychologists examine cognitive profiles, they often describe any features that deviate from the expected flat profile.

It is true, mathematically, that the expected profile IS flat. However, this does not mean that flat profiles are common. There is a very large set of possible profiles and only a tiny fraction are perfectly flat. Profiles that are nearly flat are not particularly common, either. Variability is the norm.

Sometimes it helps to get a sense of just how uneven cognitive profiles typically are. That is, it is good to fine-tune our intuitions about the typical profile with many exemplars. Otherwise it is easy to convince ourselves that the reason that we see so many interesting profiles is that we only assess people with interesting problems.

If we use the correlation matrix from the WAIS-IV to randomly simulate multivariate normal profiles, we can see that even in the general population, flat, “plain-vanilla” profiles are relatively rare. There are features that draw the eye in most profiles.

WAISIVProfilesIf cognitive abilities were uncorrelated, profiles would be much more uneven than they are. But even with moderately strong positive correlations, there is still room for quite a bit of within-person variability.

Let’s see what happens when we look at profiles that have the exact same Full Scale IQ (80, in this case). The conditional distributions of the remaining scores are seen in the “violin” plots. There is still considerable diversity of profile shape even though the Full Scale IQ is held constant.

WAISIVProfiles80Note that the supplemental subtests have wider conditional distributions because they are not included in the Full Scale IQ, not necessarily because they are less g-loaded.