Cognitive Assessment, Principles of assessment of aptitude and achievement

How to Assess Aptitudes If You Are a Mono-g-ist

For the mono-g-ist, the assessment of aptitudes is rather simple: measure g and be done with it. Other abilities may have a little predictive validity beyond g, but not enough to make it worth all the additional effort needed (Glutting, Watkins, Konold, & McDermott, 2006). This advice is simple enough, but how does one measure g well?

The first step is to select a set of highly g-loaded tests. The term highly g-loaded simply means to correlate strongly with statistical g. This raises an important question. If the existence of g is in doubt, how can we know if a test correlates with it? To the poly-G-ist, this might sound like studying the environmental impact of unicorn overpopulation. The problem is resolved by distinguishing between two different meanings of g. First, there is theoretical g, a hypothetical entity thought to have causal relationships with many aspects of daily functioning. This is the g that many doubt exists. Second, there is statistical g, which is not in question. It is typically defined by a statistical procedure called factor analysis (or a closely related procedure called principal components analysis). All scholars agree that statistical g can be extracted from a correlation matrix and that virtually all cognitive tests correlate positively with it to some degree. Thus, a g-hating poly-G-ist can talk about a g-loaded test without fear of self-contradiction. A highly g-loaded test simply has a strong correlation with statistical g. A highly g-loaded test, then, is by definition highly correlated with many other tests. This means that it is probably a good predictor of academic achievement tests, which are, for the most part, also highly g-loaded. A cognitive test with a low g-loading (e.g., WJ III Planning or WISC-IV Cancellation) does not correlate with much of anything except itself. Mono-g-ists avoid such tests whenever possible (but Poly-g-ists love them—if they can be found to be uniquely predictive of an important outcome).

The second step to estimate g is to make sure that the highly g-loaded tests you have selected are as different from each other as possible in terms of item content and response format. To select highly similar tests (e.g., more than one vocabulary test) will contaminate the estimate of g with the influence of narrow abilities, which, to the mono-g-ist, are unimportant.

Fortunately, cognitive ability test publishers have saved us much trouble and have assembled such collections of subtests to create composite scales that can be used to estimate g. Such composite scores go by many different names[1] but I will refer to them as IQ scores. These operational measures of g tend to correlate strongly with one another, mostly in the range of 0.70 to 0.80 but sometimes as low as 0.60 or as high as 0.90 (Kamphaus, 2005). Even so, they are not perfectly interchangeable. If both tests have the traditional mean of 100 and standard deviation of 15, the probability that the two scores will be within a certain range of each other can be found in the Table below.[2] For example, for a person who takes two IQ tests that are correlated at 0.80, there is a 29% chance that the IQ scores will differ by 10 points or more.

What is the probability that a person’s scores on two IQ tests will differ by the specified amount or more?

 

Probability if the IQ tests correlate at r =

Difference

0.60

0 .70

0 .80

0 .90

> 5

0.71

0.67

0.60

0.46

> 10

0.46

0.39

0.29

0.14

> 15

0.26

0.20

0.11

0.03

> 20

0.14

0.09

0.03

0.003

> 25

0.06

0.03

0.01

0.0002

 

If a person has two or more IQ scores that differ by a wide margin, it does not necessarily mean that something is wrong. To insist on perfect correlations between IQ tests is not realistic and not fair.[3] However, when a child has taken two IQ tests recently and the scores are different, it raises the question of which IQ is more accurate.


[1] Full Scale IQ (WISC-IV, SB5, UNIT), Full Scale (Leiter-R, CAS), General Intellectual Ability (WJ III), General Conceptual Ability (DAS-II), Composite Intelligence Index (RIAS), Composite Intelligence Scale (KAIT), Fluid-Crystallized Index (KABC-II), and many others.

[2] This table was created by calculating the standard deviation of the difference between two correlated normally distributed variables and then applying the cumulative probability density function of the normal curve.

[3] “If I were to command a general to turn into a seagull, and if the general did not obey, that would not be the general’s fault. It would be mine.” – Antoine de Saint-Exupéry, The Little Prince

This post is an excerpt from:

Schneider, W. J. (2013). Principles of assessment of aptitude and achievement. In D. Saklofske, C. Reynolds, & V. Schwean (Eds.), Oxford handbook of psychological assessment of children and adolescents (pp. 286–330). New York: Oxford

Advertisements
Standard

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s