CHC Theory, Cognitive Assessment

Intelligence and the Modern World of Work: A Special Issue of Human Resource Management Review

Charles Scherbaum and Harold Goldstein took an innovative approach to editing a special issue of Human Resource Management Review. They asked prominent I/O psychologists to collaborate with scholars from other disciplines to explore how advances in intelligence research might be incorporated into our understanding of the role of intelligence in the workplace.

It was an honor to be invited to participate, and it was a pleasure to be paired to work with Daniel Newman of the University of Illinois at Urbana/Champaign. Together we wrote an I/O psychology-friendly introduction to current psychometric theories of cognitive abilities, emphasizing Kevin McGrew‘s CHC theory. Before that could be done, we had to articulate compelling reasons I/O psychologists should care about assessing multiple cognitive abilities. This was a harder sell than I had anticipated.

Formal cognitive testing is not a part of most hiring decisions, though I imagine that employers typically have at least a vague sense of how bright job applicants are. When the hiring process does include formal cognitive testing, typically only general ability tests are used. Robust relationships between various aspects of job performance and general ability test scores have been established.

In comparison, the idea that multiple abilities should be measured and used in personnel selection decisions has not fared well in the marketplace of ideas. To explain this, there is no need to appeal to some conspiracy of test developers. I’m sure that they would love to develop and sell large, expensive, and complex test batteries to businesses. There is also no need to suppose that I/O psychology is peculiarly infected with a particularly virulent strain of g zealotry and that proponents of multiple ability theories have been unfairly excluded.

To the contrary, specific ability assessment has been given quite a bit of attention in the I/O psychology literature, mostly from researchers sympathetic to the idea of going beyond the assessment of general ability.  Dozens (if not hundreds) of high-quality studies were conducted to test whether using specific ability measures added useful information beyond general ability measures. In general, specific ability measures provide only modest amounts of additional information beyond what can be had from general ability scores (ΔR2 ≈ 0.02–0.06). In most cases, this incremental validity was not large enough to justify the added time, effort, and expense needed to measure multiple specific abilities. Thus it makes sense that relatively short measures of general ability have been preferred to longer, more complex measures of multiple abilities.

However, there are several reasons that the omission of specific ability tests in hiring decisions should be reexamined:

  • Since the time that those high quality studies were conducted, multidimensional theories of intelligence have advanced, and we have a better sense of which specific abilities might be important for specific tasks (e.g., working memory capacity for air traffic controllers). The tests measuring these specific abilities have also improved considerably.
  • With computerized administration, scoring, and interpretation, the cost of assessment and interpretation of multiple abilities is potentially far lower than it was in the past. Organizations that make use of the admittedly modest incremental validity of specific ability assessments would likely have a small but substantial advantage over organizations that do not. Over the long run, small advantages often accumulate into large advantages.
  • Measurement of specific abilities opens up degrees of freedom in balancing the need to maintain the predictive validity of cognitive ability assessments and the need to reduce the adverse impact on applicants from disadvantaged minority groups that can occur when using such assessments. Thus, organizations can benefit from using cognitive ability assessments in hiring decisions without sacrificing the benefits of diversity.

The publishers of Human Resource Management Review have made our paper available to download for free until January 25th, 2015.

Broad Abilities in CHC Theory

Broad Abilities in CHC Theory

Advertisements
Standard
CHC Theory

Fluid and Crystallized Intelligence in the Classroom and on the Job

Fluid intelligence is the ability to solve unfamiliar problems using logical reasoning. It requires the effortful control of attention to understand what the problem is and to work toward a logically sound answer. People with high fluid intelligence are able to figure out solutions to problems with very little instruction. Once they have found a good solution to a problem, they are able to see how it might apply to other similar problems. People with low fluid intelligence typically need hands-on, structured instruction to solve unfamiliar problems. Once they have mastered a certain skill or solution to a problem, they may have trouble seeing how it might apply in other situations. That is, their newfound knowledge does not generalize easily to other situations.

— Schneider & McGrew (2013, p. 772)

Gf in the Classroom and on the Job

Crystallized intelligence is acquired knowledge. When people solve important problems for the first time, they typically remember how they did it. The second time the problem is encountered, the solution is retrieved from memory rather than recreated anew using fluid intelligence. However, much of what constitutes crystallized intelligence is not the memory of solutions we personally have generated but the acquisition of the cumulative wisdom of those who have gone before us. That is, we are the intellectual heirs of all of the savants and geniuses throughout history. What they achieved with fluid intelligence adds to our crystallized intelligence. This is why even an average engineer can design machines that would have astounded Galileo, or even Newton. It is why ordinary high school students can use algebra to solve problems that baffled the great Greek mathematicians (who, for lack of a place-holding zero, could multiply large numbers only very clumsily).

Crystallized intelligence, broadly speaking, consists of one’s understanding of the richness and complexity of one’s native language and the general knowledge that members of one’s culture consider important. Of all the broad abilities, crystallized intelligence is by far the best single predictor of academic and occupational success. A person with a rich vocabulary can communicate more clearly and precisely than a person with an impoverished vocabulary. A person with a nuanced understanding of language can understand and communicate complex and subtle ideas better than a person with only a rudimentary grasp of language. Each bit of knowledge can be considered a tool for solving new problems. Each fact learned enriches the interconnected network of associations in a person’s memory. Even seemingly useless knowledge often has hidden virtues. For example, few adults know who Gaius and Tiberius Gracchus were (Don’t feel bad if you do not!). However, people who know the story of how they tried and failed to reform the Roman Republic are probably able to understand local and national politics far better than equally bright people who do not. It is not the case that ignorance of the Gracchi brothers dooms anyone to folly. It is the case that a well-articulated story from history can serve as a template for understanding similar events in the present.

— Schneider & McGrew (2013, pp. 772–773)

Gc in the Classroom and on the Job
Gf Gc Typology

The pictures are previously unpublished (and not to be taken too seriously).

Definitions from:

Schneider, W. J. & McGrew, K. S. (2013). Cognitive performance models: Individual differences in the ability to process information. In S. Ortiz & D. Flanagan (Sec. Eds.), Section 9: Assessment Theory, in B. Irby, G. Brown, & R. Laro-Alecio & S. Jackson (Vol Eds.), Handbook of educational theories (pp. 767–782). Charlotte, NC: Information Age Publishing.

Standard
CHC Theory, Cognitive Assessment, R

Interactive 3D Multidimensional Scaling of the WJ III

I have been playing around with interactive 3D images (with the rgl package in R) and thought that it would be fun to present a multidimensional scaling (MDS) of the WJ III NU. Kevin McGrew has produced a number of beautiful images with MDS. My favorite is this one, not just because it is gorgeous, but because of the theoretical insights it communicates.

I simply took the correlation matrix from the WJ III NU standardization sample (ages 9 to 13) and subtracted each correlation from 1 to produce a distance measure. I performed classic MDS in R with the cmdscale function, allowing 3 dimensions. I colored each test with my guess as to which CHC factor it belongs.

If you click the static image below, you can play with it (Firefox and Chrome worked for me but Internet Explorer and Safari did not.):

WJ III MDS in 3D

WJ III MDS in 3D

R code used to generate this image

Standard