Intelligence and the Modern World of Work: A Special Issue of Human Resource Management Review

Charles Scherbaum and Harold Goldstein took an innovative approach to editing a special issue of Human Resource Management Review. They asked prominent I/O psychologists to collaborate with scholars from other disciplines to explore how advances in intelligence research might be incorporated into our understanding of the role of intelligence in the workplace.

It was an honor to be invited to participate, and it was a pleasure to be paired to work with Daniel Newman of the University of Illinois at Urbana/Champaign. Together we wrote an I/O psychology-friendly introduction to current psychometric theories of cognitive abilities, emphasizing Kevin McGrew‘s CHC theory. Before that could be done, we had to articulate compelling reasons I/O psychologists should care about assessing multiple cognitive abilities. This was a harder sell than I had anticipated.

Formal cognitive testing is not a part of most hiring decisions, though I imagine that employers typically have at least a vague sense of how bright job applicants are. When the hiring process does include formal cognitive testing, typically only general ability tests are used. Robust relationships between various aspects of job performance and general ability test scores have been established.

In comparison, the idea that multiple abilities should be measured and used in personnel selection decisions has not fared well in the marketplace of ideas. To explain this, there is no need to appeal to some conspiracy of test developers. I’m sure that they would love to develop and sell large, expensive, and complex test batteries to businesses. There is also no need to suppose that I/O psychology is peculiarly infected with a particularly virulent strain of g zealotry and that proponents of multiple ability theories have been unfairly excluded.

To the contrary, specific ability assessment has been given quite a bit of attention in the I/O psychology literature, mostly from researchers sympathetic to the idea of going beyond the assessment of general ability.  Dozens (if not hundreds) of high-quality studies were conducted to test whether using specific ability measures added useful information beyond general ability measures. In general, specific ability measures provide only modest amounts of additional information beyond what can be had from general ability scores (ΔR² ≈ 0.02–0.06). In most cases, this incremental validity was not large enough to justify the added time, effort, and expense needed to measure multiple specific abilities. Thus it makes sense that relatively short measures of general ability have been preferred to longer, more complex measures of multiple abilities.

However, there are several reasons that the omission of specific ability tests in hiring decisions should be reexamined:

• Since the time that those high quality studies were conducted, multidimensional theories of intelligence have advanced, and we have a better sense of which specific abilities might be important for specific tasks (e.g., working memory capacity for air traffic controllers). The tests measuring these specific abilities have also improved considerably.
• With computerized administration, scoring, and interpretation, the cost of assessment and interpretation of multiple abilities is potentially far lower than it was in the past. Organizations that make use of the admittedly modest incremental validity of specific ability assessments would likely have a small but substantial advantage over organizations that do not. Over the long run, small advantages often accumulate into large advantages.
• Measurement of specific abilities opens up degrees of freedom in balancing the need to maintain the predictive validity of cognitive ability assessments and the need to reduce the adverse impact on applicants from disadvantaged minority groups that can occur when using such assessments. Thus, organizations can benefit from using cognitive ability assessments in hiring decisions without sacrificing the benefits of diversity.

The publishers of Human Resource Management Review have made our paper available to download for free until January 25th, 2015.

Broad Abilities in CHC Theory

Exploratory model of cognitive predictors of academic skills that I presented at APA 2014

I have many reservations about this model of cognitive predictors of academic abilities that I presented at APA today (along with co-presenters Lee Affrunti, Renée Tobin, and Kimberley Collins) but I think that it illustrates an important point: prediction and explanation of cognitive and academic abilities is so complex that it is impossible to do in one’s head. Eyeballing scores and making pronouncements is not likely to be accurate and will result in misinterpretations. We need good software that can manage the complex calculations for us. We can still think creatively in the diagnostic process but the creativity must be grounded in realistic probabilities.

The images from the poster are from a single exploratory model based on a clinical sample of 865 college students. The model was so big and complex I had to split the path diagram into two images:

Exploratory Model of WAIS and WJ III cognitive subtests. Gc = Comprehension/Knowledge, Ga = Auditory processing, Gv = Visual processing, Gl = Long-term memory: Learning, Gr = Long-term memory: Retrieval speed, Gs = Processing speed, MS = Memory span, Gwm = Working memory capacity, g = Anyone’s guess

Exploratory model of cognitive predictors of WJ III academic subtests. Percentages in error terms represent unexplained variance.

Beyond IQ, a new blog about psychological assessment

Beyond IQ is an excellent new blog about psychological assessment. It was created about a month ago by Smadar Sapir Yogev, an educational psychologist working in Jerusalem.

Smadar Sapir Yogev

Smadar posts in both Hebrew and in English on a wide range of assessment-related topics. She has created an excellent nine-part series on CHC theory, of which six parts have been published. My favorite in the series so far is the presentation on long-term memory, which presents CHC theory memory abilities alongside other aspects of memory processes identified by cognitive psychologists.

I look forward to reading Smadar’s future posts on a diverse range of topics.

CHC Theory Slides

In response to a query, I made some slides about CHC Theory with simplified definitions. Feel free to do with them whatever you like. The slides in PowerPoint look better but for the sake of convenience I also put them on Prezi:

Where does Emotional Intelligence fit into CHC Theory?

A new study by MacCann, Joseph, Newman, and Roberts (2013) about the place of emotional intelligence in CHC Theory is worth reading. I highlight some of its findings and discuss other matters in this video:

Fluctuations in attention are related to fluid but not crystallized intelligence

Attentional Control and Fluid Intelligence

There are many defensible ways to slice the ability domain. In a previous post, I put fluid intelligence, working memory capacity, and processing speed together in a conceptual grouping called Controlled Attention. I did not do this capriciously but on my review of the available evidence. However, the precise nature of the ways in which these abilities depend on attentional control is still being explored.

In what I consider to be an important paper, Unsworth and McMillan (2014) provide direct evidence that fluid intelligence test performance is related to moment-to-moment fluctuations of one’s attentional state. The paper consists of three experiments designed to tease apart various explanations of the positive correlation between test item performance and self-rated attentional state measured before each item (ranging from 1 = not at all focused on the task to 10 = totally focused on the present task).

Overall findings

1. Test performance was not negatively affected by having to complete attentional state ratings.
2. Self-rated attentional state predicted performance on fluid intelligence test items but not on crystallized test items.
3. Participants with the most variability in self-rated attentional state from item to item performed more poorly on fluid intelligence test items than did people with more stable levels of self-rated attentional state. Thus, attentional control, in accordance with theory, appears to be an important component of fluid intelligence.

One of my suspicions was that is that participants might justify poor perceived performance on a previous item by claiming low levels of attention before the next item. It might be easier on one’s self esteem to claim, “That last item was hard because I am feeling scattered, not because I am not smart.” However, this explanation is undermined by the fact that self-rated attentional state predicted performance on fluid intelligence test items whether the items were in ascending level of difficulty or in random order. Even so, it would have been nice to have seen analyses showing that attentional state predicted performance on the next item more strongly than it “predicts” performance on the previous item.

Fluid and Crystallized Intelligence in the Classroom and on the Job

Fluid intelligence is the ability to solve unfamiliar problems using logical reasoning. It requires the effortful control of attention to understand what the problem is and to work toward a logically sound answer. People with high fluid intelligence are able to figure out solutions to problems with very little instruction. Once they have found a good solution to a problem, they are able to see how it might apply to other similar problems. People with low fluid intelligence typically need hands-on, structured instruction to solve unfamiliar problems. Once they have mastered a certain skill or solution to a problem, they may have trouble seeing how it might apply in other situations. That is, their newfound knowledge does not generalize easily to other situations.

— Schneider & McGrew (2013, p. 772)

Crystallized intelligence is acquired knowledge. When people solve important problems for the first time, they typically remember how they did it. The second time the problem is encountered, the solution is retrieved from memory rather than recreated anew using fluid intelligence. However, much of what constitutes crystallized intelligence is not the memory of solutions we personally have generated but the acquisition of the cumulative wisdom of those who have gone before us. That is, we are the intellectual heirs of all of the savants and geniuses throughout history. What they achieved with fluid intelligence adds to our crystallized intelligence. This is why even an average engineer can design machines that would have astounded Galileo, or even Newton. It is why ordinary high school students can use algebra to solve problems that baffled the great Greek mathematicians (who, for lack of a place-holding zero, could multiply large numbers only very clumsily).

Crystallized intelligence, broadly speaking, consists of one’s understanding of the richness and complexity of one’s native language and the general knowledge that members of one’s culture consider important. Of all the broad abilities, crystallized intelligence is by far the best single predictor of academic and occupational success. A person with a rich vocabulary can communicate more clearly and precisely than a person with an impoverished vocabulary. A person with a nuanced understanding of language can understand and communicate complex and subtle ideas better than a person with only a rudimentary grasp of language. Each bit of knowledge can be considered a tool for solving new problems. Each fact learned enriches the interconnected network of associations in a person’s memory. Even seemingly useless knowledge often has hidden virtues. For example, few adults know who Gaius and Tiberius Gracchus were (Don’t feel bad if you do not!). However, people who know the story of how they tried and failed to reform the Roman Republic are probably able to understand local and national politics far better than equally bright people who do not. It is not the case that ignorance of the Gracchi brothers dooms anyone to folly. It is the case that a well-articulated story from history can serve as a template for understanding similar events in the present.

— Schneider & McGrew (2013, pp. 772–773)

The pictures are previously unpublished (and not to be taken too seriously).

Definitions from:

Schneider, W. J. & McGrew, K. S. (2013). Cognitive performance models: Individual differences in the ability to process information. In S. Ortiz & D. Flanagan (Sec. Eds.), Section 9: Assessment Theory, in B. Irby, G. Brown, & R. Laro-Alecio & S. Jackson (Vol Eds.), Handbook of educational theories (pp. 767–782). Charlotte, NC: Information Age Publishing.

WAIS-IV & WIAT-III Multidimensional Scaling in 3D

Here is a static image of a multidimensional scaling of the WAIS-IV and WIAT-III. It is interactive when you click it (Works best in Firefox & Chrome, not in IE or Safari).

Interactive 3D Multidimensional Scaling of the the WISC-IV

Following the same procedures as I did with the WJ III, I have made an interactive 3D MDS of the WISC-IV. The image below is static but if you click it, you can rotate and zoom the image interactively.

WISC-IV MDS

Interactive 3D Multidimensional Scaling of the WJ III

I have been playing around with interactive 3D images (with the rgl package in R) and thought that it would be fun to present a multidimensional scaling (MDS) of the WJ III NU. Kevin McGrew has produced a number of beautiful images with MDS. My favorite is this one, not just because it is gorgeous, but because of the theoretical insights it communicates.

I simply took the correlation matrix from the WJ III NU standardization sample (ages 9 to 13) and subtracted each correlation from 1 to produce a distance measure. I performed classic MDS in R with the cmdscale function, allowing 3 dimensions. I colored each test with my guess as to which CHC factor it belongs.

If you click the static image below, you can play with it (Firefox and Chrome worked for me but Internet Explorer and Safari did not.):

WJ III MDS in 3D

R code used to generate this image