Cognitive Assessment, Principles of assessment of aptitude and achievement

Advice for Psychological Evaluation Reports: Write about people, not tests

At its best, the end product of a psychological assessment is that a child’s life is made better because something useful and true is communicated to people who can use that information to make better decisions. How is this information best communicated? I believe that it is by the skillful retelling of the story of the child’s struggle to cope with the difficulties that led to the testing referral.

Not only are humans storytelling creatures, we are also storylistening creatures. We are moved by drama, cleansed by tragedy, unified by cultural myths, and inspired by tales of heroic struggle. Most importantly, through stories we remember enormous amounts of information. Tabulated test results are inert until the evaluator weaves them together into a coherent narrative explanation that helps children and their caregivers construct a richer, more nuanced, and more organized understanding of the problem. Compare the following assessment results.

Explanation 1

On a test in which Judy had to repeat words and segment them into individual phonemes, Judy earned a standard score of 78, which is in the Borderline Range. Only 7 percent of children performed at Judy’s level or lower on this test. This test is a good predictor of the ability to read single words isolated from contextual cues. On a test that measures this ability, Judy scored an 83, which is in the 13th percentile or in the Low Average Range. Reading single words is necessary to understand sentences and paragraphs. On a test that requires the evaluee to read a paragraph and then answer questions that test the evaluee’s understanding of the text, Judy scored an 84, which is in the Low Average Range. This is in the 14th percentile. An 84 in Reading Comprehension is 24 points lower than her Full Scale IQ of 110 (75th percentile, High Average Range). This is significant at the .01 level and only 3% of children in Judy’s age range have a 24-point discrepancy or larger between Reading Comprehension and Full Scale IQ. Thus, Judy meets criteria for Reading Disorder. More specifically, Judy appears to have phonological dyslexia. Phonological dyslexia refers to difficulties in reading single words because of the inability to hear individual phonemes distinctly. This difficulty in decoding single words makes reading narrative text difficult because the reading process is slow and error prone. Intensive remediation in phonics skills followed by reading fluency training is recommended.

Explanation 2

For most 12-year-olds as bright as Judy is, reading is a skill that is so well developed and automatic that it becomes a pleasure. For Judy, however, reading is chore. It takes sustained mental effort for her to read each word one by one. It then requires further concentration for her to go back and figure out what these individual words mean when they are strung together in complete sentences, paragraphs, and stories. It is a slow, laborious process that is often unpleasant for Judy.

Why did Judy, a bright and delightfully creative girl, fail to learn to read fluently? It is impossible to know with certainty. However, the problem that most likely first caused Judy to fall behind her peers is that she does not hear speech sounds as clearly as most people do. It is as if she needs glasses for her ears: The sounds are blurry. For example, although she can hear the whole word cat perfectly well, she might not recognize as easily as most children do that the word consists of three distinct sounds: |k|, |a|, and |t|. For this reason, she has to work harder to remember that these three sounds correspond to three separate letters: |k|=C, |a|=A, and |t|=T. With simple words like cat, Judy’s natural ability is more than sufficient to help her remember what the letters mean. However, learning to recognize and remember larger words, uncommonly used words, or words with irregular spellings is much more difficult for Judy than it is for most children.

Many children with the same difficulty in hearing speech sounds distinctly eventually learn to work around the problem and come to read reasonably well. However, Judy is a perceptive and sensitive girl. These traits are typically helpful but, unfortunately, they allowed her to be acutely aware, from very early on, that she did not read as well as her classmates. She clearly remembers that her friends and classmates giggled when she made reading errors that were, to them, inexplicable. For example, for a while she earned the nickname “Tornado Girl” when she was reading aloud in class and misread “volcano” as “tornado.” She came to dread reading aloud in class and felt growing levels of shame even when she read silently to herself. She began to avoid reading at all costs. She did not read for pleasure, even when the texts were easy enough for her to read because she felt, in her words, “dumb, dumb, and dumb.” Over the next several years, she fell further behind her peers. By avoiding reading, she never developed the smooth, automatic reading skills that are necessary to make reading a pleasurable and self-sustaining activity.

Although Judy’s ability to hear speech sounds distinctly is still low compared to her 12-year old peers, this weakness is not what is holding her back now. Indeed, her current ability to hear speech sounds distinctly is actually better than that of most 6 and 7 year-olds, most of whom learn to read without difficulty. With extra help, Judy can learn to decode words phonetically. However, in order for her to develop her reading fluency and reading comprehension skills to the level that she is capable, she will need to engage in sustained practice reading texts that are both interesting for Judy and are at the correct level of difficulty. She is likely to be willing to read only if she is helped to manage the sense of shame she feels when she attempts to read a book. This may require the collaboration of a reading specialist and a behavior specialist with expertise in the cognitive-behavioral treatment of anxiety-related problems.

Comparing Explanations

I am reasonably confident that most readers would find the second explanation to be much more useful than the first. The second explanation is not better than the first simply because it is more detailed. Explanation 1 could have been supplemented with more details if I had taken the time to fill it with even more information about test results. The second explanation is not better simply because it avoids statistical jargon that is difficult for parents and teachers to understand. Even if the jargon were removed from the first explanation and inserted into the second, the second explanation would still be better.

The second explanation is better because it is more about Judy than about her performance on tests. The narrative explanation of how her reading problem developed and how it was maintained is better because it leads to better treatment recommendations. More importantly, it leads to recommendations that will be understood and remembered by Judy’s parents and teachers. One of the problems with the first explanation is, ironically, that it is not difficult to understand if it is properly explained. Most parents and teachers will nod their heads as they hear it. However, they are likely to forget the explanation as soon as they leave the room. Most of us are not accustomed to thinking about people in terms of sets of continuous variables. Without a narrative structure to hold them together, assessment details slip through the cracks of our memories quickly. It is unfortunate that a forgotten explanation, no matter how accurate, no matter how brilliant, is as helpful as no explanation at all.

This post is an excerpt from:

Schneider, W. J. (2013). Principles of assessment of aptitude and achievement. In D. Saklofske, C. Reynolds, & V. Schwean (Eds.), Oxford handbook of psychological assessment of children and adolescents (pp. 286–330). New York: Oxford.

Standard
Cognitive Assessment, Principles of assessment of aptitude and achievement

Allowing yourself to be wrong allows you to be right…eventually

The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge.

– Stephen Hawking

It is wise to remember that you are one of those who can be fooled some of the time.

– Laurence J. Peter

We human beings are so good at pattern recognition that sometimes we find patterns that are not even there. I have never seen a cognitive profile, no matter how unusual and outlandish, that did not inspire a vivid interpretation that explained EVERYTHING about a child. In fact, the more outlandish, the better. On a few occasions, some of the anomalous scores that inspired the vivid interpretations turned out to be anomalous due to scoring errors. In these humbling experiences, I have learned something important. I noticed that in those cases, my interpretations seemed just as plausible to me as any other. If anything, I was more engaged with them because they were so interesting. Of course, there is nothing wrong with making sense of data and there is nothing wrong with doing so with a little creativity. Let your imagination soar! The danger is in taking yourself too seriously.

The scientific method is a system that saves us from our tendencies not to ask the hard questions after we have convinced ourselves of something. Put succinctly, the scientific method consists of not trusting any explanation until it survives your best efforts to kill it. There is much to be gained in reserving some time to imagine all the ways in which your interpretation might be wrong. The price of freedom is responsibility. The price of divergent thinking is prudence. It is better to be right in the end than to be right right now.

This post is an excerpt from:

Schneider, W. J. (2013). Principles of assessment of aptitude and achievement. In D. Saklofske, C. Reynolds, & V. Schwean (Eds.), Oxford handbook of psychological assessment of children and adolescents (pp. 286–330). New York: Oxford.

Standard
Cognitive Assessment, Principles of assessment of aptitude and achievement

Advice for psychological evaluation reports: Render abstruse jargon in the vernacular

PRIMUS DOCTOR: Most learned bachelor whom I esteem and honor, I would like to ask you the cause and reason why opium makes one sleep.

BACHELIERUS: ….The reason is that in opium resides a dormitive virtue, of which it is the nature to stupefy the senses.

—from Molière’s Le Malade Imaginaire (1673)

A man thinks that by mouthing hard words he understands hard things.

—Herman Melville

The veil of ignorance can be weaved of many threads, but the one spun with the jangly jargon of a privileged profession produces a diaphanous fabric of alluring luster and bewitching beauty. Such jargon not only impresses outsiders but comforts them with what Brian Eno called the last illusion: the belief that someone out there knows what is going on. Too often, it is a two-way illusion. Like Molière’s medical student, we psychologists fail to grasp that our (invariably Latinate) technical terms typically do not actually explain anything. There is nothing wrong with technical terms, per se; indeed, it would be hard for professionals to function without them. However, with them, it is easy to fall into logical traps and never notice. For example, saying that a child does not read well because she has dyslexia is not an explanation. It is almost a tautology, unless the time is taken to specify which precursors to reading are absent, and thus, make dyslexia an informative label.

An additional and not insubstantial benefit of using ordinary language is that you are more likely to be understood. This is not to say that your communication should be dumbed down to the point that the point is lost. Rather, as allegedly advised by Albert Einstein, “Make everything as simple as possible, but not simpler.”

This post is an excerpt from:

Schneider, W. J. (2013). Principles of assessment of aptitude and achievement. In D. Saklofske, C. Reynolds, & V. Schwean (Eds.), Oxford handbook of psychological assessment of children and adolescents (pp. 286–330). New York: Oxford.

Standard
Cognitive Assessment, Principles of assessment of aptitude and achievement

Time lavished on hypothesis fishing trips is stolen from children we no longer have time to help.

We do what we must, and call it by the best names we can.

– Ralph Waldo Emerson

Cheetahs are the fastest animals on land but do not always catch their prey. For about 60 seconds or so, cheetahs give it their all. After that, they give up. Why? For a cheetah to persist, the expected rewards must justify the caloric expenditure, the risk of injury, and the considerable strain on their bodies that sprinting inevitably causes. In the wild, there is no glory in Pyrrhic victories. Sometimes it is better to cut your losses, even though you could “succeed” with more effort.

There is something analogous that happens in cognitive assessment. For a time, it is worthwhile to vigorously pursue a hypothesis, to clarify an anomalous finding, or to explain a curious behavior. However, when answers are not forthcoming, there is a point at which it is wise to give up, even before all alternatives have been exhausted. The time saved can be devoted to other questions about the child that may be important. It is perfectly acceptable to write in reports that, given the available data, it is not yet possible to distinguish between alternative hypotheses about a child. It is perfectly acceptable to speculate about those hypotheses, provided that those speculations are clearly labeled as such and that it is explicitly stated that the true explanation might not be included in the list of speculations.

This post is an excerpt from:

Schneider, W. J. (2013). Principles of assessment of aptitude and achievement. In D. Saklofske, C. Reynolds, & V. Schwean (Eds.), Oxford handbook of psychological assessment of children and adolescents (pp. 286–330). New York: Oxford University Press.

Standard
Cognitive Assessment, Principles of assessment of aptitude and achievement

Dr. Procrustes does not need to see you; he has your test scores.

I rock at the Tower of Hanoi—you could give me a stack of as many discs as you like and I can move the whole stack from one peg to the other without any hesitation and without a single error. I don’t mean to be immodest about it, but it’s true. My performance is like 11.8 standard deviations above the mean, which by my calculations is so rare that if a million people were born every second ever since the Big Bang, there is still only a 2.7% chance that I would have been born by now—I feel very lucky (and honored) to be here.

You would be forgiven for thinking that I had excellent planning ability…but not if you voiced such an opinion out loud, within earshot of my wife, causing her to die of laughter—I would miss her very much. No, it is not by preternatural planning ability that I compete with only the gods in Tower of Hanoi tournaments-in-the-sky. In fact, the first time I tried it, my score was not particularly good. I am not going say what it was but the manual said that I ranked somewhere between the average Darwin Award winner and the person who invented English spelling rules. After giving the test some thought, however, I realized that each movement of the discs is mechanically determined by a simple rule. I will not say what the rule is for fear of compromising the validity of the test for more people. The rule is not so simple that you would figure it out while taking the test for the first time, but it is simple enough that once you learn it, you will be surprised how easy the test becomes.

All kidding aside, it is important for the clinician to be mindful of the process by which a child performs well or poorly on a test. For me, the Tower of Hanoi does not measure planning. For others, it might. Edith Kaplan (1988) was extremely creative in her methods of investigating how people performed on cognitive tests. Kaplan-inspired tools such as the WISC-IV Integrated provide more formal methods of assessing strategy use. However, careful observations and even simply asking children how they approached a task (after the tests have been administered according to standard procedures) is often enlightening and can save time during the follow-up testing phase. For example, I once read about an otherwise low-performing boy who scored very well on the WISC-IV Block Design subtest. When asked how he did so well on it, he said that he had the test at home and that he practiced it often. The clinician doubted this very much but his story turned out to be true! His mother was an employee at a university and saw someone from the Psychology Department throwing outdated WISC-III test kits into the garbage. She was intrigued and took one home for her children to play with.

I once gave the WAIS-III to a woman who responded to the WAIS-III Vocabulary subtest as if it were a free association test. I tried to use standard procedures to encourage her to give definitions to words but the standard prompts (“Tell me more”) just made it worse. Finally, I broke with protocol and said, “These are fabulous answers and I like your creativity. However, I think I did not explain myself very well. If you were to look up this word in the dictionary, what might it say about what the word means?” In the report I noted the break with protocol but I believe that the score she earned was much more reflective of her Lexical Knowledge than would have been the case had I followed procedures more strictly. I do not wish to be misunderstood, however; I never deviate from standard procedures except when I must. Even then, I conduct additional follow-up testing to make sure that the scores are correct.

This post is an excerpt from:

Schneider, W. J. (2013). Principles of assessment of aptitude and achievement. In D. Saklofske, C. Reynolds, & V. Schwean (Eds.), Oxford handbook of psychological assessment of children and adolescents (pp. 286–330). New York: Oxford University Press.

Standard
CHC Theory, Cognitive Assessment, Principles of assessment of aptitude and achievement

Needing Glasses for the Ears: Explaining Phonological Dyslexia to Parents

ME: Tell me about why you decided to be evaluated.

ADULT CLIENT: I think I might have HDHD and dahlexia.

ME (unsure): You think you might have ADHD and dyslexia?

ADULT CLIENT (embarrassed): Right, ADHD…but I think I have dahlexia, too.

Auditory processing problems are among the hardest problems to explain. If you were to give parents a simple, technically correct definition of auditory processing (e.g., the ability to perceive patterns in sound), you are likely to be misunderstood. Parents know that “auditory” has something to do with hearing and are likely to think that their child has a hearing problem or has difficulty understanding speech. Auditory processing is the ability that I spend the most time explaining so that I do not cause confusion. If a child has auditory processing problems, I say something like this to the parents:

Auditory processing is not the ability to hear. Your daughter can hear just fine. The problem that she has is difficult to explain so I am going to start by comparing her problem to vision problems (She does not have a vision problem, either. I am just comparing the two problems.). Near-sighted people are not blind. Up close, they see well. Things that are far away, however, are blurry. Making lights brighter does not help; near-sighted people need glasses. In the same way that someone who is near-sighted is not blind, your daughter is not deaf; she is not even hard of hearing. However, for her, speech sounds are a little blurry for her. It is as if she needs glasses for her ears, to make sounds clearer. Unfortunately, no such thing exists. A hearing aid would not help because it is not the volume of the sound that is the problem.

The problem is that the sounds in words are hard for her to distinguish. I’ll explain what I mean. Words are made of different sounds blended together. We usually think of the word cat as one big blob of sound – /cat/. However, whenever we need to, we can break cat into three separate sounds – /c/ /a/ /t/. With a word like cat, this is easy to do and even your daughter does not have much trouble with it. However, when she hears a long word or a word with a lot of consonants bunched together, it is hard for her to break the word into individual sounds. For example, the word strength has only one syllable but it has six sounds – /s/ /t/ /r/ /e/ /ŋ/ /θ/. With cat there are three letters, one for each sound. With strength it gets complicated because the n and the g form a single sound /ŋ/ and the t and the h make the sound /θ/. If I pronounce both the n and the g separately – /stren/ /g/ /θ/ – it sounds strange. With a word like strength, your daughter can hear the first sound and the last sound but gets lost in the middle and starts leaving sounds out or guessing wrong sounds.

Now, if you say the word strength out loud, she can hear it and she understands it. She is not confused. She can even pronounce the word correctly. Why? I’ll make another comparison to vision. When you are driving and you see a road sign from far away, you might not be able to see every letter on the sign distinctly. However, you might be able to make out the shape of the word and because you know what different signs are likely to say, you can tell someone what the word on the sign is. This is sort of like what you daughter can do. She hears the word and can say what it is based on the overall features of the word. However, she has difficulty hearing each of the sounds as distinct from each other.

If the only thing that were wrong was that she could not split words into different sounds, there would be no cause for concern. However, it turns out that this ability to hear the sounds in words as distinct rather than as big blobs of sound is really important to learn to read. If you can hear the different sounds in words, you can hear why the words are spelled as they are (if they are words with regular spelling). If you are reading and you come across a new word, you can sound it out like they do on Sesame Street.

Children who have difficulty hearing speech sounds distinctly, often have trouble learning to spell and to read. Most children learn their letters and the sounds they make and then can figure out how to read and spell most words (at least the ones with regular spelling). Without the ability to sound out a word, learning to read and spell depends mostly on memorizing each word one-by-one. New words have to be taught explicitly to the child. Some children with this problem figure out how to work around it; some have help. When children with this problem fall behind in their ability to read, we call the problem dyslexia.

I want to be clear what dyslexia is and what it is not. You may have heard that dyslexia is when children see words backwards. This is not true. I have been doing this for a long time and I have seen many children with dyslexia. Not one of those children saw anything backwards. However, many of them jumble their letters and, like younger children, sometimes they write their letters backwards. This is not due to seeing things backwards. Instead, these mistakes are due to ordinary memory errors. If you give me a long list of groceries to buy, I might remember most of the items on the list but I might not remember them in the right order. When children with dyslexia learn a new word, they might remember which letters were in the word but might forget their order. They will probably remember the first and last letters but might mix up the middle letters. If they could sound out the words, they would be able to see that the order was wrong but that does not happen as often as it otherwise would.

The reason that young children often write letters backward is that letters are very unusual. Most things have the same name no matter what angle we view them from [I demonstrate with a pen, rotating it and turning it]. This pen is called a pen no matter what I do with it. Letters are not like that. The letter b changes its name, depending on how it is rotated or flipped. It can be a d, a p, or even a q. A backwards j does not even have a name. This is weird for children and it takes a while for them to get the hang of it. Children with dyslexia have a bit of a problem remembering which sounds go with which letters and thus continue making these sorts of errors longer than do most children. The problem is blurry sounds, not backwards vision.

Of course, I would probably not say all this in one long speech but try to make the discussion more interactive.

This post is an excerpt from:

Schneider, W. J. (2013). Principles of assessment of aptitude and achievement. In D. Saklofske, C. Reynolds, & V. Schwean (Eds.), Oxford handbook of psychological assessment of children and adolescents (pp. 286–330). New York: Oxford.

Standard
CHC Theory, Cognitive Assessment, Principles of assessment of aptitude and achievement

Explaining the difference between vision and visual-spatial processing to parents.

Vision is the ability to see something and visual-spatial processing helps you make sense of what you see.

Vision is the ability to see what is there. Visual-spatial processing is the ability to see what is not there, too, in a sense. With good vision you can see what something looks like; with good visual-spatial processing you can imagine what something would look like if you turned it around or if you were standing somewhere else or if something else was covering part of it.

With good vision you can see objects; with good visual-spatial processing you can see how they might fit together.

With good vision you can see a bunch of lines and splotches of colors; with good visual-spatial processing you can see how those lines and splotches of color form meaningful patterns.

This is the ability that sculptors, painters, designers, engineers, and architects need. It comes in pretty handy for the rest of us too.

This post is an excerpt from:

Schneider, W. J. (2013). Principles of assessment of aptitude and achievement. In D. Saklofske, C. Reynolds, & V. Schwean (Eds.), Oxford handbook of psychological assessment of children and adolescents (pp. 286–330). New York: Oxford.

Standard
CHC Theory, Cognitive Assessment, Principles of assessment of aptitude and achievement

IQ Tests: Is Knowledge of Useless Knowledge Useless?

Now I wish I could write you a melody so plain

That could hold you dear lady from going insane

That could ease you and cool you and cease the pain

Of your useless and pointless knowledge

– Bob Dylan “Tombstone Blues”

There is much pleasure to be gained from useless knowledge.

– Bertrand Russell

When critics look though the items in a general verbal information test, they, with some justification, sometimes sneer at the usefulness of the content. Is there any money in being able to list off the names of the planets? Can I oppose injustice, armed with my knowledge of state capitals? Will any babies be saved because I know who Julian the Apostate was? Probably not.

Many (most?) facts I have learned are unlikely to ever be of practical use. If I knew which ones they were, I might happily surrender them to forgetfulness. However, because it is impossible to know what might be useful in the future, I will hang onto my useless and pointless knowledge for a little while longer, thank you very much.

When Francis Bacon wrote parenthetically that “knowledge itself is a power…” in the context of an argument attempting to discredit the theological beliefs of certain religious sects, he probably did not mean the phrase in the sense that it is invoked today (i.e., that knowledge confers power). However, the phrase “knowledge is power” has survived because it resonates with our experience and pithily expresses something that is increasingly true in an age that gives increasing returns to those who can profit from information.

Good items in a test of General Information should not be about random facts. Easy items should not be merely easy (e.g., “What is the color of the sky?”). Rather, they should test for knowledge of information considered essential for living independently in our culture. A person who does not understand why dishes should be washed is not ready to live unsupervised. More difficult items should not be merely difficult (e.g., “What is the largest city in Svalbard? How many teeth does an orca whale have?”). Rather, they should measure knowledge that is relevant to what is considered core aspects of our culture (e.g., “Why do banks loan people money? Why do people still learn Latin and ancient Greek? Who was Isaac Newton? What is the purpose of the United Nations?”).

Just as language development consists of many narrow abilities, there are many sub-categories in General Information. Typically these sub-categories consist of academic domains such as knowledge of the humanities and knowledge of the sciences. These categories have further subdivisions (e.g., physics, chemistry, biology, and so forth – and each of these, in turn have further subdivisions).

General Information consists of knowledge that each person in a culture is expected to be familiar with (or would be admired if he or she knew). However, much (if not most) of a person’s knowledge is not of this sort. For example, although it is expected that everyone in this culture should know what airplanes are, only pilots are expected to know how to fly them. In CHC Theory, knowledge that is expected to be known only to members of a particular profession or enthusiasts of a particular hobby, sport, or other activity is classified as Domain-Specific Knowledge (Gkn). Most subject-specific academic achievement tests (e.g., European History, Geology, Contemporary American Literature) would be considered measures of Gkn, not Gc. That is, typically (but not always) achievement measures are the relevant outcomes we wish to explain, not explanatory aptitudes. In contrast, measures of General Information (e.g., WISC-IV Information) are intended to be estimates of the body of knowledge from which a person can draw to solve a wide range of problems.

Like Lexical Knowledge, General Information has a bi-directional relationship with reading comprehension. Very little of what is written is fully self-contained; authors presume that readers have considerable background knowledge and often do not bother to explain themselves. Drout (2006) describes how difficult and amusing it is to explain to non-native speakers of English what newspaper headlines such as “Under Fire from G.O.P., White House Hunkers Down” mean.[1] Children who know more understand more of what they read. Understanding more makes reading more enjoyable. Reading more exposes children to more knowledge, much of which is inaccessible via oral culture.


[1]Why would anyone be under a fire?

It means being shot at.

People are shooting at the White House?

No, it is just a vivid way of saying that the G.O.P. is criticizing the administration, which is symbolized by the White House, where the president lives.

Who is the G.O.P.?

It stands for the Grand Old Party.

If they are old, why haven’t I heard of them?

They are the Republicans.

Oh! Is the Democratic Party the new party?

No, they have been around longer than the Republicans. The nickname GOP was first used when the party was only a few decades old.

I don’t understand.

I don’t either, really. I just know that “old” is an affectionate way of describing something you have liked for a long time.

Could I say that I enjoy old ice cream?

No, that doesn’t sound right. You should probably just avoid using “old” that way.

What does “hunker”mean?

I really have no idea. I just know that when you are under fire, you should hunker down.

If you don’t know what it means, how do you know what to do?

True! I just looked it up. It means “to squat, to sit on your haunches.” I guess it all makes sense now.

So, when the Republicans criticize the president, he sits on his haunches?

That is an amusing image, but no. It means that you stick to your guns…er…I mean…

This post is an excerpt from:

Schneider, W. J. (2013). Principles of assessment of aptitude and achievement. In D. Saklofske, C. Reynolds, & V. Schwean (Eds.), Oxford handbook of psychological assessment of children and adolescents (pp. 286–330). New York: Oxford.

Standard
CHC Theory, Cognitive Assessment, Principles of assessment of aptitude and achievement

Why do IQ tests measure vocabulary?

If Lexical Knowledge (understanding of words and their uses) is simply memorizing the definitions of fancy words, then, at best, it is a trivial ability valued by academics, pedants, and fuddy-duddies. At worst, its elevation by elitists is a tool of oppression. There is some truth to these views of Lexical Knowledge but they are myopic. I will argue that vocabulary tests are rightfully at the center of most assessments of language and crystallized intelligence. Some words have the power to open up new vistas of human experience. For example, when I was thirteen, learning the word “ambivalence” clarified many aspects of interpersonal relationships that were previously baffling.

A word is an abstraction. The need for labels of simple categories is perfectly clear. Knowing the word anger (or its equivalent in any other language) frees us from having to treat each encounter with the emotion as a unique experience. Being able to communicate with others about this abstract category of experience facilitates self-awareness and the understanding of interpersonal relations. We can build up a knowledge base of the sorts of things that typically make people angry and the kinds of reactions to expect from angry people.

It is less obvious why anger has so many synonyms and near-synonyms, some of which are a bit obscure (e.g., iracund, furibund, and zowerswopped!). Would it not be easier to communicate if there were just one word for every concept? It is worthwhile to consider the question of why words are invented. At some point in the history of a language, a person thought that it would be important to distinguish one category of experience from others and that this distinction merited a single word. Although most neologisms are outlived even by their inventors, a few of them are so useful that they catch on and are used by enough people for enough time that they are considered “official words” and are then taken for granted as if they had always existed.[1] That is, people do not adopt new words with the primary goal of impressing one another. They do it because the word succinctly captures an idea or a distinction that would otherwise be difficult or tiresome to describe indirectly. Rather than saying, “Because Shelly became suddenly angry, her sympathetic nervous system directed her blood away from her extremities toward her large muscles. One highly visible consequence of this redirection of blood flow was that her face turned white for a moment and then became discolored with splotches of red.” It is simply more economical to say that “Shelly was livid with rage.” By convention, the use of the word livid signals that Shelly is probably not thinking too clearly at the moment and that the next thing that Shelly says or does is probably going to be impulsive and possibly hurtful.

Using near synonyms interchangeably is not merely offensive to word nerds and the grammar police. It reflects, and possibly leads to, an impoverishment of thought and a less nuanced understanding of the world. For example, jealousy is often used as a substitute for envy. They are clearly related words but they are not at all the same. In fact, in a sense, they tend to be experienced by people on opposite sides of a conflicted relationship. Envy is the painful, angry awareness that someone else enjoys some (probably undeserved) advantage that we covet. Jealousy is the angry, often vigilant, suspicion we may lose our beloved to a rival. Unaware of this distinction, it would be difficult to benefit from or even make sense of the wisdom of Rochefoucauld’s observation that “Jealousy is born with love, but does not die with it.”

Lexical Knowledge is obviously important for reading decoding. If you are familiar with a word, it is easier to decode. It is also obviously important for reading comprehension. If you know what a word means, it is easier to comprehend the sentences in which it appears. It is probably the case that reading comprehension also influences Lexical Knowledge. Children who comprehend what they read are more likely to enjoy reading and thus read more. Children who read more expose themselves to words that rarely occur in casual speech but the meaning of which can be inferred from how it is used in the text. Finally, Lexical Knowledge is important for writing. Children with a rich understanding of the distinctions between words will not only be able to express what they mean more precisely, but their knowledge of certain words will enable them to express thoughts that they might not otherwise have had. For example, it seems to me unlikely that a student unfamiliar with the word “paradox” would be able to write an essay about two ideas that appear to be contradictory at first glance but at a deeper level are consistent with each other.


[1] Of course, dictionaries abound with antique words that were useful for a time but now languish in obscurity. For example, in our more egalitarian age, calling someone a cur (an inferior dog because it is of mixed breed) is not the insult that it once was. It is now used mostly for comedic effect when someone affects an aristocratic air. My favorite example of a possibly soon-to-be antique word is decadent, which is nowadays almost exclusively associated with chocolate.

This post is an excerpt from:

Schneider, W. J. (2013). Principles of assessment of aptitude and achievement. In D. Saklofske, C. Reynolds, & V. Schwean (Eds.), Oxford handbook of psychological assessment of children and adolescents (pp. 286–330). New York: Oxford.

Standard
CHC Theory, Cognitive Assessment, Principles of assessment of aptitude and achievement

Fluid Intelligence, Defined

Mentioning fluid intelligence at cocktail parties as if it were a perfectly ordinary topic of conversation carries with it a certain kind of cachet that is hard to describe unless you have experienced it for yourself. Part of Gf’s mystique can be attributed to Cattell’s (1987) assertions that Gf is linked to rather grand concepts such as innate ability, genetic potential, biological intelligence, mass action, and the overall integrity of the whole brain.[1] Heady stuff indeed!

At the measurement level, Gf tests require reasoning with abstract symbols such as figures and numbers.[2] Good measures of Gf are novel problems that require mental effort and controlled attention to solve. If a child can solve the problem without much thought, the child is probably making use of prior experience. Thus, even though a test is considered a measure of fluid intelligence, it does not measure fluid intelligence to the same degree for all children. Some children have been exposed to matrix tasks and number series in school or in games. Fluid intelligence is about novel problem solving and, as Kaufman (1994, p. 31) noted, wryly pointing out the obvious, a test is only novel once. The second time a child takes the same fluid intelligence test, performance typically improves (by about 5 points or 1/3 standard deviations, Kaufman & Lichtenburger, 2006). This is why reports that fluid intelligence can be improved with training (Jaeggi, Buschkuehl, Jonides, & Perrig, 2008) cannot be taken at face value.[3] Just because performance has improved on “Gf tests” because of training does not mean that Gf is the ability that has improved.

At the core of Gf is the narrow ability of Induction. Inductive reasoning is the ability to figure out an abstract rule from a limited set of data. In a sense, inductive reasoning represents a person’s capacity to acquire new knowledge without explicit instruction. Inductive reasoning allows a person to profit from experience. That is, information and experiences are abstracted so that they can be generalized to similar situations. Deductive reasoning is the ability to apply a rule in a logically valid manner to generate a novel solution. In CHC Theory, deductive reasoning is called General Sequential Reasoning. Although logicians have exquisitely nuanced vocabularies for talking about the various sub-categories of inductive and deductive reasoning, it will suffice to say that everyday problem solving typically requires a complex mix of the two.

Inductive and deductive reasoning can be found in multiple places in CHC Theory. Whenever inductive and deductive reasoning are applied to quantitative content, they are called quantitative reasoning. For mysterious reasons, inductive and deductive reasoning with quantitative stimuli tend to cluster together in factor analyses. Inductive and deductive reasoning also make an appearance in Gc. Whenever inductive and deductive reasoning tasks rely primarily on past experience and previous knowledge, they are classified as measures of crystallized intelligence. Many researchers have supposed that the Similarities subtest on Wechsler tests contains an element of fluid reasoning because inductive reasoning is used to figure out how two things or concepts are alike. If the question is something like, “How are a dog and a cat alike?” then it is very unlikely that a child arrives at the correct answer by reasoning things out for the first time. Instead, the child makes an association immediately based on prior knowledge.

Researchers are not satisfied with accepting Gf as a given. They wish to know the origins of Gf and to understand why some people are so much more adept at abstract reasoning than other people are (Conway, Cowan, Bunting, Therriault, & Minkoff, 2002). One hypothesis that is still being explored is that fluid reasoning has a special relationship with working memory. Working memory is the ability to hold information in mind while using controlled attention to transform it in some way (e.g., rearranging the order of things or applying a computational algorithm). Many researchers have noted that tests of fluid reasoning, particularly matrix tasks (e.g., WISC-IV Matrix Reasoning), can be made more difficult by increasing the working memory load required to solve the problem. Kyllonen and Christal (1990) published the provocative finding that individual differences in Gf could be explained entirely by individual differences in working memory. Many studies have attempted to replicate these finding but have failed. Most studies find that Gf and working memory are strongly correlated (about 0.6) but are far from identical (Kane, Hambrick, Tuholski, Wilhelm, Payne, & Engle, 2004).

Just as we have distinguished between statistical g and theoretical g, it is important to note that there is a difference between the Gf that is measured by Gf tests and the Gf that is written about by theorists. Some of Cattell’s hypotheses about Gf have stood the test of time, whereas others have not held up very well. For example, the heritability of Gf is not higher than that of Gc, as Cattell’s theory predicts. I mention this because it is probably not justified to claim that because a child scores well on Gf tests, the child has high innate talent or that the child’s biological intelligence is high.

Most of the effects of Gf on academic achievement are mediated by Gc (i.e., better reasoning leads to more knowledge which leads to higher achievement). However, Gf seems to have a special relationship with complex problem solving in mathematics. Because Gf tests measure abstract reasoning, it is unsurprising that they would predict performance in an abstract domain such as mathematics (Floyd, Evans, & McGrew, 2003).


[1] Horn (1985) tended to de-emphasize the biological/genetic interpretation of fluid intelligence.

[2] Test developers have tried to create Gf measures with verbal content (e.g., WJ-R Verbal Analogies or SB5 Verbal Fluid Reasoning) but find that verbal Gf tests do not always load on the same factor as traditional Gf tests (Canivez, 2008; Woodcock, 1990). It is possible that the KAIT Logical Steps subtest may be the only commercially available verbal Gf test that does not have substantial loadings on Gc (Flanagan & McGrew, 1998; Immekus & Miller, 2010), possibly because it does not use the verbal analogy format.

[3] See Moody (2009) for a discussion of other methodological problems that may have compromised the validity of the Jaeggi et al (2008) study.

This post is an excerpt from:

Schneider, W. J. (2013). Principles of assessment of aptitude and achievement. In D. Saklofske, C. Reynolds, & V. Schwean (Eds.), Oxford handbook of psychological assessment of children and adolescents (pp. 286–330). New York: Oxford.

Standard