Psychometrics

Reliability is where the light is. Validity is where the keys are.

There is a tired old joke about the drunk who lost his keys on the dark side of the street but is looking for them under the lamppost because “That’s where the light is.”

Reliability is where the light is. Validity is where the keys are.

Reliability is relatively easy to estimate compared to validity. Researchers and test developers make a very big deal out of high reliability coefficients because “A test cannot be valid if it is not reliable.” However, the fact that a measure is highly reliable is irrelevant if it does not allow us to make accurate inferences about the thing we wish to measure. Furthermore, if a measure is shown to have validity, its reliability is already implied.

To switch metaphors, reliability is thin gruel if validity is on the table. I think that with good models such as those offered by Weiss and colleagues (2013), validity is at least on the menu, if not already laid out for the feast. Reliability is at best an appetizer. It is nice to have, but if the main course is ample, you can skip it without worries.

This post is an excerpt from:

Schneider, W. J. (2013). What if we took our models seriously? Estimating latent scores in individuals. Journal of Psychoeducational Assessment, 31, 186–201.

Advertisements
Standard

2 thoughts on “Reliability is where the light is. Validity is where the keys are.

  1. Brant says:

    I don’t have access to the complete article, so I apologize if the answer is obvious: Why is .90 the established level of reliability for psychologists/diagnosticians for making individual decisions? Why is .86 or .84 not sufficient?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s