Thinking Critically About Hiring Assessments: 8 Questions to Ask Yourself

Assessments can provide tremendous value in the hiring process if used correctly. To determine whether or not you're getting the maximum value from assessments, ask yourself the following questions. 

Note:  The items with asterisks are taken directly from the Department of Labor’s Guide to Testing and Assessment (page numbers are cited)

 

*1)   Are we using assessments in a purposeful manner?

 
The DOL states: “Often, inappropriate use results from not having a clear understanding of what you want to measure and why you want to measure it. As an employer, you must 

first be clear about what you want to accomplish with your assessment program in order to select the proper tools to achieve those goals” (page 9-1).

*2)   Are we using a “whole person approach”? Or are we making decision on only one data point, or one dimension of a candidate? (DOL page 9-1)

*3)   Is the population we’re assessing with the instrument (i.e. our job candidates) similar to the group on which the instrument was developed or normed?  (DOL page 9-2)

*4)   Have the tools we’re using been demonstrated to be valid for the specific purpose(s) for which they are being used in our organization? (DOL page 9-3) In other words, can we show that scores on our assessment(s) are directly correlated with some outcome of interest in our workplace (i.e. turnover, job performance, sales volume, customer satisfaction, absenteeism, etc.)?

*5)     Do the tools we’re using measure stable traits and have high test-retest reliability?

The DOL states: “If a person takes the same test again, will he or she get a similar score, or a very different score? A reliable instrument will provide accurate and consistent scores. To        meaningfully interpret test scores and make useful career or employment-related decisions, use only reliable tools” (page 9-2).

6)        Are we measuring constructs with high predictive validity?  I.e. Integrity, General Mental Ability, or a combination of several constructs?

7)        Are we using assessments in the correct point in our hiring process for maximum benefit?

8)        Are the tools we’re using “working,” and how do we know? Can we tie assessment results to metrics that are important to our company? Can we calculate ROI?

If the answer to any of the above questions is "no" or "I don't know," you may want to take a critical look at your hiring process.  Contact me with questions, or to request a copy of the Department of Labor's "Testing and Assessment: An Employers' Guide to Good Practices."

DiSC Does it Right!

If you’ve ever taken a test where the results described your personality in terms of four possible styles, you’ve taken a 4-quadrant (4Q) personality assessment. Perhaps you found out that you’re blue... or C... or 4...or a golden retriever... or a blueberry banana muffin (ok, I made up that last one).  But, what exactly are 4Q personality assessments designed to measure? And, what is the best use of this kind of tool?

The following is a summary of a report published by Inscape Publishing, the distributors of the DiSC Personal Profile System. This is an excellent explanation of what DiSC (and by extension, other 4-quadrant personality assessments) measures and how that information should, and should not, be interpreted and applied.

DiSC is a measure of “surface traits” or characteristic ways of behaving in a particular environment. It helps individuals recognize the environmental cues to which they are reacting and the strategies they are using to adjust to their environment. It is not designed to describe human characteristics that are not readily observed. It is designed to describe an individual in relation to his or her environment.

DiSC addresses behavioral responses based on the individual’s emotional reaction to a particular environment. This model is not designed to support inferences about what an individual is like at the core of his/her personality or to predict how he/she will behave in the future. Further, it does not attempt to determine how effective the person’s behaviors are. It is not appropriate to suppose one can match a person to an “ideal” environment and count on the match to remain intact.

The appropriate use of DiSC is to increase self-awareness of behavior in a particular situation so one can better understand where they may want to adjust their behavior to work more effectively with others or to better adapt to a situation. In the DiSC, respondents are considered the experts on themselves and the instrument is self-scored and self-interpreted.

Inscape says DiSC is used in business settings for employee development and team building. They do not recommend the use of this tool for the purpose of employee selection/hiring, because it is not designed to predict how effective someone will be in a job. If you are using a 4Q assessment for hiring, you may want to reconsider for two reasons: 1) there are some risks in using these tools for hiring from a compliance standpoint, and 2) there are other tools that are far more effective (have higher predictive validity) for use in a hiring context.

Look for the panel discussion “Personality Assessments Gone Wild” at the upcoming KY SHRM Conference!

 
Indented content taken directly from a research report published by Inscape Publishing titled “A Comparison of the Personal Profile System and the Myers-Briggs Type Indicator.” The full document can be found at: http://www.discprofile.com/reliability-and-validity.htm

Cybervetting

(Based on a presentation by Dr. Michael Zickar at the July 2013 IPAC Conference)

Defined as reviewing content found on Social Media or other online sources when screening perspective employees, this is an area where the practice has significantly outpaced the research.  Since it is fairly naïve to categorically tell hiring practitioners not to do it, some researchers are focusing their attention on providing some structure to the practice and determining what kinds of information, if any, can be shown to be valuable predictors of future job performance.

One proposed model of how to organize information found online is to classify it into one of four categories: Professional, Proactive (or “helping” behavior), Deviant, and Irrelevant. For example, previous employment information found on LinkedIn would be considered Professional, posts about fundraising activities the candidate is involved in would be categorized as Proactive, comments about illegal activity or substance abuse would be classified as Deviant, and pictures of kids, opinions about movies, etc. would be deemed Irrelevant (likely the largest category since very little information found online is directly related to job performance).

The cautions of using such information are many. For one, how do you assign meaning to information you glean? For example, if someone writes a lot of product reviews on Amazon, do you infer that the person is exceedingly helpful? Overly critical? Or merely verbose with a lot of time on their hands? Also, how do you account for having a wealth of online data on one candidate and virtually none for another? Can you ascribe meaning to this in-and-of itself? Does the lack of an online presence show prudence and restraint? Is it an indication that someone has something to hide? How do you reconcile that the amount of information you are able to uncover about someone may be due largely to something that could potentially be discriminatory, like belonging to a particular generation, or the uniqueness of someone’s name?

In conclusion, cybervetting is a highly subjective and unstructured process with no data (at this time) to support that the information gleaned has any direct relation to job performance.  Hiring practitioners would be wise to use caution, and to rely on screening measures that have proven predictive validity, like structured interviews, cognitive ability tests, and integrity tests, and other multi-measure assessments.

For Crying Out Loud!

It’s 3:43am and my 7 month old is wailing, as she’s done for the past 210 nights. I’ve read all the books and talked to doctors and well-meaning friends. I know, intellectually, that picking her up and giving her a bottle will be counterproductive to helping her learn to fall asleep on her own. But sometimes that long-term strategic thinking is difficult to maintain when I’m TIRED and all I want to do is STOP THE SCREAMING!!

It’s now 4:02 and I’m struck by the irony of the situation. I frequently have conversations with my HR colleagues who know they should take the time to create a more strategic, predictive hiring process. But the hiring manager is screaming for the position to be filled quickly. And even though this short-term thinking will result in having to re-fill the position in a few months, there is so much pressure and such insistence… sometimes it’s just easier to give in.

I suspect we will both reach a point where we say “no more!”  It doesn’t make sense to keep putting a band-aid on the situation when what we really need to do is address the problem with the precision of a surgeon—a full diagnostic work up, then the creation and execution of a plan the fixes the problem once and for all. We need to stop spinning our wheels and gain some traction on the problem so we can finally get some rest for crying out loud!

When you get to this point, I can help you analyze your hiring needs and find an instrument that will measure the right things to be effective in solving your problem… And I won’t even ask you to come over at 3am and help me solve mine!

Race Horses and Prospective Employees: Picking Winners

This is one of the best times of year to live in the Bluegrass! The historic “Run for the Roses”—The  Kentucky Derby—is upon us.  Every track attendee has a strategy for picking ponies. Some will rely on highly subjective criteria and gut feel. Others have a much more objective strategy based on sophisticated data and statistical odds. Either way, it’s a gamble.

To a human resources manager, this predicament sounds all too familiar. Shy of having a crystal ball, we can’t know with a certainty which prospective employees will be winners, which will fall in the middle of the pack, and which will throw their jockey, bump other riders, and be disqualified!  In choosing how to place our bets, we have various tools and strategies at our disposal. Which method(s) you use will determine how much success you have in systematically picking winners.

What’s in a name?

A favorite horse-picking strategy of novice gamblers: “I’m betting on #3 because it has a cool name!”  While we may not hire someone because we like their name, the thought process isn’t all that different. I may like a horse’s name because it alludes to my favorite vacation spot, rock star, or adult beverage. In other words, I like it because it resonates with me on a personal level. In an interview, we do the same thing—we look for common ground.  And, when we find it, we have a sense of connection with that person and make certain assumptions about them as a result. Doing this is human nature, but is not necessarily the most reliable way to choose a top performing candidate…or horse.

A Horse of a Different Color

Another beloved way to pick a horse: “I bet on the grays!”  Like it or not, many studies have shown that this happens in an interviewing context as well—we do make judgments based on appearance. However, we can fall victim to this hiring strategy in a much more subtle way as well.  Some personality assessments assign one of four colors (or letter, numbers, etc.) to describe someone’s personality. Deciding that you want a “blue” or a “red” for a given position can be a dangerous proposition. These types of assessments describe one’s preferences in terms of how they tend to go about completing a job; not whether or not they’ll be effective at it. The end result… grays don’t necessarily win any more often than horses of different colors. Assessment that measure constructs with a greater ability to differentiate between top and bottom performers can help you pick winners more often.

Watch and Learn

Favorite strategy #3: “I like that one! He’s feisty!”  Before a race, track-goers have the opportunity to observe the horses in the paddock as they’re being saddled and mounted. Some are very compliant. Others give spectators an exciting show! However, do the spunky ones who buck while being saddled and balk when being loaded into the gate win more often? Does their display of energy translate into an independent spirit that can’t be bridled? Or an excitable creature who doesn’t manage their pace well and who will be exhausted by the first turn?

Observing behavior is an important strategy in hiring as well. We observe during an interview. We ask behavioral-based questions. Sometimes we even run candidates through simulated work scenarios. The question is—are we able to make reliable inferences from our observations that accurately predict future behavior? And the answer is… sometimes. Much depends on the skill of the interviewers and, more importantly, the skill of the interviewees!

Look at the Track Record

And now we reach the strategy of the experienced gambler—look at the data. How many times has this horse won? On what kind of track (turf, mud, etc.)? What is the probability that the horse will win today?  Of course, we do this as human resources professionals as well. Whenever we check references, we’re asking “what is this employee’s track record?” (think about where that term came from!).  Have they been successful before, under similar conditions? Do they have what it takes to become a champion in our organization?

The Best Bet!

Certain types of assessment instruments can help you get additional, objective data to help you calculate the odds that you’re picking a winner. The types of instruments with the highest proven predictive power are intelligence tests, integrity tests, and behavioral tests. The more information you have, the better chance you have of selecting the right candidate, so a multi-measure tool that incorporates these as well as other constructs is ideal. The cost of picking the wrong horses in your $2 exacta box probably will not break your bank or your spirit. The cost of choosing poor (or even mediocre) employees, however, can be astronomical.

Enjoy your Derby, everyone! And, I hope you pick a winner!

About the Author: Whitney Martin is an Assessment Strategist with ProActive Consulting.  Contact her at whitney@consultproactive.com or 336-202-2385 to explore the most effective assessment solutions for your particular organizational objectives.