Wednesday, September 17, 2014

The Problem with Using Personality Assessments for Hiring: As featured in Harvard Business Review

This article was published by Harvard Business Review on 8/27/14 and can be found at:

A decade ago, researchers discovered something that should have opened eyes and raised red flags in the business world.

Sara Rynes, Amy Colbert, and Kenneth Brown conducted a study in 2002 to determine whether the beliefs of HR professionals were consistent with established research findings on the effectiveness of various HR practices. They surveyed 1,000 Society for Human Resource Management (SHRM) members — HR Managers, Directors, and VPs — with an average of 14 years’ experience.

The results? The area of greatest disconnect was in staffing— one of the lynchpins of HR. This was particularly prevalent in the area of hiring assessments, where more than 50% of respondents were unfamiliar with prevailing research findings.

What HR Managers Get Wrong chart

Several studies since have explored why these research findings have seemingly failed to transfer to HR practitioners. Among the causes are the fact that HR professionals often don’t have time to read the latest research; the research itself is often present with technically complex language and data; and that the prospect of introducing an entirely new screening measure is daunting from multiple angles.

At the same time, anyone who has ever been responsible for hiring, much less managing, employees knows that there is a wide variation in worker performance levels across jobs. Therefore, it is critical for organizations to understand what differences among individuals systematically affect job performance so that the candidates with the greatest probability of success can be hired.

So what are the most effective screening measures?
Extensive research has been done on the ability of various hiring methods and measures to actually predict job performance. A seminal work in this area is Frank Schmidt’s meta-analysis of a century’s worth of workplace productivity data, first published in 1998 and recently updated. The table below shows the predictive validity of some commonly used selection practices, sorted from most effective to least effective, according to his latest analysis that was shared at the Personnel Testing Counsel Metropolitan Washington chapter meeting this past November:

Most effective screening measures chart

So if your hiring process relies primarily on interviews, reference checks, and personality tests, you are choosing to use a process that is significantly less effective than it could be if more effective measures were incorporated.

And yet that’s how many companies operate. According to a 2011 NBC News article, the use of personality assessments are on the rise, growing as much as 20% annually. Especially problematic is the widespread use of Four Quadrant (4-Q) personality tests for hiring, something I see regularly in my consulting work.

A 4-Q assessment is one where the results classify you as some combination of four different options labeled as letters, numbers, colors, animals, etc. They originated around 450 BC when Empedocles noticed that he could group people’s behavior into four categories which he labeled earth, water, fire, and air. Hippocrates made the same observation, but (coming from a medical background) labeled the categories blood, phlegm, black bile, and yellow bile. Since then, hundreds of iterations of these tools have been developed, all essentially based on the same premise and theory.

Generally speaking, 4-Q tools consist of a list of adjectives from which respondents select words that are most/least like them, and are designed to measure “style,” or tendencies and preferences. While they can seem highly insightful — not to mention being widely available and inexpensive — they have some severe shortcomings when used in high stakes applications such as hiring.

For one, they tend to be highly transparent, enabling a test taker to manipulate the results in a way that they feel will be viewed favorably by the administrator. Also, since they are designed to measure “states” (as opposed to more stable “traits”), there is a significant chance that the results will change over time as the individual’s context changes (most publishers of 4-Q tests recommend that individuals re-take them at fairly frequent intervals for this reason).

This begs the question: How can an individual’s assessment results be used to predict future job performance if there is a reasonable chance that their scores will change over time?

When using any assessment, managers need to step back and ask themselves one basic question before giving it to a potential employee: Is this test predictive of future job performance? In the case of 4-Qs, probably not. They can provide tremendous value for self-discovery, team building, coaching, enhancing communication, and numerous other developmental applications. But due to limited predictive validity, low test-retest reliability, lack of norming and an internal consistency (lie detector) measure, etc., they are not ideal for use in hiring.

The strongest personality assessments to use in a hiring context are ones that possess these attributes:
  • Measure stable traits that will not tend to change once the candidate has been on the job for some length of time.
  • Are normative in nature, which allows you to compare one candidate’s scores against another’s to determine which individual possess more (or less) of a particular trait.
  • Have a “candidness” (or “distortion” or “lie detector”) scale so you understand how likely it is that the results accurately portray the test-taker.
  • Have high reliability (including test-retest reliability) and have been shown to be valid predictors of job performance.
Even when using a tool that meets the criteria outlined above, personality constructs are not the most predictive measure available. Personality tests are most effective when combined with other measures with higher predictive validity, such as integrity or cognitive ability.

Using well-validated, highly predictive assessment tools can give business owners and managers a significant leg up when trying to select candidates who will become top producers for the organization. However, all assessment approaches are not created equal. And some will not offer a significant return on your investment. Accordingly to a 2014 Aberdeen study [registration required], only 14% of organizations have data to prove the positive business impact of their assessment strategy. Knowing which types of assessments will be most effective in accomplishing the specific objectives you have identified for your organization will enable you to select a tool with a measurable impact on the bottom line.

A Re-cap of the National SHRM Conference Through the Eyes of a First Time Attendee

This article was published in the Fall/Winter 2014 Edition of Kentucky SHRM Magazine

It’s 4:30am on the Tuesday of the SHRM Conference in Orlando. And the power has gone out. I know this because I’ve just been awoken by the most annoying of sounds. You know that BEEP that the smoke detector makes when it’s disconnected from power? BEEEEEP. I’m stumbling around my hotel room looking for my Ipod and headphones so I can drown out the BEEP.  Then it sinks in. No power means… what? No wake-up call? No hairdryer? And—so God help me—the Starbucks better have their own back-up generator! Since I’ve been robbed of sleep, I’m going to need caffeine before I present my concurrent session at 7:00am.

Let’s back up. This is my first time at the national SHRM conference. Walking into a room that holds 14,000 people is a powerful moment that I won’t soon forget. Slightly overshadowing that, though, was the sense of awe experienced when walking into a vendor hall that housed over 600 booths, some of them as large as a small city and as elaborate as a nearby Disney theme park attraction. Considering how many mini-Ipad raffles I entered, I’m bound to win one of them, right?

The general sessions were all very engaging and motivating.  On Sunday, attendees were treated to an inspiring and very vulnerable talk from Good Morning America host Robin Roberts. She encouraged people to “make your mess your message,” meaning that everyone is dealing with some challenge in their personal life and we can support and encourage others by being transparent and open with our struggles. Monday, the general session was presented by Tom Friedman, author of one of my favorite books, The World is Flat. He warned that “average is officially over,” and only those who approach their career with creativity and an entrepreneurial spirit will thrive in the new reality. On Tuesday, fellow Kentuckian and YUM! Brands CEO, David Novak coached us on how to recognize and reward employees so as to “take them with you.” Closing out the conference on Wednesday, former first lady Laura Bush shared about her life experiences and updated attendees on the current activities of her various famous family members.

With hundreds of concurrent sessions to choose from, attendees had no shortage of options and could truly tailor their learning experience. When asked what his favorite session was, Kentucky SHRM’s State Conference Director, Perry Sholes from Lexington, said Identity Loyalty—Unlocking the Keys to Creating Productive, Hardworking and Appreciative Employees. Taryn Pearson, HR Manager at PetFirst Healthcare in Southern Indiana enjoyed attending the session called HR Department of One. “It’s great to know that other HR Managers do this on their own and are successful,” she said. “The speaker also provided a lot of valuable resources for people like me who are an HR department of one.”

Of particular interest to many attendees this year was the new certification program being rolled out by SHRM as a replacement for (or alternative to?) the PHR and SPHR certifications traditionally provided through HRCI. Several sessions addressed this topic and John Bachman, LSHRM past president, worked to make sure he was hearing both sides of the story. John attended both SHRM sponsored and HRCI sponsored events on this issue. “It is hard to tell which story is THE story,” John says. “I feel like a child in a divorce; we were “raised” to love and support both, but now they are split. I’m not taking sides; I plan to be active in both and I wish them both great future success.”       

On Monday, approximately 30 conference attendees from Kentucky gathered for lunch.  After 24 hours of feeling disconnected and alone among a sea of strangers, it was very refreshing to be among friends. Throughout the rest of the conference, I met several more Kentuckians that hadn’t heard about the Kentucky lunch. Just a tip—if you plan to attend the 2015 SHRM Conference in Vegas, make sure to reach out and find out when/where the Kentucky gathering will take place. This is an invaluable opportunity to connect with local colleagues and truly maximize your conference experience.

That brings us back to Tuesday morning. When I was notified in late October that I’d been selected to present a concurrent session on Hiring Assessments, I was humbled, honored, and thrilled. When I realized that my session was to take place at 7:00am… I was baffled. Surely no more than 4 people would show up at 7:00am, right? Instead, I was grateful to speak to a gracious and enthusiastic crowd of about 400. And that’s nothing—Louisvillian Erika Tedesco from Hosparus presented at 7:00am Monday morning regarding Onboarding and Orientation Programs and had over 550 in her session.  Rounding out our Kentucky presenter team was Sandy Allgeier, a 10 year veteran of the SHRM faculty. Sandy presented a preconference workshop on being a HR Business Partner, a session she’s facilitated for the past 3 years. The Bluegrass State was well represented in the Orange State!

In reflecting on my first experience at the national SHRM conference, I truly found it thrilling (albeit, at times, overwhelming) to be a part of something so large and energetic. Right now, all I can think about is how I can make sure to be part of next year’s conference… Vegas, baby! Of all the renowned conference speakers, it was perhaps Tim McGraw (who performed for attendees Tuesday night) who summed up the conference experience up best—“I like it! I love it! I want some more of it!”