Tuesday, March 7, 2017

The Death of Guess: Using Data to Make Better Hiring Decisions

As featured on HotelExecutive.com: http://hotelexecutive.com/business_review/5103/using-data-to-make-better-hiring-decisions


As new properties explode on the scene and traveler choices abound, hotels know they have to pull out all the stops to make every guest experience a positive one. Are staff friendly are courteous? Are rooms clean? Are meals excellent? Are bills accurate? We rely on our employees to execute their jobs, not just correctly, but with enthusiasm. And, if they don't, business suffers. We do our best to hire good people (in a competitive market), we give them a little training, and then we HOPE they create raving fans.

Ever heard the expression "hope is not a strategy"? This phrase often pops into my mind when talking with HR practitioners about their hiring processes. Lurking beneath administratively burdensome screening systems that create an appearance of rigor, unstructured interviews, "gut feel," and other subjective criteria continue to weigh heavily in hiring decisions-we take our best guess and hope things will work out. However, while strong intuition and a good ability to "read" people are attributes that can prove beneficial in many contexts, they should not be the linchpin of employee selection decisions.

A profound change is occurring in the HR profession: business leaders are calling for HR to adopt a more evidence-based approach to decision-making. The frequency with which the terms metrics, analytics, and big data are creeping into HR circles is a testament to this. "We see CEOs and others wanting better data and not just a headcount report, but how is talent driving business results?" says Scott Pollak, a principal at PwC Saratoga in a Harvard Business Review report. With 57% of companies reporting their intention to have integrated, multi-source analytics in place in the next two years (according to another Harvard Business Review study), there is a push to incorporate more scientific, evidence-based practices in the people-functions in our businesses.

Based on some informal research, I've determined that exactly 98.64% of HR practitioners have, at best, a mild distaste for statistics. Therefore, there's an excellent chance that you may be experiencing some anxiety because of this demand for a more data-oriented approach to executing HR. Perhaps, too, this is why only 14% of businesses currently have data to show the business impact of their assessment strategy, according to an Aberdeen research study. With payroll and benefits representing one of the largest line items on virtually every company's operating statement, effective selection is one of the principal areas where HR can have a significant impact on the bottom line. But what kind of assessment instrument should you use in order to systematically select the best employees? The answer is a firm it depends. Follow the steps below to create a highly predictive, evidence-based, and quantifiably valuable selection system that works for your organization.

The 4 V's of Hiring

1. Vision. If companies had an unlimited budget (and candidates had endless time and patience) we could assess virtually anything-skills, knowledge, personality, values, attitudes, and the list goes on. But what are you really trying to accomplish? "We want to hire better people" is not nearly a clear enough goal. It is imperative to take the time, perhaps doing some research with internal stakeholders, to hone in on the ultimate goal for the organization.

Whether you're trying to impact staff retention, sales volume, early hire failure rate, guest satisfaction, customer referral rates, employee engagement, productivity, theft, absenteeism, safety incidents, or drug use in the workplace, there are different assessment instruments designed specifically to measure constructs that can directly impact these, and countless other, organizational issues or goals. Once your objective is clear, you can determine what you can measure that will help to predict that outcome specifically.

2. Validity. Take a moment and think about the best hiring decision you ever made. Jot down the steps in the process that moved that individual from a faceless applicant in the crowd to the super-star employee that exceeded all expectations. Now, think about the worst hiring decision you ever made-the "wolf in sheep's clothing" who turned disastrous. Did the hiring process that allowed the bad hire to get through the door differ from the one used when hiring the star? In all likelihood, the steps in both cases were nearly identical. After all, we develop our hiring processes with the goal of hiring the best employees every time-it's just that sometimes it works and sometimes it doesn't. Why? Simply because we're basing our hiring decisions on a combination of data points that lacks the ability to predict future job performance reliably.

Extensive research has been conducted on the predictive validity-the overall ability to predict job performance-of different hiring methods and measures, and some are just flat-out better predictors than others. Unfortunately, many practitioners continue to rely heavily on some of the least predictive measures, including interviews, reference checks, and four-quadrant personality assessments (not to mention "gut feel"). Well-developed assessments, including integrity tests, mental ability tests, and multi-measure tests (which incorporate a variety of constructs) can greatly increase your odds of making outstanding hiring decisions more often. The key is to find an instrument that measures the most appropriate combination of constructs to predict the well-defined outcomes you identified in the first (Vision) step.

3. Verification. Once you know what you're trying to accomplish (Vision) and the kinds of instruments that have a high degree of accuracy (Validity) in getting you there, you still need to choose an instrument from the hundreds (if not thousands) that are available in the marketplace. Considering the aforementioned fact that most HR people don't choose their profession due to their love of statistics, this part can be daunting. However, it's also necessary if you don't want to fall victim to the supreme sales skills of a vendor. If necessary, ask an Industrial-Organizational Psychologist from your local university to assist you in sifting through the technical validation documents provided by the vendor. Any tools used pre-hire must meet certain criteria as it relates to reliability, validity, adverse impact, and a number of other factors. Test publishers should be able to provide ample data showing how rigorous they were in developing their instruments.

4. Value. Now that you've developed a highly predictive selection process that is virtually guaranteed to move the needle on the metrics most essential to the business, it's time to quantify the impact that your expertise is having on the bottom line! You do this by demonstrating that the use of a particular tool is statistically correlated with what you're trying to predict. In other words, as test scores go up, turnover goes down, or as test scores go up, upsell conversion rates increases. Often, this can be achieved through either a concurrent or predictive validation study. Can you show a statistical correlation between how people score on the test you're using and how they perform on the job? If not, you may want to revisit the value you're getting from that tool.

It can't be stated strongly enough: your hiring process is destined for failure if the Vision step is not the bedrock for every decision subsequently made. As food for thought, let's contemplate an example we're all familiar with: pizza. Imagine your organization employs 20,000 pizza delivery drivers with an average annualized turnover of 170%. You know you can do a better job of "hiring better," but what specifically does that mean? Consider each of the following lines of thought:

A. As HR people, we understand the costs associated with turnover, so we begin to screen for factors such as reliability, work ethic, or other constructs that might enable us to hire longer-term employees. We'd measure our success by tracking reduction in turnover.

B. On second thought, since training for pizza delivery drivers is fairly minimal, perhaps turnover isn't actually that big of a deal. After all, all we really need are people who can zip to an address and hand someone a box, right? So let's screen for sense of urgency and sense of direction! We'd measure our success by tracking the rate of on-time deliveries.

C. But wait! Our drivers shouldn't zip too fast. After all, we need them to obey speed limits and demonstrate cautious driving practices. And, pizza delivery drivers handle a lot of cash, so we might also want to screen for integrity and safety. We'd measure our success by tracking cash discrepancies and driving incident rates.

D. And, gee, wouldn't it really differentiate our company (especially in the rather commoditized pizza market) if our drivers were also really friendly and engaging with customers? So, let's screen for friendly personalities and a passion for customer service. We'd measure our success by tracking customer satisfaction.
Each of the above is a completely valid and important strategy. Understanding which outcomes are going to have the greatest impact on the business will enable you to determine which combination of constructs is most important to measure.

The Death of Guess
Are you feeling pressure to incorporate more data-supported or evidence-based methods in your job? Is there anything you can do to increase the predictive validity of your hiring process? If pressed, could you tell your C-Suite exactly how accurate your selection system is and quantify the return on investment (ROI) for your efforts? The time has come for HR to embrace more data-driven, scientific methods for making key people decisions. Will you rise to the challenge? Well-placed and appropriately leveraged assessment instruments can be a relatively simple, cost-effective way for organizations of any size to infuse more objective data into people decisions-and to quantify the impact that the HR function makes on the metrics that matter most to the business.

Thursday, May 12, 2016

Putting Talent to the Test: Getting the Greatest Value from Assessments, Both Pre- and Post-Hire

As featured in April's TD Magazine: https://www.td.org/Publications/Magazines/TD/TD-Archive/2016/04/Putting-Talent-to-the-Test

Both Human Resources (HR) and Learning and Development (L&D) functions are tasked with the same goal: maximizing employee performance. Responsibility for the components of the typical employee lifecycle—hiring, onboarding, training, coaching, developing, evaluating performance, succession planning—are often divided down the middle between HR and L&D. It is intuitive to think that people strategies should be closely aligned and integrated between these two functions, each leveraging the intellectual capital acquired by the other. Unfortunately, all too often, organizations witness HR and L&D operating in relative isolation, focused on their individual departmental objectives rather than their shared mission.

The reality is that, for employees to thrive, a symbiotic relationship must exist between HR and L&D departments. HR can hire exceptional talent that withers on the vine if L&D is not nurturing and growing that talent. And, reciprocally, L&D can provide exquisitely designed development programs that fail to achieve results if the right people are not “on the bus.” So how can HR and L&D professionals leverage shared strategies for achieving their collective mission? As top leaders demand more evidence-based people decisions, leveraging the data gleaned from employee assessments is a powerful way to increase both the value and effectiveness of both functions.

 First: Hire the Right People
While this charge typically falls to those in the HR arena, L&D certainly has a vested interest. After all, how many times is the L&D team tasked with “fixing,” via coaching or training, an employee who frankly isn’t cut out for the position they’re in? Foundational to hiring success is a systematic and scientific process that maximizes the odds of hiring the best people as often as possible. This is the value proposition of pre-hire assessments.

There is ample research demonstrating that the use of well-developed assessment tools greatly increases the odds of hiring top performing employees over more subjective (yet heavily relied on) selection methods such as interviews and reference checks. However, there are several key factors to consider in order to create a selection assessment system that packs a predictive punch.

a.      Be clear about the goal

“We want to hire better people” is not a clear enough objective. Whether you are trying to impact turnover, employee engagement, sales volume, customer/patient satisfaction, productivity, theft, absenteeism, safety incidents, scrap rates, product quality, or some other outcome, there are different assessment instruments scientifically designed to impact these, and countless other, organizational goals.

Many traditional HR and L&D metrics—time to fill, time to productivity, retention, cost per hire, performance in a training class—can be positively impacted by integrating an appropriate type of assessment instrument. However, it’s essential to know which are the most important metrics to move the needle on. For example, if inbound call on-hold times or order pick rates have a greater financial impact on the organization than employee retention, make sure you’re focused on the metrics that matter most. Once your objective is clear, you can determine what constructs you can measure that will be most predictive of these mission critical outcomes.

      b.      Decide what to measure

Virtually any human characteristic can be measured with an assessment, including personality, skills, abilities, problem solving capabilities, knowledge, intelligence, interests, values, beliefs, and more. However, in most cases, companies don’t have the luxury, in terms of time or money, of assessing all these things (nor do they have candidates who would tolerate such a grueling endeavor). Therefore it’s important to identify which measurable characteristics will be most predictive of the outcomes you’re trying to achieve.

 Job analysis is one tool that enables you to think beyond roles, responsibilities, job descriptions, and job duties, and pinpoint what truly differentiates between someone who can merely do the job adequately and someone who blows expectations out of the water. L&D staff can play a valuable role in conducting job analyses, as they are often intimately familiar with critical roles as a result of designing custom training curriculum and/or being aware of where employees struggle to perform.

c.       Use the most predictive screening methods

Extensive research has been done on the predictive validity—the overall ability to predict job performance—of different hiring methods and measures. The table below reports the relative validity of some of the most commonly used selection methods based on a meta-analysis of a century’s worth of workplace productivity data1.

 
Graphology (Handwriting Analysis)
.02
Personality Tests
.22
Emotional Intelligence
.24
Reference Checks
.26
Integrity Tests
.46
Cognitive Ability Tests
.65
Multi-Measure Tests (i.e., Cognitive Ability + Personality + Interests)
.75+

This means that if your hiring process relies primarily on interviews, reference checks, and even personality tests, you are electing to use a process that is significantly less effective than it could be. The litmus test for any selection method should be “is the information gleaned from this step in the process clearly predictive of future job performance?” If the answer is no, there is probably no point in using it.

     d.      Select good instruments

Candidly, many people who choose a career in HR or L&D don’t do so because of their love of statistics. Therefore, it’s understandable that sifting through a highly technical validation document may be daunting; however, it’s also necessary. An assessment instrument must meet certain criteria related to reliability, validity, job relevance, adverse impact, and a number of other factors in order to be legally defensible. Test publishers should be able to provide ample data showing how rigorous they were in developing their instruments. If necessary, seek assistance in critically scrutinizing this information (consultants, academics, or Industrial/Organizational Psychologists are some potential resources).

If you suspect that your company’s selection system is not hitting on all of the above four cylinders, partner with your HR colleagues to determine if there are changes that can be made to increase the predictive power of the process.

Second: Leverage pre-hire assessment data post-hire
While substantial value is gained from using assessments for selection, it is unfortunate when the benefit ends there. By this point, the organization has made an investment, in both the tool and the candidate, and should maximize that investment. “The failure to value pre-hire data as part of post-hire talent management activities represents a wasted opportunity to realize a deeper level of predictive value from assessment tools,” according to Bersin & Associates’ Prehire Assessment Primer2. Furthermore, according to a 2015 Aberdeen study, companies that correlate pre-hire assessments with ongoing employee performance results are 24% more likely to have a greater number of employees who exceed expectations3.

Many pre-hire assessments lend themselves to a variety of post-hire applications, including:

·         Enabling managers to develop individually nuanced strategies for motivating, rewarding, and incentivizing each of their employees.

·         Facilitating a proactive dialog between new hires and their manager on potential challenge areas and possible solutions.

·         Creating highly targeted employee development plans based on identified deficits in knowledge, skills, or behavioral tendencies.

·         Tailoring the content and/or format of a training or orientation class based on the personalities, learning styles, or existing knowledge base of the participants.

·         Helping new employees assimilate with their team or department, and maximizing team effectiveness.

·         Developing succession plans or career paths that consider future job-fit and identify needed development to prepare employees for success in their next role.

Currently less than a quarter of line managers think their L&D department is critical to achieving business goals, according to 2015 research by Bersin by Deloitte4. Being able to use assessment data to tailor employee development initiatives that are aligned with business goals can go a long way in demonstrating L&D’s value.

Caution: One size does not fit all
The ultimate goal of a well-developed pre-hire assessment system is predicting future job performance. This may mean utilizing an assessment that measures more stable, “hard wired” characteristics that are not likely to change over time. It is important to understand the nature of what the tool is measuring to ensure appropriate use of this information post-hire. For example, imagine Alex is being groomed for a supervisory position. Her assessment results show that she scores very low on assertiveness and decisiveness. Can the L&D team train her to be an assertive and decisive person? Probably not. However, there are two things they can do to help Alex prepare for her new role:

1)      Increase self-awareness. Having a candid conversation with Alex about the aspects of the new role that may cause her to step out of her comfort zone will allow her to assume the role with eyes wide open.

2)      Develop adaptive strategies. While Alex still may shudder at the idea of confrontation or making a snap decision, there are certainly learnable skills, techniques, and strategies that will enable her to be more comfortable, confident, and functional in situations requiring these behaviors.

On the other hand, many of the assessments commonly used in L&D applications—four-quadrant personality assessments, or communication or leadership style assessments, for example—are not ideal for use pre-hire. While there are many tools available that lend themselves nicely to multiple applications, it’s critical to make sure all parties understand the exact nature of the tool, what it’s measuring, and appropriate and inappropriate ways to internalize and apply the information provided.

Third: Demonstrate Your Value
All too often, both HR and L&D functions are seen as cost centers, constantly asked to justify their existence and expenditures. With payroll and benefits representing one of the largest line items on virtually every company’s operating statement, incorporating assessments and tying results to key organizational metrics can be a highly compelling way to demonstrate the value that both departments add to the bottom line.

The need to demonstrate fiscal impact is heightened further due to an increased focus on incorporating metrics, analytics, and big data into decision making. “We see CEOs and others wanting better data and not just a headcount report, but how is talent driving business results?” says Scott Pollak, a principal at PwC Saratoga in a Harvard Business Review report5. With 57% of companies reporting their intention to have integrated, multi-source analytics in place in the next two years6, there is a push to incorporate more scientific, evidence-based practices in the people-functions in our businesses. Yet, currently only 14% of businesses have data to show the business impact of their assessment strategy7.

Which metrics matter most? Retention? Customer or patient satisfaction? Production or sales volume? Both HR and L&D need to have a clear understanding of where the business, and the C-Suite, are focused. Once it is clear what outcomes have the biggest impact on the viability of the organization, HR and L&D can partner to ensure assessment data is being leveraged to the fullest, both pre- and post-hire, to achieve those objectives and show, in financial terms, the success of their efforts toward recruiting, retaining, and developing a high-performing workforce.

Tuesday, August 4, 2015

Creating an Evidence-Based Coaching Culture

By Whitney Martin, MS and Hope Zoeller, Ed.D

Coaching means different things to different people and can vary in its effectiveness depending on the methods and strategies used. How can you be intentional in creating a coaching culture to ensure a maximum level of success? How can you incorporate objective data into the coaching process to increase impact?
 

The benefits of investing and developing a coaching culture are great. According to a survey conducted by the Institute of Leadership and Management, 95% of respondents experienced direct benefits to the organization, and 96% experienced benefits to the individual. A wide range of specifically cited improvements included: communication and interpersonal skills, leadership and management, conflict resolution, personal confidence, attitudes and motivation, management performance, as well as preparation for a new role or promotion.
 

So, how do you create this type of high-performance coaching culture in your organization?

  1. Define "coaching." Coaching isn't telling people what to do differently. Unfortunately, many organizations still perceive coaching as a tool for correcting poor performance. However, good coaching is about achieving a high-performance culture, not managing a low-performance one. Coaching is a cooperative, interactive process of working together on improved performance and leadership development. You also need to define what a "coaching culture" means. A coaching culture involves regular feedback and communication from the manager to COACH the employee to sustain, improve, or modify behaviors and skills.
     
  2. Develop a Shared Vision for the Coaching Culture. Once a coaching culture is in place, Senior Management needs to define what results can be achieved, what will people feel, and ultimately how coaching will influence positive behavior changes with an observable and measurable impact on the business. One example of a coaching culture vision is: "To create an environment that elevates individual and team performance by integrating coaching techniques and principles into the organization; and aligns coaching with the business strategy by educating and engaging leaders in the coaching process."
     
  3. Get Senior Management Buy-in and Participation. Without support from the top, the change to a coaching culture is doomed to fail before it begins. Once you identify an executive level sponsor(s), you need to create measurements for change. To create a sustainable environment for a coaching culture to thrive means fully integrating coaching into strategic HR processes and systems and communicating progress on a regular basis. Also, Senior Managers need to receive coaching to truly understand the power of coaching and to fully support it. However, it should not be limited to Senior Executives. While the focus of coaching may vary, all levels of employees, definitely all managers and leaders in an organization, can and should benefit from coaching.
     
  4. Integrate Measurement Tools. Various types of measurement tools can play an invaluable role in several facets of the coaching process. For example, 360-degree feedback tools can provide essential information at both the "micro" and "macro" level. At the "micro," or individual level, the information gleaned from a 360 can form the basis of the coaching strategy and individual development plan. It can answer questions like:
·      "Is this leader's self-perception in line with what is being experienced by those he or she works with?"

·      "Are this leader and their boss in alignment regarding focus and priorities for this person's job?"

·      "What specific behaviors are contributing to the perception that a particular competency is an area for development?

·      "And, after coaching has occurred, "Is progress being made?"

If 360-degree feedback is gathered on multiple managers in the organization, the data can be looked at in aggregate to answer important "macro" level questions, such as "are their certain developmental needs that appear to be systemic in the organization?" "Do our leaders' strengths align with our organizational culture, priorities, and unique selling proposition?" Other types of assessment tools can also shed light on whether individuals are optimally placed within the organization in roles where they are able to best leverage their strengths.

Coaching should not be a siloed initiative. The greatest impact can be achieved when organizations connect coaching to the strategic plan. This requires a commitment to the initiative for a minimum of two to three years. Return on Investment measures also need to be agreed upon by Senior Management. These measures must be relevant to the organization, connected to business strategy, and able to be tracked effectively.
For those organizations willing to embark on the journey, the impact on business can be tremendous, dramatically impacting your culture and your people for the better. It's amazing what individuals can achieve with the right level of support and development. When coaching is successfully integrated into the culture of an organization, it creates a competitive advantage and unlocks untapped talent and potential.

Wednesday, June 3, 2015

Hiring under the Microscope: Improving the Science of Selection

As featured in Kentucky SHRM Magazine, Spring/Summer 2015


Over the last two decades, the chatter in HR circles has concerned “becoming more strategic” and “getting a seat at the table.” However, a golden opportunity has been missed, as a linchpin function of HR—one with a profound impact on the bottom line—has been largely ignored. In 2002 Rynes, Colbert, and Brown[1] conducted research to determine whether the beliefs of HR professionals were consistent with established research findings on the effectiveness of various HR practices. As it turns out, the area of greatest disconnect was in staffing (particularly related to hiring assessments), where fewer than 50% of respondents were familiar with prevailing research findings.

As we roll into 2015, the HR chatter has turned to metrics, analytics, and big data. Yet again, though, personnel selection is late to show up to the party. A 2014 Aberdeen study[2] found that only 14% of businesses have data to show the business impact of their assessment strategy. With payroll and benefits representing one of the largest line items on virtually every company’s operating statement, effective selection is one of the top areas where HR can have a significant impact on the bottom line. It’s time for organizations to put hiring under the microscope.

Virtually everyone has acknowledged the fact that getting the right people in the right jobs is critical to business success. But how to get the “right” people continues to elude many: I have talked with organizations that have tried to incorporate everything from horoscopes to a deck of playing cards into their selection process, all in a vain attempt at systematically identifying which candidates have the best chance of becoming strong employees. I believe the following steps will help HR improve the science of selection.

1)      Clarity. HR needs to think more strategically about the desired outcomes of their selection systems.

“We want to hire better people” is not a clear enough goal. What is it you are actually trying to impact—turnover, retention, sales volume, customer satisfaction, morale, productivity, theft, absenteeism, safety incidents, drug use in the workplace, etc.? There are different assessment instruments designed specifically to address these, and countless other, issues or goals. Once your objective is clear, you can determine what constructs you can measure that will be predictive of that outcome.

2)      Validity. Predictive validity should be a driving factor (if not THE driving factor) in creating selection systems.

Extensive research has been done on the predictive validity—the overall ability to predict job performance—of different hiring methods and measures. The table below[3] reports the relative validity of some of the most commonly used selection practices based on a meta-analysis of a century’s worth of workplace productivity data.

Graphology (Handwriting Analysis)
.02
Personality Tests
.22
Emotional Intelligence
.24
Reference Checks
.26
Integrity Tests
.46
Cognitive Ability Tests
.65
Multi-Measure Tests (i.e., Cognitive Ability + Personality + Interests)
.75+

 

This means that if your hiring process relies primarily on interviews, reference checks, and even personality tests, you are electing to use a process that is significantly less effective than it could be. There is only one question that matters when deciding to incorporate a selection method: is the information gleaned from this tool predictive of future job performance? If the answer is no, there is no point in using it, regardless of how cheap, easy, or popular it is.

3)      Scrutiny. HR needs to get savvier when selecting tools.

 

Most HR people don’t choose their profession because they love numbers, so it’s understandable that sifting through a highly technical validation document may be daunting; however, it’s also necessary. A tool must meet certain criteria as it relates to reliability, validity, adverse impact, and a number of other factors. Test publishers should be able to provide ample data showing how rigorous they were in developing their instrument. If necessary, HR can seek help in critically scrutinizing this information (consultants and academics are two potential resources).

 

4)      Metrics. Selection-system outcomes should be tied to organizational metrics.

HR should be able to demonstrate that the use of a particular tool has had a direct impact on some organizational outcome of interest. In other words, as test scores go up, turnover goes down, or as test scores go up, sales volume increases. Many times this can be achieved through either a concurrent or predictive validation study.

As an example, one of my clients has 100 sales people, who are publicly ranked on a scale of 1 to 100 based on objective performance metrics. The company leadership decided that, more than anything, their goal was to avoid hiring the bad ones. We were able to create a benchmark (incorporating mental ability, behavioral characteristics, and occupational interests) that was a clear differentiator between top and bottom performers. Using this tool, they would have correctly identified five out of eight of their top performers, but perhaps more importantly, they would have conclusively avoided hiring nine out of ten of their bottom performers. This is an example of a concurrent validation study that demonstrates how assessment results are directly tied to sales success.

Organizations that choose to rely on less predictive selection methods are unnecessarily creating a competitive disadvantage for themselves. HR practitioners have an opportunity to increase their indispensability to the organization by creating scientific, evidence-based selection systems that are demonstrably linked to bottom-line outcomes of interest to the C-suite. An HR leader who can effectively do that will be happy to be placed under the microscope!

 

As you ponder “What’s Next in Human Resources?” here are some questions for reflection:

·         Are you feeling pressure to incorporate more data-supported or evidence-based methods in your job? If not, could it be coming? Have you been hearing more about “metrics,” “analytics,” or “big data”? Is now your chance to get ahead of the curve?

·         Is there anything you can do to increase the predictive validity of your hiring process? Are the steps in your current process yielding measurable results? Are there tools out there that could help your process be both more effective and more efficient?

·         Can you calculate the impact that your hiring processes have on the organization’s bottom line? If pressed, could you tell your C-Suite exactly how accurate your selection system is, and quantify the ROI of your efforts? Is there an opportunity to further solidify your strategic value to the organization by improving in this area?



[1] Sara L. Rynes, Amy E. Colbert, and Kenneth G. Brown, “HR Professionals’ Beliefs about Effective Human Resource Practices: Correspondence between Research and Practice,” Human Resource Management 41, no. 2 (2002): 149–74.
[2] M. Lombardi, “Measurement Strategies for Assessment Success,” Aberdeen Group (2014). http://v1.aberdeen.com/launch/report/research_report/9043-RR-measuring-assessment-success.asp.
[3] Based on data shared by Frank L. Schmidt in a Nov. 6, 2013, address to the Personnel Testing Council of Metropolitan Washington as an update to ——— and John E. Hunter, “The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings,” Psychological Bulletin 124, no. 2 (1998): 262–74.

Wednesday, September 17, 2014

The Problem with Using Personality Assessments for Hiring: As featured in Harvard Business Review

This article was published by Harvard Business Review on 8/27/14 and can be found at: http://blogs.hbr.org/2014/08/the-problem-with-using-personality-tests-for-hiring/

A decade ago, researchers discovered something that should have opened eyes and raised red flags in the business world.

Sara Rynes, Amy Colbert, and Kenneth Brown conducted a study in 2002 to determine whether the beliefs of HR professionals were consistent with established research findings on the effectiveness of various HR practices. They surveyed 1,000 Society for Human Resource Management (SHRM) members — HR Managers, Directors, and VPs — with an average of 14 years’ experience.

The results? The area of greatest disconnect was in staffing— one of the lynchpins of HR. This was particularly prevalent in the area of hiring assessments, where more than 50% of respondents were unfamiliar with prevailing research findings.

What HR Managers Get Wrong chart

Several studies since have explored why these research findings have seemingly failed to transfer to HR practitioners. Among the causes are the fact that HR professionals often don’t have time to read the latest research; the research itself is often present with technically complex language and data; and that the prospect of introducing an entirely new screening measure is daunting from multiple angles.

At the same time, anyone who has ever been responsible for hiring, much less managing, employees knows that there is a wide variation in worker performance levels across jobs. Therefore, it is critical for organizations to understand what differences among individuals systematically affect job performance so that the candidates with the greatest probability of success can be hired.

So what are the most effective screening measures?
Extensive research has been done on the ability of various hiring methods and measures to actually predict job performance. A seminal work in this area is Frank Schmidt’s meta-analysis of a century’s worth of workplace productivity data, first published in 1998 and recently updated. The table below shows the predictive validity of some commonly used selection practices, sorted from most effective to least effective, according to his latest analysis that was shared at the Personnel Testing Counsel Metropolitan Washington chapter meeting this past November:

Most effective screening measures chart

So if your hiring process relies primarily on interviews, reference checks, and personality tests, you are choosing to use a process that is significantly less effective than it could be if more effective measures were incorporated.

And yet that’s how many companies operate. According to a 2011 NBC News article, the use of personality assessments are on the rise, growing as much as 20% annually. Especially problematic is the widespread use of Four Quadrant (4-Q) personality tests for hiring, something I see regularly in my consulting work.

A 4-Q assessment is one where the results classify you as some combination of four different options labeled as letters, numbers, colors, animals, etc. They originated around 450 BC when Empedocles noticed that he could group people’s behavior into four categories which he labeled earth, water, fire, and air. Hippocrates made the same observation, but (coming from a medical background) labeled the categories blood, phlegm, black bile, and yellow bile. Since then, hundreds of iterations of these tools have been developed, all essentially based on the same premise and theory.

Generally speaking, 4-Q tools consist of a list of adjectives from which respondents select words that are most/least like them, and are designed to measure “style,” or tendencies and preferences. While they can seem highly insightful — not to mention being widely available and inexpensive — they have some severe shortcomings when used in high stakes applications such as hiring.

For one, they tend to be highly transparent, enabling a test taker to manipulate the results in a way that they feel will be viewed favorably by the administrator. Also, since they are designed to measure “states” (as opposed to more stable “traits”), there is a significant chance that the results will change over time as the individual’s context changes (most publishers of 4-Q tests recommend that individuals re-take them at fairly frequent intervals for this reason).

This begs the question: How can an individual’s assessment results be used to predict future job performance if there is a reasonable chance that their scores will change over time?

When using any assessment, managers need to step back and ask themselves one basic question before giving it to a potential employee: Is this test predictive of future job performance? In the case of 4-Qs, probably not. They can provide tremendous value for self-discovery, team building, coaching, enhancing communication, and numerous other developmental applications. But due to limited predictive validity, low test-retest reliability, lack of norming and an internal consistency (lie detector) measure, etc., they are not ideal for use in hiring.

The strongest personality assessments to use in a hiring context are ones that possess these attributes:
  • Measure stable traits that will not tend to change once the candidate has been on the job for some length of time.
  • Are normative in nature, which allows you to compare one candidate’s scores against another’s to determine which individual possess more (or less) of a particular trait.
  • Have a “candidness” (or “distortion” or “lie detector”) scale so you understand how likely it is that the results accurately portray the test-taker.
  • Have high reliability (including test-retest reliability) and have been shown to be valid predictors of job performance.
Even when using a tool that meets the criteria outlined above, personality constructs are not the most predictive measure available. Personality tests are most effective when combined with other measures with higher predictive validity, such as integrity or cognitive ability.

Using well-validated, highly predictive assessment tools can give business owners and managers a significant leg up when trying to select candidates who will become top producers for the organization. However, all assessment approaches are not created equal. And some will not offer a significant return on your investment. Accordingly to a 2014 Aberdeen study [registration required], only 14% of organizations have data to prove the positive business impact of their assessment strategy. Knowing which types of assessments will be most effective in accomplishing the specific objectives you have identified for your organization will enable you to select a tool with a measurable impact on the bottom line.

A Re-cap of the National SHRM Conference Through the Eyes of a First Time Attendee

This article was published in the Fall/Winter 2014 Edition of Kentucky SHRM Magazine


It’s 4:30am on the Tuesday of the SHRM Conference in Orlando. And the power has gone out. I know this because I’ve just been awoken by the most annoying of sounds. You know that BEEP that the smoke detector makes when it’s disconnected from power? BEEEEEP. I’m stumbling around my hotel room looking for my Ipod and headphones so I can drown out the BEEP.  Then it sinks in. No power means… what? No wake-up call? No hairdryer? And—so God help me—the Starbucks better have their own back-up generator! Since I’ve been robbed of sleep, I’m going to need caffeine before I present my concurrent session at 7:00am.

Let’s back up. This is my first time at the national SHRM conference. Walking into a room that holds 14,000 people is a powerful moment that I won’t soon forget. Slightly overshadowing that, though, was the sense of awe experienced when walking into a vendor hall that housed over 600 booths, some of them as large as a small city and as elaborate as a nearby Disney theme park attraction. Considering how many mini-Ipad raffles I entered, I’m bound to win one of them, right?

The general sessions were all very engaging and motivating.  On Sunday, attendees were treated to an inspiring and very vulnerable talk from Good Morning America host Robin Roberts. She encouraged people to “make your mess your message,” meaning that everyone is dealing with some challenge in their personal life and we can support and encourage others by being transparent and open with our struggles. Monday, the general session was presented by Tom Friedman, author of one of my favorite books, The World is Flat. He warned that “average is officially over,” and only those who approach their career with creativity and an entrepreneurial spirit will thrive in the new reality. On Tuesday, fellow Kentuckian and YUM! Brands CEO, David Novak coached us on how to recognize and reward employees so as to “take them with you.” Closing out the conference on Wednesday, former first lady Laura Bush shared about her life experiences and updated attendees on the current activities of her various famous family members.

With hundreds of concurrent sessions to choose from, attendees had no shortage of options and could truly tailor their learning experience. When asked what his favorite session was, Kentucky SHRM’s State Conference Director, Perry Sholes from Lexington, said Identity Loyalty—Unlocking the Keys to Creating Productive, Hardworking and Appreciative Employees. Taryn Pearson, HR Manager at PetFirst Healthcare in Southern Indiana enjoyed attending the session called HR Department of One. “It’s great to know that other HR Managers do this on their own and are successful,” she said. “The speaker also provided a lot of valuable resources for people like me who are an HR department of one.”

Of particular interest to many attendees this year was the new certification program being rolled out by SHRM as a replacement for (or alternative to?) the PHR and SPHR certifications traditionally provided through HRCI. Several sessions addressed this topic and John Bachman, LSHRM past president, worked to make sure he was hearing both sides of the story. John attended both SHRM sponsored and HRCI sponsored events on this issue. “It is hard to tell which story is THE story,” John says. “I feel like a child in a divorce; we were “raised” to love and support both, but now they are split. I’m not taking sides; I plan to be active in both and I wish them both great future success.”       

On Monday, approximately 30 conference attendees from Kentucky gathered for lunch.  After 24 hours of feeling disconnected and alone among a sea of strangers, it was very refreshing to be among friends. Throughout the rest of the conference, I met several more Kentuckians that hadn’t heard about the Kentucky lunch. Just a tip—if you plan to attend the 2015 SHRM Conference in Vegas, make sure to reach out and find out when/where the Kentucky gathering will take place. This is an invaluable opportunity to connect with local colleagues and truly maximize your conference experience.

That brings us back to Tuesday morning. When I was notified in late October that I’d been selected to present a concurrent session on Hiring Assessments, I was humbled, honored, and thrilled. When I realized that my session was to take place at 7:00am… I was baffled. Surely no more than 4 people would show up at 7:00am, right? Instead, I was grateful to speak to a gracious and enthusiastic crowd of about 400. And that’s nothing—Louisvillian Erika Tedesco from Hosparus presented at 7:00am Monday morning regarding Onboarding and Orientation Programs and had over 550 in her session.  Rounding out our Kentucky presenter team was Sandy Allgeier, a 10 year veteran of the SHRM faculty. Sandy presented a preconference workshop on being a HR Business Partner, a session she’s facilitated for the past 3 years. The Bluegrass State was well represented in the Orange State!

In reflecting on my first experience at the national SHRM conference, I truly found it thrilling (albeit, at times, overwhelming) to be a part of something so large and energetic. Right now, all I can think about is how I can make sure to be part of next year’s conference… Vegas, baby! Of all the renowned conference speakers, it was perhaps Tim McGraw (who performed for attendees Tuesday night) who summed up the conference experience up best—“I like it! I love it! I want some more of it!”