Fantasy or fantastic? Be thoughtful about new assessments

UNLEASHHR Technology2018 10 31
Fantasy or fantastic? Be thoughtful about new assessments
Share on Linkedin

The most effective HR professionals keep their finger on the pulse of new product developments. And there’s a lot to be aware of. In particular, the rise of AI (artificial intelligence) capabilities is resulting in the emergence of new, technologically-driven tools that can help with talent selection. For example, we’ve seen emerging technology that has incorporated AI in gamified assessments, scoring digital interviews, and scoring unstructured data from a candidate’s social media profile or resumé. However, being able to discern the ‘fantastic’ new developments from the ones that are mostly ‘fantasy’ requires a thoughtful approach from HR professionals. And that’s where the expertise of industrial-organizational (I-O) psychologists can really help.

Getting talent selection right is critical to sustained organizational performance. And psychological testing and assessment methods are tried and true ways to measure ability, personality, values, interests, knowledge, and skill – all critical to hiring success. Today, the tried and true methods are being challenged by a wave of new, exciting selection tools.  As you’re perusing the virtual shop window of shiny new offerings, and even when you’re evaluating existing talent assessment solutions, consider the following to ensure they adhere to industry-accepted standards and, crucially, that they work!

Know what you’re measuring with talent assessments

At the risk of oversimplifying AI’s benefits, primarily it helps us with prediction. And prediction is hard, especially when it comes to human behavior. Whether it’s your manager, direct report, friend, partner, child – it’s frustrating when their behavior doesn’t meet your expectations. New data mining and AI capabilities mean we can now build models that increase our ability to predict behavior in areas like employee retention, employer Net Promoter Score (eNPS), and quality of hire in the selection process.

However, before you even start evaluating thousands of data points from a recorded interview, parsing resumes, or scraping someone’s digital footprint, make sure you know exactly what you’re measuring. Whether it’s customer service behaviors, empathy, key experiences, or teamwork, you need to know what you’re looking for and why it may predict performance. Importantly, we need to be able to explain why a prediction is relevant to a particular outcome. However enticing a new and shiny assessment toy looks, ask yourself,  does it have the same reliability and validity as traditional assessment methods? Understanding what you want to measure needs to be the start point for determining how we go about measuring it.

Ensure your talent assessment results are consistent and precise

Games are more fun than tests, aren’t they? What if taking a test could feel like playing a game? One trend around candidate experience is gamified assessments or game-based assessments. These methods are appealing to both hiring managers and candidates because it avoids assessments feeling like a test. Gamified assessments can screen candidates in an engaging way and can also be used as a tool to attract candidates. And it’s not just about making the experience more enjoyable, incorporating gaming elements in an assessment process can provide opportunities to measure characteristics, behaviors, or skills we wouldn’t otherwise measure with traditional assessments.

However, when evaluating these ‘fun’ assessment experiences, it is important to understand what exactly they are measuring. If you take a game and it indicates you’re outgoing and resilient, then that should reflect your personality and be consistent with other methods of assessment that have been proven to measure those same characteristics. If your gaming results don’t make sense or are inconsistent with other assessments, then it may be a fun game, but it may not be a very reliable measure.

Beyond the ‘it measures what it says it measures’ validation, we should also be mindful that an assessment is consistent. For example, when you’re measuring stable characteristics (things that don’t change much), you should receive consistent results. In other words, if you play the game multiple times, your score on stable characteristics should be broadly the same. In short, all methods should measure job related characteristics (e.g., personality, ability, skills) in a precise and consistent manner.

This is not only important from an HR practitioner perspective as they look for accuracy, it also matters to job candidates that they are being assessed fairly against others and over time.

Be sure your talent assessment approach works for your organization

Simply put, not all organizations are created equal. Each organization has its own employment brand, hiring needs, and talent pools. When evaluating what assessments methods to use, you need to understand why they may (or may not) work for your organization. If we can’t explain what is being measured and why it relates to job performance (or another important outcome) then why would we expect it to continue to predict outcomes across jobs, industries, and organizations?  This is a critical area for HR professionals as explanations of measures and job performance might need to be provided to a regulatory agency or a candidate to justify selection.

Simply because they’re new, emerging methods may not yet have received the same amount of broader empirical research as traditional psychological assessment methods. For this reason, if you have satisfied yourself about points one and two above, it may be necessary to conduct a study within your organization to help provide explanations and additional support for the new tool.

There are tremendous and exciting opportunities with both emerging and existing assessment approaches. However, to ensure fair and effective selection decisions, emerging methods should be evaluated with the same professional and legal standards as traditional psychological assessments. Successfully separating the fantastic from the fantasy rests with thoughtful HR professionals and expert I-O psychologists.

Learn more about evaluating assessments in the age of big data and AI and see how IBM Kenexa Employee Assessments can help transform your approach to identifying and hiring the best-fit talent for your business using proven behavioral science techniques.

About the Author

Kevin Impelman received his Ph.D. and M.S. in industrial and organizational psychology from the University of North Texas and his B.A. in psychology from Southern Methodist University. Kevin is a licensed psychologist in the state of Texas. Kevin is a member of the Society for Industrial and Organizational Psychology (SIOP) and the American Psychological Association (APA).

Share on Linkedin