Assessing Assessments


Anyone who consistently reads this blog knows that I continue to harp on the need for firms to increase their use of formal assessments for C-suite and CEO succession decisions.  I have pointed out how the HR@Moore Survey of CHROs finds that less than a majority of firms use formal testing such as personality, cognitive ability, and behavioral assessments to gain information on C-suite and potential C-suite executives.  However, a number of firms do use such assessments, and on our surveys they have reported the exact assessments they use (by vendor).  That led one of my colleagues, Robert (Rob) Ployhart, to explore just how good these assessments might be.

Rob has developed a world-wide reputation for being one of the leading researchers in the area of assessment.  So his interest was piqued by seeing the variety of assessments CHROs reported using and he wondered about the psychometric evidence for the quality of these assessments.  For instance are they reliable (i.e., consistent over time, raters, or items) and are they valid (accurate assessments of characteristics necessary for successful job performance)?  If so, the vendors should be able to produce the empirical evidence supporting their use.  Thus, he spent the last semester having his students from his "Selection" class seek out the reliability and validity data for the most popular executive assessments.

In a few months he will release a full report of his findings.  However, the preliminary findings make me wonder if I should continue to call for the increased use of assessments.  In essence, what he found was that very few either could, or were willing to share any evidence of the reliability and/or validity of their assessment techniques.  This is troubling, as it may suggest that as interesting as these assessments might be, they may be more useful for separating your company from its money than actually increasing its ability to predict future success of candidates.

Believing that "sunlight is the best disinfectant," Rob and the Center for Executive Succession will post the results of his research, including which vendors refused to provide information on the psychometric quality of their products.  We will then encourage all CHROs to use this information as a guide for choosing among the variety of assessment tools out there.  It will be a dynamic site that can be updated as vendors share the empirical data supporting the quality of their assessments.  The goal will not be to sell any or even promote any, but rather to provide objective information for all those trying to choose among the plethora of products available.  The goal is to help CHROs distinguish between those techniques that have strong empirical data backing their use versus those whose sales pitch rests only on slick words.