How Special Are Executives? How Special Should Executive Selection Be? Observations and Recommendations

Full text

Industrial and Organizational Psychology
,2(2009), 163– 170.
Copyright c
2009 Society for Industrial and Organizational Psychology. 1754-9426/09
COMMENTARIES
How Special Are Executives? How
Special Should Executive Selection Be?
Observations and Recommendations
DENIZ S. ONES
University of Minnesota
STEPHAN DILCHERT
Baruch College, CUNY
Hollenbeck (2009) suggests that executive
selection decisions are often wrong and
believes that selection of executives should
be differentiated from selection at lower lev-
els. In addition, he asserts that by focusing
on competencies, rather than characteris-
tics, ‘‘we are doing it backwards.’’ We
agree with Hollenbeck that sound personnel
selection should start with and be based on
personal characteristics rather than amor-
phous, often ill-defined competencies. Yet,
this principle applies to all selection not just
executive selection. In order to determine
whether executive selection should truly be
a special process, two key questions must
be asked and answered.
How Unique Are Determinants
of Executive Success?
In nearly a century of applied psychological
research, our field has learned that certain
Correspondence concerning this article should be
addressed to Deniz S. Ones, Department of Psy-
chology, University of Minnesota; Stephan Dilchert,
Department of Management, Zicklin School of Busi-
ness, Baruch College, City University of New York.
E-mail: deniz.s.ones-1@tc.umn.edu
stephan.dilchert@baruch.cuny.edu
Address: Department of Psychology, University of
Minnesota, 75 East River Road, Minneapolis, MN
55455 or Department of Management, Box B9-240,
Zicklin School of Business, Baruch College, One
Bernard Baruch Way, New York, NY 10010
characteristics, such as cognitive ability, if
assessed using reliable psychometric mea-
sures, predict performance well for all kinds
of employees (Ones, Dilchert, Viswesvaran,
& Salgado, in press; Ones, Viswesvaran, &
Dilchert, 2005). Hunter’s (1981) reanalysis
of cognitive test score validities reported
by Ghiselli (1973) yielded a mean opera-
tional validity of .53 for predicting man-
agerial performance. Salgado et al. (2003)
reported an operational validity of .67 for
general mental ability tests predicting job
performance among managers, based on
European validation studies. A recent meta-
analysis reported a true score correlation of
.33 between paper-and-pencil intelligence
measures and objectively assessed leader-
ship effectiveness (Judge, Colbert, & Ilies,
2004). One of the best established find-
ings from the literature on validity gener-
alization is that the validity of cognitive
ability increases as the complexity of jobs
increases. Increased knowledge and infor-
mation processing demands of high com-
plexity jobs also underlie the importance
of cognitive ability for executives. Few
would doubt that executive jobs are among
those highest in complexity. Consequently,
cognitive ability is a key determinant of
executive job performance. The commonly
voiced objection that executives among
163
at https://www.cambridge.org/core/terms. https://doi.org/10.1111/j.1754-9434.2009.01127.x
Downloaded from https://www.cambridge.org/core. University of Minnesota Libraries, on 04 Mar 2021 at 18:16:37, subject to the Cambridge Core terms of use, available

164
D.S. Ones and S. Dilchert
themselves don’t differ much in their cog-
nitive capacities, which supposedly ren-
ders ability tests useless for selection, can
be easily refuted using data (see below).
Industrial–organizational (I –O) psychology
scholars may not have been able to per-
suade all practitioners or executives them-
selves that our tools work and that they are
relevant, but the fact is that they are, if duly
utilized.
Generalizable findings show that the
personality characteristics of Conscien-
tiousness and integrity predict job perfor-
mance and other valued work behaviors
[e.g., avoiding counterproductive work
behaviors (CWB)] very well, regardless
of the job (Ones, Dilchert, Viswesvaran,
& Judge, 2007). Ones, Viswesvaran, and
Schmidt (1993) presented data indicat-
ing that integrity tests predict both over-
all job performance as well as avoiding
counterproductive behaviors for individ-
uals in high complexity jobs (ρs=.51
for job performance and .68 for CWB).
Meta-analyses focusing on managerial sam-
ples also indicate that the assertiveness
and activity/energy facets of Extraversion,
the achievement facet of Conscientious-
ness, and global Conscientiousness are the
most potent predictors of managerial per-
formance (Hough, Ones, & Viswesvaran,
1998). Even how well the highest kind
of executives, namely U.S. presidents, per-
form is predicted outstandingly by measures
assessing cognitive ability, Conscientious-
ness (especially achievement orientation),
and Extraversion (assertiveness and energy
facets, see Rubenzer, Faschingbauer, &
Ones, 2000).
Thus, it is an empirically justified,
economically mandated strategy to select
managers, including (and we would say
especially
) executives, for the characteris-
tics listed above. Not surprisingly, similarly
consistent and large-scale empirical support
for the criterion-related validity of compe-
tencies is lacking.
The straightforward (in that it is data
driven) answer to the question ‘‘how
unique are determinants of executive per-
formance?’’ is not very! Characteristics that
predict job performance in general do so
even for highest level managers. After it
has been settled that the determinants of
executive success are not unique, we must
now answer the question whether execu-
tives themselves may be special in terms of
these characteristics.
How Special Are Executives?
To provide a precise answer to this question
(at least in terms of cognitive ability and per-
sonality traits), three interrelated questions
must be considered: Are executive profile
patterns similar to those of lower level man-
agers? How high (or low) are executives in
terms of their standing on various ability and
personality traits? And last, how variable are
executives as a group on these same traits?
Again, it doesn’t require conjecture to
address these issues. We can use norma-
tive samples and large-scale primary data
sets to answer these questions. For this pur-
pose, we compiled data on managers and
executives at different hierarchical levels on
a number of relevant individual differences
traits: general mental ability (normative data
for the Wonderlic Personnel Test), criti-
cal thinking (normative data for the Wat-
son–Glaser Critical Thinking Appraisal),
deductive reasoning (normative data for
the eTest Deductive Reasoning Test), ver-
bal ability (large-scale primary data on
the Wesman Personnel Classification Test),
and the Big Five factors: Emotional Stabil-
ity, Extraversion, Openness, Agreeableness,
and Conscientiousness (large scale primary
data from the Global Personality Inventory).
Extracting mean scores and group standard
deviations (
SD
) on each of these measures
for a variety of managerial groups, we can
answer the questions of what truly distin-
guishes executives from other managers as
well as the general working population.
Ability and Personality Profiles of
Executives Closely Resemble Managers
at Lower Levels
To illustrate this point, consider Figures 1
and 2, which provide cognitive ability and
at https://www.cambridge.org/core/terms. https://doi.org/10.1111/j.1754-9434.2009.01127.x
Downloaded from https://www.cambridge.org/core. University of Minnesota Libraries, on 04 Mar 2021 at 18:16:37, subject to the Cambridge Core terms of use, available

Executive selection: Observations and recommendations
165
Figure 1
. Cognitive ability mean-score differences between managers at different
organizational levels. All values are Cohen’s d-values referenced to general working
population norms (using population standard deviation units). Data sources: General
mental ability: Wonderlic Personnel Test normative data (Wonderlic Personnel Test
Inc., 1998). Deductive reasoning: eTest Deductive Reasoning Scale normative data
(Management Psychology Group, 2005). Critical reasoning: Watson–Glaser Critical
Thinking Appraisal normative data (Hartcourt Assessment Inc., 2006). Verbal Reasoning:
Wesman Personnel Classification Test (primary data, see Dilchert & Ones, 2008, for
sample description).
personality profiles of different levels of
management in terms of general popula-
tion standard deviation units. On almost all
traits, managers of all levels score above
general population norms. However, more
strikingly, managers at all levels exhibit
a distinct profile, characterized by higher
scores on general mental ability as well
as higher scores on Emotional Stability,
Extraversion, and Agreeableness in relation
to comparatively lower scores on Conscien-
tiousness and Openness. Even though mean
scores differ across hierarchical levels, top-
level executives follow the same general
profile pattern on cognitive ability and per-
sonality attributes as mid-level managers
and even low-level supervisors.
Executives Score Uniformly Higher Than
Lower Level Managers in Terms of Their
Ability and Personality Attributes
For executives, the normative data we
compiled for various ability measures
indicate largest deviations from the general
working population on general mental
ability (executives score 1.10 standard
deviation units higher). In the personality
domain, Emotional Stability (1.41 standard
deviation units higher), Extraversion (1.12
standard deviation units higher overall;
1.60 and 1.33 standard deviation units
for the facets energy and dominance),
and Agreeableness (1.00 standard deviation
units higher) distinguish executives from the
norm.
at https://www.cambridge.org/core/terms. https://doi.org/10.1111/j.1754-9434.2009.01127.x
Downloaded from https://www.cambridge.org/core. University of Minnesota Libraries, on 04 Mar 2021 at 18:16:37, subject to the Cambridge Core terms of use, available

166
D.S. Ones and S. Dilchert
N
N
NN
N
Figure 2
. Personality mean-score differences between managers at different organizational
levels. All values are Cohen’s
d
-values referenced to general working population norms
(using population standard deviation units). Data sources: Global Personality Inventory
(ePredix, 2001, primary data, see Dilchert & Ones, 2008, for sample description).
Although the same traits distinguish man-
agers at lower levels, executives consis-
tently outscore other managers on every
trait.1Executives score 0.50 and 0.77
standard deviation units higher than mid-
and first-line managers on general mental
ability, respectively. This was the largest
difference among levels found on the cog-
nitive ability measures charted in Figure 1.
In terms of their personality characteris-
tics, Figure 2 clearly shows that there is
a perfect gradation from lowest to highest
level executives in terms of each of the Big
Five dimensions of personality. Top execu-
tives score higher than regular executives,
who in turn score higher than mid-level
1. We could only find one notable exception, even
after investigating 31 different personality facets
and six ‘‘dark side’’ traits in addition to the ability
and personality traits presented in Figures 1 and 2.
Top executives score lower on ‘‘attention to detail’’
than other levels of management.
managers, first-line managers, and supervi-
sors. But again, profiles run parallel to one
another.
There Is Substantial Variability in Cognitive
Ability and Personality Scale Scores of
Executives and Executive Applicant Pools
Previous work on cognitive ability and per-
sonality indicates that applicants to a large
variety of jobs (including managerial posi-
tions) display substantial variability, typi-
cally mimicking variability in the general
population (Ones & Viswesvaran, 2003;
Sackett & Ostgaard, 1994). The same is
true for executives and applicants to exec-
utive positions. Figures 3 and 4 display
variability of cognitive ability and Big Five
personality factor scores for different man-
agerial levels. Using the same measures and
normative data as before, standard devi-
ation of the respective managerial groups
have been indexed back to those of the
at https://www.cambridge.org/core/terms. https://doi.org/10.1111/j.1754-9434.2009.01127.x
Downloaded from https://www.cambridge.org/core. University of Minnesota Libraries, on 04 Mar 2021 at 18:16:37, subject to the Cambridge Core terms of use, available

Executive selection: Observations and recommendations
167
Figure 3
. Variability in cognitive ability of managers at different organizational levels
compared with the general working population. All values are standard deviation ratios
(u-values) expressing the variability of one group compared with another. Values of 1.00
indicate that the variability among managers and executives is the same as in the general
working population; values below 1.00 indicate that the variability among managers is
lower than in the reference group, whereas values above 1 indicate that managers are
more variable. Data sources and sample sizes are the same as those in Figure 1.
general working population. The result-
ing
u
-values (see Hunter & Schmidt, 1990,
p. 184) express restriction of range (values
below 1.00) or range enhancement (values
above 1.00). For cognitive ability measures,
executives are most homogenous on critical
reasoning and general mental ability. Yet,
for the latter trait, the degree of reduced vari-
ability among executives is moderate, only
14% lower than the general population.
For the Big Five dimensions of personality,
executives are much less variable. Here,
the homogeneity of executives compared
with the general working population is quite
striking. Reductions in score variability are
49% for Openness, 48% for Extraversion,
45% for Emotional Stability, 38% for Agree-
ableness, and 28% for Conscientiousness.
Lower level executives as well as mid-
level managers (those who typically fill the
pipelines for executive positions), and even
first-line managers and supervisors, display
similar levels of reduced variability in their
personality characteristics.
Implications of Our Observations
for Executive Selection
As a field, we are now at a point where we
can have a positive impact on individual
well-being and organizational productivity
through practices rooted in strong empirical
evidence. The observations we have made
above have important implications for
executive selection.
Focusing on profile patterns of appli-
cants, rather than absolute trait levels (as is
often the case when considering individual
‘‘fit’’), misleads executive selection efforts.
There are at least two reasons for this.
at https://www.cambridge.org/core/terms. https://doi.org/10.1111/j.1754-9434.2009.01127.x
Downloaded from https://www.cambridge.org/core. University of Minnesota Libraries, on 04 Mar 2021 at 18:16:37, subject to the Cambridge Core terms of use, available

168
D.S. Ones and S. Dilchert
Figure 4
. Variability in personality of managers at different organizational levels compared
with the general working population. All values are standard deviation ratios (
u
-values)
expressing the variability of one group compared with another. Values of 1.00 indicate
that the variability among managers and executives is the same as in the general working
population; values below 1.00 indicate that the variability among managers is lower than
in the reference group, whereas values above 1 indicate that managers are more variable.
Data sources and sample sizes are the same as those in Figure 2.
First, cognitive ability and personality pro-
files of lower levels of management follow
the same patterns as those of executives.
Executive profile patterns reflect trait con-
figurations created by occupational attrac-
tion–selection–attrition effects. Second, the
profile patterns needed to predict criteria of
executive success do not necessarily corre-
spond to mean profile patterns found among
executives. For example, the best marker
of executives’ personality profiles is Emo-
tional Stability, a trait that displays only
modest predictive validity for managerial
performance.
Selection should focus on the relative
standing of the applicant being evaluated
on traits that are relevant to performance. In
the case of executive selection, where large
applicant pools are the exception, and indi-
vidual candidates are often evaluated on
their own without direct comparison, the
use of proper normative data is impera-
tive. This is true not only for the use of
psychometric tests but also for standard-
ized interviews, assessment centers, and the
like. For example, on general mental ability,
executives score over 1 standard deviation
unit higher than the general population (and
over half a
SD
higher than mid-level man-
agers). It is not enough to provide a ‘‘hire’’
recommendation into an executive position
for a candidate who scores well above aver-
age. It is important to know precisely how
high the candidate’s trait standing is and
how it compares to typical applicants and
incumbents in the position he or she aspires
to. Unless assessors have at their disposal
appropriate managerial and executive nor-
mative data (and use them in decision mak-
ing), recommendations and evaluations will
at https://www.cambridge.org/core/terms. https://doi.org/10.1111/j.1754-9434.2009.01127.x
Downloaded from https://www.cambridge.org/core. University of Minnesota Libraries, on 04 Mar 2021 at 18:16:37, subject to the Cambridge Core terms of use, available

Executive selection: Observations and recommendations
169
be shots in the dark rather than on target.
Sadly, few cognitive ability tests, and even
fewer personality measures, offer appropri-
ate norms for managers and executives.
This may well be one of the reasons why
psychometric tests are held in low regard
or used infrequently in these populations.
Information that illustrates how outstanding
executives really are in terms of their abil-
ity and personality trait levels is rare. One
of the most important additions that our
field can make to the executive selection
toolbox is the systematic accumulation of
executive and managerial norms for psy-
chometrically sound, valid individual dif-
ferences measures, in order to make scores
more meaningful and encourage their use.
Another improvement will be the manner
in which executive selection occurs. Most
often, only one or maybe a handful of
candidates are evaluated and considered
for a given position. The differences
between the average executive and low-
level managers are such that only 2 in 10
low-level managers will score at or above
the level of the average executive in terms
of general mental ability. Only 2 or 3 in
10 first-line managers have the energy and
assertiveness (Extraversion) that would be
congruent with the levels of these attributes
found among most senior executives. The
data tell a similar story for many of the
individual differences traits examined.
As a consequence, either much larger
pools of candidates should be considered
for executive positions or those select few
who are being considered should be bet-
ter preselected to arrive at relative homo-
geneity on relevant attributes. The latter
is too often erroneously taken for granted,
which limits organizations’ recruitment and
selection efforts, increasing the chances of
making hiring mistakes. The empirical evi-
dence we summarized in Figures 3 and 4
tells a different story—especially for cog-
nitive ability. We agree with Hollenbeck
that too often the focus is on ‘‘what one
has done,’’ rather than looking into the
future and what a person is likely to achieve
based on his character. However, the latter
approach requires wider and stronger exec-
utive search practices that do not limit
candidate pools based on prior achieve-
ment but instead take into account the
large variability of personal characteristics
and individual differences that exist among
executives.
The fact that managers and executives
are more variable in
cognitive ability
than commonly believed suggests that
there is an opportunity (we would say
obligation) to select based on this trait.
The fact that high-level managerial job
incumbents are more homogenous in
terms of their
personality characteristics
suggests that (a) applicant pools supplying
executive candidates may already be quite
homogenous in personality characteristics,
and/or (b) executive selection is already
targeting these traits and may be doing
so to the exclusion of other relevant
attributes (e.g., cognitive abilities). Our
foremost recommendation is to start to more
heavily select executives on general mental
ability, a variable that has traditionally been
neglected in executive selection. As a field,
we cannot afford to give up on our most
popular tools just because some consider
them unfashionable, especially if there are
no viable substitutes.
As Hollenbeck aptly points out, I–O psy-
chology ‘‘has the character, competence,
and competencies’’ to contribute to execu-
tive selection. We would like to add that our
field also has the data that must inform how
we make selection decisions. If Hollenbeck
is right and most executive selection deci-
sions are wrong, then perhaps it is again
time to listen to what the data are telling us.
References
Dilchert, S., & Ones, D. S. (2008). Personality and
extrinsic career success: Predicting managerial
salary at different organizational levels.
Zeitschrift
f¨ur Personalpsychologie, 7
, 1– 23.
ePredix. (2001).
Global personality inventory technical
manual
. Minneapolis, MN: Author.
Ghiselli, E. E. (1973). The validity of aptitude tests
in personnel selection.
Personnel Psychology, 26
,
461– 477.
Hartcourt Assessment Inc. (2006).
Watson-Glaser
critical thinking appraisal short form— updated
normative information.
Retrieved November 30,
at https://www.cambridge.org/core/terms. https://doi.org/10.1111/j.1754-9434.2009.01127.x
Downloaded from https://www.cambridge.org/core. University of Minnesota Libraries, on 04 Mar 2021 at 18:16:37, subject to the Cambridge Core terms of use, available

170
D.S. Ones and S. Dilchert
2008, from www.pearsonassess.com/hai/Images/
dotCom/HTC/WG/NormInformationShort.pdf.
Hollenbeck, G. P. (2009). Executive selection—
What’s right ... and what’s wrong.
Industrial
and Organizational Psychology: Perspectives on
Science and Practice, 2
, 130– 143.
Hough, L. M., Ones, D. S., & Viswesvaran, C. (1998,
April).
Personality correlates of managerial perfor-
mance constructs.
Poster session presented at the
8th Annual Conference of the Society for Industrial
and Organizational Psychology, Dallas, TX.
Hunter, J. E. (1981).
False premises underlying the
1978 Uniform Guidelines on Employee Selection
Procedures: The myth of test invalidity
. Paper
presented to the Personnel Testing Council of
Metropolitan Washington, Washington, DC.
Hunter, J. E., & Schmidt, F. L. (1990). Methods of
meta-analysis: Correcting error and bias in research
findings. Thousand Oaks, CA: Sage.
Judge, T. A., Colbert, A. E., & Ilies, R. (2004). Intelli-
gence and leadership: A quantitative review and
test of theoretical propositions.
Journal of Applied
Psychology, 89
, 542– 552.
Management Psychology Group. (2005).
Business
Check List technical manual
. Atlanta, GA: Author.
Ones, D. S., Dilchert, S., Viswesvaran, C., & Judge,
T. A. (2007). In support of personality assessment
in organizational settings.
Personnel Psychology,
60
, 995– 1027.
Ones, D. S., Dilchert, S., Viswesvaran, C., & Salgado,
J. F. (in press). Cognitive abilities. In J. L. Farr & N.
Tippins (Eds.),
Handbook of employee selection
.
Mahwah, NJ: Erlbaum.
Ones, D. S., & Viswesvaran, C. (2003). Job-specific
applicant pools and national norms for personality
scales: Implications for range-restriction correc-
tions in validation research.
Journal of Applied
Psychology, 88
, 570– 577.
Ones, D. S., Viswesvaran, C., & Dilchert, S. (2005).
Cognitive ability in personnel selection decisions.
In A. Evers, O. Voskuijl, & N. Anderson (Eds.),
Handbook of selection
, pp. 143–173. Oxford, UK:
Blackwell.
Ones, D. S., Viswesvaran, C., & Schmidt, F. L. (1993).
Comprehensive meta-analysis of integrity test
validities: Findings and implications for personnel
selection and theories of job performance.
Journal
of Applied Psychology, 78
, 679– 703.
Rubenzer, S. J., Faschingbauer, T. R., & Ones, D. S.
(2000). Assessing the U.S. presidents using the
revised NEO Personality Inventory.
Assessment, 7
,
403– 420.
Sackett, P. R., & Ostgaard, D. J. (1994). Job-specific
applicant pools and national norms for cognitive
ability tests: Implications for range restriction
corrections in validation research.
Journal of
Applied Psychology, 79
, 680– 684.
Salgado, J. F., Anderson, N., Moscoso, S., Bertua, C.,
de Fruyt, F., & Rolland, J. P. (2003). A meta-analytic
study of general mental ability validity for different
occupations in the European Community.
Journal
of Applied Psychology, 88
, 1068– 1081.
Wonderlic Personnel Test Inc. (1998).
Wonderlic Per-
sonnel Test and Scholastic Level Exam user’s man-
ual
. Libertyville, IL: Wonderlic Personnel Test, Inc.
at https://www.cambridge.org/core/terms. https://doi.org/10.1111/j.1754-9434.2009.01127.x
Downloaded from https://www.cambridge.org/core. University of Minnesota Libraries, on 04 Mar 2021 at 18:16:37, subject to the Cambridge Core terms of use, available