Performance Flashcards
(Borman et al., 1993; Schmidt et al., 1986).
Previous research indicates that general ability measures are predictive of performance on all jobs, with general ability exerting its primary influence indirectly through job knowledge.
Coward and Sackett (1990)
conducted a definitive study involving 174 independent samples with a mean sample size of 210 (a database of 36,540 individuals). They found no evidence for nonlinear relationships between ability and performance
(Adler et al., 2016)
Supervisors and employees both dread performance
appraisals (as echoed by Murphy, 2019)
this article recaps the points made by the panelists who participated
in the debate. The arguments for eliminating ratings include these: (a) the disappointing interventions, (b) the disagreement when multiple raters evaluate the same
performance, (c) the failure to develop adequate criteria for evaluating ratings, (d) the weak relationship between the performance of ratees and the ratings they receive,
(e) the conficting purposes of performance ratings in organizations, (f) the inconsistent efects of performance feedback on subsequent performance,
The
arguments for retaining ratings include (a) the recognition that changing the rating
process is likely to have minimal efect on the performance management process as
a whole, (b) performance is always evaluated in some manner, (c) “too hard” is no
excuse for industrial–organizational (I-O) psychology, (d) ratings and diferentiated
evaluations have many merits for improving organizations, (e) artifcial tradeofs
are driving organizations to inappropriately abandon ratings, (f) the alternatives
to ratings may be worse, and (g) the better questions are these: How could performance ratings be improved
(Pulakos et al., 2015)
the great majority of appraisal systems in organisations are viewed as ineffective.
Performance management systems do not seem to fare much better; the conclusion that performance management is broken is shared among many researchers and practitioners.
For example, no solution
has been found to ameliorate the seemingly
intractable problem of leniency in ratings.
DeNisi & Murphy, 2017
Reviews of research on both performance appraisal and performance management noted that there is little if any evidence that these systems have any real impact on the performance or effectiveness of employees
Murphy (2019)
First, many stakeholders believe that PM is important and beneficial to measure
performance and to use that information to drive decisions. Second, it is widely believed that performance
feedback is valuable and that it helps to improve employee motivation and performance. Third, there does not
seem to be any clear alternative to the type of evaluation system most organizations use; virtually, every system
that has been proposed to replace traditional performance appraisal (e.g., performance management systems,
evaluation systems based on objective performance, and productivity measures) has fared as badly, if not worse.
RE: power law & why it makes appraisal systems lose their luster.
The stars
should be easy to spot without appraisal systems, and virtually, everyone else will be performing at such a lower level that differentiating
among these more average performers will be virtually pointless.
- WOW - Ufortunately, the people
who need and would benefit from performance feedback (e.g., poor performers) are often actively avoid feedback
(e.g., Moss et al., 2003; Moss, Sanchez, Brumbaugh, & Borkowski, 2009).
- WOW - Ufortunately, the people
There is considerable evidence that in order to work, performance feedback must
be accepted by recipients as fair and valid (Anseel & Lievens, 2009)
First, performance ratings, even when they are truly accurate, are often seen by recipients as unduly harsh. There
is extensive evidence (e.g., Campbell, Campbell, & Ho-Beng, 1998; Harris & Schaubroeck, 1988; Meyer, 1980; Thornton, 1980) that people view their own performance more favourably that do their supervisors, their peers, or other
external raters.
the tendency for people to view ratings they receive from others as unfairly low has been
identified by Murphy et al. (2018) as one of the principal structural sources of failure in performance appraisal systems.
Murphy et al. (2018)
discuss the “death spiral” of performance appraisal systems, describing ways in which disappointing experiences with
performance evaluation lead to higher levels of cynicism and disengagement, and show how these negative experiences feed upon themselves.
Yet, there is evidence that developmental feedback can be useful when the recipient is new to the
task or is a newcomer in the organisation (Li, Harris, Boswell & Xie, 201; Nurse, 2005; Reilly et al., 1996
Because the feedback coaches provide is
typically focused on learning and development rather than evaluation, rewards, and sanctions, employees may be
more receptive to this form of help and guidance than they are to traditional performance evaluations.
perf eval should not be done for more than one reason at a time!
Kluger and Nir (2010)
suggest using feedforward.
The feedforward interview focuses on (a) articulating what has gone well, by eliciting description of positive experiences from the target; (b) understanding how the strengths of the target and the context in which the event or
experience contributed to that positive experience; and (c) helping the target to apply those strengths and/or experiences to future challenges.
Bouskila-Yam and Kluger
(2011) proposed a fundamental reorientation of performance appraisal and feedback.
they
proposed that performance reviews should focus on what people do well and should work towards establishing goals
that are based on strengths rather than weaknesses.
Several surveys have shown that performance evaluations are most commonly used in organisations for two
purposes—that is, to provide information that can be useful for making training and development and to serve as
input for decisions about salary, promotions, layoffs, and dismissals
Cleveland et al. (1989)
suggested that there were four broad purposes
for evaluating job performance: (a) to make distinctions between individuals, such as identifying the best candidates
for salary increases or promotions; (b) to make distinctions within individuals, such as identifying individual strengths
and weaknesses for the purpose of determining training and development needs and priorities; (c) to support HR systems in organisations, such as validating personnel tests, evaluating the success, or training programmes; and (d) documentation, such as providing a record to support decisions such as promotions or dismissal.
O’Boyle & Aguinis (2012)
conducted 5 studies involving 198 samples including 633,263 researchers, entertainers, politicians, and amateur and professional athletes. Results are remarkably consistent across industries, types of jobs,
types of performance measures, and time frames and indicate that individual performance is not normally distributed.
When performance data do not conform to the normal distribution, then the conclusion is that the error “must” lie within the sample
not the population. Subsequent adjustments are made (e.g., dropping outliers) in order to make the sample “better reflect” the “true” underlying
normal curve.
he normal distribution has been used to model a variety of phenomena including human traits such as height (Yule, 1912) and intelligence
(Galton, 1889), as well as probability distributions (Hull, 1928), economic
trends such as stock pricing (Bronzin, 1908), and the laws of thermodynamics (Reif, 1965).
Ferguson (1947) noted that “ratings for a large and representative
group of assistant managers should be distributed in accordance with the
percentages predicted for a normal distribution” (p. 308)
The normality
assumption persisted through the years, and researchers began to not only
assume job performance normality but forced it upon the observed distributions regardless of the actual observed distributional properties. For
example, in developing a performance appraisal system, Canter (1953)
used “a forced normal distribution of judgments” (p. 456) for evaluating
open-ended responses. Likewise, Schultz and Siegel (1961) “forced the
[performance] rater to respond on a seven-point scale and to normalize
approximately the distribution of his responses”
Specifically, when performance scores deviate from normality, the cause is attributed to leniency bias, severity bias, and/or a halo error (Aguinis, 2009; Schneier,
1977).
Rating systems where most employees occupy the same category
with only a few at the highest category are assumed to be indicative of
range restriction and other “statistical artifacts” (Motowidlo & Borman,
1977).
Jacobs (1974) argued
that in sales industries (automotive, insurance, stock) performance is not
normal because a small group of incumbents who possess the expertise and salesmanship dominate activity.
Whereas a value exceeding three standard deviations from the mean is often thought to be an outlier in the
context of a normal curve (e.g., Orr, Sackett, & Dubois, 1991), a Paretian distribution would predict that these values are far more common
and that their elimination or transformation is a questionable practice.
There are important differences between Gaussian and Paretian distributions. First, Gaussian distributions underpredict the likelihood of extreme events. For instance, when stock market performance is predicted
using the normal curve, a single-day 10% drop in the financial markets should occur once every 500 years (Buchanan, 2004). In reality,
it occurs about once every 5 years (Mandelbrot, Hudson, & Grunwald,
2005)
Third, a key difference between
normal and Paretian distributions is scale invariance. In OBHRM, scale
invariance usually refers to the extent to which a measurement instrument
generalizes across different cultures or populations.
Germane to OBHRM in particular is that if performance operates
under power laws, then the distribution should be the same regardless of
the level of analysis. That is, the distribution of individual performance
should closely mimic the distribution of firm performance. Researchers
who study performance at the firm level of analysis do not necessarily
assume that the underlying distribution is normal (e.g., Stanley et al.,
1995).
if individual performance is found to
also follow a power law distribution, as it is the case for firm performance
(Bonardi, 2004; Powell, 2003; Stanley et al., 1995)
Of a total of 198 samples
of performers, 186 (93.94%) follow a Paretian distribution more closely
than a Gaussian distribution.
*Our results suggest that the distribution of
individual performance is such that most performers are in the lowest
category. Based on Study 1, we discovered that approximately 66% to 80% of performers in domains are in the lowest category/band for their appropriate performance metric.
Extending to the second standard deviation, the difference in productivity between
the 97.73rd percentile and median researcher should be four, and this additional output is valued at $22,652. However, the difference between the
two points is actually seven. Thus, if SDy is two, then the additional output
of these workers is $39,645 more than the median worker.
And leadership/mentoring of stars;
Thus, greater attention
should be paid to the tremendous impact of the few vital individuals.
Despite their small numbers, slight percentage increases in the output of
top performers far outweigh moderate increases of the many.
In addition to the study of leadership, our results also affect research
on work teams.
Future research:
We
may expect the group productivity to increase in the presence of an elite
worker, but is the increase in group output negated by the loss of individual output of the elite worker being slowed by non-elites? It may also be
that elites only develop in interactive, dynamic environments, and the isolation of elite workers or grouping multiple elites together could hamper
their abnormal productivity.
CWB
(i.e., harmful behaviors targeted at the organization or its members) has
always been assumed to have a strong, negative relation with the other two
components, but it is unclear if this relationship remains strong, or even
negative, among elite performers. For example, the superstars of Study 4
often appeared as supervillains in Study 5. Do the most productive workers
also engage in the most destructive behavior? If so, future research should
examine if this is due to managers’ fear of reprimanding a superstar, the
superstar’s sense of entitlement, non-elites covering for the superstar’s
misbehavior out of hero worship, or some interaction of all three.
JUICY:
inally, going beyond any individual research domain, a Paretian distribution of performance may help explain why despite more than a century
of research on the antecedents of job performance and the countless theoretical models proposed, explained variance estimates (R2) rarely exceed
.50 (Cascio & Aguinis, 2008b). It is possible that research conducted over
the past century has not made important improvements in the ability to predict individual performance because prediction techniques rely on means
and variances assumed to derive from normal distributions, leading to
gross errors in the prediction of performance
Random assignment will only balance the groups when the distribution of the outcome is normally distributed (when the prevalence
of outliers is low). In the case of Paretian distributions, the prevalence
of outliers is much higher.
. As a result, a single high performer has an
important impact on the mean of the group and ultimately on the significance or nonsignificance of the test statistic. Likewise, the residual
created by extreme performers’ distance from a regression line widens
standard errors to create Type II errors. Interestingly, the wide standard
errors and unpredictable means caused by extreme performers should result in great variability in findings in terms of both statistical significance
and direction. This may explain so many “inconsistent findings” in the
OBHRM literature (Schmidt, 2008).
Techniques
exist that properly and accurately estimate models where the outcome is
Paretian. Poisson processes are one such solution, and although not well
established in OBHRM research, they do have a history in the natural
sciences (e.g., Eliazar & Klafter, 2008) and finance. In addition, Bayesian techniques are likely to provide the greatest
applicability to the study of superstars.
The Matthew effect (Ceci & Papierno, 2005; Merton, 1968) states that those already in an advantageous
position are able to leverage their position to gain disproportionate rewards
Likewise, compensation systems such as pay for performance and CEO compensation are an especially divisive issue, with
many claiming that disproportionate pay is an indicator of unfair practices
(Walsh, 2008).
As
we described earlier, a Pareto curve demonstrates scale invariance, and
thus whether looking at the entire population or just the top percentile, the
same distribution shape emerges. For selection, this means that there are
real and important differences between the best candidate and the second
best candidate.
And the challenge of keeping superstars as other orgs want them!
In Studies 1, 2, 4, and 5, we found only one sample (NCAA rushing)
for which individual performance was better modeled with a Gaussian
distribution than a Paretian distribution.
Consider two measurement-related reasons for the potential better
fit of a Gaussian distribution. First, a measure of performance may be
too coarse to capture differences between superstars and the “simply adequate” (Aguinis, Pierce, & Culpepper, 2009). Specifically, in Study 3,
performance was measured as whether an official was elected or not, and
the measure did not capture differences among performers such as by how
many votes an individual won or lost an election.
supervisory ratings, are one of the most popular ways to operationalize
performance - Normality is introduced by the scale or rater evaluation training.
Now, consider three situations and reasons why the underlying performance distribution, not just observed performance scores, may actually fit
a Gaussian as opposed to a Paretian model. First, it may be the case that, in
certain industries and certain job types, superstars simply do not emerge.
For example, the manufacturing economy of the 20th century strove not
only for uniformity of product but also uniformity of worker. Quotas,
union maximums, assembly lines, and situational and technological constraints all constrained performance to values close to the mean.
However, industries and organizations that rely on manual labor, have limited technology, and place
strict standards for both minimum and maximum production are likely
to lead to normal distributions of individual performance. As we move
into the 21st century, software engineers, consultants, healthcare workers,
and educators make up an increasingly large part of the economy; but, for
the foreseeable future, farmers, factory workers, and construction crews
will continue to play an important role, and these types of jobs may best
be modeled with a normal distribution (e.g., Hull, 1928; Tiffin, 1947).
Second, research is needed on the deleterious
effects of superstars. For example, does the presence of a superstar demotivate other workers to such an extent that total organizational output
decreases.
When and how do these individuals reach the elite group? What is
the precise composition of this elite group—do individuals rotate in and
out of this group, or once in the top group, they remain in the top for
most of their career? What individual, group, and cultural factors predict
an individual’s membership in the top-performing group over time?
Murphy & Cleveland, 1995)
Already argued performance was not normally distributed.
Borman & Bush (1993)
Taxonmoy of manageria performance consisting of 18 dimensions such as planning & organization
Campbell (1993)
Effectiveness
Productivity: ration of effectiveness (output) to the cost of achieving that level of effectiveness (input) (Mahoney 1988)
Pulakos et al. (2000)
Taxonomy of ADAPTIVE performance:
In Study 1, over 1,000 critical incidents
from 21 different jobs were content analyzed to identify an 8-dimension taxonomy of adaptive performance. Study 2 reports the development and administration of an instrument, the Job Adaptability
Inventory,
The goal of this research is to develop a taxonomy of adaptive
job performance along the lines of the job performance model
developed by Campbell et al. (1993).
The 8 components found were:
Handling emergencies or crisis situations - appropriate & proper urgency
Handling work stress
Solving problems creatively - turning probs upsidedown & insideout
Dealing with uncertain
and unpredictable
work situations - not needing things to be black and white; refusing to be
paralyzed by uncertainty or ambiguity.
Learning work tasks,
technologies, and
procedures
Interpersonal adaptability
Cultural adaptability
Physically oriented adaptability
Katz (1964)
Extra-role performance behaviors that contribute to org goals but are outside of the person’s job description.
Podsakoff et al. (1997) found wee related to effectiveness at work.
OCBs
Smith et al., 1983
Organ (1997)
taxonomy of OCBs:
Altruism
Civic virtue: things that support the org and being a good representative outside.
Conscientiousness (thinking ahead)
Courtesy - asking others how a big project is coming along.
Sportsmanship - not complaining, even is justified.
Befort & Hattrup (2003)
More experienced managers seem to appreciate the contribution of OCBs to org performance more.