Self-Assessment of Training Impact at Work: Validation of a Measurement Scale

Abstract

This paper describes the development and validation of a measurement scale for Training Impact at Work
Impact is defined as the training long-term effect on work performance, motivation and/or attitudes. Two
weeks after training, a questionnaire with 12 impact evaluation items was applied to participants from 226
courses that had been offered by a Brazilian public organization. The 1.270 valid answers were submitted to
factor analyses (PAF, direct-oblimin) and to reliability analyses (Cronbach’s Alpha). A two sub-scale structure
was found (a=0.86; r=.56) accounting for 60 percent of the impact variability. A single factor structure was
also found and it is similarly reliable (a=.90), accounting for 45 percent of the variability. Both structures are
useful, reliable and valid

Full text

Interamerican Journal of Psychology
Sociedad Interamericana de Psicología
rip@ufrgs.br
ISSN (Versión impresa): 0034-9690
LATINOAMERICANISTAS
2004
Gardênia Abbad / Jairo Eduardo Borges Andrade / Lúcia Henriques Sallorenzo
SELF-ASSESSMENT OF TRAINING IMPACT AT WORK: VALIDATION OF A
MEASUREMENT SCALE
Interamerican Journal of Psychology, año/vol. 38, número 002
Sociedad Interamericana de Psicología
Austin, Latinoamericanistas
pp. 277-284
Red de Revistas Científicas de América Latina y el Caribe, España y Portugal
Universidad Autónoma del Estado de México
http://redalyc.uaemex.mx

277
ARTICULOS
R. interam. Psicol. 38(2), 2004
Revista Interamericana de Psicología/Interamerican Journal of Psychology - 2004, Vol. 38, Num. 2 pp. 277-284
This paper reports an experience of developing and
validating a measurement scale of Training Impact at Work.
This research has made possible the creation of a
psychometrically valid tool applicable in studies related
to the effects of training on individual performance.
According to Borges-Andrade and Abbad (1996) and Abbad
(1999), this sort of investigation is rare both in national
and foreign literature. Few attempts have been made in
connection with organizing measurements to determine
the level of training effectiveness involving trained
individuals. The lack of reliable, valid measurements of
learning, reaction (satisfaction with training) and training
impact at work may be one of the factors that have been
hindering research progress in the training evaluation area.
Training impact at work is one of the main criterion
variables of training evaluation models and corresponds to
the third level of evaluation in more traditional approaches
Self-Assessment of Training Impact at Work:
Validation of a Measurement Scale
Gardênia Abbad
1
Jairo Eduardo Borges-Andrade
Lúcia Henriques Sallorenzo
Universidade de Brasília, Brasil
Abstract
This paper describes the development and validation of a measurement scale for Training Impact at Work
Impact is defined as the training long-term effect on work performance, motivation and/or attitudes. Two
weeks after training, a questionnaire with 12 impact evaluation items was applied to participants from 226
courses that had been offered by a Brazilian public organization. The 1.270 valid answers were submitted to
factor analyses (PAF, direct-oblimin) and to reliability analyses (Cronbach’s Alpha). A two sub-scale structure
was found (a=0.86; r=.56) accounting for 60 percent of the impact variability. A single factor structure was
also found and it is similarly reliable (a=.90), accounting for 45 percent of the variability. Both structures are
useful, reliable and valid.
Keywords: Labour; program evaluation; measurement; test construction.
Auto-Avaliação de Impacto do Treinamento no Trabalho: Validação de uma Escala
Resumo
Este artigo descreve o desenvolvimento e a validação de uma escala de avaliação do Impacto do Treinamento
no Trabalho. Impacto é definido como o efeito do treinamento a longo prazo no desempenho, motivação e/ou
atitudes. Duas semanas após o treinamento, foi aplicado um questionário com 12 itens de avaliação de impac-
to, em participantes de 226 cursos oferecidos por uma organização pública brasileira. As 1.270 respostas
válidas foram submetidas à análise fatorial (PAF, direct-oblimin) e de confiabilidade (Alpha de Cronbach).
Dois tipos de estruturas fatoriais foram encontrados: uma com duas escalas (a=0,86; r=0,56) e outra unifatorial
(a=0,90) explicando, respectivamente, 60% e 45% da variabilidade de impacto. Ambas as estruturas são úteis,
confiáveis e válidas.
Palavras-chave: Trabalho, avaliação de programa; medidas; construção do teste.
such as those of Kirkpatrick (1976, 1977) and Hamblin
(1978). Job impact is an indirect result of training, and it
is defined as the effect of training on the trainees’
performance, motivation and/or attitudes.
Kirkpatrick’s (1976, 1977) and Hamblin’s (1978)
approaches suggest that criterion variables such as reactions,
learning, job performance (impact) and results (or
organization change and final value) keep a highly positive
relation among them. However, according to Alliger and Janak
(1989), specialized literature on training evaluation has shown
that those relations are not always significant or not always in
the direction foreseen by such approaches. Research results
have revealed situations in which a participant - despite showing
contentment (a favorable reaction) with training and having
obtained good scores in learning evaluations - would not
apply his or her newly acquired skills at work. In this case,
such a trainee has failed to produce a positive training
transfer, not necessarily because of poor memory, poor
retention capability, and poor generalization, or due to
shortcomings on the training program, but because
opportunities to put the things he or she has learned into
practice in the work environment have been missing.
1 Address: SQN 205, Bloco C, Apt. 201 - 70843-030 Brasília, DF, Brasil.
Telephones: (55-61) 272-0043 or 307-2625. Ext.: 222. Fax: (55-61) 347-
7746. E-mail: gardenia@unb.br, gardenia@brturbo.com

278
ARTICULOS
R. interam. Psicol. 38(2), 2004
GARDÊNIA ABBAD, JAIRO EDUARDO BORGES-ANDRADE & LÚCIA HENRIQUES SALLORENZO
Researchers have given little attention to the evaluation
of assumptions related to Kirkpatrick’s model (1976,
1977) and have neglected the importance of doing further
studies on the interrelation of the criterion variable.
Goldstein (1991) has defended the idea that reaction
measurements are of little avail as learning predictors and
make sense only when related to training needs.
Furthermore, according to Goldstein as well as
Tannenbaum and Yukl (1992), learning should be
considered only as a needed - although not sufficient -
condition for transfer or impact at work.
Results of recent research presented by Tannenbaum
and Yukl (1992) have not confirmed the significant relation
between learning and transfer (impact) measurements as
well as between reaction measurements and other criteria
(learning, impact, and results). Some results have shown,
however, that the reaction level played a moderating role
on the motivation/learning relation. In view of this,
instructional program efficiency should not be determined
by evaluating only one variable level, as it has commonly
been done in the area.
Such findings related to the weak relationships between
reaction, learning, and impact are not conclusive. Few attempts
have been made to identify predicting variables of each of these
evaluation levels and to build up reliable measurements of
different evaluation levels have been made. In this context, the
purpose of this paper is to present a reliable measurement of
training impact at work which can make possible a careful and
more precise evaluation of relationships among criterion
variables of training effectiveness.
Method
This article describes the development and validation
of a Training Impact at Work Scale. The tool has been
developed in three stages – development, semantic
validation, and statistical validation. The Training Impact
at Work Scale described in this paper is part of the data
collecting tools used by Abbad (1999) in a research effort
analyzing 226 courses from March 1997 to June 1998 at a
training agency in a Brazilian public organization.
Developing and Semantically Validating the Scale
Training Impact at Work is an indirect result of training
and it is defined as the effect of training on trainees’
performance, motivation and/or attitudes. The definition
of impact adopted in this paper, as well as the working up
of the measurement tool in this construct, has been highly
influenced by the methodological and theoretical features
of the works of Basarab and Root (1992), Leitão (1996),
Lima, Borges-Andrade, and Vieira (1989), Noe (1986),
Noe and Schmitt (1986) and Paula (1992), from whom
the questions have been taken.
Training Impact at Work has been measured by means
of self-assessment at two different moments - two weeks
and two months after training. On both occasions, the
questionnaires had the same dimensions, items, and scales,
and their differences were in connection only with the
orientations and time of application. A Green Format
questionnaire was sent to participants two weeks after the
end of the course, while a Blue Format was sent three
months after the end of the course.
The aforementioned questionnaires contained - in their first
part - items related to Training Impact at Work, and also items
about Situational Supporting Factors, Material Support to
Transfer of Training, and Consequences Associated to the Use
of Newly Acquired Skills at Work. The last three parts,
belonging to the category called Supporting Transfer, are not
the focus of this paper. Furthermore, the questionnaires
included also instructions for filling in, deadline, and guidelines
on how they should be returned to the training agency. Impact
evaluation items were associated to a 5-point Likert scale where
1 corresponded to I totally disagree with this statement and 5
to I totally agree with this statement. The questionnaires were
submitted to semantic validation in a sample of 14 individuals
that was similar to the one of training participants.
Statistical Validation Procedures
The 12-item questionnaire on Training Impact at Work
(IMTT) was applied on two different occasions to a sample
of workers from the organization that has served as a
training agency headquarters - two weeks and three months
after training. Before completing the Green Format and
the Blue Format questionnaires, sample participants had
already completed two other questionnaires. The only
differences between the Green and the Blue formats were
related lo guidelines, color and time of application. The
12 impact items, however, were the same.
Two thousand nine hundred and nine Green Format
IMTT questionnaires - and the same number of Blue Format
questionnaires - were sent via internal mail to all workers who
had participated in training activities. Directions required
participants to return their fully completed questionnaires to
the training agency within no later than 15 working days.
The validation sample for the Training Impact at Work
scale included workers who attended 171 courses during
1997 as well as 57 courses during 1998. Those courses
were divided in specific areas as follow: Auditing - 25, Law
Studies - 20, Public Administration - 10, Entrepreneurial
Administration - 2, Organizational Management - 65,
Expression and Communication – 7, Computer Technology -
80, International Studies - 3, Administrative Activities - 6,
Supervising - 1, and Grammar and Writing - 8. The average
number of participants was 18.4 (SD=5.77). Small classes
consisted of 6 workers; large ones consisted of 32. The
total average of class hours was 18.03 (SD=0.36), and the
daily amount was 3.04 hours (SD=0.19).

279
ARTICULOS
R. interam. Psicol. 38(2), 2004
SELF-ASSESSMENT OF TRAINING IMPACT AT WORK: VALIDATION OF A MEASUREMENT SCALE
More than half of the sample (55.7%) were males,
74.5% of them were middle-aged (30 to 50 years of age)
and most of them (74.1%) had an university degree.
Regarding functional characteristics, the sample had a
balanced structure as to job position (55.9% had a job
position due to an university degree; the remaining ones
were high school graduates). For most of the sample
(65.5%), seniority ranged from one to ten years.
Firstly, as Tabachnick and Fidell (1996) recommend,
data obtained through questionnaires were submitted to
descriptive and exploratory statistical analyses. Secondly,
they were submitted to an analysis of the principal
components, a factor analysis (PAF, oblique rotation - direct
oblimin), as well as an analysis of their internal consistency
(Cronbach’s Alpha). Such analyses have allowed examining
the data empirical structure, validating the questionnaires’
content and evaluating their reliability.
Results
This section presents the results gathered from the
statistical validation process of the Training Impact at Work
questionnaires, for both formats.
Training Impact at Work Scale Validation - Green Format
The return rate of the Training Impact at Work - Green
Format questionnaire was approximately of 43.66%. This
corresponds to a final sample of approximately 1,270 valid
answers per item. Answers were submitted to an analysis
of their principal components in line with Kaiser’s method
– eigenvalue > 1. As verified by KMO (.93), the data
matrix could be factorized. Figure 1 shows the scree plot
containing the scale principal components.
Results issued after analyzing the principal components
reveal an empirical structure consisting of two highly
correlated factors (r=.56), which explain 60.06% of total
variance in answers. Such component items refer to
training impact on performance and attitudes. However,
taking the scree plot (Figure 1) and the high magnitude of
bivariate correlation among components, they suggest that
the scale has a predominantly unifactorial structure. For
this reason, data were analyzed one more time by using the
principal axis method (PAF, oblique rotation - direct oblimin),
thus forcing the solution into one factor only. The scale has
also been submitted to a reliability analysis (Cronbach’s Alpha).
A summary of such results can be found in Table 1.
Table 1 shows the results of such analyses in terms of
items, descriptive statistics, factorial loadings and the
Cronbach’s Alpha value. A high level of reliability has been
obtained and the items express the participant’s perception
of effects produced by training on performance and
motivation at work. This factor explains 45.15% of the total
variance in answers.
Figure 1. Principal components of the training impact
at work - Green Format Scale
Table 1
Empirical Structure of the Training Impact at Work - Green Format Factor (
α
=.90)
Code
Imp6
Imp3
Imp5
Imp9
Imp8
Imp7
Imp10
Imp1
Imp11
Imp2
Imp12
Imp4
Description of Items
The quality of the work I do has improved.
I make fewer mistakes at work.
I do my work faster.
My self-confidence has increased.
My motivation for working has improved.
The quality of the work I do has improved in tasks not related to the course.
I suggest more frequent changes in work routine.
I often make use of skills learned during training.
I feel more receptive to changes.
I take advantage of opportunities to practice my newly acquired skills.
My workmates can learn from me.
I can remember well the course content.
Loading
0.79
0.76
0.75
0.75
0.68
0.67
0.66
0.64
0.63
0.62
0.61
0.42
M
3.67
3.61
3.63
3.56
3.67
3.15
2.97
3.46
3.27
4.10
3.09
3.89
SD
1.00
1.10
1.00
1.10
1.10
1.10
1.10
1.20
1.10
0.90
1.10
0.90

280
ARTICULOS
R. interam. Psicol. 38(2), 2004
In a second phase, the data matrix was analyzed, thus
forcing the analysis solution (PAF) in two factors, as it is
suggested by the scree plot, format (see Figure 1). These
two components explained 60.06% of total variance in
answers and were highly correlated (r=.67). The first scale
(where α=.86) refers to training impact on performance
and consists of 7 items (Imp3, Imp1, Imp6, Imp2, Imp5,
Imp4, and Imp12), while the second scale refers to training
impact on attitudes (where α=.86) and consists of 5 items
(Imp11, Imp9, Imp8, Imp10, and Imp7) (Table 1). The two
solutions reveal that both the unidimensional structure and
the bidimensional one are useful, reliable, and valid.
Therefore, depending on the evaluation purposes, they can
be used altogether or separately.
Training Impact at Work Scale Validation - Blue Format
The answers provided by approximately 710 trainees to
evaluation items on the Blue Format were submitted to the
same statistical validation process used with the Green Format.
As verified by KMO (0.94), the data matrix could be factorized.
The results of the analyses of the principal component
indicate an empirical structure with two highly correlated
factors (r=.60), which explain 65.58% of total variance in
answers. However, taking the scree plot (similar to Figure
1) as well as the high magnitude of bivariate correlation
among components, a predominantly unifactorial structure
can be observed. For this reason, data have been analyzed
in two stages, by using the same techniques described in
connection with the Green Format.
GARDÊNIA ABBAD, JAIRO EDUARDO BORGES-ANDRADE & LÚCIA HENRIQUES SALLORENZO
The results obtained from the analyses have shown that
the empirical structure of the Blue Format is, when taken
alone, almost identical to the one obtained with the Green
Format. Table 2 presents the Blue Format Impact empirical
structure and descriptive statistics.
It can be noted from Tables 1 and 2 that the internal
consistency rate of items increased and that the relative
position of some of them changed in scale when the second
questionnaire was applied. This is true, for example, for
items “Imp7” and “Imp10”, whose relative importance was
increased, and items “Imp1” and “Imp8”, whose importance
decreased when the second questionnaire was applied.
In a second stage, the data matrix was analyzed, and the
analysis solution (PAF) was forced in two factors, as
suggested by the scree pIot format. These two components
explained 59.02% of total variance in answers and kept a
high magnitude correlation (r=.71) among them. The first
scale (where α=.90) refers to training impact on performance
and consists of 7 items (Imp1b, Imp3b, Imp5b, Imp6b, Imp2b,
Imp4b, and Imp12b), while the second scale refers to training
impact in attitudes (where α=.89) and consists of 5 items
(Imp11b, Imp9b, Imp8b, Impl0b, and Imp7b) (Table 2). The
results confirm those obtained with the Green Format
questionnaire, showing that the unidimensional and
bidimensional empirical structures composing the Training
Impact at Work scale have seemingly been consistent and stable.
A matched-samples t test, confirming the stability of
measures for items constituting such structures, has
revealed that after three months Training Impact at Work
Table 2
Empirical Structure of the Training Impact at Work - Blue Format Factor (
α
=0.93)
Code
Imp6b
Imp3b
Imp5b
Imp9b
Imp7b
Imp10b
Imp2b
Imp11b
Imp8b
Imp12b
Imp1b
Imp4b
Description of Items
The quality of the work I do has improved
I make fewer mistakes at work
I do my work faster
My self-confidence has increased
The quality of the work I do has improved in tasks not related to the course
I suggest more frequent changes in work routine
I take advantage of opportunities to practice my newly acquired skills
I feel more receptive to changes
My motivation for working has improved
My workmates can learn from me
I often make use of skills learned during training
I can remember well the course content
Loading
0.83
0.80
0.80
0.78
0.72
0.72
0.71
0.71
0.69
0.68
0.66
0.51
M
3.64
3.58
3.56
3.49
3.10
2.93
3.99
3.23
3,64
3.07
3,45
3.71
SD
1.07
1.13
1.10
1.14
1.10
1.08
0.93
1.1
1,05
1.12
1,22
0.94

281
ARTICULOS
R. interam. Psicol. 38(2), 2004
SELF-ASSESSMENT OF TRAINING IMPACT AT WORK: VALIDATION OF A MEASUREMENT SCALE
self-evaluations were quite similar to the ones collected
two weeks after course completion. However, two
significant differences between such variable averages have
been found - the first being related to the item “I can
remember well the course content” (Imp4, t=5.41, df=548,
p=.005); the second being related to “I take advantage of
opportunities to practice” (Imp2, t=2.41, df=540, p=.016). In
both cases, averages associated with such variables were
significantly lower in the second application of the same items.
Discussion
The evaluation questionnaire for Training Impact at
Work presented in this paper can be used in research
involving the global effects of training on trainees’ behavior.
The 1 2-item scale is more comprehensive than the
bidimensional one with two sub-scales. Both are reliable
structures, and both can be applied, depending on research
purposes - to investigate the global impact of training on
participants’ behavior or on more specific impacts (on
participants’ productive performance or attitudes).
The unidimensional scale was used in a number of nation
wide projects (Borges-Andrade, Azevedo, Pereira, Rocha,
& Puente, 1999; Borges-Andrade, Gama, & Oliveira-Simões,
1999; see also Britto, 1999, e.g.). It had its empirical structure
preserved and its validity confirmed in different organizational
environments.
The present study has not validated a questionnaire for
heteroevaluations of training impact at work. Greater
objectivity and reliability of results would have been ensured
if similar questionnaires had been used with supervisors or
peers. Although the data collecting questionnaire has obtained
fine validity and reliability rates, it needs improvement.
When he evaluated the courses offered by three
Brazilian organizations with headquarters in the Federal
District – a financial institution, an airport administration
enterprise and a private telephone company – Meneses
(2002) and Meneses and Abbad (2003) compared the
empirical structure of self- and heteroevaluations of
training impact at work using a scale similar to the one
described in this study. The items in the heteroevaluation
scale applied to supervisors and peers differ from the self-
assessment items only by the facts that they were written
using the third person and not the first one.
Concerning heteroevaluations scales of training impact
at work, as seen by supervisors or peers, Meneses (2002) found
factorial structures, in his research, that were very similar to
the ones obtained in Abbad’s (1999) impact self-evaluations.
The 12 items included in Meneses (2002) heteroevaluation
scale of training impact explained 50% of the total variance of
answers and reached a confiability index of 0,91. Similar results
were issued from a different study led by Borges-Andrade
and cols. (1999) in Banco do Brasil, referring to statistical
validation of the impact heteroevaluation tool. The final
scale had the same number of items (12) and explained
56% of the total variance of answers obtained, with a
confiability index of 0,94. This is a demonstration of the
scale stability. Posterior studies would make a correlation
between self- and heteroevalutions of training impact at
work and test the validity of the tool by using external
criteria such as heteroevaluation or measurements of
individual performance at work.
The aforementioned described results of factorial
analysis have indicated that item “Imp4” (I can remember
well the course content) is the one showing the lowest
factorial loading in the selfevaluation scale of training
impact. Abbad (1999) has made stepwise multiple
regression analysis having trainees’ answers to such item
as a criterion variable and has found that most variables
explaining retention (ability to remember content) were
different from those found in training impact at work
models, thus indicating that content retention, despite being
correlated with impact, should not be mistaken with it.
Such differences in predicting the two variables - ability
to remember content and impact - should be interpreted in
the light of conceptual and methodological features
involving the development of the measurement
questionnaire used in this research. The first conceptual
issue refers to the use of the expression ability to
remember content in the questionnaire. In view of the way
the statement has been made, the ability to remember
content is neither a necessary nor a sufficient condition to
transfer of training. Participants, for example, may not
remember a content, but they may be able to put their newly
acquired skills into practice at work. In some cases,
participants can use them without the need to bring to mind
all the theoretical content learned in the classroom.
Therefore, computer technology learners, for example,
do not have to remember the software theoretical points
taught by teachers in order to be prepared to make correct
use of new procedures at work. In such cases, being exposed
to teaching is not needed even to learn. This is true, not
necessarily because imparted knowledge is not a formal
prerequisite to learn how softwares should be used, but because
the points that must be remembered are not verbal information
on procedures, but behaviors, steps to be taken, and strategies
involved in knowing how to complete the procedure.

282
ARTICULOS
R. interam. Psicol. 38(2), 2004
GARDÊNIA ABBAD, JAIRO EDUARDO BORGES-ANDRADE & LÚCIA HENRIQUES SALLORENZO
Conversely, participants may remember content, but they
may not know how to explain it. In such a case, not taking into
account an eventual lack of adequate instructional planning,
participants may be said to have learned what, but not how.
Such trainees do not know how to behave the prescribed way.
However, they can say what must be done.
In the context of transfer and impact, being able to retain
is knowing how to do something after training is completed.
Item “Imp4”, which is used to measure retention, does not
suggest the use of such concept This variable would be more
intensely correlated with impact, if the corresponding item
were asked about the participants’ ability to act in harmony
with effectiveness standards and criteria taught during the
course.
Referring again about the hypothetical example
involving a computer technology course, trainees who have
learned how to work out different tables, lines and fonts
by using a word processor must remember steps and results
of actions in order to reproduce them correctly in their
work environment. Here, the expression “ability to
remember content” does net seem adequate for two
reasons. The first is that the questionnaire statement seems
to refer only to trainees’ ability to say things about the
content, not to their ability to make evident performance
learned during training. The second problem involves the
idea beneath the object to be remembered. Item “Imp4”
seems to have made participants to focus on the content
of theoretical explanations.
In line with previously described conceptual analyses,
the aforementioned item should check if, after training,
participants still knew how to do the things taught during
the course. But it did not. Apparently, the item seems to
have limited the concept of retention to being able lo
remember things said by the instructor about course
themes.
For such reasons, the concept of retention in research
involving transfer of training and training impact at work
should be better defined or even revised. A major benefit
of redefining would be preventing confusions in the use of
such concepts Tracey, Tannenbaum and Kavanagh (1995),
as well as, Rouiller and Goldstein (1993), for example,
have confused retention with transfer. To measure transfer,
both researches used measures based on instructional
purposes, which, in a few instances, can measure - at the
utmost - skill retention. The researchers ensured retention
measure, but failed to include generalization and changes
that skills acquired during training can suffer in the work
environment.
Analyses have shown that retention is the direct
effect of training measured in terms of participants’
ability to perform actions learned during the course with
the same level of competence that was evident in learning
tests. Transfer of training is the application of such
abilities in the work environment, while training impact
at work involves measuring the effects of transfer of
training on participants’ performance and attitudes. Such
revision would make room for a better understanding
of cases in which retention and transfer were present,
but not for a tangible impact of training on work
performance.
In research in this area, learning and retention are
measures revealing participants’ performance in tests
evaluating the extent to which instructional objectives
have been reached, and they are usually applied at the
end of courses. On the other hand, an evidence of
retention would be noticed if, some time after training,
participants were submitted to a test similar to the one
applied at the end of the course and could obtain the
same scores or similar ones. To prove that transfer of
skill has taken place, participants should show that they
knew how to use their new skills in their work
environment, even in activities whose characteristics
and demands are different from tasks used to evaluate
short-term learning and retention. In this sense, transfer
of training is the direct result of training involving
maintenance (or long-term retention) and generalization,
as Baldwin and Ford (1988) have suggested. Transfer of
training, however, is not enough for training impact at work.
There are other factors - such as those related to the
psychosocial support to transfer that influence the
amplitude of training impact on participants’ performance
and attitudes.
Solely applying new skills at work is no guarantee
of visible effects on participants’ global performance
and attitudes. In order to provide evidence that training
has produced positive effects on their work, participants
should manifest relevant improvement in their products
or work processes as well as changes in their attitudes.
The dispositional nature of performance learned
during training would require a longitudinal assessment
of its manifestations, so that long-term observation
could be done in connection with the relation between
new and old skills as well as with articulations between
such abilities and the situations and environments
surrounding them. This study has made possible only
the analysis of training impact in two transversal
samples. In the future, longitudinal sample research on
impact needs to be done and an analysis of how

283
ARTICULOS
R. interam. Psicol. 38(2), 2004
SELF-ASSESSMENT OF TRAINING IMPACT AT WORK: VALIDATION OF A MEASUREMENT SCALE
knowledge acquired during training articulates with the
individual’s previous experience should be made in
order to ensure better performance. It seems also
necessary to investigate possible unfavorable impacts
of new knowledges on further performances of training
participants.
Bransford and Schwartz (1999) claim that traditional
studies in the area of transfer of training cannot explain
how a set of pre-existing abilities (the use of word
processors, for example) affect people’s ability to learn
a second set of similar abilities (the use of a different
word processor, for example), because, traditionally,
studies focus the effects of training solely on knowing
how and knowing what.
In this context, longitudinal studies would also
facilitate understanding the effects of training on
knowing what, knowing how and knowing with.
According to Bransford and Schwartz (1999), knowing
with refers to the set of previous knowledge and
experience accumulated by the individual from which
he or she perceives, interprets and judges present and
future situations. In this case, the different kinds of
knowledge would not be independent. Knowing how
would depend not only on the ability to interpret but
also on the way the individual uses his or her previous
experience (from knowing with). Also, for applying new
skills in a smart, efficient way, the individual should
develop metacognitive abilities to learn from new
experiences and to adapt to the environment in order to
be successful.
Additional studies need to be done to improve the
way training impact at work is measured. A research
agenda is suggested here and should include:
1. Developing and validating evaluation
questionnaires of training impact at work that may
contain specific items for long-term retention
evaluation (based on instructional objectives, for
example), general items for impact evaluation such
as those in the questionnaire that has been described
in this paper, in addition to these related to the
amplitude of training effects on performance and
attitudes.
2. Using heteroevaluations of training impact at work
aiming at comparing them to selfevaluations, as well
as examining eventual discrepancies or similarities.
3. Using a variety of data collecting methods
including interview and observation in longitudinal
investigations on the effects of training on the
individual’s behavioral repertoire at work.
References
Abbad, G. (1999). Um modelo integrado de avaliação do impacto do
treinamento no trabalho– IMPACT. Tese de Doutorado não
publicada, Universidade de Brasília. Brasília, DF, Brasil.
Alliger, G. M., & Janak, E. A. (1989). Kirkpatrick’s levels of training criteria:
thirty years later. Personnel Psychology, 42, 331-342.
Baldwin, T. T., & Ford, J. K. (1988). Transfer of training: a review and
directions for future research. Personnel Psychology, 41, 63-105.
Basarab, D. J., & Root, D. K. (1992). The training evaluation process.
Boston, USA: Kluwer Academic.
Borges-Andrade, J. E., & Abbad, G. (1996). Treinamento no Brasil: reflexões
sobre suas pesquisas. Revista de Administração, 31, 112-125.
Borges-Andrade, J. E., Azevedo, L. P. S., Pereira, M. H. G. G., Rocha, K. C.
P., & Puente, K. E. P. (1999). Impacto de treinamentos no trabalho: o
caso do Banco do Brasil [Resumo]. In Sociedade Brasileira de Psico-
logia (Org.), Resumos de Comunicação Científica, XXIX Reunião
Anual de Psicologia (p.53c). Ribeirão Preto, SP, Brasil: SBP.
Borges-Andrade, J. E., Gama, A. L. G., & Oliveira-Simões, J. T. (1999).
Impacto do treinamento no trabalho: um estudo de caso na Eletronorte
[Resumo]. In Sociedade Brasileira de Psicologia (Org.), Resumos
Comunicação Científica, XXIX Reunião Anual de Psicologia
(pp.53b-53c). Ribeirão Preto, SP, Brasil: SBP.
Bransford, J. D., & Schwartz, D. L. (1999). Rethinking transfer: a simple
proposal with multiple implications. In A. Iron-Nejad, & P. D. Pearson
(Eds.), Review of research in education (pp. 61-100). Washington,
DC, USA: American Educational Research Association.
Britto, M. J. P. (1999). Avaliação de impacto de treinamento na área de
reabilitação: preditores individuais e situacionais. Dissertação de
Mestrado não publicada, Universidade de Brasília. Brasília, DF, Brasil.
Goldstein, I. L. (1991). Training in work organizations. In Dunnette &
Hough (Eds.), Handbook of industrial and organizational
psychology (2nd ed., pp.507-619). California, USA: Consulting
Psychology.
Hamblin, A. C. (1978). Avaliação e controle de treinamento. São Paulo,
Brasil: McGraw-Hill do Brasil.
Kirkpatrick, D. L. (1976). Evaluation of training. In R. L. Craig (Ed.), Training
and development handbook (2nd ed., pp.18.1-18.27). New York, USA:
McGraw-Hill.
Kirkpatrick, D. L. (1977). Evaluating training programs: evidence vs. proof.
Training and Development Journal, 31(11), 9-12.
Leitão, J. S. S. (1996). Clima organizacional na transferência de treinamen-
to. Revista de Administração-USP, 31(3), 53-62.
Lima, S. M. V., Borges-Andrade, J. E., & Vieira, S. B. A. (1989). Cursos de
curta duração e desempenho em instituições de pesquisa agrícola.
Revista de Administração, 24(2), 36-46.
Meneses, P. P. M. (2002). Auto-eficácia, locus de controle, suporte à
transferência e impacto de treinamento no trabalho. Dissertação de
Mestrado não publicada, Universidade de Brasília, DF, Brasil.
Meneses, P. P. M. & Abbad, G. (2003). Preditores individuais e situacionais
de auto e heteroavaliação de impacto do treinamento no trabalho.
Revista de Administração Contemporânea, 7(Edição Especial), 185-
204.
Noe, R. A. (1986). Trainees’ attributes and attitudes: neglected influences
on training effectiveness. Academy of Management Review, 11(4),
736-749.
Noe, R. A., & Schmitt, N. (1986). The influence of trainee attitudes on
training effectiveness: test of a model. Personnel Psychology, 39,
497-523.

284
ARTICULOS
R. interam. Psicol. 38(2), 2004
GARDÊNIA ABBAD, JAIRO EDUARDO BORGES-ANDRADE & LÚCIA HENRIQUES SALLORENZO
Gardênia da Silva Abbad. B. Sc. and Psychologist by the University of Brasilia (UnB), Brazil and M. Sc. and
Doctor in Social and Organizational Psychology by the University of Brasilia (UnB), Brazil. She is a Full Professor
at the Dept. of Social and Work Psychology, Institute of Psychology, UnB, where she does research, teaching (at
the undergraduate, master and doctoral levels) and consulting services. Her areas of interest are learning in
organizations, training, and development and organizational behavior. E-mail: gardenia@unb.br,
gardenia@brturbo.com
Jairo Eduardo Borges-Andrade. B. Sc. and Psychologist by the University of Brasilia (UnB), Brazil, and M.
Sc. and Ph.D. in Instructional Systems, by The Florida State University, Tallahassee, USA. He is a Full Professor
at the Dept. of Social and Work Psychology, Institute of Psychology, UnB, where he does research, teaching (at
the undergraduate, master and doctoral levels) and consulting services. His areas of interest are: training and
development and organizational behavior. E-mail: jeborges@linkexpress.com.br
Lúcia Henriques Sallorenzo. Psychologist by the University of Brasilia (UnB), Brazil and M. Sc. in Social and
Organizational Psychology by the University of Brasilia (UnB), Brazil. She is a Full Professor at the Dept. of
Psychology, Catholic University of Brasilia (UCB), where she does research, teaching (at the undergraduate
levels). Her areas of interest are learning in organizations, distance training and development and organizational
behavior. E-mail: lucias@ucb.br
Paula, S. M. A. (1992). Variáveis preditoras de impacto de treinamento
no trabalho: análise da percepção dos treinandos de duas
organizações. Dissertação de Mestrado não-publicada, Universidade
de Brasília. Brasília, DF, Brasil.
Rouiller, J. Z., & Goldstein, I. L. (1993). The relationship between
organizational transfer climate and positive transfer of training. Human
Resource Development Quarterly, 4(4), 377-390.
Tabachnick, B. G., & Fidell, L. S. (1996). Using multivariate statistics (3rd
ed.). New York, USA: Harper & Row.
Tannenbaum, S. I., & Yukl, G. (1992) Training and development in work
organizations. Annual Review of Psychology, 43, 399-441.
Tracey, J. B., Tannenbaum, S. I., & Kavanagh, M. J. (1995). Applying
trained skills on the job: The importance of the work environment.
Journal of Applied Psychology, 80, 239-252.