the success of training
EVALUATION OF TRAINING, like motherhood and apple pie, is
inherently a good thing. But, because short term priorities always crowd out their longer
term competitors, it's typically something we plan to do better next year - after all,
we've got away with it so far, so another year won't hurt!
And even if training evaluation is undertaken, it is usually at the easiest
and lowest level - the measurement of student reactions through happy sheets. Reactions
are important and the happy sheets serve a purpose, but will they be enough to back up
your arguments when there is a need for a greater investment in training, when major
changes need to be made in direction, when there is stiffer competition for resources,
when times get tough?
Why evaluate training?
Let's summarise the main arguments for better evaluation of training:
To validate training as a business tool
Training is one of many actions that an organisation can take to improve its
performance and profitability. Only if training is properly evaluated can it be compared
against these other methods and expect, therefore, to be selected either in preference to
or in combination with other methods.
To justify the costs incurred in training
We all know that when money is tight, training budgets are amongst the first to be
sacrificed. Only by thorough, quantitative analysis can training departments make the case
necessary to resist these cuts.
To help improve the design of training
Training programmes should be continuously improved to provide better value and
increased benefits for an organisation. Without formal evaluation, the basis for changes
can only be subjective.
To help in selecting training methods
These days there are many alternative approaches available to training departments,
including a variety of classroom, on-job and self-study methods. Using comparative
evaluation techniques, organisations can make rational decisions about the methods to
Criteria for measuring training success
The form of evaluation that we undertake is determined by the criteria that
we choose, or are told to use, to measure success:
One way of measuring the success of training is the good old bums on
seats. Although by no means a true measure of the effectiveness of training, student
numbers do reflect the fact that the training is addressing a need and that the design and
methodology is meeting expectations.
Direct costs are those costs that are incurred directly as a result of a training
programme external design and development, consultancy fees, travel expenses and so
on. If the programme did not take place, these costs would not be incurred. Many
organisations only ever take direct costs into consideration when measuring training
Indirect costs are costs that may or may not be directly associated with a training
event, but which would have been incurred anyway, whether or not the training took place.
Examples are salaries of in-house trainers and students and the costs of rooms and
equipment. Any analysis of the true costs of training will include both direct and
Efficiency is a measure of the amount of learning achieved relative to the amount of
effort put in. In practical terms this means the amount of time it takes to complete a
piece of training. Efficiency has a direct relation to cost the more efficient a
training method is, the less it will cost.
Performance to schedule
Sometimes with a training programme, time is of the essence the
training needs to be completed by a given date if a particular business objective is to be
achieved. In these situations, the extent to which a training programme performs to
schedule is a critical measure of success.
If you are a training provider operating externally to a client organisation, then
income received is a vital measure of your success. Its the financial equivalent of
bums on seats the more courses you run or places you fill, the greater
the benefit. Some internal training providers may also cross-charge their clients,
although, because this correspondingly increases the cost to the organisation, this is not
regarded as a benefit when assessing return on investment.
The extent to which trainees mix
A justification often made for training, particularly group events, is that it
provides an opportunity for students who work in different departments or regions to meet
with each other, share experiences and make contacts. Because this is a valued outcome of
training, it needs to be considered when comparing training methods. Similarly, some
training may be regarded as a perk, a benefit of some value, even if this is not directly
related to learning.
Reactions are what you measure with the happy sheet. Reactions are
important because, if students react negatively to your courses, they are less likely to
transfer what they learned to their work and more likely to give bad reports to their
peers, leading in turn to lower student numbers.
Learning, in terms of new or improved skills, knowledge and attitudes, is the primary
aim of a training event. Learning can be measured objectively using a test or exam or some
form of assessed exercise. If a student has to achieve a certain level of learning to
obtain a pass mark, then the number of passes may be used as an evaluation
measure. Another important aspect of learning is the degree of retention how much
of the learning has stuck after the course is over.
If a student has learned something from a course, you hope that this will be reflected
in their behaviour on the job. If a student employs what they have learned appropriately,
then their work behaviour will meet desired criteria. Behaviour can be measured through
observation or, in some cases, through some automated means. To assess behaviour change
requires that the measurements are taken before and after the training.
If, as a result of training, students are using appropriate behaviours on the job,
then you would expect that to have a positive impact on performance. A wide variety of
indicators can be employed to measure the impact of training on performance numbers
of complaints, sales made, output per hour and so on. It is hard to be sure that it is
training that has made the difference without making comparisons to a control group
a group of employees who have not been through the training.
Return on investment as a measure
Return on investment (ROI) is a measure of the monetary benefits obtained
by an organisation over a specified time period in return for a given investment in a
training programme. Looking at it another way, ROI is the extent to which the benefits
(outputs) of training exceed the costs (inputs).
ROI can be used both to justify a planned investment and to
evaluate the extent to which the desired return was achieved. However, it can not measure
all aspects of training success:
- whether students liked the training or not
- the numbers of students participating in the training
- the extent to which students' personal objectives were
The process of calculating ROI
To calculate ROI you must first make estimates or obtain measurements of
the costs and benefits associated with a training programme. As you will see, the
calculation of ROI is then a relatively simple process. Let's start with the costs