Measuring before and after

Evaluating online learning

Evaluation - why bother?

What are we measuring?

Measuring reactions

Measuring learning

Measuring application

Measuring business results

Measuring return on investment

Measuring before and after

Measuring before and after
TO MEASURE THE EFFECT of training, you need to take each indicator in turn and generate before and after data – what the level was before the training and what it was afterwards. The difference between the two provides an indication of the effect of the training.

For example, sales in a certain region may be $10m a month leading up to a training programme and $12m a month after. The 20% increase in productivity could be attributed to the training and entered into the ROI analysis.

Control groups
It is not always so clear cut that training has been the cause of a change in a performance indicator. Say that a number of measures were taken simultaneously to cure a problem. How do you know which of the measures was responsible for any positive results? Similarly, at the same time as a training programme is run, there may be a completely unexpected downturn in a market, meaning that performance actually goes down after the training. How can you separate the effect of the training from the effect of the downturn?

In these situations, the interaction between these variables can be significantly reduced by the use of a control group. A control group is a subset of the target population that does not receive the training. By comparing the results of those who have received the training with those of the control group, you can separate the effect of the training on performance.

For example, here are some statistics for a particular performance indicator:

	Before	After	Change
Group receiving training	10	16	+60%
Control group	10	12	+20%
Effect of training			+40%

It can be seen from this table that performance has risen by 20% even without training. The increase in performance that can be attributed to the training is therefore 40%.

Where next?
Sometimes it seems that evaluation is becoming a show of machismo. "What level is your company up to? Oh dear, you mean you're still using happy sheets? We've just graduated to level 5 - soon we'll be measuring the effects of our training on the global economy."

It's true to say that evaluation does get trickier the more you move up the levels. It's also true to say that financial impact is an increasingly important issue. Remember that all levels of evaluation are important for different reasons, but that the only ones that count are those where you can produce valid, reliable results. Get evaluating - you may even find that your online learning's been a success.

END

MORE FEATURES | SINGLE PAGE VERSION