A process for selecting training methods

by Clive Shepherd


Do you sometimes feel there’s 'a madness in your methods'? How do you currently select from the bewildering array of training methods available? Stick with the devil you know? Try a bit of everything on the assumption that something's bound to work? Clive Shepherd believes that the selection of training methods is more complex than we might think and requires a thorough and systematic approach. Here he describes the model that he has developed to bring a 'method to the madness'.

Contents

The problem with decision making
Enter the tool
Defining the problem
What are the options?
Criteria for assessing alternatives
Ways of comparing methods
Making assumptions
Totalling up costs
Weighing up the options
The Training Methods Selector Tool


The problem with decision making

Back in the early 80s, I attended a course by Kepner Tregoe on Problem Solving and Decision Making. They taught us how to make a decision systematically: to define and weigh our objectives; to score each of our alternatives against these objectives; to multiply our scores by our weights and then review our results - a systematic analysis that would help to yield a conclusive result.

I set about trying this out. I was agonising over whether to buy the red sports car with a hard ride and no space or that comfy family saloon with lots of space and a body like armour. I entered in my data, steam was coming out of the calculator, I anticipated an end to my indecisiveness. And then the scores came out. 247 points ... to each option!. I vowed from that point on to go with my intuition. No more analysis paralysis. Which car did I buy? Guess.

Of course we make decisions all the time, perhaps hundreds a day, without stopping to think. We can cope with some quite thorny problems without resort to a committee, a computer or even a pen and paper. But every now and then we have a major decision to make, where there are so many factors to consider, so many seemingly conflicting criteria, so many permutations. In those circumstances you really do need a thorough and systematic approach.

In my work over the past 18 years as practitioner, purveyor and would-be prophet of technology for training, I have time and time again been asked to justify why videodisc, CD-ROM, web-based training or whatever, should take over from the so-called ‘traditional methods’. Everyone’s looking for a clear-cut answer. However, as someone who’d spent a good deal of their life using traditional methods and finding them pretty effective, at least some of the time, I found it hard to give a black and white response. It depends on this, that, the other and more besides. It’s a complex decision.

The fact is, selecting training methods can be hard, particularly when there’s a lot of trainees, a lot of money and a lot of your job at stake. That’s why I decided to repent my sins and develop a tool for selecting training methods.

Enter the tool

As someone who prefers working on screen to scrappy bits of paper, I turned to the computer to develop the tool. This proved to be a good choice as the model became increasingly complicated, with a lot more calculations than you would want to do by hand. However, the principles behind the model are simple enough and many of you will find that you can work with it without making a further contribution to Bill Gates’ fortune.

The tool is designed for anyone who is responsible for finding solutions to training problems and who’s interested in undertaking a thorough review of the available training methods. The tool will be most useful when the training problem is complex, when there are many alternative options or when you wish to critically examine your assumptions.

Defining the problem

The first step in the analysis is predictable – you need a clear statement of the problem you are trying to address:

• the name of the overall training programme
• the specific target audience
• the broad learning objective

Don’t try to use the tool to analyse more than one audience or objective - if you have a complex programme to plan, the best way of dealing with it is to break it down into its elements and deal with them one by one. However, if you want to get home this weekend, don’t be too fine-grained in the objectives you analyse. There’s a balance to be struck.

To get a feel for the scale of the problem, you should also make an estimate of the numbers to be trained and the date by which this must be achieved – both of which can have a profound influence on the selection you make.

What are the options?

There are, of course, any number of possible training methods. My model works with twelve, divided into three categories:

Self-instruction:

Web-based with on-line support: here the training is delivered over an intranet or the Internet, with a facility for remote interaction with a tutor or subject-matter expert. Because bandwidth is normally limited, you have to assume that video and audio would not be available.

Stand-alone multimedia PC: in this case, the training is delivered off the hard disk or, more likely, CD-ROM, with audio and video capabilities.

Multimedia PC / on-line hybrid: this combines the multimedia capabilities of CD-ROM with the potential for updates of volatile information, collaboration and tutorial support that can be achieved through the on-line connection.

Multimedia PC in small groups: this is the same as the second example above, but in this case the training is delivered to a small group.

Workbook alone: the training is delivered through a workbook, with support from no other media.

Video and workbook: As above, but with the aid of a video cassette (or TV broadcast).

Audio and workbook: The workbook is supplemented by audio from tape, CD or radio.

On-job instruction:

One-to-one instruction: here the training, coaching or mentoring is carried out for a single trainee.

Instruction in small group: As above but with a number of trainees.

Classroom training:

Class with only basic visual aids: this is the standard classroom situation with the normal resources such as flip charts, OHPs, white boards, etc.

Class with full audio-visual support: in this case the classroom is equipped with facilities such as a video projector connected to a PC as well as TV or video.

Class with a/v & actual equipment: the difference here is that the equipment used in the workplace - PCs, machinery, etc. - is available in the classroom for trainees to practise on.

Criteria for assessing alternatives

The pressures on a training manager from the target audience, his or her own priorities and those of the boss, do vary enormously. Sometimes the quality of the learning outcome is the over-riding criterion, on other occasions time is ‘of the essence’.

The model allows you to apply weights to the criteria that are used to compare the various methods, according to your own priorities. At the bottom end, a weighting of zero means that this issue is ignored in the overall score, whereas a weighting of four means that the issue has four times its normal effect on the outcome. There are five criteria:

Direct cost: the cost of trainee travel plus the purchase of services, facilities, materials or equipment specifically for the training.

Indirect cost: the cost of using existing labour and facilities plus the cost of trainees being away from work to travel to and receive the training.

Efficiency: the time taken to deliver the training.

Effectiveness: the degree to which the training is likely to achieve its learning objective.

Mix: the degree to which trainees get to mix and make contacts with eachother. This might seem to be of lesser importance to the other criteria, but I have included it because of the number of times senior managers have specified it as an important secondary reason for doing the training.

Ways of comparing methods

As we have seen, the criteria to be used for evaluating alternatives can vary enormously. On top of this, the differences between the various methods are numerous and quite subtle. As a result, there is a wide range of questions that has to be answered to specify exactly what the situation is and how the various alternatives match up.

Population variability

The first area to address is the variability within the population. Ask yourself (against a scale from 0 to 4) to what extent the trainees ...

• vary in terms of their current level of knowledge or skill of the subject
• vary in their aptitude for achieving the required knowledge and skills
• vary in terms of the learning they will need

The implication here is that high variability suggests an individualised or maybe a small group approach, where trainees can work at their own pace and the material can be organised in a more modular fashion.

Population preferences

Of course trainees come with the own prejudices, experiences and learning styles. Their preferences do matter. Using the 0-4 scale, ask yourself whether they tend to prefer to learn on their own, to receive individual instruction or to learn in a group.

Nature of the learning to be achieved

Often the potential effectiveness of a training method is governed by the nature of the learning objective. Ask yourself, using the same 0-4 scale, to what extent the learning is likely to be more effective if ...

• trainees can be away from job pressures for a continuous period
• trainees can interact with other trainees
• the training is undertaken in short sessions
• there are opportunities for live demonstration and practice
• trainees can receive the training at their own pace
• trainees can easily backtrack and review any part of the training
• the training is delivered consistently on every occasion
• trainees are able to ask questions of a subject matter expert

Media requirements

Another issue to consider is the requirement for specific media within the training process. First ask yourself, using the 0-4 scale, the extent to which each of the following are required to present the training effectively: text, voice, body language, still graphics/ photos, the actual equipment used on the job and animations/video sequences.

Then look at the extent to which the following media facilities are required for trainees to practise the required skills effectively: voice recording, video recording, computer simulations, the use of actual equipment.

Logistical issues

Practical considerations are also important. On the 0-4 scale, ask yourself to what extent …

• it is difficult to gather adequate numbers at any one time for classroom training
• it is difficult for trainees to travel to a central training site
• it is expensive for trainees to travel to a central training site
• it is difficult for trainees to be released for sufficient time to do the training in one go
• the training content will need to be regularly updated

Available resources

It’s rare to have to make a decision on training methods without any history of previous training. There are usually resources at our disposal, of one sort or another, that we can apply to the new situation in order to bring the costs down.

People
How many experienced personnel do you have in-house in the following categories: training designers, developers of CBT materials, developers of a/v materials, on-job instructors/coaches, classroom training presenters?

Equipment
How many of the following items of equipment do you have available: intranet-enabled PCs, multimedia PCs, video players and monitors, audio cassette or CD players?

Facilities
How many classrooms do you have with basic visual aids, with full audio-visual support or with actual equipment for each trainee to practise with.

Existing materials
To what extent do you already have suitable materials available: training designs, audio-visual materials, computer-based training materials?

Off-the-shelf solutions
Of course there is always the option of buying an off-the-shelf solution in preference to creating one in-house or having it created for you. You need to know if suitable products are available off-the-shelf and what they would cost per trainee in each of the following categories:

• computer-based training materials
• stand-alone workbooks
• videos plus workbook
• audio cassettes/CDs plus workbook
• classroom course materials
• externally-run classroom courses

Making assumptions

There’s a number of assumptions that need to be fed into the black box before it can fully digest what it knows about your training situation:

Labour costs: the average cost per hour of internal and external labour.

Design and development times: the time in hours required to design one hour of training and to develop each hour of CBT, audio and video.

Cost of new equipment: the cost of intranet-ready PCs, multimedia PCs, VCRs with monitors and audio cassette or CD players.

Class and group sizes: the size of a typical small training group and the size of a typical class.

Cost of classroom facilities: cost per hour, both internal and external of a basic classroom, a classroom with a/v equipment and a classroom equipped with actual equipment.

Travel and accommodation: the average cost of travel and accommodation and the average time taken to travel to a training event.

The final set of assumptions is a little more contentious. There is a fair amount of evidence to show that individualised instruction is likely to be more efficient than classroom training in terms of the time needed to complete the training. This is because self-pacing,  the structuring of the material into modules and the intensity of interaction means that all of the learning time is productive. The model assumes the following efficiencies over classroom training, although these can be over-ridden:

• 50% for self-instructional methods
• 40% for one-to-one instruction
• 25% for small groups (CBT or on-the-job instruction)

Totalling up costs

The black box can now be left to calculate the direct and indirect costs for implementing your training solution, for each of the twelve training methods. Based on the assumptions that you have made, and what you have specified about the numbers to be trained, the deadline and available resources, costs can be calculated under these headings: design, CBT development, audio development, video development, instructors, classrooms, travelling costs, costs of travelling time, hardware/equipment and off-the-shelf resources. There is insufficient space here to specify exactly how these calculations are made, but the process is purely arithmetic and involves no rocket science.

If you have specified that direct and indirect costs are the primary criteria by which you wish to measure the training alternatives, then this is as far as you need to go. The model will provide you with the following results:

£000	Indirect Cost	Direct Cost	Total
Self-instruction:
Web-based with on-line support	120	35	155
Stand-alone multimedia PC	124	35	159
Multimedia PC / on-line hybrid	124	35	159
Multimedia PC in small groups	174	35	209
Workbook alone	102	22	124
Video and workbook	110	22	132
Audio and workbook	108	23	131
On-job instruction:
One-to-one instruction	242	22	264
Instruction in small group	202	22	224
Classroom training:
Class with only basic visual aids	357	397	754
Class with full audio-visual support	361	397	758
Class with a/v & actual equipment	364	397	761
Weighting	2	1

Fig1: Sample results table showing cost comparisons

Weighing up the options

So far we’ve only been able to compare our training options against two criteria. The other three criteria – efficiency, effectiveness and mix – are not as quantifiable, so we need to make full use of the ratings we entered if we are to make a numeric comparison.

Many of the questions that we asked in comparing methods – population variability, population preferences, the nature of the learning to be achieved, media requirements and logistical issues – have obvious inferences that lead you towards or away from particular options. By assigning the ratings to each of the methods, it is possible for the model to obtain an overall percentage score for each method against each of the criteria. To complete the picture, it is also possible to translate the cost totals into percentages, so that the methods can be compared against all criteria in a consistent manner. The final task is to multiply each of these scores by the weightings that were given to each criterion and obtain a grand total percentage score for each method. Phew!

	Effic %	Effec %	Mix %	Total %
Self-instruction:
Web-based with on-line support	100	60	33	72
Stand-alone multimedia PC	80	65	0	65
Multimedia PC / on-line hybrid	100	69	33	75
Multimedia PC in small groups	53	62	50	59
Workbook alone	61	53	0	57
Video and workbook	61	62	0	60
Audio and workbook	61	58	0	59
On-job instruction:
One-to-one instruction	80	50	0	54
Instruction in small group	53	48	50	53
Classroom training:
Class with only basic visual aids	0	44	100	25
Class with full audio-visual support	0	48	100	26
Class with a/v & actual equipment	0	51	100	27
Weighting	3	4	1

Fig2: Sample results table showing percentage comparisons

The training methods selector tool

Having followed the workings of the model through to this point, you will appreciate how useful a computer is likely to be. This is particularly true when you want to carry out a series of analyses of different parts of a training programme or to conduct a number of ‘what ifs’ with the same basic data. The Training Methods Selector software gathers your inputs using standard form filling techniques, allows you to make your own assumptions (if you don’t want to accept the defaults) and then displays your results for printing out. It has been developed both as a Microsoft Excel spreadsheet and as a series of web pages that can be installed on an intranet.

Real life is never quite as clear cut as the algorithms required for computer software. No doubt the tool will initially be a blunt one. What's needed is feedback on how it helps in real situations, so it can be enhanced to be more useful to more people. Try the tool out. Any help you can give will be much appreciated.