The following may be a wakeup call or reality check for some of you...if so I apologize ahead of time. In any case, there is little excuse for fraud, waste or abuse of authority. To consume an employers resources indiscriminately without a proven return is unethical at the least and criminal at the worst.
Recently, a professional associate asked me a question he had been trying to answer for some time. Because we shared the development of many answers during a major training development project, he thought I might have an answer. My friend wanted to know how a manager can determine whether or not a training program is paying off for the students and the employer. I am a long-time training developer, administrator and manager who has supervised up to 1300 people in project accomplishment and I have also been searching for a valid way to interest management in making their training investment pay off. The question answers itself IF management approaches the development of training in the same way a major capital project is attacked.
First the FACTUAL trainee performance objectives for the training program are defined by finding out what the trainees already know and comparing that to what they need to know after training. The difference between possessed and required knowledge/skill constitutes the training requirements. It is vital that the department/unit manager whose people are being trained and the manager responsible for the allocation of training dollars be present and participate in all steps of training development.
Training requirements are then grouped logically and converted into behavioral statements which indicate the action required, the standards of performance and any constraints or provisions (usually called givens) made for the trainee. When the training requirements have been restated as performance objectives which are meaningful, measurable and observable, they should be reviewed with workers, modified if required and then certified as the course content. The important things to consider here are not the needs of the entity but the needs of the trainee population to enable success in achieving the goals of the entity.
After extensive lead-time planning has established the validity of the need for training and the structure of the program, the program is administered and evaluated. If management and those to be trained have contracted the objectives of the program fairly and completely the program should assure some return on the investment. The problem is... few corporations are willing to invest in a TQA approach to training development and that is why there continues to be a question regarding ROI.
When the training is thoroughly planned to achieve results and the execution of the training is honestly evaluated, there has to be a return. The perception is widely held that training is just one more expensive evil that must be addressed during the conduct of business. Because there is little substance to the commitment, management lacks a clear idea of what training should be administered with the result being they never know if that training has had a payoff. Allow me to restate the most important underlying principle of assuring a ROI; "It is vital that the department/unit manager whose people are being trained and the manager responsible for the allocation of training dollars be present and participate in all steps of training development".
When Bill asked me my feelings on ROI, I felt kind of like the man who was asked the following question: "How do I get to Carnegie Hall?" The glib answer, "Practice, practice, practice", may or may not meet the needs of the questioner however it does offer one answer.
There is, in my experience and my philosophy of training investment and management, only one answer to the question and though simplistic in construct, it is nevertheless the only answer which meets the reality test. A corporate entity, which invests heavily in training, can be assured of the return on investment based on results, results and only results.
PRODUCTION
Does the training increase production? Show me the numbers before and after
training. Did the numbers stay at or near the level of initial improvement
or did they return to pre-training levels? Were the objectives structured
properly and achieved? Was the early analysis accurate? Did management
participate is determining the production goals and then assure that the
training supported those goals?
PROBLEM SOLVING
Was the training designed to correct a problem and did the problem diminish
or disappear after training? Did the problem soon reoccur within the same
group that received the training? Was the initial problem or group of problems
identified and clearly stated? Did the early analysis include all potential
sources of the problem and concurrent solutions in the form of performances
and competencies to be taught? Did management participate is determining
the solution and then assure that the training supported it?
BEHAVIOR MODIFICATION
Did HRD solve a personnel problem through behavior modification
and did that problem stay solved? Did the HRD problem return as though
it had never been addressed?
Did you send your folks away to some fancy, schmancy, high-dollar seminar and did they come back just feeling all warm and fuzzy or did they come back more competent at their work? Warm and fuzzy is short term and though it feels nice...SHOW ME SOME CONCRETE RESULTS. Easy to say,...but with behavior modification the changes are more internal and results more difficult to measure. Therein lies the problem... if the changes cannot be measured and there is no specific difference in the trainees personal associations or work production, what exactly took place?
Some of industry's largest chunks of HRD resources are spent on behavioral issues. I still maintain that the only CONCRETE way to determine a ROI is through results...if the course was intended to modify the trainees behavior, find a way to measure that AT WORK and not at the boondoggle seminar out-briefing. Even if there seems to be some change in the trainee... is it a long-term change or do the same old behaviors return? Don't tell me how big the budget was or how many specialists were involved in the development of your training program... DID IT GET THE JOB DONE?... well did it?
MOST IMPORTANT OF ALL...WERE THE COMPETENCIES PUT TO USE ON THE JOB AND RETAINED LONG-TERM OR IS IT OBVIOUS THAT THE TRAINING WILL HAVE TO BE REPEATED NOT ONCE BUT MANY TIMES?
If your managerial instincts indicate that a small, medium, or capital investment-level training project is needed, your instincts should also tell you that you had better be able to defend the expense at any level once the training is complete.
The results mentioned can only be determined through the application of integrity, dedication to the truth as opposed to what we want to happen, and a firm resolve to follow through to the completion of the data acquisition process.
Just as a pure scientist applies control group-blind testing to prove an hypothesis, training managers must be willing to determine skill and knowledge levels of two like sample groups, train one group and evaluate the competencies again. Then measure the performance level difference between the two groups. Roughly the same result can be achieved using one study group through the use of pre and post testing. The training manager must also conduct a LONG-TERM follow up which retests the trained group for the presence of continued positive competencies indicating a ROI. While it takes a bit of doing to determine the facets of this search for results, the effort will be worthwhile in the acquisition of the assurance that your training project produced long-term results. Here are two major pitfalls to avoid.
1. Never leave the design, execution and evaluation of training completely in the hands of others. Managers must be completely involved and conversant with the design, content and methods of presentation of subjects being taught and are entitled to expect changes in knowledge and/or performance after their employees have gone through the training. Just as parents who pay taxes have the right to see their children receive a quality public education, those same parents must also be responsible to assure that the facts are living up to their expectations. Never put the fox in charge of the hen house.
2. Never assume that knowledge and skills have been acquired. Verify that the training responds to a proven need which is met and achieved by the training process. Initially, students must be provided with measurement and feedback on acquisition of knowledge by using performance oriented, competency based testing of the new skill (the-student-actually-does-something-and-the-instructor-observes-and-grades-performance).
Managers must have proof in the form of certified, numerically-based scores which clearly illustrate the success or failure of the trainee population as a group and as individuals. Fearing management action on low testing results, unions often make evaluation a negotiating point at contract renewal, often insisting that answers to tests be present in direct conjunction with the tests. In addition, political pressures within the corporation/union relationship often result in the reduction of performance standards to the catch-all application of the term 100% mastery, thereby eliminating specific performance standards. In conventional classrooms, few trainees can achieve 100% mastery of a subject after one classroom session. In environments where correct answers are given in the printed documentation to both the interim measurement AND the final test, trainees can achieve 100% mastery of reading the correct answer with a general retention rate of zip. This natural expression of human nature, that is, going directly to the correct answer as opposed to having to work for it, occurs in many classrooms where instructors come directly from the workforce or middle management and see no real need for accountability of trainee performance.
By comparison, training methods such as Computer Based Training (CBT) require the trainees to interact with an impersonal terminal and be competing only against themselves. Training is presented in small, easily digestible chunks, and is followed immediately with interim measurements. During these mini-tests, trainees are evaluated on the previous few minutes of material. If they do not select the correct answer, they are branched back to the material they missed. Trainees stay in the measurement / branching loop until they get the correct answer. CBT measures not only trainee achievement using an adjunct data base, but usually informs the trainee of results immediately. Testing results are usually available from that same data base for the use of training managers on a student-by-student basis.
So the answer seems to be an old one. An investment without some return is no investment at all. Managers who do not help to shape, administer and courageously evaluate training efforts are professionally negligent for they have foolishly left the quality of the workforce in the hands of outside "experts", the unions or the trainees themselves.
The answer is that training, just like the workforce, cost control centers, and management must earn its place in the company by being productive and accountable...anything less does not constitute a ROI.
__________________________________
Mike has over thirty years of experience as an instructional developer
and lives in Atlanta. Thanks Mike!