培训的投资回报-5.ppt

ROI of Training,Session 6 Isolating the Results/Effects Of the Training Intervention,Pre-Amble,Isolating the results/effect of the training effort is the most difficult and also most important element of building the case for value (ROI); Because of this, value at the higher levels is seldom determined; However, the credibility not only of your study to show value, but also the training function is based on this element.,Pre-Amble,The complexity of isolating the effect, is a function of the variables that are in play at any moment in time in an organization,Pre-Amble,The Challenge,Pre-Amble,An acceptable degree of accuracy,How then do you isolate the effects?,Control groups Trend line analysis of performance data Participants estimation of impact Supervisors estimation of impact Managements estimation of impact Use of previous studies Use of customer input Subordinates report of other factors Estimating the impact of other factors,Isolating the Effects of the Intervention,Control groups Trend line analysis of performance data Participants estimation of impact Supervisors estimation of impact Managements estimation of impact Use of previous studies Use of customer input Subordinates report of other factors Estimating the impact of other factors,Isolating the Effects of the Intervention,Isolating the Effects of the Intervention,ELIMINATE,Control groups Trend line analysis of performance data Participants estimation of impact Supervisors estimation of impact Managements estimation of impact Use of previous studies Use of customer input Subordinates report of other factors Estimating the impact of other factors,Isolating the Effects of the Intervention,Isolating the Effects of the Intervention,CALCULATE AND ATTRIBUTE THE REST,Control Groups,Most accurate isolation approach; Involves the use of an experimental group that is impacted by the intervention and a control group that is not; Composition of both groups should be as identical as possible and, if feasible, the selection of participants for each group should be on a random basis.,Control Group Designs,Question is What Do You Control?,Case Study,Federal Express ROI analysis using control groups as a good example of the state of the art in measuring ROI. The study focused on 20 employees who went through an intense, redesigned two-week training program soon after being hired to drive company vans. Their performance was compared with a control group of 20 other new hires whose managers were told to do no more or less on-the-job training than they normally would. Performance was tracked for the two groups for 90 days in categories such as accidents, injuries, time-card errors, and domestic air-bill errors. The ten performance categories were assigned monetary values by experts from engineering, finance, and other groups. The intervention demonstrated that the performance of the highly trained employees was superior to that of the group that did not receive the upgraded training and also resulted in a 24% return on investment.,Also look at CLC Best Practice Document,Federal Express demonstrates: Practicality Non disruptive; Relatively accurate; Line involvement (and commitment); ROI is possible if you know what you are aiming for (measures),Also look at CLC Best Practice Document,Practical Exercise,Accra Wholesale Brickworks, find that there production targets are not being met. The Company is midsize but established in the City. It employs a production team consisting of three foreman, that supervise approximately 8 workers each and these teams take responsibility for procurement of the raw materials, mixing, filling the moulds, drying and baking. The production function is managed by a section manager.,Practical Exercise,Practical Exercise,Using the control group method: How would you selects your experimental group vis your control group What will you control How will you apply the Trend line Analysis Technique?,Trend-line Analysis,Basic format less accurate than control group Draw a trend-line using previous performance data as a base When intervention is conducted, actual performance is plotted and compared to the trend line Improvement in actual performance over what the trend line predicted can then be reasonably attributed to the intervention The primary advantage of this approach is that it is simple, inexpensive, and takes very little effort; However not accurate as it does not isolate into other factors; However a good first order analysis.,Trend-line Analysis,Trend line Use Case Study (Micro Electronics Incorporated),Practical Exercise,Estimate of Impact by Key Role Players,Easy to implement; Based on the assumption that participants are capable of determining or estimating how much of a performance improvement/impact is related to the intervention; Because their actions have produced the improvement, participants may have very accurate input on the issue; Although an estimate, this value will usually have considerable credibility with management because participants are at the center of the change or improvement; High face validity.,Estimate of Impact by Key Role Players,Easy to implement; Based on the assumption that participants are capable of determining or estimating how much of a performance improvement/impact is related to the intervention; Because their actions have produced the improvement, participants may have very accurate input on the issue; Although an estimate, this value will usually have considerable credibility with management because participants are at the center of the change or improvement; High face validity.,Typical Research Questions,The individuals who do not respond to the questionnaire or provide non- usable data on the questionnaire are assumed to report no improvement/impact; Extreme data and unrealistic claims are omitted from the analysis; Only annualized values are used, it is assumed that there are no benefits from the program after the first year of implementation; The confidence level, expressed as a percent, is multiplied by the improvement value to reduce the amount of the improvement by the potential error; If value is expressed it is also factored by the confidence value.,Adjustments to improve validity,Estimate of Impact by Key Role Players,Estimate of Impact by Key Role Players,Estimate of Impact by Key Role Players,The process can be expanded to include:,Supervisors,Management,Other stakeholders,Expert Input,Select carefully, based on their knowledge of the process and situation; If capacity does not exist in the organization; Will probably use techniques that most training practitioners can implement; Organization culture will determine whether the outcome will be accepted; If linked to skills transfer it could be worthwhile; Often top management place more confidence in outside experts than in their own staff.,Obtaining Input From a Target Group,An example is the subordinates of the supervisors trained in people handling skills; It could be possible for them to estimate how much of an improvement can be attributed to the intervention; They can also provide input in terms of what other factors might have contributed to the improvement; The input is usually obtained through surveys or interviews. When the survey results show significant changes in participant behavior after training and no significant change in the general work climate, the improvement in work performance, therefore, must be attributed to the changes in supervisor behavior since other factors remained constant; This approach has some disadvantages. Data from subordinates is subjective and may be questionable because of the possibility for biased input; Often subordinates may have difficulty in determining changes in the work climate. However, in some cases, subordinates are aware of the factors that caused changes in their work unit and can provide input about the magnitude or quantity of changes. When combined with other methods to isolate impact, this process has increased credibility.,Obtaining Input From a Target Group,In some situations it could be feasible to calculate the impact of factors (other than the intervention) that influenced the improvement and then conclude that the intervention is credited with the remaining portion; In terms of this approach, the intervention claims credit for improvement that cannot be attributed to other factors; The method is appropriate when the other factors are easily identified and the appropriate mechanisms are in place to calculate their impact on the improvement; The process can be very credible if the method used to isolate the impact of other factors is credible.,Which Strategy?,Control group feasibility; Availability of historical data; Expertise to conduct study; Capability and willingness to estimate impact; Can non-training effects be isolated? What are the constraints in terms of using the data collection technique and isolation strategy.,Credibility of Data is Influenced By:,Reputation of the source of the data; Reputation of the source of the study; Motives of the researchers; Personal bias of audience; Methodology of the study; Assumptions made in the analysis; Realism of the outcome data; Type of data; Scope of analysis.,