Determining ROI for Leadership Development - U.S. Air Force*

* A Human Technology/Human Productivity Research Collaborative

Client Situation: The Air Force spends significant dollars to send a civilian to a high reputation Master's program at Stanford, Harvard, MIT, and the government's war colleges. The costs include tuition, housing for the year, plus the employee?s salary. They wanted to know if they should continue or abandon the program.

Organizational Diagnosis: For this project, the diagnostic component was to determine what data was possessed that could be used post hoc to evaluate the program effects. The organization had substantial information in personnel records and also competency assessment data.

Action Planning and Implementation: We designed an evaluation framework that included both tangible and intangible measures. Because this was a post hoc evaluation, we used a comparison control group that consisted of employees who had been accepted to the program, but who had not ultimately attended for reasons unrelated to motivation or capability. The research team collected data on six measures: Applicability of Training, Impact on Career Progression & Mobility, Impact on Competency Proficiency, Reputational Effects, Impact on Awards, Promotion, and Retention, and Cost impacts - Critical Incidents Reports. These six were examined with the five-level evaluation framework developed by Kirkpatrick and modified by Phillips. The evaluation levels include:

• Level 1: Reaction, assesses Participants? reaction to the program and their plans for implementing what they learned.
• Level 2: Learning, measures the extent to which Participants increased skills and knowledge, or shifted their attitudes.
• Level 3: Behavior, measures the extent to which a change in behavior occurred on the job as a result of the program.
• Level 4: Results, measures the impact the changed behavior has had on business or organizational indicators such as reduced costs, improved productivity, increased morale or favorable feedback from coworkers and customers.
• Level 5: Return on Investment, measures the financial and intangible value added to the organization for its dollar investment in training and/or development.

We designed and conducted surveys and interviews with CCDP participants, their supervisors, their customers, and a control group to gather data for quantitative and qualitative metrics that comprise a comprehensive ROI model. Personnel records provided another source of data.

Results and Measures: Seven reports were written, one for each of the six measures, and an integrated report that combined all results into an ROI model. A Benefit-to-Cost Ratio of $14.71 for each $1 spent per Participant was found. In terms of percentage return on investment, a 1,371% return for the money spent on each person sent to CCDP was determined based on financial findings.