Atom Wave methodology for measuring ROI of Leadership and Management Development is based on Jack Philips' methodology, expanded by one more level of our own, measuring Sustainability of results. The six levels of ROI evaluation are the following:
Level 1: How did the participants like the program?
Level 2: What did participants learn to do differently?
Level 3: How applicable was what they learnt?
Level 4: What is the business impact of applying the new techniques?
Level 5: What is the ROI of the business impact of this Intervention?
Level 6: How much do they continue to use the techniques they learnt after a period of time?
In the table beside, there are the six levels of evaluation and the tools we use to measure each of them.
Practice sessions, role-plays and different activities ensure learning of the new techniques. Yet, the most reliable methodology we have is a BEFORE and AFTER video recording of a simulated situation. Depending on the client needs, we design a series of scenarios related to the people management skills and every participant gets an individual recording at the start of each program and one at the end of it (each program usually has a duration of six months).
The table depicts the results of the skill level of a group of 14 middle managers in Coaching and Feedback.
According to multiple studies, the respondents are not satisfied from their organization's leadership development trainings. For us, at Atom Wave, this makes perfect sense. When it comes to Leadership Development, there is attitude and behavior changes involved and these need practice and support into implementation in order to be achieved. Therefore, our program design has the following elements:
Combination of training with coaching to each participant for a 6 month period in order to allow participants to embed the changes in their every day practice.
Each participant selects one particular member of his/her team in order to focus and apply a development plan during the program so that progress can be more visible.
As content designers, we focus on making training content 100% relevant to the participants’ every day work and thus design a series of tailored tools (Development Plans / Forms / scenarios)
The key question to answer in relation to the expected result from a development intrervention is the following: What will participants be doing differently and the organization will know that this program has an impact on business results?
In one of our programs about Solution Focused Coaching and Feedback, there were participating 14 managers. They set and achieved total of 50 objectives that had tangible, visible results. These results were in the following five categories:
1. Sales Increase
2. Productivity increase
3. Better management Skills
4. Attitude Change
5. Communication and Collaboration improvement
Brief example: Participating manager as a result of the program, realized she had a centralized management style and she needed to change that. After working on this topic during her coaching sessions, she changed the way she was working with her team, thus giving them more space for initiatives and decision making.
#1 One of her team members got promoted within three months, the organization gets developed a new manager.
#2 Participating manager gained at least one hr daily by being less controlling, which is translated to additional time to sales for the organization.
In each program, there are lots of valuable improvements/ results. We divide these into those that tangible and intangible.
Tangible results are directly related to measurable results: Sales increase, Cost saving, Time saving, Productivity increase and these are the ones that can be calculated in a ROI. Intangible results are visible results where others can spot the progress such as improved collaboration, customer service, increased motivation, better communication but are not calculable.
For ROI calculations, we need to focus only on Tangible results and of those there may be some that are again hard to link with a development intervention.
In the program mentioned above, out of the 14 participating managers, 9 (64%) reported significantly increased sales results both for the selected coachee and for the rest of the teams. Yet, these are not calculable in terms of Euros as it is not easy to tell how much does each product sale bring to the organization in the financial sector.
Therefore, we calculated time related savings.
The team of 14 participants had a total of 16.5 saved hrs daily from delegating better, developing skills and increasing productivity or saving time. Calculating the value of these 16.5 hrs daily at a 33 euro/hr which is the AVG cost of each manhour in this organization, gave us:
Program Benefit: 16.5 hrs daily X 33 euro manhour cost X 230 working days = 125,235 euro
Program Cost: Program Cost + Attendance time cost + Transport of participants' cost + Administration and HR cost=58.334 euro.
Thus we have according to the
ROI formula: ((Program Benefits - Program Costs)/Program Costs)X100% = ((€125,235 – €58,334)/€58,334 x 100%=115% ROI
Sustainability of training interventions is dependent on all the previous stages. When a participating manager gets to master a new way of doing things ie. Solution Focused coaching, and when s/he gets to use it and finds out it works, then the change happens more naturally.
In the above mentioned study, we sent an Evaluation questionnaire, six months after the program completion and twelve months after its launch.
As the results indicate, the implementation level has remained high and in most instances like in Leadership, participants increased the implementation of the new know-how.
ROI of 195% and successful roll-out to 300 managers of Solution Focused Coaching Train the Trainer.
According to the 2014 McKinsey article of Gurdjian et al, Leadership Development programs success factors are: