How do you measure the impact of training?

Share This Post

If you are going to EduTECH next week be sure to say hello to Scott Frasard joining us from the USA who shared some thoughts on measuring the impact of training when we were in Austin in March 2016.

training eval

Scott’s recent role is as the Global Manager Training for ebay where they measure everything from order of satisfaction all the way up through return of investment.

This blog post follows an approach that could be applied to any training activity, accredited or not and overwhelmingly it is all about understanding the problem you are trying to fix and matching the measures to what it would be like if it was all working well.

Why is measurement and the return on investment important?

Well for some it is an opportunity to be academic or scientific in a workplace setting.  To be able to isolate training’s effect from other things such as culture, maybe the leaders or the people themselves so we can actually separate out here’s what training did versus any other kind of effect.

Are there some key metrics that are a number one priority to measure?

It really has everything to do with the training and each training activity is unique in terms of what problem it is trying to solve.  What we do is determine whether or not it solves their problem which may or may not include a specific metric such as customer satisfaction, average handle time, productivity improvements, reduced down time or that sort of thing.  It really comes down to what that training was designed to do and then you determine whether or not it actually achieved it.

What tools can you use to measure the impact of training?

There are a variety of tools that you can use to indicate for example when new recruits are ready to work on their own without a whole of support and guidance from mentor or supervisor or a coach.  Coming out of training they go through a period of on the job training, what we call nesting, and this measures their knowledge, skills and abilities as well as some personal attributes that are important to the line of work that they are going into.  You can also measure fit with the very specific teams that they’re going to work in and this helps us identify strengths and weaknesses.  This is a really robust process of identifying what those measurement elements should be and it’s a process of analysing and validating yes these are the right ones and are known to be critical for success.

What is the evidence that you are using to measure the impact?

There are elements about critical knowledge, about the business they’re involved in, the tools that they use like the technology and their ability to use them effectively.  As well as things about the person him or herself – like dealing with ambiguity, decision making, expectations on what they can deal with like customers, challenges and complaints.

What are some of the trends into the future?

Insights and reports for the person and the supervisor who’s going to be receiving these new employees so they’re able to see straight away if there any problem areas that they need to address on day one.  Circling summary data back to the training organisation and if we identify issues that leads us to make changes with the training or instructional design trying a different approach to address the gap.  We also work with our talent acquisition so they are able to see outcomes of their work as they are recruiting people for these positions.

Into the future, one of the things that we’re really excited about is in working with our coaching organisation and the supervisors to track these people over time.  Again with a new set of measurements about once a quarter and predictive analysis to show whether it’s a single or set of variables that work together to be able to predict future performance.

Which if you think about it from a workforce management perspective, if you know that any given person that’s going to be do well 6 months from now or is going to be struggling then you know how to work with that person or you have a better idea rather than leaving it to chance.  It also will help identify those people that are maybe borderline but salvageable and differentiate those from other people who are just not going to be successful.

It is important to ask – what can we do with this data, what can we do for our training, our talent acquisition and our leadership, and how can we use this information to make things better?

PS. Thanks Scott for your time and insights!

May 2016


Subscribe To Our newsletter

behind the scenes

More To Explore


Pitch Night

An old school pitch night was held at Stone and Chalk Adelaide at Lot 14 on a chilly night with 100+ people where The Hon