To anyone who wants to gain a foundation in mentoring, and managers and team leaders who want to develop or refine their mentoring skills.
EVALUATING & ENDING
THE MENTORING PROGRAM
How to assess a mentoring program
Continuously evaluating mentoring programs ensures that they maintain high standards. Without evaluation, program participants may become complacent and not see the point in continuing.
Evaluation planning and measurements
When approaching the evaluation, it helps to focus on why you're evaluating the program and to sort information into 3 categories.
First, you're interested in the success of the program's processes – what elements of the design worked and what didn't. Second you'll want to know how the program affected the people involved.
Finally, you should analyze data to assess the short-term and long-term effects on the organization.
When measuring program process outcomes, you should revisit your intentions for establishing the mentoring program. Then you can measure how successful it was in achieving its intended objectives.
When analyzing how the program affected the people involved, the main areas to consider are participant experiences and perceptions of the program. Try to take these into account in both ongoing evaluations and in more formal evaluations. The elements you use for this category of your evaluation could include mentor and mentee perceptions of the program, reported levels of motivation, or levels of job satisfaction among program participants.
The last category to include on your evaluation of a program is the effects on the organization as a whole. Often these will only become evident after the program has been operational for a period of time. The organizational effects will generally be focused on more in summative evaluations.
Measurable outcomes for organizational effects might include hard, bottom-line metrics such as sales turnover, market share, or employee retention. They could also include effects on softer metrics such as workplace culture, project turnaround times, or numbers of promotions.
To arrive at a satisfactory and comprehensive evaluation, data should be gathered from participants, managers, and program developers. To make the analysis richer, you should try to use both quantitative and qualitative assessments. Quantitative data usually involves numerical or statistical data. It provides more descriptive insights into an issue or area of analysis. It can be gathered from interviews or observations.
Whatever type of data you need, make sure you collect it regularly. You can use various methods to do so:
comparison of personal development plans to results achieved,
participant logs or diaries,
direct observation of mentees,
group discussions, and solid, statistical data used as basis for evaluations.
Assessing people's reactions
There are four levels you should consider when evaluating the impact of the program fully.
Analyze the reactions of the people involved.
Assess whether the desired learning has been achieved.
Investigate whether that learning has transferred to new behavior in the workplace.
Try to identify how the program benefits the organization overall.
Mentoring is primarily about people and how they learn and evolve as a result of working with other people. With this in mind, the first of the four steps – analyzing reactions of people involved – may be the most important step.
Measuring how people feel can be difficult because of the subjective and intangible nature of the data. However, interviews and questionnaires that probe people's opinions can provide realistic and valuable indicators of how well the program is working. It's also critical that such evaluations are carried out during the program – or shortly after it ends – when people's reactions are fresh in their minds.
Assessing learning and behavior changes
Because mentoring often teaches abstract skills, it can be difficult to evaluate changes in learning levels. What you can do is focus on the objectives of the program and ensure to link assessments to these.
Different types of testing techniques lend themselves to the assessment of knowledge, skills, or attitude change in mentoring programs. When specific knowledge change can be defined in a program's objectives, you can use techniques such as pre-program and post-program interviews, exams, and questionnaires to test for change.
The results of these techniques can be easily compared among mentees and it helps you to assess how each mentee has improved in specific areas.
To assess whether mentees have acquired specific skills, you can observe them as they carry out tasks, perform simulated pre-program and post-program role-plays, and conduct case studies. These techniques help you assess how mentees perform when faced with a specific task as well as gauge how well they've improved their skill levels in specific areas.
Measuring changes in attitude is more challenging than measuring changes in knowledge and skill levels.
Attitudes are difficult to gauge due to their subjective nature. You can, however, compare surveys, questionnaires, and mentor or supervisor observations given at the beginning of the program to those gathered at subsequent stages. Also, appropriate psychological assessments can help to monitor changes in attitude over time in relation to specific areas.
The third level of your overall program evaluation is concerned with assessing whether learning has transferred to actual changes in behavior. This usually involves gathering the opinions and observations of those who work with the mentee, including line managers, supervisors, peers, direct reports, and clients. These are the people best placed to assess the mentee's behavior on a regular basis.
Hard measures such as sales performance, client acquisition, or number of times a particular task is carried out successfully can also be used to measure learning transfer and behavioral change.
You can gather evidence about whether learning has transferred to actual changes in behavior in a number of ways:
An increase in performance appraisal results may indicate if a positive behavioral change brought about through learning has occurred.
An increase in positive 360-degree reviews of the mentee completed by peers, those who report to the mentee, and managers of the mentee.
An increase in the number of mentors who want to continue with the program can be seen as a positive affirmation of the improvement in mentee behavior.
Finally, in appropriate situations, gleaning feedback from clients who deal with mentees can be an excellent way to pinpoint improvements in behavior. Positive client feedback can also indicate a positive transfer of learning.
The final level you need to consider in your program evaluation is assessing the program's impact on the organization as a whole. The techniques you use for this level can sometimes be combined with those used for the previous level: observing new behavior. Results of all program participants' 360-degree reviews, interviews, and questionnaires can be gathered and assimilated. This helps you to arrive at conclusions about how behavior changes are impacting the organization's culture and its bottom line.
The primary focus at this level of the evaluation is on assessing whether the mentoring program has helped the organization achieve its business aims. The ideal scenario for a program coordinator is to be able to create a cause-and-effect relationship between the program's results and the organization's strategic aims. This can be hard to do as many of the benefits provided by mentoring programs are non quantifiable, subtle, and long-term.
Before outlining metrics you can use, it's useful to note that program coordinators and decision makers need to be careful in their evaluation of organizational benefits. Try not to attribute improvements to the program where other factors may have played a bigger part.
To discover organizational changes resulting from the program, look for things such as increased sales performance and increased efficiency in the completion of tasks. You could also search for improvements in organizational culture as seen in an increase in reports of job satisfaction and staff retention. Try to discover financial benefits also, such as an increase in revenue or areas where costs have been lowered through extra efforts made by mentees.
To create a comprehensive evaluation, you should try to gather data for all four levels. This includes analyze the reactions of people involved, assess whether learning has been achieved, observe whether learning has transferred to new behavior, and identify benefits to the organization.
it shouldn't be a static report with no calls to action. It should instead be complete with a set of recommendations and practical suggestions for improvement. The evaluation needs to promote and enhance what's good about the program and eliminate extraneous elements that aren't achieving any benefits or are wasting time.
The benefits of evaluating mentoring programs include finding out whether the time and money invested in such programs are really paying off.
When assessing a mentoring program, assess participant reactions to it, and then look at whether desired learning was achieved. It's also important to investigate whether learning has transferred to changed behavior on a day-to-day basis. Finally, try to assess the impact on the company by linking program results to the business aims of the organization.