Evaluation of Maximizing Employee Performance Training Course. Bradley Drewyor, Executive Editor McGraw-Hill Education. A report submitted to

Transcription

1 Evaluation of Maximizing Employee Performance Training Course Bradley Drewyor, Executive Editor McGraw-Hill Education A report submitted to Melissa Janis MHE Learning & Development December, 2011

2 ABSTRACT The purpose of this report is to communicate the findings of the evaluation of the Maximizing Employee Performance training course offered by McGraw-Hill Education (MHE) Learning & Development. MHE places a high value on the role managers can play in maximizing corporate performance through effective management of staff. The Maximizing Employee Performance training presents people managers information and a set of tools to support feedback interactions with their direct reports. The Maximizing Employee Performance training course is a halfday training that has been developed and is administered by MHE Learning & Development staff. The main objectives of the course are to prepare participating employees to 1. demonstrate an understanding that feelings are important, and that constructive feedback through the GROW and CEDAR models creates awareness and responsibility. 2. acknowledge that tension increases over time, and demonstrate an understanding that providing coaching and feedback are essential tools for managers. 3. increase the frequency of how often they provide feedback to staff. This report details the results of an evaluation of the program against these key goals, as well as a detailed look at the results of the training course in participant mastery of other key concepts. As the report will show in detail, the findings support the training course as having positive impact on goals 2 and 3 listed above. Additionally, the results indicate the training course has potentially positive impact on goal 1, as well. The report goes on to discuss specific results and potential areas of concern for MHE Learning & Development to reevaluate the course materials in order to create promote greater mastery of course content for future offerings of the Maximizing Employee Performance training

3 EVALUATION OF MAXIMIZING EMPLOYEE PERFORMANCE TRAINING COURSE Maximizing employee performance is essential to the success of any business. MHE Learning & Development believes strongly that when employees feel what they do is important, they are motivated to succeed. However, people managers often lack the understanding and comfort level they need to provide consistent, constructive, and positive feedback. Even when employees are aware their bosses are disappointed with their performance, they are very often unable to pinpoint and discuss how they are failing to meet expectations and what they can do to begin meeting expectations. MHE Learning & Development research suggests manager s can improve employee performance by a significant percentage simply by giving feedback. Furthermore, they are looking to instill a culture where managing performance improvement in a manner that makes people feel good about what they do becomes the norm. To this end, MHE Learning & Development is offering a series of trainings to increase manager effectiveness in promoting positive employee growth and working relationships. One course from this series, Maximizing Employee Performance, focuses on providing employees techniques for delivering coaching and feedback, with a particular emphasis on giving positive feedback. The purpose of this document is to serve as a report of an evaluation of one offering of the Maximizing Employee Performance curriculum, delivered in the Columbus, Ohio, McGraw- Hill Education facility. This report provides an overview of the training course, a list of course objectives, a description of the evaluation process, a presentation of evaluation results, and a summary of learning taken from the evaluation exercise. EVALUATION OVERVIEW Description of the Program: Maximizing Employee Performance is a six-module training course that was developed and is delivered by MHE Learning and Development staff. The training course is offered through MHE, and is intended to be delivered in one half-day learning session. Participants meet in a small group, led by the trainer. Learning activities include lecture and discussion, team-based learning, and paired role play. Each student leaves the course with a copy of the presentation slides and a resource to encourage use of the GROW model and CEDAR framework for preparing and participating in feedback sessions with direct reports. Course Objectives: Each of the six modules in the course has a key question and a learning objective or objectives. These questions and objectives were used to plan the evaluation objectives for the training, but were not all explicitly addressed as part of the evaluation. The modules and objectives appear in the chart on the next page

4 # Module Objectives 1 What does a workplace look like when Build consensus on what good looks like employees are doing their very best? Create interest and ownership 2 How do you know when you need to Create commitment for taking early, take action? substantive action 3 How can you take effective action? Understand available choices and pros/cons, including the relative impact on the employee Increase comfort with having difficult conversations with employees Improve willingness and ability to ask open ended questions Increase frequency of positive feedback 4 What happens after the conversation? Understand the importance of continuing to notice Provide objective data to support perspective 5 Follow-Up Create commitment to transfer learning to the workplace 6 Wrap-up Reinforce messages and commitment to apply learning Training Participants: The Maximizing Employee Performance training course is offered to people managers at all levels and within all departments of the organization. Regardless of organizational role, people managers have a powerful affect on the performance of direct reports. This training is offered across the different locations within McGraw-Hill Education, and participants are targeted for the training if they have direct reports, but have not yet completed the training. Once targeted, spots in the course are filled on a first-come, first-serve basis. Participants are required to participate in this training face-to-face, devoting one half day of time to new learning. For this training, 12 participants committed to the training and attended the training session. Training Objectives: Upon completing the training, it is expected that participants will have built knowledge and a comfort level with the importance and intricacies of offering constructive and positive feedback to direct reports, which will result in improved and more frequent feedback to employees. As stated previously, there are six modules in the course, each with its own objectives. The evaluation detailed in this report distills these detailed objectives into three key evaluation targets. These objectives were developed by the course originator and trainer (Melissa Janis) from MHE Learning and Development and the evaluator (Bradley Drewyor). By completing this training course, employees will 2. demonstrate understanding that feelings are important, and that constructive feedback through the GROW and CEDAR models creates awareness and responsibility. 3. acknowledge that tension increases over time, and demonstrate understanding that providing coaching and feedback are essential tools for managers. 4. increase the frequency of how often managers provide feedback to staff

5 EVALUATION METHOD Evaluation Model: Based on the training objectives and the delivery method, the evaluation employed a goal-based model for targeting training objectives, identifying questions and activities, and collecting, analyzing, and interpreting data. The overarching goal for the evaluation communicated by MHE Learning and Development was to investigate if participants both grasped key concepts at the time of training and changed their management and feedback frequency based on internalizing the key concepts and making them part of their managerial toolkit. Evaluation Questions and Activities: The following chart presents the evaluation questions and activities used to observe the behaviors of participants, resulting in an evaluation of the training objectives. Evaluation activities include: (1) a pretest/posttest focusing on the main features of the training, including the GROW model and CEDAR framework, the three B s of positive feedback, and other concepts, (2) a survey, administered on the day of training, of how often they provide feedback to staff currently, (3) and a follow-up attitudinal survey to determine how the training changed their habits for providing feedback four weeks after the training. Key questions, the objectives targeted by the questions, and the activities used to observe the learning are as follows. Evaluation Questions Do students understand that feelings are important, and that constructive feedback creates awareness and responsibility in employees? Are students able to demonstrate an understanding that providing timely coaching and feedback can help to manage challenging situations before they escalate over time? Has the training resulted in an increased frequency of providing feedback to employees by participating managers? Objectives Targeted 1. Demonstrate an understanding that feelings are important, and that constructive feedback creates awareness and responsibility 2. Acknowledge that tension increases over time, and demonstrate an understanding that providing coaching and feedback are essential tools for managers 3. Increase the frequency of how often managers provide feedback to staff Activities to Observe Pretest/Posttest Pretest/Posttest Attitudinal survey on day of training; Follow-up survey Pretest/Posttest: The pretest and posttest were administered on the day of the training by the evaluator, using paper and pencil at the site of the training. Both tests consisted of the same set of six questions, which were used to evaluate each of the 12 participants on their understanding of the course materials and objectives prior to the training and their knowledge following the training. These tests were used to determine the immediate learning impact of the training for participants

6 Attitudinal Survey Day of the Training: Following the training, each participant was asked to complete a paper-and-pencil attitudinal survey consisting of four questions to determine the frequency of and value they placed on delivering timely, constructive feedback to direct reports. This information provides a baseline of information to compare with results of a follow-up survey to see how, or if, the training ultimately resulted in a change in valuation and behavior in regards to providing feedback to employees. Attitudinal Survey One Month after the Training: All participants were asked to complete an online survey to measure how, or if, their attitudes toward feedback and frequency of delivering feedback to direct reports has changed from the survey administered to participants on the day of the training. This feedback will support MHE Learning & Development in reviewing course content and learning priorities for future offerings of the course to emphasize the key learning and behavioral changes they expect to see internalized by people managers. Results Pretest/Posttest The pretest and posttest consisted of the same six questions, asked in the same order, immediately prior to the training and directly following completion of the course. All participants were asked to complete both sets of questions to demonstrate immediate knowledge transfer for course material. Results appear in the chart below for each question both on the pretest and posttest. A complete version of the pretest/posttest is available in the Appendix A. Question 1 Correct Response(s) a d 2 b 3 b Pretest Results Posttest Results Question Correct Response Pretest Results Posttest Results a 58% a 100% a 0% a 0% b 0% b 0% b 36% b 0% c 25% c 10% 4 c c 9% c 91% d 75% d 90% d 9% d 9% e 33% e 0% e 46% e 0% a 100% a 55% a 0% a 73% b 0% b 45% b 27% b 0% c 0% c 0% 5 a c 18% c 0% d 0% d 0% d 37% d 27% e 0% e 0% e 18% e 0% a 58% a 36% a 0% a 18% b 42% b 64% b 25% b 9% c 0% c 0% 6 d or e c 33% c 9% d 0% d 0% d 42% d 46% e 0% e 0% e 0% e 18% - 6 -

7 Results from the pretest to the posttest indicate positive gains were made between pretest and posttest knowledge of key course concepts. In particular, participants enhanced their knowledge to high percentages on question 1 (Feedback is effective when it creates awareness and responsibility.), question 4 (When using the CEDAR model, you start with what you expected to see.), and question 5 (When an employee offers an unacceptable excuse for poor performance, the best approach is to validate their excuse.). At the same time, question 2 showed a change from 0% correct to 45% correct, a strong growth, but not a high total percentage correct. Question 3 showed little growth from pretest to posttest, and question 6 seems to show more participants answering on the wrong end of the scale for the posttest than on the pretest, even though generally more participants answered on the correct end of the scale, as well, leaving a gap in the center. Attitudinal Survey Day of Training On the day of training, each participant was provided a four-question attitudinal survey to determine how they evaluated their attitudes toward performance evaluation prior to completing the course. The attitudinal survey was delayed until after the training to ensure participants would have the vocabulary necessary to confidently respond about their attitudes. Results are discussed below. A copy of the complete survey can be found in the Appendix B. Note: Only eight of the twelve participants completed the attitudinal survey following the training. Asked how frequently they provided positive feedback to direct reports, six of the eight participants (75%) indicated they did so often, while the other two participants (25%) indicated they did so only once in a while. When asked a similar question, but about constructive feedback, participants were evenly split (50%-50%) between often and once in a while. Next, the students were asked how consistently they did the following three things consistently when providing positive feedback: (a) Gave specific feedback; (b) Demonstrated energy when working with an employee; (c) Expressed appreciation for the situation. Five of the eight participants (62.5%) indicated they gave specific feedback, three of the eight participants (37.5%) suggested they demonstrated energy when working with an employee, and seven of the eight participants (87.5%) felt they expressed appreciation for the situation when giving positive feedback. The final question asked participants to characterize their approach to correcting an employee s performance on a spectrum from telling to asking questions. Participants all felt they were at least in the center or much further toward telling than asking questions. The responses are charted on the line below. Telling Asking Questions Attitudinal Survey Follow-Up Survey Four weeks after the day of the training, participants were asked to complete an anonymous survey built on Google Docs to evaluate their attitudes toward both employee feedback and the Maximizing Employee Performance course content in general. Continuing the trend noted above, only six of the twelve respondents completed the online survey. Questions and participant responses appear on the next page

8 - 8 -

9 In addition to the results on the previous page, participants were asked to respond to an open-ended question about how the completion of the course has affected their approach to discussing employee performance with their direct reports. The complete responses appear in the Appendix C; however, all responses generally focused on how the course encouraged more frequent, more specific feedback with employees. This response is very consistent with the results of the other questions, as participants provided specific and constructive feedback quickly (50%) and often (84% daily or weekly), but in a more conversational, rather than directive, manner. Discussion Results on the pretest and posttest demonstrated participant improvement on all questions, although some participants were less certain about the importance of employee s reactions to feedback versus the need for accurate information. While the group, in general, acknowledged a new respect for the importance of employee emotions or reactions, a few participants moved toward accuracy over reactions. Looking at the other five questions, the test results generally support the ability of the instructor to impart the intended knowledge to participants during the training. At the same time, the test results also point to some reevaluation of the specific content or stress on certain content to increase the overall mastery of concepts. Specifically, participants did not completely master the difference between responding to an employee s performance immediately versus when it starts to escalate. The intent of the course was for students to leave with an understanding that they cannot deal with every issue immediately, and they need to let direct reports sort some issues on their own, until it is clear that the issue at hand is beginning to escalate. Additionally, there could be more focus put on the idea of feedback conversations tending more toward managers asking questions instead of telling direct reports the answers, which was a major goal of the course. In looking at the two attitudinal surveys, there was definitely positive movement in terms of understanding that constructive and positive feedback should be offered frequently and quickly. While those are important goals, the attitudinal survey did demonstrate participants are still uncertain about the role of asking questions when giving feedback, and their lack of use of the CEDAR framework and GROW models following the survey are indicators that they are not using the tools to allow them to confidently, and comfortably, have conversations built around questions rather than mandates. Overall, participants remained positive about the value of the course, and they indicated that they felt the course helped them improve their management of employee performance

10 This final section of the discussion will return to the evaluation questions and objectives listed at the outset of the evaluation report. For each question and set of objectives, a quick summary of the effectiveness of the course based on the evaluation at hand are provided. Evaluation Questions Objectives Targeted Results Do students understand that feelings are important, and that constructive feedback creates awareness and responsibility in employees? Are students able to demonstrate an understanding that providing timely coaching and feedback can help to manage challenging situations before they escalate over time? Has the training resulted in an increased frequency of providing feedback to employees by participating managers? 1. Demonstrate understanding that feelings are important, and that constructive feedback creates awareness and responsibility 2. Acknowledge that tension increases over time, and demonstrate understanding that providing coaching and feedback are essential tools for managers 3. Increase the frequency of how often managers provide feedback to staff Project Costs Based on the pretest and posttest results, there was an increase in the number of participants who thought feelings were at least as important as accurate information (42% pretest to 64% posttest); however, as discussed above some students moved further in the other direction following the course materials. Participants did demonstrate on the pretest and posttest that they understood that timely response to issues was important, and the attitudinal surveys support a finding of increased understanding in the value of coaching and feedback. The attitudinal survey does support a finding of increased frequency and speed of feedback. Personnel Evaluator Bradley Drewyor: 2.5 days x $300/day $750 Trainer: Melissa Janis: 1 days x $400/day $400 Subtotal $1,150 Travel (airfare, hotel, per diem) One two-day round trip: New York City to Columbus, Ohio $500 $ 500 Presentation Auditorium Set Up $200 Continental Breakfast $ 75 Copies, Chart Paper, Other Supplies $ 25 Subtotal $ 300 Total $1,

11 Appendices Appendix A Pretest/Posttest Questions...12 Appendix B Attitudinal Survey: Day of the Training...13 Appendix C Attitudinal Survey: Follow-Up Survey Open-Ended Responses

12 Appendix A Maximizing Employee Performance Pretest/Posttest This questionnaire will serve as a quick look back at what you learned during from the training curriculum. Again, your responses will be completely anonymous and will be used as a part of an evaluation of the course. Answer the questions below to the best of your knowledge as of this moment. 1. Feedback is effective when it creates (circle two) a. awareness. b. collegiality. c. rapport. d. responsibility. e. team-work. 2. The best time to deal with an employee performance issue is (circle one) a. Immediately. b. when it starts to escalate. c. after a few weeks have passed. d. the next time you meet formally. e. during a scheduled Performance Review. 3. The most effective way to improve employee performance is to (circle one) a. tell them exactly what they did wrong. b. ask them their opinion of their performance. c. cite an example of someone who does it better than they do. d. tell them to figure it out on their own. e. tell them what will happen if they don t improve 4. When using the CEDAR model, you start with (circle one) a. how important it is they improve. b. how the conversation is confidential. c. what you expected to see. d. what you are actually seeing. e. what you are hoping to achieve. 5. When an employee offers an unacceptable excuse for poor performance, the best approach is to (circle one) a. validate their excuse. b. explain why the excuse is not valid. c. offer other solutions. d. ask questions about their excuse. e. None of the above 6. Accurate information about performance is more important than how people feel about it. (circle one) a. Strongly agree b. Agree c. Neither agree or disagree d. Disagree e. Strongly disagree

13 Appendix B Attitude Survey The following questions will be used to evaluate your attitude toward performance evaluation prior to completing the training course. Based on what you learned over the past few hours, answer each question with how you approached feedback and performance discussions with staff up to this point in your career. Statement Constantly Often 1. Prior to the course, how frequently did you give positive feedback? 2. Prior to the course, how frequently did you give constructive feedback? Once in a While Rarely Never 3. Prior to the course, when you gave positive feedback, which of the following did you do consistently? (circle all that apply) a. Gave specific feedback b. Demonstrated energy when working with an employee c. Expressed appreciation for the situation 4. Prior to the course, where on this line would you characterize your approach to correcting an employee s performance? (draw an X on the line) Telling Asking Questions

14 Appendix C Attitudinal Survey: Follow-Up Survey Open-Ended Responses 1. Completing this course reminded me of the importance of providing specific feedback for employee performance. Additionally, the course made me focus on the fact that I often end on the telling as opposed to asking questions end of the continuum, resulting in micro-management, as opposed to leadership and growth. 2. It reminded me of the importance of giving specific feedback. I think the "specific" aspect of feedback, whether \positive or negative, is generally overlooked. 3. I shared some of the information I learned with my direct reports, to raise their awareness of their own behaviours (sic). I have also been much more open and empathetic and try to understand my staff's interests and goals. 4. I have lost half of my staff in the last month so my team of 4 has shrunk to 2 and I will not be able to replace the 2 I lost so our workloads are out of control. One of my direct reports is a star top performer, and I constantly reinforce her with positive feedback because she does such a great job. My other direct report is not a top performer but his workload has doubled in the last month and he has been a good sport about it so it has been a real challenge for all of us to keep our heads above water. 5. Yes. I learned what helps the employee understand a situation or approval in a more positive manner. I enjoyed this course