Evaluating Meetings What is it? Perhaps the most important step required to move from ordinary to exceptional meetings is a shift in thinking. Rather than viewing meetings as periodic events, think of them as a key work process of the team, no different than producing a product, or delivering a service. Below is a diagram showing a variation of the PDCA cycle for stages in the meeting process. When viewed this way, meetings become a work process of the team, which can be evaluated and improved, just like any other process. PDCA for the Meeting Process Plan the meeting Purpose Meeting absolutely necessary? Who should attend? Develop Agenda with changes based upon prior evaluation Individuals Prepare After Publish record or minutes Implement actions Check on progress Plan next meeting During the meeting Review roles, agenda, ground rules Follow agenda, stay on track Record (AID) Actions, Issues, Decisions Process checks Close the meeting Clarify actions, deadlines, responsibilities Evaluate the meeting What worked? What didn t? How can we improve? To improve your meetings it is essential that you Evaluate both the meeting and team's effectiveness. How do I use It? Multiple methods are available for evaluating a meeting. Following are descriptions of a few. Plus/Delta--This is one of the simplest and most widely used evaluation techniques. Divide a flipchart into two columns, like the illustration below. Plus Delta Record team member comments about the meeting. Plus What went well before this meeting, i.e., since the last meeting? What went well during this meeting? What should we keep doing? Delta is the Greek word for change. The symbol is. Delta What didn't work? What can we do better? What should we change or do differently in the future?
Sticky Plus/Delta--This is a variation on the Plus/Delta technique that is especially effective when dealing with very large groups (more than 20). Rather than having participants state their Pluses and Deltas aloud for the scribe to capture, have participants write at least one Plus and one Delta on separate sticky notes and place them on a flipchart as they leave the meeting. POINTER: Use the Affinity Technique in Chapter 4 to cluster the feedback into themes. PDCA--Create a form or flipchart like the table below, and use it to record ideas about what has worked and what did not during each stage of the meeting process. What worked? What didn't? What can we improve? Plan During Close After Finger Vote--To quickly surface concerns of team members, ask them to rate the meeting by raising an appropriate number of fingers, 1-5, using the following: 1=hated it; 2=didn't like it; 3=OK; 4=liked it; 5=loved it Allow those who vote 1-2 fingers to clarify their concerns and state what they would propose the team to do address them during future meetings. Facilitator Feedback--Another option is to have the person assigned as Facilitator for a meeting: Observe and record team behaviors during the meeting, then Give feedback to the team or individuals at break points or at the end of the meeting. There is some danger in this approach, particularly when the team is in the early stages of development or the assigned Facilitator is not trained in giving effective feedback. To be effective, the feedback giver must evaluate without imposing his or her "way" of doing things. This approach works best when the feedback-giver is a neutral third party and follows the guidelines on the next page.
FEEDBACK Guidelines Ask before you coach--don't give feedback unless the recipient has agreed to have you do so. Not everything needs to be public--some feedback is best provided to individuals in private, rather than with the full team in public. Use your judgment. Here and now--comment on this meeting. No history. Speak for yourself--don't say "the team liked.." or "Dan felt.." Comment on actions or behaviors--don't comment on the person. For example, don't say "Tim, you are quick to cut others off." Instead, say "Tim, I noticed on three separate occasions, you cut others off before they were finished speaking." By phrasing it this way, you are commenting on what Tim did, not on how Tim is! State observations and inferences. Without judging, describe what you saw and explain the inferences or assumptions it led you to, e.g., "When we changed subjects without reaching any kind of decision, I assumed we were avoiding x." Be specific--what specific behaviors were effective and why? Which were not effective and how might it be done differently? How often did they occur? To this end, it is helpful to record the frequency of specific behaviors. If participation is a problem, record the number of times that dominant or reluctant individuals speak. Watch to see if individuals direct their comments to other individuals, versus the team as a whole. Be constructive--point out the positives. Surveys--Another approach is to have team members complete a survey such as the one on the next page to evaluate team effectiveness. This can be administered: Prior to a meeting, and the results used at the outset of that meeting, to identify potential problem areas and get the team to agree on ground rules for addressing them. At the end of a meeting, to identify opportunities for improvement at future meetings. Advantages of a survey are that it allows you to: Ensure the evaluation covers all areas typically of importance to meeting effectiveness. Non-directive approaches (like Plus/Delta) allow the participants to address any areas, and they may not recognize the importance of some meeting mechanics. Keep individual ratings confidential, and report only overall group scores. In some difficult circumstances, you are more likely to get honest feedback through an anonymous survey. Focus on specific areas by including questions to address past problems. HINT: Prior to administering a survey, you might attend one meeting to observe the team and identify specific areas to include on the survey. You might also review prior meeting evaluations to identify what problem areas the team has previously identified.
Evaluation of Team and Meeting Effectiveness Evaluate your team's effectiveness by answering the questions below. Do members of this team Performance 1=Very Good 2=Good 3=So-so 4=Poor 5=Very Poor x Importance 1=Trivial 2=Not Imp. 3=Somewhat Imp. 4= Important 5=Very Import. A) Come to meetings prepared? 1 2 3 4 5 x 1 2 3 4 5 = B) Remain present during the entire meeting? 1 2 3 4 5 x 1 2 3 4 5 = C) Participate fully and appropriately? 1 2 3 4 5 x 1 2 3 4 5 = D) Use methods to analyze problems, make decisions, etc.? 1 2 3 4 5 x 1 2 3 4 5 = E) Follow through on commitments made during meetings? 1 2 3 4 5 x 1 2 3 4 5 = F) Have one conversation at a time? 1 2 3 4 5 x 1 2 3 4 5 = G) Avoid interrupting speakers? 1 2 3 4 5 x 1 2 3 4 5 = H) Avoid revisiting a prior topic or decision? 1 2 3 4 5 x 1 2 3 4 5 = I) Address issues honestly and openly? 1 2 3 4 5 x 1 2 3 4 5 = J) Deal constructively with conflict or differences of opinion? 1 2 3 4 5 x 1 2 3 4 5 = K) Offer new, creative, or unconventional ideas? 1 2 3 4 5 x 1 2 3 4 5 = L) Assign and follow meeting roles (scribe, facilitator, time-keeper)? 1 2 3 4 5 x 1 2 3 4 5 = M) Develop and adhere to realistic agendas, with times for each topic? 1 2 3 4 5 x 1 2 3 4 5 = N) Record Actions, Issues, and Decisions agreed upon during meetings? 1 2 3 4 5 x 1 2 3 4 5 = O) Evaluate team and meeting effectiveness using Plus/Delta, etc.? 1 2 3 4 5 x 1 2 3 4 5 = = Score
Analysis of Survey Data To summarize and review the survey data, several options are available: Multiply Performance x Importance to determine the top priority areas. The way the survey scale is constructed, those with highest scores indicate poor performance and most important. Calculate an Average and Range for each item. o Calculating Average -- If 7 team members rated a particular question as follows: 2, 3, 1, 4, 3, 2, 2 Average = sum # of responses = (2+3+1+4+3+2+2) 7 = = 17 7 = 2 o Calculating Range -- The range is the highest rating minus the lowest. Range = high - low = 4-1 = 3 When assessing Average and Range scores, consider the following: Average Performance scores of 3-5 are Opportunities. Average < 3 is a Strength. Ranges of three or more indicate enough diversity of opinion that the team should discuss the matter. HINT: Label survey questions with letter A-Z. Use letters not numbers, so references to the items do not get confused with their scores. If your survey assesses both Performance and Importance, it is possible to prepare a plot to illustrate the survey data. Steps include: Prepare a table of results like the example below, which involved team ratings of eight areas, A-H. Average scores were as follows: Items Questions related to: x Avg. Score Performance y Avg. Score Importance A Meeting prep 2 2 B Stay in the room 3 2 C Participate 2 4 D Use methods 1 3 E F G H Follow through One conversation Interrupt speaker Revisit the past 4.0 4 4 4 3 3 4 1
Plot the average ratings of the items, A-H, as shown on the next page. Analyze a Performance vs. Importance plot, as shown on the previous page, beginning with the upper left quadrant and moving clockwise through all four quadrants (SOGO) as follows. Strengths (Upper left) Ratings of C and D indicate that the team performs well in areas they deem important, such as Participation and Using Methods. Opportunities (Upper right) F, G and E are in this quadrant, indicating the team has rated these items as relatively Important and low in Performance, making the key opportunities for improvement: One conversation at a time Follow-through after meetings Avoid Interruptions of the speaker. Good enough? (Bottom right) Though H and B are rated low in Performance, they are also rated less Important by the team. So while there is room for improvement in Staying in the Room and avoid Revisiting past decisions or topics, the team does not identify this as a priority. Overdone? (Bottom left) A is potentially an area that is being overworked. This begs the question Should we remove resources or effort from this area and focus attention elsewhere? In this case, it appears the team has been spending more time than necessary in Meeting preparation. Improvement, not Evaluation, is the Objective No matter what evaluation method is used the method is ineffective unless follow-up action is taken to address the identified opportunities for improvement. These might include: More emphasis on key roles, e.g., facilitator for conversation difficulties, time-keeper for time problems, scribe for follow-through problems. Better planning or utilization of the agenda. Change in recording/publishing of meeting notes. Change in the means used to monitor followthrough on action items after meetings. Establish or change ground rules. For conversation difficulties, use of the LISTEN methods to improve team discussion skills. POINTER: Refer to Ground Rules and LISTENing Skills for methods to address common team difficulties.
An example plot of survey items A-H. 5 Strengths Opportunities More 4 I M P O R T A N C E 3 2 Less 1 Overdone? Good enough? 1 2 3 4 5 Better PERFORMANCE Worse