DRIVING LEARNING IMPACT WITH HARVARD MANAGEMENTOR { perspectives } LEADERSHIP DEVELOPMENT
ABOUT CORPORATE LEARNING Harvard Business Publishing Corporate Learning partners with clients to create world-class leadership development solutions for managers at all levels. Our team leverages the management insight, thought leadership, and expertise of Harvard Business School faculty and authors from Harvard Business Review to create tailored leadership development solutions. With more than 20 years of practical experience, our innovative, technology-enabled solutions drive meaningful and lasting business results. Corporate Learning is a market group within Harvard Business Publishing. ABOUT HARVARD BUSINESS PUBLISHING Harvard Business Publishing was founded in 1994 as a not-for-profit, wholly owned subsidiary of Harvard University. Its mission is to improve the practice of management and its impact in a changing world. The company achieves its mission through its relationships with customers in three market groups: Higher Education, Corporate Learning, and Harvard Business Review Group. Through these platforms, Harvard Business Publishing is able to influence real-world change by maximizing the reach and impact of its essential offering ideas. 2015 Harvard Business School Publishing. All rights reserved. Harvard Business Publishing is an affiliate of Harvard Business School.
DRIVING LEARNING IMPACT WITH HARVARD MANAGEMENTOR For leadership development to have lasting effect, programs require the right learning design, metrics, and reporting and analytics. Harvard ManageMentor s on-demand, work-based approach and new On-the-Job component provide organizations with new ways to drive measurable learning impact. BY DIANE BELCHER, DENNIS GULLOTTI, AND MICHAEL WARECH Reaching for Impact Organizations know that for leadership development interventions to be successful, learning needs to be integrated with work and linked to the achievement of goals and business results. Leadership development measurement approaches, however, have traditionally focused on knowledge gain and are too often disconnected from actual work. The disconnect is largely a result of poor learning design. If the learning design fails to provide learners with adequate opportunity to practice and apply what they learn on the job, it becomes challenging to reliably measure learning impact. Development should be integrated with learners work to move from learning how to be a leader to actually becoming a leader. Without that mind-set shift among current managers, organizations will not effectively prepare their managers to drive business impact. HOW CAN AN ORGANIZATION AUGMENT THE TRADITIONAL MODEL OF MEASURING KNOWLEDGE GAIN BY ALSO MEASURING LEARNING IMPACT? With these challenges in mind, how can an organization augment the traditional model of measuring knowledge gain by also measuring learning impact? In order for leadership development to create measurable learning impact, it must have three fundamental elements: an optimized learning design, valid and reliable metrics and measurement, and appropriate reporting and analytics. Harvard ManageMentor is an on-demand learning and performance support resource designed to drive learning transfer through an innovative work-based approach and focus on measurable learning impact that brings these fundamental elements together. HARVARD BUSINESS PUBLISHING CORPORATE LEARNING { 1 }
Three Keys to Success Learning Design Strong leadership development programs start with a design that is oriented toward learning impact. The most impactful learning programs are focused not just on increasing knowledge, but also on helping learners improve by applying that knowledge consistently in their day-to-day activities. For example, learners can master a skill through practicing, making mistakes, and trying things out. Skill development and, ultimately, mastery do not come from passively receiving information. Instead, they come from cycles of learning, reflecting, and solving problems. That s why impactful leadership development programs incorporate concepts of doing and reflecting directly into their learning design. Harvard Business School professor David Garvin points out in Rethinking the MBA, Innovative thinking and creativity, much like swimming and biking, is a skill best learned through practice, reflection, and repeated application. Reflection plays a key part in learning from experience in fact, it is just as important as practice. Technology can be a great enabler of practice and reflection as an integral part of learning. For example, reflection can be built right into an on-demand platform making it easy for the learner to capture reflections and reference them later. Since managers are learning on the job, it is critical that the learning design focuses them on the elements they need to practice and apply the most. Programs can enhance learning by providing tools at the moment when learners need them. These resources include focused pieces of content, job aids, and activities. A flexible, modular, and on-demand approach to learning provides busy managers with exactly what they need to know, in a format that is relevant and only a tap or click away. REFLECTION IMPROVES PERFORMANCE A 2014 study by Giada Di Stefano (HEC Paris), Francesca Gino (Harvard Business School), Gary Pisano (Harvard Business School), and Bradley Staats (University of North Carolina Kenan-Flagler Business School) provides evidence that routine reflection by adults improves their performance. Specifically, the research team conducted a series of studies that compared groups focused on learning by doing with groups focused on learning by doing and reflecting. Their results showed that the groups whose experience was deliberately coupled with reflection consistently outperformed the other groups. They concluded that reflecting on what we ve done teaches us to do it more effectively the next time around. Also, When we stop, reflect, and think about learning, we feel a greater sense of self-efficacy, Gino says. We re more motivated and we perform better afterward. And we all know that learning is more impactful when it s part of an ongoing process. Learners need to be engaged in a learning experience process versus a one-time event. When given repeated opportunities to explore concepts via a wide variety of experiences, learners absorb more. measurable impact = design + metrics + reporting/analytics { 2 } driving learning impact with harvard managementor
Work-based Learning Approach Focuses on Application of Skills on the Job Harvard ManageMentor focuses on integrating learning and work, with an emphasis on application throughout. Harvard ManageMentor is oriented toward learning impact starting with the design of each management topic. Each online topic is broken into concise, bitesize learning bursts called lessons. Since every minute is precious to time-starved managers, these short learning bursts were designed so that each piece of content is important and useful, and to allow for maximum impact through practice and reflection. DEVELOPMENT SHOULD BE INTEGRATED WITH LEARNERS WORK TO MOVE FROM LEARNING HOW TO BE A LEADER TO ACTUALLY BECOMING A LEADER Each lesson has a built-in cycle of learn-practice-reflect, in which learners explore the core content, practice what they have learned in scenario-based activities, and then reflect on their learning and how they can apply it to their everyday work. Since Harvard ManageMentor is available on demand, and practice and reflection are part of the learning design, these important steps become routine for learners. The benefit of creating that routine is learners are already thinking about application when starting a lesson, which helps increase learning impact. Also, their reflections are automatically stored and referenced in other components of the learning design. Learning Metrics The well-known Kirkpatrick model of training evaluation provides the key metrics that any professional needs to consider. Historically, learning and development (L&D) organizations focus their measurements on the first two levels. While level two, knowledge gain, is important and necessary, that data alone isn t enough to proclaim that successful behavior change and learning transfer have taken place. It certainly won t allow you to measure true learning impact. While it is important to maintain these two levels as foundational metrics, we encourage companies to strive for level three and four measurement. Having valid and reliable learning metrics is a good start, but what happens when some learners, or a segment of learners, meet the desired levels and others do not? How do you determine if outside factors may have either positively or negatively impacted the results? THE KIRKPATRICK MODEL LEVEL 1: REACTION The users thoughts and feelings about the training. LEVEL 2: LEARNING An increase in knowledge or skills. LEVEL 3: BEHAVIOR The transfer of knowledge or skills to the job. LEVEL 4: RESULTS The final results of the training, such as business impact or results. HARVARD BUSINESS PUBLISHING CORPORATE LEARNING { 3 }
The Learn-Practice- Reflect model [in Harvard ManageMentor] helps our learners retain what they are learning. The On-the-Job action plans are a very effective way to apply the learning and develop the correct mind-set within the individual. Mutheer Al Kalbani, Senior Manager of HR Development, Bank Sohar S.A.O.G. Along with learning metrics, additional data needs to be captured throughout the learning experience in order to assess the level of impact certain program features may have played in the outcome of the developmental experience. For example: o owere the learners provided with adequate support and resources for applying new skills and concepts? o odid the topics and supporting materials help the learners in achieving their development goals? o odid the learners discuss their action plans and performance results with their managers? It stands to reason a learner with higher levels of support from her manager will have increased learning impact. Capturing appropriate underlying metrics allows program managers and organizations to monitor, adjust, manage, and communicate the impact of their leadership development programs more accurately, efficiently, and effectively. Harvard ManageMentor s Measurement Emphasis on Learning Impact Because Harvard ManageMentor is an on-demand resource, it automatically captures these vital level three statistics through the On-the-Job component within each topic. Each lesson within a topic has a related performance goal. Upon completion of the topic, learners rate themselves against these goals and select personal performance goals. When selecting a performance goal, learners see their reflections from each lesson, further helping make the connection from learning a new skill to applying the new skill on the job. After selecting performance goals, learners then create an action plan to apply what they have learned to their everyday work. Before starting the action plan, learners take a post-topic Impact Survey, which captures their personal insights related to the impact of the learning and their overall experience. Learners then work on their action plan over a period of time and are mentored with emailed content, tips, and reminders to keep them motivated and on track. The Harvard ManageMentor On-the-Job component helps managers translate their ideas into practice and helps them learn from experience. After completing the On-the-Job component, learners take another Impact Survey. Comparing responses between the two surveys provides key measurements for determining the magnitude of learning impact. The two brief On-the-Job surveys assess learners performance across several impact drivers, such as: { 4 } driving learning impact with harvard managementor
SUPPORT FOR APPLICATION ON THE JOB I was provided with adequate support and resources (e.g., time, money) for applying new skills and concepts to my job. MANAGER REINFORCEMENT My manager and I have discussed my action plan or my results from applying new skills and concepts from this topic. COURSE RELEVANCE Have the topic and supporting materials (videos, handouts, etc.) helped me achieve my development goals? THE ON-THE-JOB COMPONENT HELPS MANAGERS TRANSLATE THEIR IDEAS INTO PRACTICE AND HELPS THEM LEARN FROM EXPERIENCE BUSINESS ALIGNMENT Is this topic aligned to my business priorities and development goals (only asked in the post-topic survey)? Three additional questions combine to create a learning impact measure. This measurement is the net improvement across these three areas: 1. Knowledge impact: I used new skills and knowledge from this topic. 2. Job impact: I have applied skills and concepts from this topic to my job. 3. Results: The new learning I have applied has improved my actual performance on the job. This approach to learning impact delivers a new, single measure of applied learning and answers the question: What is the impact of applying skills learned on their daily work and, by extension, the business? Reporting and Analytics Learning metrics should tell a story, one that provides information to a variety of stakeholders in order to help them make informed business decisions: ooinstructional designers focused on the effectiveness of the program and how the program aligns to the learner s job. ooorganizational leaders focused on who has completed the program and if they are applying the learning in a consistent manner. ool&d teams focused on how the investment in learning is being used and if it is having an impact on business results. Dashboards and reports should support the different levels and types of information required by these stakeholders. Managers also need quick access to deeper analytics and customized reporting so they can diagnose potential issues faster. WHO LEARNS FROM REPORTING? INSTRUCTIONAL DESIGNERS Effectiveness and skills alignment ORGANIZATIONAL LEADERS Completion and skills application L&D TEAMS How the investment is creating business results Learning analytics allow for the combination and analysis of multiple types of metrics. Simple correlational work all the way through HARVARD BUSINESS PUBLISHING CORPORATE LEARNING { 5 }
HARVARD MANAGEMENTOR ONE VIEW DASHBOARD Learning impact is the net improvement across three areas: Knowledge impact Job impact Results BEING ABLE TO SEE HOW A MANAGER S SUPPORT IMPACTS LEARNING IS CRITICAL TO MAKING SMART PROGRAM ADJUSTMENTS predictive modeling may be possible, all in the service of identifying relationships, patterns, and trends to determine the effectiveness of an overall program and its various component parts. As noted above, being able to see how a certain factor like a manager s support impacts learning is critical to making smart program adjustments. Harvard ManageMentor Metrics Administrators of Harvard ManageMentor have access to a powerful and easy-to-use analytics and reporting tool that gives them the opportunity to do that kind of detailed analysis. Called the One View dashboard, it includes a series of tiles that provide a snapshot of various aspects of the program, including how effective learners are in applying skills on the job. Learning impact is a single measurement shown in the One View dashboard, for all learners and topics. Drilling in to the Learning Impact report shows the learning impact percentage over time with filters available for all topics or a specific topic, and over a specific time period. The report also includes the average score for each impact driver compared to Harvard ManageMentor s benchmarking data. Having a benchmark helps target if and where adjustments to a program should be made. For example, if management support scores are low relative to the benchmark, then program managers can take extra steps to ensure that leaders are involved in the learning by using Harvard ManageMentor s Develop Others materials to facilitate a discussion with their teams to reinforce the targeted concepts. Data-based decision making is crucial to successful talent development and management and overall business success. Harvard ManageMentor s reports do not simply present information or results; instead, they { 6 } driving learning impact with harvard managementor
help drive decision making and identify possible actions related to the developmental programming and the organization s business. SUMMARY Achieving Impact In today s global, connected, and fast-paced environment, competitiveness requires leadership development that shows its impact on performance. Traditional approaches that look at just knowledge gain are not sufficient. We need a new approach that considers: ooa learning design that is fully oriented to application, integrating learning with work and requiring learners to think about how learning new skills can improve their performance. oothe right learning metrics that don t just measure knowledge gain, but also the impact new skills are having on individual and organizational performance. INTUITIVE DASHBOARDS, REPORTING, AND ANALYTICS GIVE L&D TEAMS THE TOOLS TO BEGIN TO ASSESS THE LEVEL OF LEARNING IMPACT ooreporting and analytics that provide views for different roles and levels, and that can also help determine which aspects of learning are working or not working for which audiences so that smart adjustments can be made. Harvard ManageMentor s on-demand design facilitates rapid application via concise, goal-oriented content; built-in cycles of learn-practice-reflect; and an On-the-Job component that helps learners apply new skills and reinforce critical business concepts. In addition, Harvard ManageMentor s intuitive dashboards, reporting, and analytics give L&D teams the tools to begin to assess the level of learning impact taking place and to continually improve their leadership development programs. While the learning impact reporting in Harvard ManageMentor is a powerful way to measure the success of a program, organizations should explore additional measures to help assess the level of impact their programs are having on their businesses bottom lines. ABOUT HARVARD MANAGEMENTOR Harvard ManageMentor is the premier on-demand learning and performance support resource for leadership and management skill development. Harvard ManageMentor s high-quality content regularly updated topics, videos, tools, and on-the-job activities engage and motivate learners. Harvard ManageMentor is fueled by the latest in thinking and proven practices from Harvard Business Publishing s world-class experts and designed to drive learning transfer and business impact for organizations. HARVARD BUSINESS PUBLISHING CORPORATE LEARNING { 7 }
ABOUT THE AUTHORS DIANE BELCHER is director of continuous learning for Corporate Learning. She is responsible for the product strategy for the Harvard ManageMentor offering. Prior to joining the Harvard Business Publishing team, Diane worked for Fidelity Investments, where she held various roles in human resources and led the customer experience strategy for the Workplace Investing business. diane.belcher@harvardbusiness.org DENNIS GULLOTTI is associate director, solutions marketing for Harvard Business Publishing. He is responsible for the go-to-market strategy and execution for Harvard ManageMentor, the premier on-demand learning resource for leadership and management development. He has over 10 years of experience in talent management and corporate learning, with previous work experience that includes positions at Plateau Systems and Thomson NETg. dennis.gullotti@harvardbusiness.org MICHAEL A. WARECH, PH.D., is a senior learning solutions manager with Harvard Business Publishing Corporate Learning. Michael has over 25 years of experience designing, developing, and implementing data-driven human capital solutions that demonstrably impact an organization s bottom line. He has led numerous projects for a variety of Fortune 1000 companies and organizations. He has worked with companies to identify and articulate their learning strategies and has deep and direct experience designing leadership development solutions as well as more broad and robust learning and development curricula. michael.warech@harvardbusiness.org { 8 } driving learning impact with harvard managementor
UNITED STATES Harvard Business Publishing 60 Harvard Way Boston, MA 02163 phone: 800-795-5200 (Outside U.S. and Canada call 617-783-7888) email: corporate@harvardbusiness.org EUROPE Harvard Business School Publishing Europe Pvt. Ltd. Vernon House 23 Sicilian Avenue London WC1A 2QS phone: +0203 463 2350 email: CLeurope@harvardbusiness.org INDIA Harvard Business School Publishing India Pvt. Ltd. Piramal Tower, 7th floor Peninsula Corporate Park Ganpatrao Kadam Marg Lower Parel (West), Mumbai 400 013 phone: +91 22 6108 4043 email: infoindia@harvardbusiness.org SINGAPORE Harvard Business School Publishing Asia/Pacific Pvt. Ltd. 80 Raffles Place, #25-01 UOB Plaza 1 Singapore 048624 email: CLasia-pac@harvardbusiness.org LEARN MORE www.harvardbusiness.org 2015 Harvard Business School Publishing. All rights reserved. Harvard Business Publishing is an affiliate of Harvard Business School. MC192370515