CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE

CONTENTS 3 Introduction 5 The Learner Experience 7 Perceptions of Training Consistency 11 Impact of Consistency on Learners 15 Conclusions 16 Study Demographics 18 About This Research CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 2

CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE Introduction Consistency in training is a concept that touches nearly everything in L&D. Not surprisingly, maintaining consistency comes up again and again in Training Industry research as one of the biggest challenges to delivering an effective training portfolio to a population of learners. These learners may be employees on opposite sides of the country who need to gain proficiency in the same set of skills or learners outside the organization, such as customers seeking information about how to use a company s products or services. No matter where they hail from, however, a common underlying objective of providing learning opportunities for these individuals is that their experience doesn t vary from one day to the next, from one location to the next or from one instructor to another. Consistency is a driver and sometimes a requirement for a variety of training initiatives. For instance, certifications need reliable presentation of materials to meet specific learning goals. Learning technologies can facilitate consistency by requiring instructional designers to design content and courses within the framework of the technology platform. Considering that a company s learning management system (LMS) represents a large portion of an organization s L&D budget, one would think this functionality would be a high priority in the selection process. Unfortunately, this is not always the case. Despite all the time and money that companies invest to try to deliver a consistent learning experience, are learners able to recognize consistency from training event to training event, and do they value its importance? There is an argument that it might not matter that much that learners are simply the end users, and consistency is a strategic consideration that manifests in ways not obvious to most learners. The counterargument is that it matters quite a bit. The more learners experience training courses that are structured, the more likely they are to approach their next training expecting to learn from the structure in the same way. Drastic changes in how content is organized, accessed or delivered from event to event can confuse learners and lead to poorer retention and CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 3

transfer of knowledge and skills. This is true not because the content is bad, necessarily, but because the learners have to begin each training by trying to determine what to expect in terms of the amount of pre-work involved, the type of learning format they will use, the number of participants (small group versus lecture), and how much participation is required or expected on their part. This statement assumes that the learner is motivated to put in this effort to engage with the training. So, what kind of effect does training consistency really have on learners? To examine these issues in greater detail, Training Industry, Inc. and The Training Associates Corporation conducted a study to understand how learners feel about the consistency of the training provided by their employers. In early 2017, 283 learners completed a confidential survey that explored how they prefer to be trained, their opinion of how consistent their learning experience is across training programs and the impact that they see on their job as a result of training. WHAT KIND OF EFFECT DOES TRAINING CONSISTENCY REALLY HAVE ON LEARNERS? CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 4

The Learner Experience First, learners were asked a series of questions about the effectiveness of the training they received at their respective companies. As shown in Figure 1, 31 percent of learners rated their company s training as always or almost always effective. In contrast, 32 percent of learners rated their company s training as sometimes effective or not effective. Figure 1. Learner Ratings of Overall Training Effectiveness Figure 3. Use of Training Modalities in the Past 12 Months E-learning 52% ILT 49% Webinars/Podcasts 30% VILT 28% Videos 28% Mentoring/Coaching 2 Social/Collaborative learning Always effective Almost always effective Usually effective 1 20% 35% Blended (ILT + E-Learning/VILT) 15% Microlearning 13% Sometimes effective 25% Mobile learning 9% Not effective 7% Game-based learning Effectiveness, however, is not the only marker for the quality of the cumulative learning experience. As shown in Figure 2, 38 percent of learners rated their company s training as always or almost always consistent. However, 12 percent rated their company s training as not at all consistent. Between Figures 1 and 2, it is clear that one-third of learners may have less than stellar opinions of the training they frequently encounter. Next, learners were asked about the different types of modalities used in the training events they participated in during the last year. As shown in Figure 3, the two most common modalities learners experienced in the past 12 months were e-learning (52%) and instructor-led training (ILT; 49%). For more context, each modality was counted to gauge how common multimodal training is in learning experiences. Figure 4 shows the number of different modalities used by learners in the past year. As shown, 52 percent of companies used two to four different modalities to deliver training. For Figure 2. Learner Ratings of Overall Training Consistency Figure 4. Frequency of Multiple Modality Use Always consistent Almost always consistent Usually consistent Sometimes consistent 9% 20% 29% 30% 32% 2 1 Not at all consistent 1 2 3 4 5 6 7 8 9+ CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 5

Figure 5. Learning Modality Preferences ILT 38% Mentoring/Coaching 15% Blended (ILT + E-Learning/VILT) 9% E-learning 9% Social/Collaborative learning 7% Webinars/Podcasts 7% VILT Videos Microlearning 2% Game-based learning 2% Mobile learning 0% the 32 percent that used a single delivery method, the data showed that e-learning was by far the most prevalent, followed by ILT and webinars, mirroring the pattern in Figure 3. The modalities that learners participate in most frequently, however, may not necessarily be the modalities that learners would like to use if given a choice. As such, survey respondents were asked to indicate which training delivery method they felt was the single most effective and impactful modality. As shown in Figure 5, learners generally prefer ILT over and above all other modalities. This preference was found regardless of whether their company is effective or consistent at delivering training. Next, respondents were asked to identify what types of training they had participated in over the past 12 months. As shown in Figure 6, learners were most likely to participate in job-specific technical training. Between one-third and one-quarter of learners completed a certification program, regulatory/ compliance training, diversity or HRrelated training, or health/occupational safety training. Figure 6. Participation in Training Topics Job-specific technical 45% Certification programs 30% Regulatory/Compliance 28% Diversity/HR 27% Health/Occupational safety 25% Customer Service 2 Soft Skills 23% IT 22% Process/Quality 19% Leadership development 18% Product/Service knowledge New employee orientation 15% Management/Supervisory skills 9% Sales skills 8% Market education 5% Sales methodology 5% In sum, the collective perspectives of the learners providing data for this study represent a varied cross-section of experiences as they relate to training effectiveness, modalities, topics and overall training consistency. However, the devil is in the details, and the next section further explores the ingredients of training consistency. CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 6

Perceptions of Training Consistency Although most discussions surrounding training consistency occur with an understanding that everyone is talking about the same thing, consistency can be much more multifaceted. In this research, there were four areas of consistency for which respondents rated their experiences at their company: 1. Training content (i.e., related to the creation and management of instructional material, generally referred to as courses, curriculum, workshops, seminars, webinars or programs) 2. Training delivery (i.e., methods associated with the transfer of information) 3. Learning technology (i.e., technologies used for creating, managing and delivering training) 4. Training administration (i.e., activities and processes associated with managing the logistics and day-to-day operations of the training function) GREAT CONTENT CAN LANGUISH IN POOR DELIVERY. Figure 7. Consistency Perceptions of Training Content 5% 8% 5% 1 2 40% 1 25% 3 22% 13% 25% 39% 15% 1 28% 3 29% 32% 1 33% 30% 19% 22% 25% 29% Look and feel of training content Structure of training content Appropriate level of detail in training content Appropriate amount of training content in any given program Training content appears up to date Training content seems well researched Training content is job-relevant Always effective Almost always effective Usually effective Sometimes effective Not effective CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 7

For each specific area, respondents were asked to rate the effectiveness of their companies at delivering a consistent learning experience across training programs. The first area of consistency concerned the training content that learners encounter. As shown in Figure 7, learners felt their companies were most effective at ensuring training content is related to their jobs (a combined 51 percent of learners rated their company as always effective and almost always effective ). About 45 percent of learners also agreed that the training content they re exposed to is consistently both well researched and current. However, when it comes to the look, structure and amount of content, learners were less emphatic. Roughly 25 percent of learners rated their companies as marginally effective or ineffective at providing content that is consistent across these factors. Some learning leaders, when considering the consistency of their training programs, may think of consistency as only applying to content. However, great content can languish in poor delivery. Accordingly, respondents next rated the consistency of the delivery of training they have experienced at their companies. As shown in Figure 8, the strengths of most companies training delivery centers around aspects of classroom learning. Between 43 and 46 percent of learners rated the engagement and expertise of instructors, along with the quality of classroom facilities, as always effective or almost always effective. Companies Figure 8. Consistency Perceptions of Training Delivery Instructor delivering training is engaging Expertise of the instructor Quality of inperson training facilities Quality of virtual classroom platform Matching training modality with content Collecting feedback on improvements Physical surroundings conducive to virtual training 2% 3% Always effective 1 13% 1 15% 22% 2 20% 13% 19% 18% 29% 32% 29% 3 27% 28% 2 28% 23% 28% 30% Almost always effective 3 Usually effective Sometimes effective Not effective CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 8

were seen as less consistent when it came to aspects of virtual training, but learners were particularly split on their employers effectiveness at collecting feedback for improving future training programs. Next, respondents rated the consistency of their experience with the learning technologies their companies use. As shown in Figure 9, companies fare best when using forms of virtual training that function as delivery platforms and when offering training without technology issues caused by company intranets or software limitations. When it comes to using social tools for collaboration, however, many companies appear to be lagging behind, as only 25 percent of learners rated their companies as always effective or almost always effective. It is also notable that across all four categories of consistency, learning technology garnered the lowest endorsements of effectiveness from learners. Lastly, ratings were made on the consistency of the administrative activities concerning training. As shown Figure 9. Consistency Perceptions of Learning Technology 1 22% 28% 15% 13% 2 3 8% 1 18% 2 1 23% 27% 20% 2 29% 18% 10% Using/ navigating an LMS Using virtual training as delivery platforms (e.g., videos, e-learning) Online social and collaboration tools to support training programs Reinforcement tools (e.g., post-training quizzes/ activities) The ability to complete training without experiencing technology issues Always effective Almost always effective Usually effective Sometimes effective Not effective CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 9

in Figure 10, registering for training (48%) and tracking subsequent course completion (5) were the two areas where most companies seem to be providing a consistent experience for learners. Cumulatively, Figures 7 through 10 show that consistency can be meaningfully broken down into many constituent parts and that learners are capable of making distinctions among different aspects of their learning experience and the consistency thereof. But why should companies care? LEARNERS GENERALLY PREFER ILT OVER AND ABOVE ALL OTHER MODALITIES AS THE MOST IMPACTFUL AND EFFECTIVE MODALITY. Figure 10. Consistency Perceptions of Training Administration 19% 29% 2 5% 2 28% 8% 22% 32% 25% 1 7% 1 23% 29% 15% 1 1 18% 25% 2 10% The ability to schedule/ register for training Navigating an individual learning plan Tracking training completion The use of pre-/ post-test to support learning Internal marketing to publicize training programs Always effective Almost always effective Usually effective Sometimes effective Not effective CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 10

Impact of Consistency on Learners To this point, this report has described learners perceptions about the effectiveness and areas of consistency of training at their respective companies. But are such perceptions merely an interesting footnote to training strategy, or do they have a tangible impact on employee performance? This section makes the case for the latter by illustrating the insights gained from an analysis of the relationships between consistency and training outcomes. First, survey respondents were asked about the ongoing learning and development opportunities at their organizations. As shown in Figure 11, less than one-fifth of companies are always providing professional development opportunities. It is notable that 15 to 24 percent of learners responded rarely across all items, and, in particular, 12 percent of learners felt that their company does not provide them with training that could prepare them for CONSISTENCY IS SIGNIFICANTLY RELATED TO A LEARNER S MOTIVATION, PERCEPTIONS OF TRAINING EFFECTIVENESS AND DISPOSITION TOWARD PROFESSIONAL DEVELOPMENT. positions in the future. On a five-point scale, with 5 being always and 1 being 1 Bulut, C., & Osman, C. (2010). The effects of organizational training on organizational commitment. International Journal of Training and Development, 14(4), 309-322. Figure 11. Perceptions of Professional Development Opportunities 5% 7% 10% 25% 19% 1 19% 2 27% 15% 22% 20% 25% 3 3 3 33% 38% Provides professional development opportunities that help me be more effective at my daily job tasks Makes professional development opportunities available that upskill me for more advanced positions in the company Provides professional development opportunities for all employees Provides professional development opportunities for employees with jobs similar to mine Provides professional development opportunities primarily for senior/ management positions Always effective Almost always effective Usually effective Sometimes effective Not effective CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 11

never, the combination of responses averaged 3.20. These results indicate that across all survey respondents, the majority of companies sometimes offer professional development opportunities to learners. Learners motivation for training is the extent to which they are willing to improve their knowledge, skills and abilities (KSAs) by engaging with training offerings. This concept was measured using eight items 1. As shown in Figure 12, most learners are highly motivated to engage with training, as evidenced by the relatively high proportion of strongly agree and agree responses across items. On a five-point scale, with 5 being strongly agree and 1 being strongly disagree, the combination of responses averaged 3.84. These results suggest that most learners are motivated to improve their KSAs. Why describe the perceived opportunities and level of motivation of learners? Because, as we ll see, these measures are related to the consistency of training as it is understood by learners in their companies. The results that follow are derived from a correlational analysis of many of the collected ratings and scales used in this research to test whether significant relationships exist between training consistency and the attitudes and experiences of learners. This analysis averaged each of the component sets of ratings for training consistency to create four values that align to each component of training consistency (i.e., companies consistency in the content, delivery, technology and administration of training). These averaged ratings are Figure 12. Motivation for Training I try to learn as much as I can from training programs. I tend to learn more from training programs than most people. I am usually motivated to learn the skills emphasized in training programs. I am willing to exert considerable effort in training programs in order to improve my skills. I believe I can improve my skills by participating in training programs. I believe I can learn the material presented in most training programs. Participation in training programs is of little use to me because I have all the knowledge and skills I need to successfully perform my job. I am willing to invest effort to improve skills and competencies related to my current job. I am willing to invest effort to improve skills and competencies in order to prepare myself for a promotion. 8% 5% 2% 0% 2% 7% 1 13% 13% 18% 18% 18% 23% 22% 29% 33% 35% 42% 30% 30% 33% 22% 3 3 Strongly Agree Agree Neutral Disagree Strongly disagree 49% 48% 50% 48% 5 48% 49% CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 12

based on the data presented in Figures 7 through 10. LEARNERS AT COMPANIES WITH EFFECTIVE AND CONSISTENT L&D PRACTICES WERE SIGNIFICANTLY MORE LIKELY TO BE MORE SATISFIED IN THEIR JOBS. All four of these areas of consistency were found to be significantly related to learners ratings of professional development opportunities, with correlations ranging from 0.57 to 0.63. The size of these correlations denotes large relationships, 2 which usually implies that such results are not a hiccup in the numbers. In other words, there is a correlational relationship between consistency across all facets of training and learners perceptions that their company provides professional development through training. So, when learners see frequent opportunities to better themselves, these learners are likely to work at companies that offer a consistent L&D experience. All four of the areas of consistency were also found to be significantly related to learners ratings of overall training effectiveness, with correlations ranging from 0.43 to 0.58. These are considered medium-to-large correlations, demonstrating a significant relationship between aspects of training consistency and the general effectiveness of a company s L&D offerings. In other words, the more consistent a company s training is, the more effective it tends to be. Lastly, all four of the areas of consistency were found to be significantly related to learners ratings of motivation for training, 2 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates. CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 13

with correlations ranging from 0.28 to 0.35. These are considered medium relationships, so while they are not as strong as the relationships between consistency and professional development opportunities, they nonetheless represent a significant relationship between consistency and learner motivation. Further, the relationships between consistency and motivation are stronger than the correlation between training effectiveness and motivation, which was significant but comparatively small in magnitude (r = 0.15). This finding suggests that a learner s motivation may have more to do with the experience of training than with the ultimate effectiveness of the KSAs acquired. Next, survey responses were entered into a series of analysis of variance (ANOVA) tests, which determined whether there were significant differences among the average scores for groups of learners. This analysis created the groups of learners by crossing ratings of training effectiveness (Figure 1) with ratings of TRAINING CONSISTENCY IS MULTIFACETED. training consistency (Figure 2). This resulted in a group of 68 learners who rated their respective company s training as both effective and consistent and an opposing group of 72 learners who rated their company s training as both ineffective and inconsistent. Compared to companies that provide ineffective and inconsistent training programs, results of this analysis showed that learners at companies with effective and consistent L&D practices were significantly more likely to: Be motivated to make efforts to improve themselves and their job performance by participating in training 3 Be more satisfied with their jobs 4 Have much greater awareness of the benefits and opportunities afforded by training for their own professional development 5 To summarize, consistency is significantly related to a learner s motivation, perceptions of training effectiveness and disposition toward professional development. Moreover, companies whose training programs are both consistent and effective are more likely to have motivated learners who are satisfied with their jobs and see training as a means to professional development. Consistency in training and L&D practices is a systemic endeavor, and its effects stretch across entire organizations. 3 F(1,138) = 17.53, p <.001, averages by group: 4.09 (consistent/effective) vs. 3.71 (inconsistent/ineffective) 4 F(1,136) = 44.58, p <.001, averages by group: 4.44 (consistent/effective) vs. 3.43 (inconsistent/ineffective) 5 F(1,138) = 116.10, p <.001, averages by group: 3.91 (consistent/effective) vs. 2.39 (inconsistent/ineffective) CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 14

Conclusions Consistency has multiple levels because it can be applied in many ways: consistency in a specific training event; learning path consistency for an employee in a particular department; and enterprise consistency at the strategic, crossfunctional level. Consistency is ultimately systemic, because it cannot be removed from the context in which it occurs; consistency both affects and is affected by the organization providing the training. Throughout the results of this research, it is clear that learners understand consistency as something worthy of their attention. Nine percent saw the training they re offered as being always consistent, and 12 percent saw their GREATER CONSISTENCY CAN BOOST LEARNERS ENGAGEMENT. training as anything but. This leaves nearly four-fifths of learners experiences falling somewhere in between the poles of this continuum. In other words, companies have ground to make up when it comes to the consistency of their training. Why should companies close the gap? What benefit do learners perceptions of consistency hold for the companies they work for? As we ve shown in the preceding pages, the benefits are plenty, based on the statistical findings. For instance, ratings of training consistency (i.e., companies that are more consistent in the content, delivery, technology and administration of training) were significantly related to learners ratings of the professional development opportunities they see available to them and to their motivation to learn. These findings suggest that a company that invests in the consistency of its training will tend to have learners who are more engaged both with the training and with their career trajectory at the company. This consistency also tends to be prevalent in companies with effective training programs. Further, companies with effective and consistent training have learners who are more motivated and satisfied with their work and have a positive regard for professional development, compared to learners at companies struggling with effectiveness and consistency in their L&D offerings. If learners were generally unaware of or indifferent to their training experiences, the data would have painted a drastically different picture. As stated in the introduction to this report, consistency has been repeatedly identified in Training Industry research as a challenge for learning leaders. The results of this report underscore the downstream effects of meeting this challenge (or falling short) from the viewpoint of the learners at the receiving end of training programs. So, how consistent has your company s training been lately? CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 15

Study Demographics Figures 13 through 16 provide context on the 283 survey respondents who participated in this research. These figures summarize the companies the respondents represent and the functional areas/departments supported by their roles within the organization. Figure 13. Organizational Size 2 2 15% 10% 1 8% 3% < 100 101-500 501-1k 1k-5k 5k-10k 10k-20k 20k-50k 50k+ Figure 14. Industries Represented Health Care/Medical/Pharma 20% Durable Goods/Consumables Government Education 1 10% Technology/Telecom Banking/Finance/Insurance 8% 8% Manufacturing Entertainment/Hospitality Nonprofit Business Services/Consulting Training and Development Utilities Construction 3% 2% 2% Other CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 16

Figure 15. Functional Areas/Departments Represented 33% 18% 1 8% Customer Service IT L&D Operations R&D Finance/ Accounting HR Sales Marketing/ Advertising Other Figure 16. Job Roles Represented 40% 18% 15% 1 5% 5% Associate Specialist Analyst Manager Trainer Consultant Instructor Executive Level Instructional Designer CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 17

About This Research About The Training Associates Founded in 1994, The Training Associates is the industry recognized leader of learning and development talent and solutions. TTA s talent offerings feature over 25,000 L&D resources specializing in adult learning for all technologies and business professional skills. Our services include our ability to undertake and manage projects of any size or scope related to the design, development, and delivery of training. For more information, visit www.thetrainingassociates.com, call 800.241.8868, contact us at info@ttacorp.com, or connect with us on Twitter and LinkedIn. About Training Industry Our focus is on helping dedicated business and training professionals get the information, insight and tools needed to more effectively manage the business of learning. Our website, TrainingIndustry.com, spotlights the latest news, articles, case studies and best practices within the training industry. For more information, visit www.trainingindustry.com, call 866.298.4203, or connect with us on Twitter and LinkedIn. Training Industry, Inc. research captures the collective wisdom of learning professionals, revealing fresh data on trends and practices in the evolving training market. Copyright 2017 by The Training Associates and Training Industry, Inc. All rights reserved. No materials from this study can be duplicated, copied, republished or reused without written permission from The Training Associates or Training Industry, Inc. The information and insights contained in this report reflect the research and observations of Training Industry, Inc. and the Training Associates Corporation. CONSISTENCY OF TRAINING AND THE LEARNING EXPERIENCE 2017 18

Copyright 2017 by The Training Associates and Training Industry, Inc. All rights reserved. No materials from this study can be duplicated, copied, republished or reused without written permission from The Training Associates or Training Industry, Inc. The information and insights contained in this report reflect the research and observations of Training Industry, Inc. and the Training Associates Corporation.