Work Based Learning E-Learning Impact Survey JISC ADVANCE RSC WALES ILT/e-Learning Survey Russell Symmons Alyson Dacey March July
EXECUTIVE SUMMARY This document reports on a survey carried out by JISC Advance Regional Support Centre for Wales (RSC Wales) in the first half of. The aim of the survey was to compare the outputs with an identical survey carried out in the same period 2008 thus providing evidence to demonstrate that the Work Based Learning sector in Wales is increasingly adopting the innovative use of technology to enhance core business functions supported by activities undertaken by the RSC. Invitations to participate in the survey were sent to senior managers of 80 Work Based Learning organisations. In total there were 25 unique organisational responses (31%). Questions were based on the nationally recognised WBL e-learning Positioning Statement, which forms part of the RSC s Information Learning Technology (ILT) Strategy Toolkit. Findings show a significant increase in the proportion of WBL providers in Wales who have introduced an e-learning strategy. It is envisaged that support in relation to evaluating and refining the strategy will be a key feature of forthcoming RSC Wales activity. Due to a significant minority of providers yet to develop an ILT strategy, this area of support will also continue to feature prominently. Providers are increasingly recognising and utilising ILT and the Internet in order to meet the needs of learners. With a significant number of providers developing and re-engineering their resources accordingly, it is anticipated that RSC Wales will continue to support such activities in a variety of formats. There is evidence of increased use of technology in both assessment and verification processes. Strategic interventions such as the introduction of e-portfolios are taking place in a considered fashion. With the exception of utilising technology to communicate with learners a consistent number of providers fail to make use of technology to support, inform or record learners progress. In all other instances of learner support, as with the uptake of e- portfolios, findings show a steady move away from solely paper based practices with organisations percolating from one phase of development to another. The evidence shows organisations are increasingly understanding and meeting staff development needs resulting in greater confidence and competence of all staff in the use of ILT. However, there seems to be a correlation between those providers yet to engage meaningfully in staff development activities with those yet to engage in utilising technology to maximise learner support and teaching and learning opportunities. The use of computers/ict equipment and increasingly robust Internet connectivity is providing greater opportunities for organisations to enhance their learning provision as evidenced by the increasing development of learning platforms. However, if 1
organisations are to successfully adopt the use of ILT or e-learning then a robust technical support service is of paramount importance and could be seen as a key action point moving forward. Providers are increasingly committed to implementing, reviewing and funding the use of ILT or e-learning. RSC Wales and JISC Advance offer a range of services to support implementation, development and review of strategy and Business Processes. Due to National Policy drivers there is the increased likelihood of greater partnership working throughout the sector and with other learning providers. RSC Wales views the development of Learning Partnership as a Regional Priority activity and is currently developing a range of activities demonstrating the benefits technology can bring in enhancing collaborative working arrangements. 2
Contents EXECUTIVE SUMMARY... 1 1. INTRODUCTION... 4 2. VISION AND STRATEGIC PLANNING... 6 3. TEACHING AND LEARNING... 7 4. ASSESSMENT AND VERIFICATION... 8 5. LEARNER SUPPORT... 9 6. STAFF DEVELOPMENT... 11 7. INFRASTRUCTURE AND EQUIPMENT... 13 8. MANAGING AND IMPLEMENTING E-LEARNING... 15 9. RELATED DOCUMENTS... 17 10. ACKNOWLEDGEMENTS... 17 3
1. Introduction This document reports on a survey carried out by RSC Wales in the first half of. The aim of the survey was to compare the outputs with an identical survey carried out in the same period 2008 thus providing evidence to demonstrate that the Work Based Learning sector in Wales is increasingly adopting the innovative use of technology to enhance core business functions supported by activities undertaken by RSC Wales. Organisational Details Invitations to participate in the survey were sent to senior managers of 80 Work Based Learning organisations. In total there were 25 unique organisational responses (31%) compared with 39% in 2008. FIGURE: 1-1 45% 40% 0.39 35% 30% 25% 20% 15% 10% 0.16 0.13 0.22 0.1 2008 5% 0% Single location Multi-site FE College Local Authority Not-for-profit 0 Other Figure 1:1 makes a comparison of the type of provider that completed the survey. The sample is heavily biased towards the Private Training Provider (64%) with Local Authorities (8%), FE Colleges (16%), Not for Profit Organisations (8%) and Other (4%). Survey Methodology The survey questionnaire was developed using Lime Survey, a web based tool that enables users to create on-line surveys, and was broken down into the following themes: Organisational Details Vision and Strategic Planning Teaching and Learning Assessment and Verification 4
Learner Support Staff Development Infrastructure and Equipment Managing and Implementing ILT and e-learning Questions were based on the nationally recognised WBL e-learning Positioning Statement, which forms part of the RSC s ILT Strategy Toolkit 1, and largely of a quantitative nature. Opportunity was given for amplification or explanation of responses providing a qualitative aspect to the research. 1 http://rscwbl.jiscinvolve.org/inspection/ 5
2. Vision and Strategic Planning Organisations that adopt a holistic approach when developing a vision into a strategic plan address several key points that assist with the transition and form the basis for continual improvement. Such topics include knowledge sharing, relevance, measuring impact and external influences. Findings show that since 2008 there has been a significant increase in the proportion of providers in Wales with a clear vision regarding the use of ILT or e-learning and in most cases this vision is communicated throughout the organisation. FIGURE 2-1 ILT or e-learning Strategy 60% 50% 40% 30% 20% 10% 0% Not Yet Started Early Stages Developing Established Embedded 2008 Similarly, the number of organisations working towards realising the vision and developing an ILT strategy has also increased markedly as shown in fig 2.1. In such cases the evidence suggests that the ILT Strategy is being cross referenced to other organisational strategic plans, that there is a considered approach to its management and some coherence with local, regional and national strategic priorities. Currently a third of providers are undertaking some review and evaluation activities of the impact and effectiveness of the ILT strategy with a similar number beginning to consider the issue. This represents a considerable shift in activity since 2008. As the introduction of ILT strategies become more established it is anticipated that this number will continue to grow accordingly. ILT Strategy Development was seen as a key finding in the 2008 Work Based Learning E-learning Survey. This finding was communicated to the sector at the National Training Federation for Wales (NTFW) Annual conference in October 2008 and addressed by a succession of workshops held throughout Wales. Key Findings The evidence shows a significant increase in the number of WBL providers in Wales who have introduced an e-learning strategy. It is envisaged that support in relation to evaluating and refining the strategy will be a key feature of forthcoming RSC Wales activity. Due to the significant minority of providers yet to develop an ILT strategy, this area of support will also continue to feature prominently. 6
3. Teaching and Learning Modern technologies have the potential to enhance teaching and learning in a number of ways, offering learners the opportunity to learn in a style, at a time and in a place of their choosing. Greater emphasis is also being placed on the Learner Voice by both the Welsh Assembly Government and Estyn through initiatives like the Quality and Effectiveness Framework for Post 16 Learning. It is difficult to envisage meeting such demands without the use of technology. FIGURE 3-1 Teaching and Learning Styles 50% 40% 34% 4% 4% 24% 20% 8% 4% 12% 2008 Not Yet Started Early Stages Developing Established Embedded Since 2008 there has been a significant increase in the volume of providers using e- learning to cater for varying learning styles and needs. The availability of electronic resources in largely mirrors the numbers of providers using e-learning to cater for learning styles although there is some evidence that a small number of providers have failed to maintain the pace of development. However, such an anomaly may be addressed by considering whether providers are being more analytical or whether the same organisations took part in both surveys. Findings similarly indicate a steady increase in the number of organisations using the Internet for teaching and learning during the period 2008- and consequently adapting their teaching and learning provision to offer greater flexibility. This result is further supported by observations, requests for support and anecdotal evidence gained during provider visits undertaken by RSC Wales. Key Findings The evidence shows that providers are increasingly recognising and utilising ILT and the Internet in order to meet the needs of learners. A key feature of RSC Wales activity throughout has been focused on recording and reacting to the learner voice. With a significant number of providers developing and re-engineering their resources accordingly, it is anticipated that RSC Wales will continue to support such activities in a variety of formats. 7
4. Assessment and Verification Traditionally learners have compiled paper-based portfolios satisfying the requirements of the programme they are undertaking. Although still common practice, the process can be labour intensive and cumbersome, especially as many of the documents contained may be hard copies originally produced digitally e.g. word processed documents, digital photographs etc. Incorporating the use of technology into the assessment process can alleviate these issues whilst also addressing the driving forces of sustainability and efficiency savings. Survey findings indicate that technology is being increasingly used in all areas of assessment and verification. Fig 4.1 shows a marked increase in the number of providers using ILT to record evidence. FIGURE 4-1 Portfolio Evidence 42% 33% 38% 33% 21% 21% 2008 4% 4% 0% 4% Not Yet Started Early Stages Developing Established Embedded In relation to e-assessment the data suggests that, since 2008, a small number of providers have moved positively from one stage to another with a significant proportion stating it is organisational practice for technical certificates and numeracy/literacy to be tested through e-assessment wherever possible. The vast majority of participants though, remain in the Early and Developing Stages. This finding reflects discussions undertaken between providers and RSC Wales showing organisations to be largely positive towards the concept of e-assessment but wishing to take a considered and measured approach towards implementation. The use of technology to plan verification sampling and record Internal Verification decisions is increasing rapidly with nearly a third of respondents stating it is now accepted practice. Key Findings Providers are increasingly using technology in both assessment and verification processes. Strategic interventions are taking place in a considered fashion. RSC Wales will continue to offer support relating to the use of e-portfolios and technologies beneficial to assessment and evidence gathering/recording. 8
5. Learner Support The use of technology assists organisations in disseminating information in a variety of guises as well as gathering and collating information on which future plans and decisions can be based e.g. satisfaction surveys. The resultant information can be made available in a format, location and time to suit the individual with currency of information being an additional benefit in that information can be updated as and when necessary. Similarly, a move away from reliance on paper based resources that can be lost or damaged, assists in meeting efficiency and sustainability standards. FIGURE 5-1 84% Additional Support for Learners 48% 4% 17% 26% 8% 4% 0% 0% 9% 2008 Not Yet Started Early Stages Developing Established Embedded Despite such benefits and with the exception of utilising technology to communicate with learners, the uptake of technology to provide information, advice and guidance has increased at a lower than expected rate. This is surprising given the many established communication technologies employed by providers such as web sites, email and mobile phones. This picture is largely replicated when using technology to provide additional support for learners. RSC Wales anticipated this situation and so delivered a series of 8 events throughout Wales addressing Personalised Learning and Accessibility and Inclusion. Feedback suggests that providers are now better placed to identify and realise the benefits of technology where additional support is required with uptake expected to increase accordingly. The majority of providers recognise the importance of developing the learners IT/ICT skills in order to enhance their learning experience or to move towards electronic means of assessment, with only a small minority failing to address the issue. As with the uptake of e-portfolios, the evidence suggests a steady move away from solely paper based practices with a small number of organisations percolating steadily from one phase of development to another. 9
Key Findings With the exception of utilising technology to communicate with learners a consistent 17% of providers fail to make use of technology to support, inform or record learners progress. In all other instances, as with the uptake of e-portfolios, findings show a steady move away from solely paper based practices with a small number of organisations percolating from one phase of development to another. 10
6. Staff Development Since the 2008 RSC Wales WBL e-learning survey providers have frequently sought assistance in terms of raising staff awareness of various technologies and how they may impact upon core business processes and the teaching and learning functions. Organisations seeking to adopt e-learning or the use of ILT throughout the organisation recognise staff competence and confidence to be key components for success. Fig 6.1 illustrates a profound increase in the number of providers considering staff skills in relation to the use of ILT and e-learning. In the majority of cases this recognition is built upon through formal Continual Professional Development activities and making best use of enthusiasts. FIGURE 6-1 Analysis and Understanding of Staff Development Needs 48% 50% 4% 4% 9% 30% 23% 14% 9% 9% 2008 Not Yet Started Early Stages Developing Established Embedded Survey findings further indicate an increase in confidence and competence of trainers, assessors and, particularly, administrative staff. The evidence also suggests that failing to meaningfully implement ILT staff development activities impacts accordingly on Teaching and Learning and Learner Support. Where implemented, all organisations now report instances where ILT staff development activities have had a positive impact across the organisation. In 36% of cases providers initiated monitoring systems and identified that practices across the organisation were beginning to change as a result of such interventions, with a further 9% having established reporting processes evidencing a positive impact on practice across the organisation. Key Findings The evidence suggests that organisations are increasingly understanding and meeting staff development needs resulting in greater confidence and competence of all staff in the use of ILT. However, there seems to be a correlation between those providers yet to engage meaningfully in staff development activities with learner 11
support and teaching and learning opportunities. In response to such outcomes, RSC Wales is developing activities to further support learning providers encompassing the role technology can play in enhancing teaching and learning, including VLE, Web 2.0, mobile learning, immersive environments, accessibility, and aspects of the learner voice through use of learners own technology. 12
7. Infrastructure and Equipment Redundancy and suitability of infrastructure and equipment can be a serious concern both in terms of learning and efficiency due to the fast paced and evolutionary changes in the world of technology. One such example is the move away from desk based technologies to portable or mobile technologies, something providers need to consider when developing online or electronic learning materials. All respondents continue to provide access to computers and other ICT equipment with increasingly robust Internet connectivity. Over three quarters of learning providers state that access to computers and other ICT equipment is available at most if not all of their learning and assessment locations. However, a significant FIGURE 7-1 35% Learning Platform Development 43% 22% 24% 24% 22% 13% 10% 8% 2008 0% Not Yet Started Early Stages Developing Established Embedded minority consider themselves to be in the early stages of development offering a limited service and facing some difficulty in utilising technology for teaching and learning. There has been an overall increase in the engagement with and development of learning platforms. Such findings are corroborated by discussion and support activities undertaken by RSC Wales since 2008. Similarly, learning providers have redefined what they originally perceived as a learning platform which included the organisation Intranet or shared drive to now mean Virtual Learning Environments such as Moodle, a major project in terms of design, content creation and addressing sound pedagogic principles thus taking a substantial amount of time and focus. Findings show a slight dip in the availability and flexibility of technical support for staff and learners since 2008 with the majority of providers currently developing or re-designing their support provision. This may in part be accounted for by the greater range of technology utilised by organisations and learners in when compared to 2008. 13
Key Findings The use of computers/ict equipment and increasingly robust Internet connectivity is providing greater opportunities for organisations to enhance their learning provision as evidenced by the increasing development of learning platforms. However, if organisations are to successfully adopt the use of ILT or e-learning then a robust technical support service is of paramount importance and could be seen as a key action point moving forward. 14
8. Managing and Implementing e-learning Many of the sub-topics included in Managing and Implementing e-learning relate to an organisation s ILT or e-learning strategy and so fewer organisations responded to this section reflecting the fact that not all providers have an e-learning strategy. Similarly, those providers in the early stages of developing strategy may not yet be in the position to review its implementation and effectiveness. Since 2008 there has been a positive move by organisations to implement their ILT strategies and action plans. The evidence also suggests that where providers have FIGURE 8-1 Monitoring and Reviewing Implementation 52% 24% 13% 38% 22% 24% 9% 14% 4% 0% 2008 Not Yet Started Early Stages Developing Established Embedded an ILT strategy it has been implemented is subject to review and is increasingly aligned to other organisational aims and policies. The survey results show all organisations to be committed to the development of ILT or e-learning compared to less than half in 2008. Anecdotal evidence drawn from the WBL community and the National Training Federation for Wales attribute greater awareness of technology largely to the support functions provided by the RSC Wales. There is currently little evidence to suggest greater engagement in partnership working relating to the use of ILT. However, where partnerships exist there is evidence that they are being strengthened e.g. via involvement in various forums including the Moodle Wales User Group. However, WBL providers in Wales are currently responding to National policy drivers and forming new working partnerships and collaborations. It is therefore anticipated that this will be replicated in terms of ILT in the near future. Since findings suggest an increased recognition of the need to allocate funds specifically for the development of e-learning or use of ILT. Over two thirds of senior managers now make specific funding provision for electronic learning whereas over half had no consistent approach to funding the use of ILT or e-learning in 2008. 15
Key Findings Providers are increasingly committed to implementing, reviewing and funding the use of ILT or e-learning. RSC Wales and JISC Advance offer a range of services to support implementation, development and review of strategy and Business Processes. Due to National Policy drivers there is the increased likelihood of greater partnership working throughout the sector and with other learning providers. RSC Wales views the development of Learning Partnerships as a Regional Priority activity and is currently developing a range of activities demonstrating the benefits technology can bring in enhancing collaborative working arrangements 16
9. Related Documents Welsh Assembly Government Quality and Effectiveness Framework for post-16 learning http://wales.gov.uk/topics/educationandskills/learningproviders/post16/raisingqualitya ndstandards/?lang=en Welsh Assembly Government (2008) Transforming Education and Training Provision in Wales: Delivering Skills that Work for Wales http://www.wales.gov.uk/docs/dcells/publications/100301transformationpolicyen.pdf RSC UK (2008), ILT Work Based E-Learning Positioning Statement http://rscwbl.jiscinvolve.org/inspection/ 10. Acknowledgements RSC Wales would like to thank all organisations that contributed to the survey. 17
Work Based Learning E-Learning Impact Survey JISC Advance Regional Support Centre for Wales Library and Information Services Emily Phipps Building Swansea University Hendrefoelan Swansea SA2 7NB T/Ph: 01792 295959 www.rsc-wales.ac.uk Peter Scott RSC Wales Manager p.scott@swansea.ac.uk RSC Wales WBL e-learning Advisors Russell Symmons r.symmons@swansea.ac.uk Tel 07814 698434 Alyson Dacey a.dacey@swansea.ac.uk Tel 07813 634356 18