WELCOME WEBBASED E-LEARNING FOR SME AND CRAFTSMEN OF MODERN EUROPE Authors Helena Bijnens, EuroPACE ivzw, Belgium, Johannes De Gruyter, EuroPACE ivzw, Belgium, Ilse Op de Beeck, EuroPACE ivzw, Belgium, Onno Reiners, Bitmedia, Germany, Mario Felderer, Bitmedia, Austria Introduction The eten-supported WELCOME 1 project aims to create a European wide web-based e-learning platform service for SME and craftsmen, offering modules which will be certificated and acknowledged on a European basis. Since 6 February 2006 a set of 5 WELCOME courses, with technical contents adjusted to the needs of SME and craftsmen, are being tested in 5 target countries: Germany, Austria, Poland, Hungary and Lithuania Particularly the technical platform-related aspects of the courses, the attitudes of the specific target groups towards elearning and the quality of the course contents are being screened and evaluated. Based on the evaluation of these aspects final adjustments to the courses will be made and a business plan for future deployment of the courses will be created.. This paper will start with a general introduction on the WELCOME project, followed by a more specific focus on the set-up and the format of the pilot runs. Finally, the evaluation model will be described. Project description The BBZ Hellweg e. V. in Soest, Germany, is the project lead partner and coordinator of a consortium, composed of 9 experienced partners: bit media e-learning solution GmbH & Co KG (bit media) in Austria, AGH - University of Science and Technology (AGH-UST) in Poland, Budapest Enterprise Agency (BEA) in Hungary, DIDAgroup S.p.A. (DIDA) in Italy, INGENATIC GmbH (INGENATIC) in Germany, Visaginas Business Competence Center Labour Resources Agency (LRA) in Lithuania, R. & S. KELLER GmbH (KELLER) in Germany, EuroPACE ivzw (EuroPACE) in Belgium. These experts from 7 different EU countries are dedicated to different tasks. Furthermore, additional certifying institutions are integrated in the project as subcontractors. The project started on 1st April 2005 and lasts for 18 months. It is promoted by the European Commission, DG Information Society and Media under the programme eten. The WELCOME service addresses current and anticipated re-skilling needs of technically oriented SME, craftsmen and individuals. As a result of the growing technical complexity at the workplace (introduction of new machines and technologies) in SME a continuous need to upgrade the technical competencies of employees is emerging. However, on a European scale, no e-learning courses with technical contents adjusted to the specific needs of SME are available. The WELCOME service will fill this supply gap and offers a combined set of five premium e- learning courses with technical topics specifically tailored to the needs of the target group. All courses are being adapted to local requirements. They all offer standardised placement tests for determining the course level appropriate to each person s prior education, allowing self-guided learning to be completed at the learner s own pace. The courses are adjusted to the characteristics and learning styles of adult learners, with plenty of interactive multimedia elements, highly practical and entirely up to date. Additionally all courses will include final examinations rewarding successful participants with a certificate, that aims to be accepted across the participating countries and at a later stage also throughout Europe. While the ECDL course is already well established across Europe, offering final examinations, the consortium intends to transfer the successful pass course principles (Europe wide accepted curricula, certificates and high level course modules) to the other topics. The following course topics are covered by WELCOME:
ECDL: European Computer Driving Licence (software knowledge) EBDL: European Business Driving Licence (basic business knowledge) EICTP: European Information Communication Technologies Pass (computer hardware/network knowledge) EMP: European Mechatronics Pass (Mechatronics knowledge) ECNCP: European CNC Pass (Knowledge for using and programming CNC machines) The overall aim of this market validation project is to test the feasibility of the deployment and to prepare the European roll out of the service. In accordance with this overall objective, the project is providing a detailed validation of the European e-learning market, with special reference to local market requirements and demands. The pilot-run-tested platform will feature curricula and courses, which are specifically tailored to meet the national requirements of the target groups in the addressed European countries. For all courses, certificates will be available and issued by appropriate national/ European institutions being recognized at least in all participating countries, preferably even throughout Europe. Moreover, a detailed business and deployment plan will, based on the results of the pilot runs, pave the way for the initial market deployment phase. Finally, the target users, multipliers and the interested public will be familiarised with the advantages, possibilities and opportunities offered by the service. Components of the Pilot Runs 2 Entry Tests Each course starts with an entry test for the learner. In the pilot run phase however, entry tests of the elearning contents will differ from the final courses in their look, their feel and their functionality. The entry tests will not provide an automated learning path recommendation based on the test results. In other words, the entry tests will not automatically produce a needs-based individualized course curriculum for the learner. Rather it will show performance results which leave the choice of modules entirely up to user. This shortage to generate automated curriculums is due to the limited functionality SITOS platform. However, in this test phase, tutor guidance can relieve this deficiency. Making individual recommendations is part of the tutors task. He/she can access detailed tracking & learning data of each learner which makes the manual assignment of learning objects possible. As such they can help the individual learner to decide which lessons/courses are useful or necessary. Learner Support Functions User Guide The content partners are providing a short instruction/user guide implemented in each course and each lesson where the learner can find information on how to use the lesson navigation, special features of lessons etc. Forums For efficient collaboration and communication SITOS provides the discussion forum as a feature. In the WELCOME project these forums are assigned to different user groups. So tutors and learners can communicate on a selected topic in their own national language. User group functionality will see to it that learners see only the forum for their booked (assigned) courses in their respective language group. Tutoring Services The tutors give selected participants feedback on their exercises and tests and can keep an overview on the learners progress. There are also synchronous event-chats at defined timeslots as an opportunity for social exchange within a defined learner group. At the beginning, in the middle and at the end of a tutored phase all
learners have the possibility to meet in virtual room. In this room a tutor can give instructions if he wants or the group can discuss live about themes touching the course topic. The tutors have the following qualifications: Subject matter expert: the tutor needs to make qualified responses to all subject matter issues and questions that come up from participants of the pilot run phase. Didactical qualification: the tutor should have a teaching background in the topical area in order to understand the needs of the pilot run target group and make adequate responses Native language speaker: the content will be in English. To compensate for this it is paramount to have a native language support person and environment Technology affinity: the tutor needs to be well versed in computer handling, elearning methodologies, etutoring, the platform Calendar The calendar function of SITOS informs learners on tutor phases, the online time of tutors and dates for event-chats for all courses of their respective language. FAQs The FAQ functionality is used to list answers for generally (technical) questions in the national language of user. Final Tests and Examinations For the pilot run the platform offers no examinations but final tests on the elearning content. Examinations can only occur at the end of the full curriculum development which comprises the whole project. As the pilot runs do not cover the package of courses to all the users, full examinations are not available yet.. Evaluation Model 3 Introduction In order to validate the pilots, evaluation activities are receiving a lot of weight in the WELCOME project. All results from the evaluation will be used for optimisation of the courses with respect to the envisaged implementation and deployment of the services. Furthermore, the gathered information should enable validation of the business objectives of the WELCOME service and lead to the further development of tailor specific business plan aspects (e.g. market analysis, user needs, design of the services, indications for adjustments to the training contents, price setting etc.). Not only end users but all actors and stakeholders will be targeted during the evaluation work package: test users, tutors, project partners, subcontractors. Stakeholders As the evaluation results will have a direct impact on the WELCOME partners, the subcontractors and the tutors, all conclusions and recommendations will be communicated to them. The partners will need the evaluation results in order to measure the success of the pilots, the potential scale a full service may have, the impact on the target test users and the potential return on investments. The subcontractors will need the evaluation results in order to give green light for certification of the course. The tutors will need the results to get an idea about the needs for support of the users. Parameters To obtain feedback on the optimisation of pilots and the business objectives, the pilots will be evaluated according to parameters, such as, student attitudes on elearning methodology for delivery of the course contents, course/program administration, method of course delivery, method of assessment, reliability of the technology, degree of student familiarity with the hardware/software, usefulness and quality of the course content, effectiveness of the support mechanisms (tutors, chat, forum etc.), marketing potential of the courses, need for certification of the courses, speed of study, retention rate, drop-out and success rate, general appreciation of the whole concept.
These parameters will be analysed in general terms and, depending on the relevance for the final report, also categorised by country, course, tutored or non-tutored user guidance. Doing so, we may detect feasible national differences in completing the course, potential differences between the courses with regards to success rate or even dissimilarities between those who receive tutor help and those who don t. Formative and Summative Evaluation Activities in WELCOME The WELCOME pilot courses will be evaluated both from a formative and a summative point of view 4. Formative evaluation is being enabled in this evaluation through monitoring activities, immediate learner feedback at the forums, support activities of the tutors (they are asked to maintain a logbook of requests and the answers they provided). Summative evaluation takes place at the end of a course or program with the purpose of redesigning a course or program. This type of evaluation includes attitudes towards the course/program as well as learning outcomes. The summative evaluation of the WELCOME pilot runs would consist of three questionnaires (beginning, middle, end) for three target groups involved: the test users the stakeholders and the partners. It would also assess the administrative aspects of the program/course. Amongst others, the following questions will be asked: what are the strengths and weaknesses of the course? Would you recommend this course/program to your colleagues or other test users? What would you do differently? What would you add or eliminate? How relevant and useful was the content? What are some of the benefits that you gained during the course/program? Summative evaluation will result in a final evaluation report which will be used as a basis to redesign the courses. Data collection methods Conclusions Monitoring: regular observation and logging of the online activities focussing on pedagogical issues (e.g. is the learner support sufficiently reactive/proactive; are tutors acting in a timely way), technical issues (e.g. is easy access to the course materials and support devices ensured for test users and tutors) as well as organisational issues (e.g. are problems sufficiently fast detected and corrected, is security maintained). Numerical measurements (through automated logs in the SITOS platform) that we can analyse for each country and for each module specifically. They will focus on the umber of registered users, the frequency and duration of visits, the page hits, the content for each module that is most visited or less visited, the drop-out rate, the use of the support mechanisms (number of messages in the forums, number of users in each forum), the average response time of the tutors, the visits to the user guides. Discussion forums for ad-hoc and spontaneous feedback from users Specific questionnaires customised to each target group of the evaluation (pilot users, tutors, partners, subcontractors) at specific times during the pilots (after registration and after the final test). The eten-supported WELCOME project aims to offer a European wide web-based e-learning platform service offering modules which will be certificated and acknowledged on a European basis. Five WELCOME courses are piloted since 6 February 2006. The evaluation of these pilots is based on summative and formative evaluation methods and includes data collection methods such as regular observation and logging of the online activities, numerical measurements (through automated logs in the SITOS platform), discussion forums and specific questionnaires for each target group of the evaluation: test users, tutors, partners and subcontractors. The information resulting from the evaluation activities will enable the validation of the pilots in view of their optimization with respect to the implementation and deployment of the services that the project wants to install. Furthermore, the gathered information should enable validation of the business objectives of the WELCOME service and lead to the further development of tailor specific business plan aspects (e.g. market analysis, user needs, design of the services, indications for adjustments to the training contents, price setting etc.)
References: 1. WELCOME project website at http://welcome.bitmedia.cc 2. ONNO REINERS, MARIO FELDERER (2005) Test Run Concept WELCOME, internal project document 3. Helena Bijnens, Ilse Op De Beeck, Johannes De Gruyter (2006) Checklist for Pilot Runs D.6.1, internal project document 4. THOMAS E.CYRS (1998) Evaluating Distance Learning Programs and Courses, http://www.zianet.com/edacyrs/tips/evaluate_dl.htm Authors Helena Bijnens, Johannes De Gruyter, Ilse Op de Beeck EuroPACE ivzw Kapeldreef 62 3001 Heverlee Belgium email: info@europace.be Onno Reiners, Mario Felderer bit media e-learning solution GmbH & Co KG Kaerntner Strasse 311 8054 Graz Austria office@bitmedia.cc