EUROPEAN COMMISSION DIRECTORATE GENERAL EDUCATION AND CULTURE INTERIM EVALUATION OF THE COMENIUS NETWORKS FINAL REPORT ANNEXES June 2004 Viale Castrense, 8 IT 00182 Roma. 1
Index of Annexes Interim Evaluation of Comenius Networks Final Report - ANNEXES ANNEX 1 - SPECIFICATIONS ANNEXED TO THE INVITATION TO TENDER... 3 ANNEX 2 - METHODOLOGICAL PATHS... 11 ANNEX 3 - LIST OF INDICATORS... 15 ANNEX 4 - KEY STAGES OF THE EVALUATION... 29 ANNEX 5 QUESTIONNAIRES... 30 ANNEX 6 - QUANTITATIVE RESULTS OF THE QUESTIONNAIRE SURVEY COORDINATORS AND PARTNERS... 68 ANNEX 7 QUANTITATIVE RESULTS OF THE QUESTIONNAIRE SURVEY MEMBERS... 86 ANNEX 8 - QUANTITATIVE RESULTS OF THE QUESTIONNAIRE SURVEY - NATIONAL AGENCIES... 90 ANNEX 9 - QUANTITATIVE RESULTS OF THE QUESTIONNAIRE SURVEY UNSUCCESSFUL APPLICANTS... 92 ANNEX 10 PROGRAMME AGENDA OF THE ONE-DAY SEMINAR... 97 ANNEX 11 - LIST OF PARTICIPANTS IN THE ONE-DAY SEMINAR... 101 ANNEX 12 - CONCLUSIONS OF THE SEMINAR... 104 ANNEX 13 NETWORKS GRANTS AND TOTAL APPROVED BUDGET BY YEAR... 106 ANNEX 14 - PERCENTAGE OF APPROVED BUDGET SPENT... 107 ANNEX 15 - NETWORKS APPROVED BUDGET... 108 ANNEX 16 - SOCRATES BUDGET... 110 ANNEX 17 - BIBLIOGRAPHY... 111 ANNEX 18 - STRENGTHS AND WEAKNESSES OF THE METHODOLOGY USED... 113. 2
ANNEX 1 - SPECIFICATIONS ANNEXED TO THE INVITATION TO TENDER Invitation to tender No. DG EAC/08/03, to be awarded by restricted procedure, pursuant to the calls for expressions of interest Nos DG EAC/21/00, of 22.07.2000, and DG BUDG 2000/S203-130610 Interim evaluation of the Comenius networks BACKGROUND INFORMATION The European Commission's Directorate-General for Education and Culture (hereinafter: DG EAC) is announcing a tendering procedure for intellectual services in connection with the second phase of the Socrates action programme for education. This invitation to tender is for an interim evaluation of the implementation and the initial results of Comenius Action 3 Comenius networks. It comes in the wake of the following calls for expressions of interest: a) call for expressions of interest No. DG EAC/21/00, published in the Official Journal of the European Communities No. C210 of 22/07/2000. This invitation to tender is addressed to natural and legal persons with a knowledge of English and French from sub-lists E01 (Evaluation and monitoring of Community programmes) and E02 (Evaluation and monitoring of projects sponsored under Community programmes); b) call for expressions of interest No. DG BUDG 2000/S203-130610, published in the Official Journal of the European Communities No. S203 of 21/10/2000. This invitation to tender is addressed to natural and legal persons with a knowledge of English and French from sub-lists 1 (Ex-ante, interim and/or ex-post evaluations of all types of Community activities), 2 (Other evaluation-related activities) and 3 (Preparation and organisation of training seminars and preparation of didactic material relating to evaluation) in the following fields: Education and culture, Employment and social affairs, Labour market and employment policies. CONTEXT OF THE EVALUATION In 2000, Decision No 253/2000/EC of the European Parliament and of the Council of 24 January 2000 established the second phase (2000-2006) of the Socrates Community action programme in the field of education. The main aim of the programme is to contribute to improving the quality of education in Europe through developing the European dimension in education and through promoting lifelong learning. Through its various Actions, Socrates aims to promote cooperation and mobility in the field of education and encourage innovation in the development of educational practices and materials. Socrates also aims to increase proficiency in European languages, promote active citizenship and equal opportunities and step up the fight against exclusion in all its forms, including racism and xenophobia, while also giving particular attention to persons with special educational needs. Socrates II covers the period 2000-2006 and is at present open to 30 European countries. It is built on the experience acquired in the first phase (1995-1999), but the range of activities funded and of potential participants has been extended considerably, offering scope for innovative approaches with regard to the new challenges facing education systems in the knowledge-based society.. 3
The evaluation forming the subject of this invitation to tender conforms with Article 14 of Decision 253/2000/EC, which stipulates that the Commission shall regularly monitor and evaluate the programme in cooperation with the Member States. This evaluation must be based on a) independent external evaluations designed to assess the relevance, efficiency, effectiveness and impact of actions implemented in the context of the programme and b) the Member States' reports on the implementation and impact of the programme, to be submitted to the Commission by 30/09/2003. The evaluation also conforms with the "Monitoring and evaluation policy for the second phase of the Socrates programme", as approved by the Socrates Committee in January 2002. Together with the Member States' reports, it will form the basis for the interim evaluation report on the results achieved and the qualitative and quantitative aspects of the implementation of the Socrates programme, which the Commission is required to submit to the European Parliament, the Council, the Economic and Social Committee and the Committee of the Regions at the start of 2004. It will also complement the Commission communication on the continuation of the programme, due to be submitted by 31/12/2006, and the ex-post evaluation report due to be submitted by 31/12/2007. It also forms an integral part of the European Commission's general evaluation strategy, in accordance with Article 2 of the Financial Regulation applicable to the budget of the European Communities. It comes on top of the evaluations recently launched for Comenius Actions 1 and 2, and complements the plan for an interim evaluation of the three strands of Comenius. CONTEXT OF THE ACTION Comenius is one of the eight Actions of the Socrates education programme established by Decision 253/2000/EC, which runs from 2000 to 2006. Its field is school education. Comenius seeks to enhance the quality and reinforce the European dimension of school education, promote the learning of languages in schools and promote the intercultural dimension in school education. Through its three strands, Comenius promotes transnational cooperation between schools (Comenius 1 - School partnerships), the professional development of staff involved in education (Comenius 2 -Training of staff involved in school education) and networking between schools (Comenius 3 - Comenius networks). The specific objective of Comenius Action 3 is to promote the networking of Comenius projects on subjects of mutual interest in order to promote European cooperation and innovation in school education in various thematic areas. The Comenius networks create a platform to help individuals and institutions involved in Comenius strengthen their European cooperation and maintain it beyond the period of Community support for their specific projects. They also constitute a forum for joint reflection and cooperation in identifying and promoting innovation and best practice in a given thematic area. Fourteen networks are at presented being supported under Comenius 3 (10 were selected in 2001, 4 in 2002), involving approximately 140 partners and at least 1800 members. For the years 2001 and 2002, 13 indicative priority thematic areas were selected. Six of these were addressed in the first year: intercultural education; cultural heritage; educational use of ICT; evaluation of quality in school education; combating violence, racism and xenophobia; school and the world of work. Two more were addressed in the second year: European citizenship and education on the environment.. 4
In general, the activities carried out within the networks are: activities to facilitate and enhance European co-operation, such as exchanging information, training of project coordinators, promoting new projects, disseminating project results and good practice; activities to promote educational innovation and good practice in the thematic area concerned, such as comparative analyses, case studies, formulating recommendations and organising working groups, seminars or conferences; project coordination and management activities. Each Comenius network is required to: establish a website, produce an annual report on the state of innovation in its area of activity, provide the players in Comenius with comprehensive information, organise an annual conference or seminar, participate in an annual meeting of network coordinators organised by the European Commission. To be eligible for support, a Comenius networking partnership must include at least one organisation from each of six different Socrates countries, at least one of which must be an EU Member State. The coordination of the network must be undertaken by a public authority, a higher education institution, a teacher training institute, a research centre, etc. Financial support is restricted to a maximum of three years. All the networks selected are subject to this condition. The Comenius networks are made up of partners, who are entitled to Community co-financing for their activities, and associate members, who participate in the activities but receive no Community funding. Comenius 3 is a "centralised Action" within SOCRATES. This means that projects are selected centrally by the European Commission, assisted by a panel of independent experts. The Commission also manages the Action, with the support of a Technical Assistance Office. SUBJECT OF THE EVALUATION Comenius 3 is a new Action which has not yet been subjected to an evaluation. The first project selection exercise was completed in autumn 2001, and implementation of the Action began in February 2002. The evaluator will be required to analyse the operation of all the 2001 and 2002 networks, draw the necessary conclusions and make recommendations for the period 2003/2006 as well as for a possible third phase of Socrates. MAIN QUESTIONS TO BE CONSIDERED IN THE EVALUATION The purpose of the evaluation is to furnish an analysis of the relevance, efficiency, effectiveness and initial impact of the activities funded under Comenius 3, based on quantitative and qualitative data. In accordance with the general objectives of Comenius, with the specific objectives of Comenius 3 as laid down in Decision 253/2000/EC and in the Socrates Programme Guidelines for Applicants, and with the priorities established in the annual calls for proposals, the evaluation must therefore seek to provide initial answers to the following questions: 1) Relevance: To what extent do the objectives of the Action meet the needs identified?. 5
To what extent do the activities supported meet the objectives of the Action Could one envisage alternative approaches for achieving these objectives? Nature and relative importance of the activities carried out: which of them are most conducive to achievement of the specific objectives? Do the activities supported correspond to networking activities rather than traditional transnational projects? To what extent and in what way do these activities enable innovation and best practice to be identified and encouraged in the thematic area chosen? More broadly, what are the initial results of the measures envisaged by networks to involve the players from schools and educational establishments and those from teacher training establishments in their activities? 2) Efficiency From the point of view both of the beneficiaries and the Commission, is the financial investment (in particular the amount of the grant compared with the total budget for the network) sufficient, satisfactory or excessive, having regard to the size of the partnerships and the activities carried out or envisaged? To what extent have the Community grants served as a catalyst for other sources of funding? What are the conditions that could hamper or help ensure the success of the networks and that enable the coordinating institutions to run the networks effectively? Have these conditions been met? How does the composition of the partnerships affect the functioning of the networks? Selection (from the point of view of the beneficiaries) : does two-stage selection offer an added value? to what extent is a discussion leading to adaptation of the initial proposal prior to the selection decision useful and desirable? Is enough information provided on networks' activities, especially to public authorities (government departments, regional authorities, inspectors, etc) who could draw lessons from the information for their education systems? Is the promotion of this new Action satisfactory? How could it be improved, if necessary? 3) Effectiveness What has been the foremost impact of the networking activities on staff involved in school education?. 6
What has been the result of the first annual thematic conferences (number and variety of participants, importance accorded to concerns in the field, advancement of the debate, nature and quality of productions, immediate impact, etc)? Have the two main objectives of the networks (platform/dissemination and innovation) been taken into account via the "outputs" and activities launched? To what extent do the networks make it possible to build on the experience acquired and the results achieved in the ongoing or completed Comenius 1 and Comenius 2 projects (and, more generally, the Socrates projects)? What is the added value of the Comenius networks compared with the traditional Comenius 1 and 2 partnerships? Compared with similar activities conducted in the national context, what is the European added value of the networks, in terms of content and the development of a European awareness between the players and the beneficiaries? To what extent is the impetus of new Comenius projects incorporated into the activities of the first networks? Are there the beginnings of a "sustainability" strategy to keep networks active once their contract terminates? To what extent and in what way do the networking activities help individuals and institutions involved in Comenius to maintain and strengthen their European cooperation beyond the period of Community support for their specific projects? INFORMATION SOURCES AND INDICATORS In order to answer the above questions, evaluators will need to gather the following quantitative and qualitative data: Participation For partnerships: types and quantitative analysis of thematic areas covered types and quantitative analysis of participating bodies geographical coverage: origins of the network coordinators and partners; is it easy to achieve the minimal geographical coverage required for a partnership (minimum of six different countries)? Is this the optimal coverage? types of activities and products Target audiences for the networks' activities: numbers and types Importance of the associate members, both in qualitative and quantitative terms Analysis of the selection and management procedures and their assessment by the bodies and individuals involved. The evaluation indicators established for Comenius in collaboration with the Programme Committee, together with the list of available information sources, are given in the Annex.. 7
EVALUATION ASSISTANCE The external evaluator will be monitored and assisted in his work by a small steering group, consisting of the Unit A3 official responsible for Comenius 3, the Unit A3 "Evaluation" coordinator and the Socrates programme evaluation coordinator. METHODOLOGY Tenderers must demonstrate that they fully understand the object of the evaluation. They must present a clear and precise timetable for the evaluation work to be carried out within the allotted time and provide adequate documentary evidence of the expertise of the members of the evaluation team. Comenius 3 is a new Action. The first projects were only finally selected in autumn 2001, and implementation of the Action did not really begin until February 2002. The number of networks is therefore still relatively small. Consequently, the evaluators will need to identify significant criteria for judging the implementation of the Action while not overlooking these particular circumstances. The Commission therefore recommends a simplified evaluation method which will nevertheless offer an adequate level of reliability. The method will consist of: a survey of coordinators, partners and a sample of network members, applicants who may not have been selected and other players who may be concerned (such as national authorities, associations, etc.), by means of a written questionnaire. The summarised results of this survey will provide a working basis for the seminar described below; a one-day seminar for representatives of each network and other players concerned, including independent evaluation experts appointed by the Socrates Committee (around 40 persons in total), the seminar to be organised and run by the evaluators. The purpose will be to analyse and examine in greater depth the main results and conclusions of the survey and thus provide material for the evaluator's final report. The evaluator will be required to refine this methodology or propose relevant variants. Organisation of the evaluation work must take account of possible constraints linked to the academic and school calendars. MEETINGS The tenderer should schedule a minimum of three meetings with the Commission services in Brussels: a meeting to agree on the work programme and finalise the draft questionnaire, a meeting after the survey, to review the results and the interim reports and prepare the seminar, a meeting after the presentation of the draft final report. if necessary, a fourth meeting depending on how work is progressing.. 8
Reports and documents to be submitted The evaluator must submit to the Commission the following documents: - a work document, after signature of the contract, specifying the work plan and the methodological tools to be used; - a draft questionnaire for the evaluation survey; - an interim report, in French or English, summarising the results of the documentary research and the questionnaire survey. This report will contain an abstract (in both French and English), an introduction, a presentation of the method used, a description and an analysis of the data processed and an analysis of the results, together with provisional conclusions; - a final report, describing all the work carried out and the results achieved in performance of the contract. The text of this report may be drafted in French or English and must not exceed 20 pages, plus Annexes. It will contain an abstract (in both French and English), an introduction, a presentation of the method used, a description and an analysis of the data processed and an analysis of the results, together with conclusions and recommendations for the future implementation of the Action. The draft final report must be submitted to the European Commission by no later than one month before expiry of the contract. The Commission will then inform the contractor of its acceptance of this report, or will pass on its comments. Within a month of receiving the Commission's comments, the contractor will submit the final version of the report, either taking account of the comments or putting forward alternative viewpoints. The final report will be considered as being accepted by the Commission if, within 60 days after its receipt, the Commission has not explicitly passed on any comments to the contractor. The final report will be presented to the members of the group of experts appointed by the Socrates Committee, as well as to the Socrates Committee's School Education Sub-Committee. The contractor will send the Commission the documents in question in electronic form (Word), plus three paper copies. ORGANISATION OF THE SEMINAR The contractor will bear the costs of organising the seminar, as well as the participants' travel and subsistence costs. He must: prepare the agenda for the seminar, in agreement with the Commission; find an appropriate location for the seminar, with easy access for the participants, and book the rooms, facilities and equipment; prepare and send out invitations; organise travel and accommodation for the participants and reimburse their costs;. 9
supply participants with the necessary practical information. INDICATIVE TIMETABLE FOR THE KEY EVALUATION STAGES Month 2 : -submission of the work document and the draft questionnaire -working meeting Month 5 : -submission of the draft interim report -working meeting -seminar (during first half of month) Month 7 : -submission of the draft final report -working meeting Month 8 : -submission of the final report. 10
ANNEX 2 - METHODOLOGICAL PATHS Evaluation questions Qualitative-methodological implications Expected results of the evaluation To what extent do the objectives of the Action meet the needs identified? To what extent do the activities supported meet the objectives of the Action? Could one envisage alternative approaches for achieving these objectives? Nature and relative importance of the activities carried out: which of them are most conducive to achievement of the specific objectives? Do the activities supported correspond to networking activities rather than traditional transnational projects? To what extent and in what way do these activities enable innovation and best practice to be identified and encouraged in the thematic area chosen? More broadly, what are the initial results of the measures envisaged by networks to involve the players from schools and educational establishments and those from teacher training establishments in their activities? From the point of view both of the beneficiaries and the Commission, is the financial investment (in particular the amount of the grant compared with the total budget for the network) sufficient, satisfactory or excessive, having regard to the size of the partnerships and the activities carried out or Relevance: the evaluation made clear whether the Action meets the needs and the problems identified. In particular, the extent to which the activities are respondent to the objectives of the Action has been assessed (for instance, the extent to which the networking activities contribute to develop the potential of Comenius 1 and 2 projects). The evaluation process established whether the activities under study are relevant to the policies of the European Union in the field of education and in what way such activities can affect the policy tools adopted. Efficiency: The examination of financial investment, management procedure, composition of the Networks in relation to their functioning, selection process from the point of view of beneficiaries and promotion and communication allowed The analytical work has been instrumental in collecting, selecting and processing the data and the information about Comenius Networks. The evaluation aims at providing decision-makers at various levels with analytical tools and with insightful observations about policy and management strategies in the field of education. The evaluation aims at providing decision-makers at various levels with analytical tools and with insightful observations on how to implement the management framework of the Action.. 11
Evaluation questions Qualitative-methodological implications Expected results of the evaluation envisaged? To what extent have the Community grants served as a catalyst for other sources of funding? What are the conditions that could hamper or help ensure the success of the networks and that enable the coordinating institutions to run the networks effectively? Have these conditions been met? How does the composition of the partnerships affect the functioning of the networks? Selection (from the point of view of the beneficiaries): does two-stage selection offer an added value? to what extent is a discussion leading to adaptation of the initial proposal prior to the selection decision useful and desirable? Is enough information provided on networks activities, especially to public authorities (government departments, regional authorities, inspectors, etc) who could draw lessons from the information for their education systems? Is the promotion of this new Action satisfactory? How could it be improved, if necessary? the evaluator to find out which stage of the process needs to be modified or strengthened. As far as the structure of each Network is concerned, special attention has been paid to the role of the coordinating institutions, since it is of paramount importance to the success of each Network.. 12
Evaluation questions What has been the foremost impact of the networking activities on staff involved in school education? What has been the result of the first annual thematic conferences (number of participants, importance accorded to concerns in the field, advancement of the debate, nature and quality of productions, immediate impact, etc)? Have the two main objectives of the networks (platform/dissemination and innovation) been taken into account via the "outputs" and activities launched? To what extent do the networks make it possible to build on the experience acquired and the results achieved in the ongoing or completed Comenius 1 and Comenius 2 projects (and, more generally, the Socrates projects)? What is the added value of the Comenius networks compared with the traditional Comenius 1 and 2 partnerships? Compared with similar activities conducted in the national context, what is the European added value of the networks, in terms of content and the development of a European awareness between the players and the beneficiaries? Are there the beginnings of a "sustainability" strategy to keep networks active once their contract terminates? To what extent and in what way do the networking activities help individuals and institutions involved in Comenius to maintain and strengthen their European cooperation beyond the period of Community support for their specific Qualitative-methodological implications Effectiveness: After assessing to what extent results are consistent with the objectives of the Action, the evaluator appreciated the European added value of the Networks. Sustainability: Indications that the partnerships are likely to continue after the end of the Community support have been sought. The beginnings of a sustainability Expected results of the evaluation The analysis of issues such as nature of the activities undertaken, networking, innovation and best practices allowed the evaluator to understand whether results are respondent to the objectives of the Action or whether other approaches can be introduced. The evaluation tries to provide the Commission with an understanding of the actors commitment to maintaining and strengthening their European cooperation.. 13
Evaluation questions Qualitative-methodological implications Expected results of the evaluation projects? strategy has been assessed in relation to the results and the outcomes achieved in the 2000-2002 period.. 14
ANNEX 3 - LIST OF INDICATORS Type of coordinating institution questions GENERAL INFORMATION QUESTIONNARE FOR COORDINATORS Official indicators of Comenius 3/key questions number and type of institutions involved number of thematic network projects number of countries. Additional indicators/key questions provided by the evaluator Please indicate the main thematic area addressed by the Network (one answer) number of networks per theme list of themes, number of study areas Please indicate the effective start date for the network activities I = time span between the date for the start of the activities and its effective start If there was a delay in the planned date for the start-up of activities, KQ = level of difficulty encountered in the start phase briefly indicate the reason number of institutions involved in the Please indicate the number of partners involved in the Network activities of the network and cooperating in a network Has the make-up of the partnership group been modified during the I = number of partnership modified realisation phase? Please indicate the number of members involved in the Network. number of institutions involved in the activities of the network and cooperating in a network Is the number of members different with respect to that indicated in the application? I = number of members modified Have you ever taken part, in the past, in other activities financed by the Socrates program or by another European Community program? I = number of coordinators with experience in Socrates and other EC programmes. 15
OBJECTIVES Please indicate the main objective of the Network Do you think you meet these objectives when implementing the activities of your network? Does the implementation of the activities of your network concretely contribute to achieving the transversal policies of the European Union? PROMOTION AND COMMUNICATION How did you learn about Action 3 of Comenius? KQ = relevance of the specific objectives of the network compared to the general objectives of the action KQ = relevance of the activities of the network compared to the general objectives of the network KQ = contribution of the activities to the fulfilment of each of the transversal priorities I = number of information tools used for gathering info on Comenius 3 From which institution did you learn about Action 3 of Comenius? I = types of institutions disseminating information on Comenius 3 How successfully do you think that institutional bodies (EC, national agencies, national authorities) promote Comenius 3? Do you feel there should be increased communication and promotion regarding Comenius 3? Which channels of dissemination do you think should be used the most? Which institutions should increase the promotion of Comenius 3? SELECTION Do you believe that the two-phase selection process for proposals : Have the independent experts comments and recommendations been useful for the final proposal? Do you feel that the experts comments and recommendations following the pre-selection phase were helpful in terms of: KQ = opinion on promotion activities KQ = opinion on promotion activities KQ = opinion on the future strategy for the promotion of the Action KQ = opinion on the future strategy for the promotion of the Action KQ = opinion on selection procedures KQ = opinion on selection procedures KQ = opinion on selection procedures. 16
MANAGEMENT What languages are used to communicate within the network? Please indicate the number of coordinating meetings held during the last year What instruments of communication are used to hold these meetings? Have specific tools for the network management been put in place? (steering Committee, finalised resources, contact list, other..) Have you had any problems in the implementation of the Network activities? Do you think that the structure you work with provided you with expertise and equipment? PARTNERSHIP Do you feel that it was easy to satisfy the geographic coverage required by Comenius 3 for the selection of partners? (minimum of six partners from six different countries) How did you find your partners? From which institution did you receive assistance in finding the partners? Had the partners contacted any knowledge of Action Comenius 3? I = number/type of languages used for communicating within the Network I = number of coordinating meetings held during the last year I = type of instruments used for hold this meeting I = type of specific tool adopted for the network management KQ = level of difficulty encountered in the implementation of the activities KQ = qualitative data on conditions enabling the coordinating institutions to run effectively KQ = level of difficulty encountered in the establishment of the partnership KQ= way of coming into contact with partners I = number of subjects receiving assistance for the selection of partners I = number of subjects knowing the Action Do you think that a partnership made up of different institutions contribute to the achievement of the network objectives? added value of partnership During which phase was the cooperation with the partners most effective? Are you satisfied with the cooperation with the partners? How do you judge the involvement of the partners in the Network activities? KQ = level of active involvement of partners KQ = quality of cooperation KQ = Opinion on the involvement of the partners in Network's activities. 17
ACTIVITIES AND OUTCOMES Please mention the main activities carried out by your organisation within the Network besides the coordination (max. of 3 responses) Interim Evaluation of Comenius Networks Final Report - ANNEXES number and nature of the activities, number and nature of events organised, number of other institutions involved in events organised by the network Within the main results obtained/to obtain which do you think are the most significant? (max. of 3 responses with scores of 1 -maximum -to 3 minimum) Please indicate which are the benefits of the participation in the Network for your institution (max. of 3 responses with scores of 1 - maximum -to 3 minimum) Which/how many languages are utilised in the implementation of the activities? CONFERENCES How many conferences have you organised since the start of the activities? What was the total number of participants in this (these) conferences? How many countries did the participants come from? In your opinion what factors had a positive influence on the success of your conference? In your opinion what factors had a negative influence? CONTACT SEMINARS How many contact seminars have you organised since the start of the activities? What was the total number of participants in this (these) contact seminars? How many countries did the participants come from? In your opinion what factors had a positive influence on the success of your contact seminars? KQ = qualitative data on relevance of the activities and outputs in particular: number and nature of events organised, number and nature of material produced and sisseminated, number of projects resulting from events organised by the network, level of evaluation,extent of synergies with other action of socrates and with other EC programmes I = first impacts of the network and project outputs on the institution I = number/type of languages used for carry out the activities I = number of conferences organised I = number of people taking part in conferences I = number of countries participating in these conference KQ = opinion on the overall effectiveness of these conferences KQ = opinion on the overall effectiveness of these conferences I = number of contact seminars organised I = number of people taking part in contact seminars I = number of countries participating in these contact seminars KQ = opinion on the overall effectiveness of these seminars In your opinion what factors had a negative influence? KQ = opinion on the overall effectiveness of these seminars. 18
WEBSITE When did the Network s website go on line? Please provide a brief description of the main contents of the Website Which/how many languages are utilised? I = time span between the date for the start of the activities and its effective start KQ = type of dissemination activities per web I = Type of languages used in the dissemination of information How many visitors has the Website had to date? INNOVATION In what way do you feel that the Network has contributed to generating innovation within the selected thematic area? KQ= contribution of school partnership networks as forum of relections and innovation in specific areas of study Do you feel that the same level of innovation could be obtained even KQ= added value of the network and without the contribution of the network established under Comenius 3 relevance of the activities compared to the? general objectives of the action FINANCIAL INFORMATION Please indicate the percentage of the total approved budget received expenditure budget compared to the that has been spent as of the date of the response attributed budget I = number of institutions/ people benefiting from dissemination activities through web From which sources did the network receive additional funding? I = number and type of additional sources of funding If your proposal had been not financed under Comenius 3, would the Network have been started anyway? added value of the EU funding How do you judge the grant received from the Commission? KQ = opinion on the grant received by the Commission. 19
DISSEMINATION OF RESULTS How are the results disseminated outside of the network? type of dissemination tools What do you think of the partners contribution to the dissemination of the results? How did you identify the potential target groups of the dissemination? type of institutions/ people benefiting from dissemination activities KQ = qualitative data on contribution of partners to the dissemination of the results Which are the main target groups of the dissemination? During the implementation of the activities did you modify the Network target group? What role is played by the use of ICT in the dissemination of the results? What has been done to make your Network visible to public authorities (government departments, regional authorities, etc)? SUSTAINABILITY Do you feel that the activities of the network will be able to continue even following the end of the contract with the Commission? Do you think you will continue to cooperate with the partners even after the end of the activities planned for the Network? What could the European Commission do to support any future activities of the Network? Having reached this phase in your activities, what advice /suggestions can you provide national authorities responsible for education with? establishment of sustainable networks KQ = qualitative data on identification of potential target groups I = number of institution that have modified their target group KQ = role of ICT in the dissemination of the activities KQ = Extent of the visibility and image of the Networks to public authorities KQ = qualitative data on the sustainability of the Network activities, number of Networks continuing their activity after the end of the contract KQ = role and importance of the EC in the life of the institutions KQ = contribution of the Action to the implementation of national programmes. 20
Type of partner organisation questions GENERAL INFORMATION QUESTIONNARE FOR PARTNERS Official indicators of Comenius 3/key questions number and type of institutions involved number of partners number of countries Additional indicators/key questions provided by the evaluator Have you ever taken part, in the past, in other activities financed by the Socrates program or by another European Community program? PROMOTION AND COMMUNICATION How did you learn about Action 3 of Comenius? From which institution did you learn about Action 3 of Comenius? How successfully do you think that institutional bodies (EC, national agencies, national authorities) promote Comenius 3? Do you feel there should be increased communication and promotion regarding Comenius 3? Which channels of dissemination do you think should be used the most? Which institutions should increase the promotion of Comenius 3? MANAGEMENT Please indicate the problems you have met in implementing the activities of the working plan I = number of partners with experience in Socratres and other EC programmes I = number of information tools used for gathering info on Comenius 3 KQ = opinion on promotion activities KQ = opinion on promotion activities KQ = opinion on the future strategy for the promotion of the Action KQ = opinion on the future strategy for the promotion of the Action KQ = opinion on the future strategy for the promotion of the Action KQ = type of problems related to management. 21
SECTION PARTNERSHIP What role did you play in setting up the partnership? During which phase was the cooperation with the partners most effective? Are you satisfied with the cooperation of the other partners of the Network? How do you judge your level of involvement in the activities of the Network? How successful do you feel the Coordinator has been in managing the activities? ACTIVITIES AND OUTCOMES Please mention the Network activities you are involved in number and nature of the activities, number and nature of events organised, number of other institutions involved in events organised by the network KQ = level of interaction among partners in the establishment of the partnership KQ = level of cooperation among partners KQ = quality of cooperation KQ = self evaluation of the level of involvement of partners KQ = evaluation of the network management on the part of the coordinator Within the main outputs obtained/to obtain which do you think are the most significant? (max. of 3 responses with scores of 1 -maximum -to 3 minimum) WEBSITE Have you contributed to the implementation of the Website? If yes, could you describe your contribution to the Website? What is your opinion of the quality of the Website? KQ = qualitative data on relevance of the activities and outputs in particular: number and nature of events organised, number and nature of material produced and sisseminated, number of projects resulting from events organised by the network, level of evaluation,extent of synergies with other action of socrates and with other EC programmes KQ = level of involvement of partners in the design of the website KQ = Method and means of web dissemination KQ = evaluation of the quality level of the website. 22
INNOVATION In what way do you feel that the Network has contributed to generating innovation within the selected thematic area? KQ= contribution of school partnership networks as forum of relections and innovation in specific areas of study Do you feel that the same level of innovation could be obtained even KQ= added value of the network and without the contribution of the partnership established under Comenius relevance of the activities compared to the 3? general objectives of the action DISSEMINATION OF RESULTS Which mode do you use most frequently for disseminating the results type of dissemination tools of your activities? What role do you play as a partner in the dissemination of the results KQ = role of partners in the dissemination of the results in your country and in other EU countries? FINANCIAL INFORMATION Please indicate the percentage of the total community grant already expenditure budget compared to the received that has been spent as of the date of the response attributed budget How do you judge the grant received from the Commission? SUSTAINABILITY Do you feel that the activities of the network will be able to continue even following the end of the contract with the Commission? Do you think you will continue to cooperate with the partners even after the end of the activities planned for the Network? What could the European Commission do to support any future activities of the Network? Having reached this phase in your activities, what advice /suggestions can you provide national authorities responsible for education with? establishment of sustainable networks KQ = opinion on the grant received by the Commission KQ = qualitative data on the sustainability of the Network activities, number of Networks continuing their activity after the end of the contract KQ = role and importance of the EC in the life of the institutions KQ = contribution of the Action to the implementation of national programmes. 23
Type of organisation questions GENERAL INFORMATION QUESTIONNARE FOR NETWORK MEMBERS Official indicators of Comenius 3/key Additional indicators/key questions provided by the evaluator questions number and type of institutions involved,number of partners,number of countries. Please indicate the name of the network of which you are a member - I = number of members per network Have you ever taken part in other activities funded by the Socrates program or by another European-Community program? PROMOTION AND COMMUNICATION How did you learn about Action 3 of Comenius? From which institution did you learn about Action 3 of Comenius? How successfully do you think that institutional bodies (EC, national agencies, national authorities) promote Comenius 3? Do you feel there should be increased communication and promotion regarding Comenius 3? Which channels of dissemination do you think should be used the most? Which institutions should increase the promotion of Comenius 3? I = number of members with experience in Socrates and other EC programmes I = number of information tools used for gathering info on Comenius 3 KQ = opinion on promotion activities KQ = opinion on promotion activities KQ = opinion on the future strategy for the promotion of the Action KQ = opinion on the future strategy for the promotion of the Action KQ = opinion on the future strategy for the promotion of the Action. 24
ACTIVITIES AND OUTCOMES Please give a brief description of your contribution to the activities of the Network Please indicate how you participated in the activities of the Network Have you ever taken part in an event/seminar organised by the Network? Please indicate what benefits you/your institution receive from participation in the Network Please indicate what kind of relation you have with the Coordinator of the Network Please indicate what kind of relation you have with the partners of the Network WEBSITE How often do you consult the Website of your Network? What is your opinion of the quality of the Website? KQ = self evaluation of the contribution to the activities of the network KQ = modes of participation in the activities I = number of events /seminars which the members took part in I = first impacts of the network and project outputs on the institution KQ = type of relation with the coordinator KQ = type of relation with the partners I = frequency of consultation of the website KQ = evaluation of the quality of the website. 25
QUESTIONNARE FOR UNSUCCESSFUL APPLICANTS questions Official indicators of Comenius 3/key questions Additional indicators/key questions provided by the evaluator GENERAL INFORMATION number and type of institutions Type of organisation involved,number of thematic network projects,number of countries. Please indicate which thematic area was proposed for the establishment of the Network I = number of applicants not selected per theme Have you ever taken part, in the past, in other activities financed by the Socrates program or by another European Community program? PROMOTION AND COMMUNICATION How did you learn about Action 3 of Comenius? From which institution did you learn about Action 3 of Comenius? How successfully do you think that institutional bodies (EC, national agencies, national authorities) promoted Comenius 3? Do you feel there should be increased communication and promotion regarding Comenius 3? Which channels of dissemination do you think should be used the most? Which institutions should increase the promotion of Comenius 3? I = number of unsuccessfull applicants with experience in Socrates and other EC programmes I = number of information tools used for gathering info on Comenius 3 KQ = opinion on promotion activities KQ = opinion on promotion activities KQ = opinion on the future strategy for the promotion of the Action KQ = opinion on the future strategy for the promotion of the Action KQ = opinion on the future strategy for the promotion of the Action. 26
QUESTIONNARE FOR NATIONAL AGENCIES questions Official indicators of Comenius 3/key questions Additional indicators/key questions provided by the evaluator PROMOTION AND COMMUNICATION Please indicate the modes most frequently used by your agency to disseminate information on Comenius 3 I = modes used for disseminating information What are the potential targets for the information? Rank their importance KQ = list of potential target group Please indicate the types of requests/questions received by your agency KQ = type of requests received by the agency Do you think that there should be greater communications and promotion on Comenius 3? KQ = opinion on promotion activities Which channels of dissemination do you think should be used the most? Please indicate how you assisted potential promoters looking for new partners FINANCIAL INFORMATION KQ = opinion on the future strategy for the promotion of the Action KQ = role played by the NA in supporting new organisations Please indicate the amount of financing allocated by your agency to participate in seminars/events organised by the Comenius 3 networks I = expenditure budget for seminars and events RECOMMENDATIONS What recommendations do you feel could be made to the European Commission regarding Comenius 3 Action? What recommendations do you feel could be made to the currently supported networks? KQ = role and importance of the EC in the life of the institutions I = opinion on the activities carried out by the Networks. 27
QUESTIONNARE FOR EUROPEAN EDUCATION NETWORKS, ASSOCIATIONS AND NON-GOVERNMENTAL ORGANISATIONS IN THE FIELD OF EDUCATION questions Official indicators of Comenius 3/key questions Additional indicators/key questions provided by the evaluator GENERAL INFORMATION Type of organisation number and type of institutions with knowledge of the Action Have you ever taken part in other activities funded by the Socrates program or by another European-Community program? I = number of other institutions with experience in Socrates and in other EC programmes PROMOTION AND COMMUNICATION Do you know the Comenius 3 Action? If yes, does your organisation promote or disseminate information on Comenius 3? If yes, through what channels? How successfully do you think that institutional bodies (EC, national agencies, national authorities) promoted Comenius 3? Do you feel there should be increased communication and promotion regarding Comenius 3? Which channels of dissemination do you think should be used the most? Which institutions should increase the promotion of Comenius 3 Are you involved in a Comenius 3 Network? I = number of other institutions that know the Comenius 3 Action I = number of other institutions that promote the Comenius 3 Action I = types of communicaton tools used for disseminating information on the Comenius 3 Action KQ = opinion on the future strategy for the promotion of the Action KQ = opinion on the future strategy for the promotion of the Action KQ = opinion on the future strategy for the promotion of the Action KQ = opinion on the future strategy for the promotion of the Action I = number of other institutions involved in a Comenius 3 Action Does any of your member participate in the activities of a Comenius 3 Network? RECOMMENDATIONS What recommendations do you feel could be made to the European Commission with regard to action 3 of Comenius? Please give your opinion about the activities carried out by the Comenius Networks I = number of other institutions involved in a Comenius 3 Action KQ = role and importance of the EC in the life of the institutions I = opinion on the activities carried out by the Networks. 28