TERMS OF REFERENCE FOR AN INSTITUTIONAL CONSULTANCY

Transcription

1 TERMS OF REFERENCE FOR AN INSTITUTIONAL CONSULTANCY Project title: Evaluation on the contribution of the UNICEF school construction component to the overall programme goals of access, quality and systems capacity enhancement. Section: Basic Education and Youth Development Key partner: Ministry of Education, Science and Technology Location: Lilongwe, Malawi Duration: 50 days Start date: TBD End date: 4 th December 2016 Closing date: Apply before or on 13 October 2015 BACKGROUND AND JUSTIFICATION School infrastructure development component under the child friendly schools programme of the current UNICEF MCO country programme has been designed to contribute chiefly to two key components of the national Education Sector Implementation Plan (ESIP) I. improved access to equitable education II. improved and relevant quality education The education sector in Malawi also aims to establish standards, norms and effective processes for school improvement generally through the national Primary School Improvement Programme (PSIP). In that framework, school construction is meant to enhance participation, capacity, sustainability and efficiency of the sector. Access and quality of education in Malawi has long been affected by an extreme shortage of classrooms. While primary enrolment increased by 45% between 2004 and 2013, the number of primary classrooms rose by only 12%. A recent study (USAID, 2014) identified classroom shortage as one of the major factor leading to student absenteeism, repetition and dropouts. Analysis of 1

2 PET-QSD Survey data also confirms this. Hence, the target of the Malawian education sector as laid out in the ESIP II (2013/ /18) is to construct at least 20,000 classrooms to meet increasing population of school age children. It is within, and in response to, this context that the Basic Education and Youth Development (BEYD) section seeks to meet the objectives of improved educational access and quality for all children through a broad School Construction component contributing to both volume and quality of school infrastructure in Malawi. School infrastructure development includes multiple inputs such as building of classrooms, latrines and teachers houses as well as contribution to the development of school construction norms and standards including capacity for school construction. The proposed evaluation is important in assessing the extent to which the School Construction Component enables UNICEF Malawi to contribute to those broad objectives for key learner outcomes in education. UNICEF has utilised the Child Friendly School (CFS) quality framework as guidance to meet the MCO commitment to the education sector. This includes emphasizing the importance of inclusiveness and equity for marginalised children. In the context of Malawi this has reflected in a strong focus on girls education due to the many factors related to school environment and infrastructure that hinder girls from enrolling, enjoying and completing basic education. As an example girls drop out of school in large numbers when reaching puberty due to lack of adequate water and sanitation facilities. The CFS framework also highlights the need to consider school infrastructure as a holistic approach to improved learning environment; including both hard-ware and soft-ware aspects. 1 This evaluation will assess the extent to which the School Construction component of the Child Friendly Schools Programme has contributed to a) the development of standards, and capacity for their implementation within the wider BEYD goals of strengthening institutional capacity and b) the learning environment. PURPOSE OF THE EVALUATION 1 Hard-ware aspects include building of classrooms while soft-ware aspects include issues such as maintenance and attitudes towards the school environment. 2

3 The purpose of the evaluation is threefold: 1) to determine school infrastructure contribution to the basic education programme objectives of improving access and quality through child friendly school construction and design 2) to assess effectiveness and efficiency of the inputs and processes 3) to identify lessons learnt and provide recommendations for future of school infrastructure and other strategies in the new country programme The evaluation is to improve UNICEF s accountability and provide for lessons learnt on schools infrastructure projects. Findings from the evaluation will support UNICEF and its key partner the Ministry of Education to make evidence based decisions for future programming and planning within the field of education. GOAL To determine the contribution of the School Construction Programe as carried out under the current Country Programme to the BEYD education programme objectives to improve learner outcomes (access and quality) through child friendly education and capacity enhancement. SPECIFIC OBJECTIVES 1. To assess the contribution of the school construction component to the BEYD programme objectives of improving access to quality basic education for all, especially of the most vulnerable and girls 2. To assess the achievement of the school construction component in contributing to the child friendly education approach in in focus areas and targeted schools; 3

4 3. To assess the effectiveness and efficiency of the inputs and process of the school infrastructure project in meeting the above leaner outcomes, standards and capacity enhancement objectives; 4. To identify lessons learnt and make recommendations for the new country programme ( ). SCOPE OF WORK Within the OECD evaluation framework (relevancy, effectiveness, efficiency, impact and sustainability) the key dimensions to be evaluated will be threefold focusing on expected outputs linked to learner outcomes; processes; and the chosen inputs and particularly the child friendly education and the school improvement approach. The impact evaluation is to focus mainly on 35 primary schools constructed in 5 districts of Malawi under the BEYD Child Friendly Schools Programme between 2012 and Whereas the type and level of construction and subsequent interventions vary between school sites, the consultant will have to apply relevant and applicable assessment, examination and review methods as appropriate in order to fulfil the outlined scope of work. Each dimension is laid out below with key supporting questions and tasks intended to assist and guide (but not limited to) the evaluator in his approach. 1. Assess the focus of the School Construction Component in line with the BEYD education outcome objectives of increasing equitable access and also quality of education where possible given that the country programme still has one year to go; Were the programme objectives appropriate in the overall problem context; needs and priorities? What value (and for whom) did the School Construction add relative to other education initiatives? Given progress to date, are the School Construction objectives still relevant? 4

5 2. Examine efficiency, effectiveness and sustainability of the School Construction Component inputs and processes within the child friendly school quality framework; How well did the various activities transform the available resources into results? What measures were taken to ensure cost-effectiveness in procurement and implementation? What measures were taken to ensure effective financial implementation, monitoring and reporting? How well did the School Construction predict and react to risks? To what extent did the project enable local participation and ownership of the project? To what extent will maintenance be sustained by local partners/beneficiaries once the funding comes to an end? 3. Examine key outputs and standards in light of the child friendly quality framework To what extent did the School Construction deliver expected results? To what extent has the design enabled the improvement of the school environment? How well has the School Construction component contributed to the improvement of standards and systems; especially in terms of institutional capacity? To what extent was progress made against key objectives of improving access and quality of education? Increased enrolment Decreased drop out Improved pass rates 4. Assess the extent to which the School Construction Component of the programme has ensured capacity of key stakeholders through participation, training and guidelines; 5

6 The evaluation will examine impact of the approach in terms of contributing to establishment of systems/institutional capacity; The evaluation will consider extent to which policies and strategies have been established including cross-sectoral and gender responsive approaches; The valuation will examine extent to which capacity of stakeholders and ensured ownership including participation, maintenance and monitoring. 5. Review the overall safety and inclusiveness of the school environment, the level ground and environment. How well did stakeholders (head teachers, teachers, school management committee members and students) in the targeted schools participate in the process; by providing leadership harmonization, utilization and maintenance of the built environment? What are the key factors most likely to provide an understanding of success or bottlenecks associated with the implemented interventions in the targeted schools? Improving access and quality education and achieving a child friendly school environment can never depend on infrastructure or school construction alone. It relies on multiplicity of interrelated in and out-of-school factors that are socio-economic and cultural in nature. When attempting to evaluate the impact of certain factors or environment and not others it is important to be able to apply methods that take into account and control for variables known to be directly related to the quality of education. This can include variables such as availability of qualified teachers, access to teaching and learning materials, socio-economic background and other relevant factors within the context of Malawi. The methodology of the evaluation will have to take this into consideration. The evaluation shall include an overall performance rating for each of the above five evaluation criteria (see scope of work) on the basis of the following scale: Highly satisfactory (fully according to plan or better); 6

7 Satisfactory (on balance according to plan, positive aspects outweighing negative aspects); Less than satisfactory (not sufficiently according to plan, taking account of the evolving context; a few positive aspects, but outweighed by negative aspects): Highly unsatisfactory (seriously deficient, very few or no positive aspects). Each rating should be stated as part of the conclusions for each of the five criteria. METHODOLOGY The consultants are expected to present, in detail, their approach, methodology and tools, with an action plan and time frame that addresses the expected outputs, with reference to the overall and specific objectives as well as budget. The following are to be taken into consideration: 1. The consultant will undertake a desk review 2 of construction related education programmes in Malawi to better understand and document the educational context; including challenges and best practices that arise with school construction. The consultant will have to be become well acquainted with all the materials which UNICEF provides. It will have to draw up questionnaire and interview questions, coding plans, and how he/she will analyse the data (analytical framework). This review will provide for a necessary bottleneck analysis that explains the relationship between school infrastructure and key domains of education quality such as access, demand, enabling environment and overall learning outcomes. 2. Taking into consideration that there are numerous and multiple factors that can influence the outcome of a programme a mixed method approach which uses both qualitative and quantitative data collection and analysis is recommended. While quantitative methods and analysis acknowledges the fact that there are objective realities that can impact directly on particular outcomes. Qualitative methods and analysis acknowledges the fact that processes, emotions and experiences are real and play a crucial role in the way people 2 The consultant to ensure all key documents are taken into consideration. The following specific documents are particularly recommended: UNICEF strategies; Sector policy and strategies including ESIP documents; School Improvement Programme documents; school quality standards including those for school infrastructure; MCO/UNICEF country programme documents; Review and evaluation studies;, etc. 7

8 interact and make use of institutions, cultures and social structures. The balance between the two methods is vital for improved understanding and learning of the subject. The evaluation will therefore need to: analyse fixed outcomes and factors through correlations and patterns providing a quantitative analysis of impact and results (quantitative) provide a deeper understanding of the outcome through in-depth interviews and perspectives of the actual beneficiaries of the programme (qualitative) Quantitative data collection and analysis: When choosing a quantitative data collection method the consultant needs to consider that i) control groups were not establish for the programme in the beginning of the programme ii) some school sites have ongoing construction and interventions due to the fact that the country programme has one year to go. The consultant will therefore need to choose a valid method 3 that fits the scope of work in order to be able to assess inputs, outcomes and impact of school construction. Data is to be collected from 35 primary schools and should focus on capturing the relationship between the built environment (school construction) and the following (but not limited to) key education indicators: Enrolment; boys and girls, OVC and SNE Dropout; boys and girls, standards 5 and 8 Pass rates; boys and girls Completion rates; boys and girls It is important to note that there may be some challenges in data reliability and disaggregation of data by gender, age group, etc. It is proposed that some of the risks could be mitigated through triangulation, cross-referencing, focus group discussions and other means. Qualitative data collection and analysis: Whereas the evaluation is to assess processes of work, its contribution to standards, norms and the success of capacity building along with 3 Methods such as before and after comparison or with-without comparison making use of regression analysis are for example valid ways of performing impact evaluations. 8

9 attitudes and participation of stakeholders it is vital to make use of qualitative data collection and analysis. Data is to be collected through informative interviews and focus groups from all key stakeholders 4 at national, district and school level involved in the School Construction. The consultant is also encouraged to make use of onsite observations in order to assess maintenance, cleanliness and utilization of the built environment. 3. Ethical clearance - the consultants are expected to identify all relevant ethical issues (i.e. consent from parents when discussing with children; gender related issues and issues of power balance) from the proposed methodology and seek ethical clearance as appropriate from the National Committee on Research in Social Sciences and Humanities (NCRSH) in accordance with the rules and regulations of conducting research in Malawi. 4. Limitations the consultant needs to take into consideration and provide workable solutions to the following challenges and possible limitation of the evaluation: a. A baseline survey was not undertaken.in utilising a before and after approach the consultant (s) school take into consideration that some children and staff may not have been at the school before the construction started. b. Some school sites have ongoing construction, as such the evaluation criteria may not be applicable in all schools c. Control groups were not established in the beginning of the programme making it difficult to compare outcomes. Hence, the consultant is expected to propose a methodology that works around this issue. The consultancy is expected to develop the above into a robust methodology and draw up a work plan for visiting all relevant sites and stakeholders to collect all relevant and necessary data. SKILLS AND QUALIFICATIONS 4 Including MoEST, EIMU, and Education Managers at district and school level, Head Teachers, Teachers, Community Members and Students. 9

10 A qualified team with at least 8 years relevant background and experience in programme evaluation, research, extensive work with the education sector and school infrastructure is expected to submit a proposal with a team of the following minimum expertise, and background: Education qualification and competencies: Team leader PhD in Education or the Social Science 8-10 years experience in programme evaluation within the field of Education sector, or social development in Malawi or sub-saharan Africa; Proven capacity to conduct research at a national scale, including capacity to work with national level partners as well as management, analysis, and interpret large sets of data Excellent analytical and writing skills. Excellent communication and negotiation skills and fluency in English written and verbal Construction and/or Environmental Specialist (s) Advanced or Master s degree in Built Environmental Studies and Psychology; Architecture or Engineering with a focus on schools environment and links to learning of children; 8 years knowledge and experience in construction design and engineering in Malawi or Sub-Saharan Africa with a focus on the comprehensive range of school infrastructure, and school ground planning and development; Skills in building economics, quantity surveying, and comparative cost analysis and experience in value for money and technical audits; Proven capacity to conduct research, the management, statistical analysis and interpretation of large sets of data, at a national scale, including capacity to work with national level partners; Excellent analytical, writing and communication skills with. Fluency in English both written and verbal is essential. 10

11 DISEMINATION PLANS The final report will be shared widely with Government partners through the Ministry of Education, donors and cooperating partners. The following activities will be implemented in this regard: 1) Report will be printed and widely distributed to key stakeholders 2) At national level, report will be presented in all relevant technical working groups as well as for development partners; 3) At district level. The report will be utilized for review and planning session including at sub district levels. 4) Finally, the report will be used as part of knowledge sharing and management process initiated by the basic education programme. 11

12 TIME FRAME The following provisions outlines a detailed scope of works that the consultant is expected to undertake. The evaluation will be undertaken between April and June Task Deliverable Description Timeframe 1 Inception Report Prepare an inception report summarising how the consultancy has understood the tasks and deliverables and how they propose to deliver on the consultancy, i.e. highlighting the methodology, the data collection tools and analysis process and the presentation of results. 5 days 2 Desk review and documentation of the field of school construction in Malawi 3 Data collection and Field report The desk review should be structured to provide a bottlenecks analysis that presents that relationship between school infrastructure and education in the form of access, demand, enabling environment and overall quality such as learning outcomes. The analysis will provide a background and support the findings of the evaluation. The consultancy is expected to produce a field report after data collecting documenting process, stakeholders met and key challenges. A PowerPoint presentation will be done to a reference group comprising UNICEF, Ministry of Education, Science and Technology and other relevant stakeholders for validation and feedback. 10 days 25 days 12

13 4 Final report and presentation of findings 1. Present draft findings in the form of a report and a power point presentation to UNICEF, Ministry of Education, Science and Technology and other relevant stakeholders for inputs, validation and feedback. 10 days 2. Present final electronic copies of the findings to UNICEF and Ministry of Education, Science and Technology in a user friendly report and power point presentation. The final report should include the following: Executive summary (non-technical including conclusions and recommendation) Introduction and background (including the bottleneck analysis) Methodology Findings (both quantitative and qualitative) Discussions (wider context) 50 days KEY RESULTS Program review that results in evidence for future interventions and programmes on child friendly school environment 13

14 Final report that will influence policy and programmatic decisions relating to enhancement of quality education within Malawi at national, sub-national and school levels DELIVERABLES Intermediary deliverables 1. Inception report and power point presentation to validate the methodology to be used in the evaluation. 2. Field report after data collection and PowerPoint presentation for validation and feedback. 3. A preliminary report highlighting major findings from the assessment and key recommendations shared and reviewed. 4. Power Point presentation on the preliminary findings with UNICEF, government and stakeholders Evaluation report The Evaluation Report which shall be compliant with the UNICEF - Adapted UNEG standards (to be provided to successful consultants) should include - but is not limited to - the following components: Table of Contents Executive Summary. Background Methodology including source of data, data collection, people and places visited Quality and reliability of data Findings Conclusions Recommendations Appendices should include: Terms of Reference List of people interviewed 14

15 List of documents consulted A tightly-drafted, to-the-point and free-standing Executive Summary is an essential component. It should be short and not more than five pages. It should focus mainly on the key purpose or issues of the evaluation, outline the main analytical points, and clearly indicate the main conclusions, lessons learned and specific recommendations. The Executive Summary shall include the Performance Rating of the main 5 evaluation criteria. Cross-references should be made to the corresponding page or paragraph numbers in the main text that follows. The Recommendations should be the subject of a separate final chapter. Wherever possible, for each key conclusion there should be a corresponding recommendation. The key points of the conclusions will vary in nature but will often cover aspects of the key evaluation criteria (including performance ratings). Final deliverables As final outputs three sets of documents will be required (electronic copies in Word & PDF)- (1) Final evaluation report, (2) Micro data in soft copy BUDGET The consultant should provide an overall budget for the activity of the evaluation. Payment will be best on deliverables as follows Inception report and presentation 30% Field report and presentation 20% Draft report and presentation 20% Final report and data sets 30% Standard UNICEF procedures will apply for invoicing and all other financial management requirements set out in the contract. Standard penalty clauses will also apply for late and poor quality deliverables. MANAGEMENT OF THE EVALUATION 15

16 The consultant will report to a reference group including the Chief of Education as well as relevant government and civil society stakeholders. A reference group is responsible for quality assurance of the evaluation at each stage providing the consultant with the criteria for the evaluation of the quality of each deliverable. A technical team will provide required logistical support, available research, documents, studies and evaluation relevant to the exercise Name Title Signature Date Charles Nabongo Chief of Basic Education Chris Davids Chief of PME, Roisin De Burca Deputy Representative Mahimbo Mdoe Country Representative Applications should be sent on or before 13 October 2015 to the following address: hrmalawi@unicef.org 16