Planning, monitoring and evaluation Formats & Project Cycle Management (Logical Framework Approach) Koen De Koster VLIR-UOS Programme Officer
Project Cycle Management Defines different phases in the project (programme) life with a well-defined process of involvement of different stakeholders, management activities and decision making procedures 2
Identification (South Proposal) to Formulation: Partner Programme (PP) PP = critical document outlining all details for the first 5 year phase: programme, projects, problem analyses, results frameworks, planning, etc. Guiding document? south concept notes Developed during pre-partner programme Key moment: formulation mission Parallel project level workshops with stakeholders Preceded by LFA training 3
Partner Programme document?
PARTNER PROGRAMME (PP) FORMAT Programme level 1. Facts and Figures 2. IUC Phase I Partner Programme 2.1. Programme Objectives 2.2. Complementarity and synergy external and internal 2.3. Sustainability strategy of the IUC cooperation 2.4. Transversal themes of the Belgian Development Cooperation 2.5. Programme level annexes Programme level annexes to the proposed Phase I Partner Programme Annex 1: Partner Programme monitoring framework and budget Annex 2: Management manual including Partner Programme policies Annex 3: Strategic plan of the university Annex 4: The partner university organogram
PARTNER PROGRAMME (PP) FORMAT Project level 1. Project n 1: <Project title> 1.1 Presentation of the project partners 1.2 Problem analysis 1.3 Project strategy 1.4 Budget and operational planning for Phase I 7. Project n 7: <Project title> 8. Project level annexes Project level annexes Project Annex 1: LFM - OP RM Project Annex 2: Project budget overview for the entire Phase I Project Annex 3: Mobility for year 1 (cfr format, own formats allowed) Project Annex 4: Problem tree Project Annex 5: Participatory Assessment of academic unit (optional)
How to arrive at a PP?
How to? VLIR-UOS mainly uses the Logical Framework Approach, combined with some Theory of Change concepts. Logical Framework Approach is a participatory P,M and E tool and process, and is wider than the format itself Ideally finalised during formulation Preparation by local PL & team + stakeholders in preparation of formulation phase Typically in a workshop setting (formulation mission) with facilitation 11
Planning, Monitoring & Evaluation Strategic evaluation: - Mid-term evaluation - Final Evaluation Programming Evaluation Identification & Selection Problem Analysis Strategic monitoring - Monitoring results (both indicators and programme theory) - Risk management Operational monitoring - Activities - financial Implementati on Formulation Strategic planning - Programme Theory - Planning for monitoring - Risk management Operational planning - Activities - Budget 12
Logical Framework Approach during formulation Analysis Phase 1. Stakeholder Analysis 2. Problem analysis (current situation) 3. Analysis of objectives (Future improved situation ; Identify domains of change) 4. Strategy analysis Comparison of various options and selection (Identify strategic priorities) Planning Phase 1. Logical framework Map pathway of change Define monitoring, evaluation and learning priorities and process 2. Activity scheduling identifying key deliverables 3. Resource scheduling 13
1. Stakeholder Analysis 1. Identification: identify stakeholders individuals, groups of people or organisations who have an interest (a stake) in the (proposed) project and hence can have a positive or negative influence or contribution 2. Analysis: 1. Analysing stakeholders interests/attitudes 2. Analysing Stakeholder Power and Influence 3. Take into account potential differences for men and women gender-sensitive analysis can unravel possible gender inequalities 3. Defining stakeholder management 14
2. Problem Analysis «identifies the negative aspects of an existing situation and analyses the cause and effect relationships between the identified problems» 1. Definition of the initial scope 2. Identification of the major problems faced by target groups and (in)direct beneficiaries 3. Visualise the problem analysis and the hierarchy of problems (cause and effect relationships) using a Problem Tree (other possible methods include: Mind Maps or Rich Picture) 16
2. Problem Analysis problem tree Effects Decreasing incomes of artisanal fisherfolk Decreasing fish stocks Low price received by artisanal fisherfolk in the village Causes Destruction of coral & mangrove habitats Illegal fishing methods applied Processed fish is of bad quality Limited access to markets 19
Rich Picture
2. Problem Analysis problem tree Insufficient integration of research methods in education Insufficient research equipment Know-how on the organisation of research is lacking Absence of fully fledged labs Lack of central government policies Low FI in sector Insufficient research capacity Inadequate quality of HR in sector Insufficient evidence based policy-making Low income of households Development problem
3. Analysis of objectives Identifying domains of change Key question: what needs to change in order to achieve the desired change? Who and what needs to change in order to achieve the desired change? Output: identification of different domains of change (different actors, different levels, etc.) hierarchy of objectives Most used method: objectives tree.
3. Analysis of objectives Ends Incomes of artisanal fisherfolk increased Rate of decline in fish stocks arrested Price received by artisanal fisher-folk increased Means Coral & man-grove habitats conserved Incidence of illegal fishing reduced Quality of fish processing improved Access to markets improved 24
4. Analysis of Strategies Identifying strategic priorities Exploring and deciding on strategic priorities within the domains of change towards the desired change (defining final scope/priorities); based on an assessment of the relevance, the feasibility and the sustainability the objectives; to concentrate on what is really important, effective and feasible. What can you realistically attain in the next 5 years? 26
4. Analysis Of Strategies In Out Incomes of artisanal fisherfolk increased Rate of decline in fish stocks arrested Price received by artisanal fisher-folk increased Coral & man-grove habitats conserved Incidence of illegal fishing reduced Quality of fish processing improved Access to markets improved 27
Sufficient, reliable and affordable energy supplies for all Rwandans Better management of demand side Electricity Generation performance Electricity Transmission and Distribution Performance Better Commercial performance Adequate forecasting of demand Improved Operations & Maintenance Better billing Phasing of extensions Working on adapted tariffs to discourage consumption during peak hours Construction of Power Plants, transmission lines, distribution lines (out of scope) Rehabilitation of Power Plants, Transmission and Distribution infrastructure Accessibility of commercial services for clients Avoidance of nontechnical losses
Objectives Deliverables related to education improvement Research deliverables Strengthened research or education capacities Infrastructure and equipment (functional) Improved Research practices Improved Education practices New knowledge is created through research + Uptake of research results by relevant stakeholders Contribution to development change
Logical Framework Approach during formulation Analysis Phase 1. Stakeholder Analysis 2. Problem analysis (current situation) 3. Analysis of objectives (Future improved situation ; Identify domains of change) 4. Strategy analysis Comparison of various options and selection (Identify strategic priorities) Planning Phase 1. Logical framework Map pathway of change Define monitoring, evaluation and learning priorities and process 2. Activity scheduling identifying key deliverables 3. Resource scheduling 30
Logical framework Intervention Logic Objectively Verifiable Indicators OVI Sources of Verification SOV Risks/Assumptions GENERAL OBJECTVE SPECIFIC OBJECTIVE(S) INTERMEDIATE RESULTS ACTIVITIES 31
1. Logical framework Map pathway of change: Define results at different levels, reflect on their internal logic, formulate assumptions/risks Define monitoring, evaluation and learning priorities and process: developing indicators to measure results (and risks)
1. Logical framework: pathway of change (programme theory) 1. Detail the pathway of change (causal pathway; cfr. objectives tree): 2. Make implicit assumptions/ risks/ preconditions underlying the intervention logic explicit 3. Do results work out differently for men/women? 4. Determine your intervention s results at different results: General objective (Impact) Specific objective(s) (outcomes) Intermediate results (outputs) 5. You can now fill in the first column of the logical framework
Examples of detailed pathways of change
Logical framework Intervention Logic Objectively Verifiable Indicators OVI Sources of Verification SOV GENERAL OBJECTVE SPECIFIC OBJECTIVE(S) INTERMEDIATE RESULTS ACTIVITIES 37
1. Logical framework Results level Outputs Specific objective(s) General objective IR s (outcome level) (impact level) Level of control Sphere of control Sphere of influence Sphere of interest What? Products or services (can also include use of outputs) Changes for beneficiaries Contribution to a change at level of society Resulting from Resulting from activities Change resulting from the use of outputs Resulting from the achievement of a combination of (other) outcomes When? Achieved during the intervention Achieved at the latest at the end of an intervention Appears (mostly) after the end of an intervention
Standard project Outputs (intermediate results) Outcome (Specific objective) Impact (general objective) Deliverables related to education improvement Research deliverables Strengthened research or education capacities Infrastructure and equipment (functional) Changes in behaviour Uptake Use of outputs Improved Research practices Improved Education practices New knowledge is created through research + Uptake of research results by relevant stakeholders Uptake Use (application of innovation) Contribution to development change Long-term effects effect of doing things differently Effects of extension Sphere of control Sphere of influence Beneficiary Why do we do it? Society Achieved during intervention Results obligation Achieved before end of intervention Best efforts obligation Achieved after the end of the intervention Contribution
Logical framework : defining M&E priorities Indicators Objectively Verifiable Indicators: to demonstrate the achievement of results at different levels (IR, SO, GO) to feed into the intervention s learning process (e.g. indicators related to assumptions/hypotheses) + Next to unique indicators, pick indicators from standard indicator list Determine how indicators will be measured (Source of Verification) Disaggregate by sex when possible
Logical framework : defining M&E priorities Risks/assumptions can negatively affect an intervention (Identification) Integrate assumptions/risks in the risk management matrix + identify other risks (FIN, COM, OP, STRAT, REP) Analyse risks Define risk responses
Risk Management Matrix RISK MANAGEMENT MATRIX > For instruction please see Annex document on Monitoring and Evaluation (pages 18 22) PROJECT TITLE: Identification Assessment Control Follow-up Date identifica - Risks tion Probability Potential impact Total risk level Response (incl. timing) Owner Follow-up 0 0 0 0 0 0 0 0 0
Standard indicators RESEARCH 1Number of articles in international peer reviewed journals 2Number of articles in national peer reviewed journals 3Number of conference proceedings (full paper) 4Number of chapters in books (based on peer review) Number of books with international distribution (author or 5editor) Number of working/technical papers/popularising 6literature/articles 7Number of Research protocols EDUCATION Number of new or substantially updated Bachelor 8programmes developed (curriculum) Number of new Bachelor programmes accredited 9 Number of new or substantially updated Master programmes 10developed(curriculum) Number of new Master programmes accredited 11 Number of new or substantially updated PhD programmes 12developed (curriculum) Number of new PhD programmes accredited 13 14Number of new courses developed Reach (number of students) of new courses (disaggregated 15by sex: male: - female: - total:) 16Number of syllabi/textbooks developed E-Learning packages developed (distance learning, CD-rom 17etc.) HR DEVELOPMENT Number of Master students supported by the project that have graduated during the reporting year (disaggregated by sex: 18male: - female: - total:) Percentage of academic staff with a Master degree by relevant institutional level (department or faculty level; 19disaggregated by sex: male: - female: - total:) Number of PhD students supported by the project that have graduated during the reporting year (disaggregated by sex: 20male: - female: - total:) Percentage of academic staff with a PhD degree by relevant institutional level (department or faculty level; disaggregated 21by sex: male: - female: - total:) EXTENSION AND OUTREACH 22Number of presentations or workshops and reach 23Number of citations in local media 24Number of training modules package developed 25Number of audio visual extension materials 26Number of spin-offs/incubators 27Number of policy advice/papers
2.-3. From logframe to activity and resource schedule (incl. Phasing) Operational Plan Intermediate Result/Activity/Sub-activity BUDGET (2) R 1 2 3 4 5 6 7 8 9 10 11 12 1.1 Hep vaccine in health care Workers AprMayJunJulyAugSep Oct NovDec JanFebMarc 1.1.1 Survey all higher education institutions in SA regarding HBV vaccination policies and training x x x x x x x x x 500 M 1.1.2 Survey all categories of HCWs for knowledge, attitudes and practices (KAP) regarding HBV vaccination x x x x x x 500 A 1.1.3 Perform serological and molecular tests on HCWs surveyed in 1. 1. 2 x x x x x x 10 000 1.1.4 Link KAP survey data of 1.1.2 to data obtained from 1.1.3 1,2 Health problems, health risk behaviours and obesity in HCWs (later in staff and student HCWs) 1.2.1 Conduct baseline survey of health risk behaviors and health problems in 3 settings x x x 500 L 1.2.2 Develop training packages for interventions x x x 500 1.2.3 Set up intervention facilities i.e. exercise and group sessions venue x x x 5000 1.2.4 Conduct intervention research in various settings respectively (RCTs, Quasi experiments and PAR) x x x nil 48
What about the programme level? PSU? No full PCM approach, but An implicit programme theory A results framework with overall objectives A set of indicators Operational plan No full PCM approach, but Risk management An operational plan and budget
IUC programme? Standard project (outputs) Programme Support Unit In targeted departments Improved Research practices Improved Education practices New knowledge is created through research + Uptake of research results by relevant stakeholders Contributes to More performant institutional policies and/or services Contribution to different developmen t changes University as a development actor Institutional capacities improved Programme level framework Transversal projects Project perspective Programme perspective
Thank you for your attention 51