Invitation to Tender: External evaluation of the STEM Insight programme 1. Summary STEM Learning Ltd. is seeking an evaluator to examine the effectiveness of the STEM Insight programme which offers staff in schools and colleges a unique chance to experience STEM-related work in industrial or university settings. Teachers participate in a placement with either an employer or a university, are supported by face-to-face and online CPD, and learn more about diverse career paths and opportunities for students. The evaluation should cover the period July 2014 to December 2016. 2. Background Originally conceived by the Institute of Mechanical Engineers (IMechE) and delivered by STEM Learning Ltd, the programme was initiated in 2014 and was previously known as the Teachers Industrial Partners Scheme and the Teachers Academic Partners Scheme. The pilot phase of the programme ran through 2014/2015 during which time 26 teachers took part in placements with Industry and Academic hosts for a period of 5 or 10 days. The teachers are supported by an online CPD programme and the programme concludes with a face to face group CPD session, post placement. The programme is delivered through a partnership within Project ENTHUSE, and is overseen by a Steering Group involving IMechE, IET, The Biochemical Society, The Institute of Structural Engineers, and BP. The two schemes were brought together in April 2016 and rebranded as STEM Insight and during 2016 a further 35 placements were completed. The STEM Insight programme provides teachers and technicians (primary, secondary and FE) with extended opportunities to experience modern STEM working environments whether in industry or in academia. Through this experience, teachers can develop a range of contexts for their teaching, and gain a first-hand understanding of the varied range of careers available and current routes through which young people can progress into and through STEM-related careers.
The programme achieves this by providing teachers with: unique chances to experience STEM work in modern employers and use this insight to enrich the teaching of STEM subjects (Insight into industry) unique chances to experience the changing nature of STEM subjects in Higher Education and the skills required for undergraduate study. (Insight into university) invaluable CPD opportunities which equip schools and colleges to respond to the Government Careers Strategy and transform careers guidance. better understanding of the breadth of STEM- related occupations within the industrial and academic sectors of the UK, raising awareness of the career paths and progression routes through to industry and Higher Education. fostering links between schools, colleges and industry and Higher Education building a community of practice, with a network of school/industry/academic experts who can support other colleagues in schools and colleges. The number of placements completed to date is: Trial period - 2014/15 (July Jan- Dec 2016 2014 to December 2015) 26 35 The key objectives of the STEM Insight programme are: For teachers Improved knowledge of STEM-related careers and educational / vocational routes, employability skills, subject and pedagogical knowledge Increased enthusiasm and motivation for teaching Career retention/progression Increased use of enrichment activities in formal and informal learning For students Improvements in knowledge of STEM careers and of corresponding educational / vocational routes and appreciation of STEM and employability skills Increased uptake of STEM subjects at A levels 2 years after the programme Increased aspirations for STEM careers Improved motivation and engagement in lessons Raised achievements in relevant STEM subject(s)
For the school/college Sharing new knowledge and resources with colleagues More links / engagements with employers, HEIs and STEM Ambassadors Increased use of industry and HEI examples in teaching Rising quality of subject teaching Improved career education in STEM subjects Improved STEM career awareness among parents For employers (HEIs and Industry) Improved understanding of how to use formal and informal learning to inspire young people to wards STEM further education and careers Better able to inform young people and parents about career pathways and employment opportunities available with STEM employers 3. Purposes of the evaluation The purposes of the evaluation are to: Investigate how successful the programme is at achieving the aims and objectives noted against each of the key performance indicators (see Appendix 1). Assess the level of impact the STEM Insight has in terms of the teacher, the student, the school or college, and the employer. Assess the outputs and impact of the programme as a whole in terms of the overall aims Provide further recommendations on how to improve management, delivery and impact of the STEM Insight programme. 4. Methodology The evaluation will be expected to use data and method triangulation and combine qualitative and quantitative methodologies. School visits and or case studies should form part of the methodology. The focus of the evaluation will be guided by the delivery targets and key performance indicators (KPIs) agreed with the STEM Insight steering group (Appendix 1). We also expect the evaluator to integrate programme monitoring information as well as evaluation and impact data, which STEM Learning collects internally (Appendix 2)
Your tender response should set out a detailed explanation of the methodology you propose to use for this evaluation; including evaluation models that you feel are particularly suitable for a programme of this kind. Proposals should indicate which aspects of the criteria in Appendix 1 will be covered by the planned methodology. 5. Intellectual Property Rights All intellectual property arising from any work will be vested in STEM Learning Ltd. 6. Deliverables and Audience: The main deliverables will be a clear, succinct report highlighting emerging findings (14 April 2017) and a final report that sets out the key findings and makes recommendations for improvements (30 April 2017). The principal audiences for the reports will be STEM Learning s leadership, who will consider the reports so that lessons can be learned and implemented for the next, expanded phase of the programme. The final report will also be shared with the programme s funders, the programme steering group, and other key stakeholders as required. 7. Timeline: Activity Milestone Open, competitive ITT issued Clarification questions responded to 6 Feb 2017 Last date for the submission of any queries. 17 Feb 2017 Deadline for receipt of submissions 20 Feb 2017 Independent evaluator appointed and evaluation started W/C 27 Feb 2017 Progress updates to STEM Learning Programme team Emerging headlines 14 April 2017 Final report 30 April 2017 Weekly updates in March 2017, then fortnightly updates 8. Costing and pricing Proposals should include a full budget plan for the whole costs including any anticipated travel, expenses and VAT. All costs should be clear and transparent, clarifying the number of days of each individual working on the activity that they provide. Decisions will be based on perceived value for money, rather than the
lowest cost proposal. Within the region of 15,000 for the contract period (March April 2017) 9. Tender content Tenders should show, as a minimum: How the aims and actions of the evaluation will be addressed to ensure effective and ethical completion of the evaluation across the UK Scotland, England, Northern Ireland and Wales. The timeline for development of instruments, data collection, analysis and reporting. The roles, responsibilities and brief CVs of personnel in the research team. A breakdown of costs by staff and non-staff categories, office costs, travel and subsistence costs. Risk assessment and mitigating actions. Contact details of two referees Where possible links to recently published reports should be included. The intention is for there to be one group of researchers undertaking the external evaluation. If two (or more) institutions wish to jointly tender, that is permissible, but the responsibilities of members of the research group need to be made clear. 10. Submission and format of proposals The deadline for receipt of submissions is 12.00 noon on Monday 20 February 2017. Late submissions will not be accepted. All submissions will be acknowledged with a notification of receipt. An electronic copy of the proposal should be received in PDF format by this deadline. This is an electronic only submission process; therefore all documentation must be submitted in PDF format as a single zipped folder if the size of the submission is greater than 10Mb. Proposals should be no longer than 10 sides of A4 (excluding CVs of people involved) and typed in Arial size 12 font. If the proposal exceeds this limit evaluators will be advised to disregard any information outside the limit. Proposals with the subject line STEM Learning Evaluation Submission should be emailed to: d.watson@stem.org.uk STEM Learning reserves the right not to award a contract.
A decision will be made as soon as possible after receiving the tenders, with a telephone clarification meeting if needed. 11. References Tenderers should indicate the names of two current or recent customers for whom similar evaluations have been carried out and who would be prepared to act as referees. 12. Selection criteria Proposals will be evaluated against the following criteria which are not listed in order of importance: Compliance with, and understanding of, requirements outlined in this tender document. Understanding of the educational landscape, STEM education and the background and context of the STEM Learning network. Effectiveness of the proposed methodology for undertaking the evaluation. Clarity and feasibility of the programme of work, work plans and risk assessment. Relevant experience of the tendering team. Ability to meet the required timetable. Management of any existing or potential conflict of interest identified by the bidder. Value for money. 13. Enquiries These instructions are designed to ensure that all tenders are given equal and fair consideration. It is important therefore that you provide all the information asked for in the format and order specified. Please contact Dianne Watson (d.watson@stem.org.uk) if you have any difficulty in providing the information requested. Pre-tender negotiations are not allowed. If you have any questions relating to the requirements then please contact Dianne Watson as above. We will aim to respond to any queries within 5 working days. If we feel that the query relates to an issue of which all prospective bidders should be made aware we reserve the right to communicate our response to all those who have requested tender documentation. If you consider any point you raise to be confidential to your own proposal please make this clear. Please note the last date for the submission of any queries is noon Monday 20 th February 2017.
Appendix One (Key Performance Indicators) KPI areas Data description Outcomes and indicators Quality of engagement Participants evaluation feedback on the quality of placements and the follow-up CPD training - Participants positive feedback (quantitative., e.g. overall quality, usefulness, and qualitative, e.g. collected via interviews or case studies) Impact on STEM educators (teaching staff and career advisors) Impact on young people Impact on schools /colleges Impact on employers (HEIs and industry) Participants (and, their colleagues) reporting on impact after the placement and after the follow-up CPD (short- and long-term impact) Participants (and their colleagues and students) reporting on impact after the placement and after the follow-up CPD (short- and long-term) National school and pupil data (NPD, Ofsted, Edubase) Participants reporting on - School use of enrichment activities in STEM teaching - Provision of career info and advice - Quality of teaching in STEM subjects National school and pupil data placement leads feedback and reporting of impact - % of participants reporting High/Medium impact - Improved knowledge of STEM-related careers and educational / vocational routes employability skills subject and pedagogical knowledge - increased enthusiasm and motivation for teaching - career retention/progression - use of enrichment activities in formal and informal learning - % of participants reporting High/Medium impact - Improvements in - knowledge of STEM careers and of corresponding educational / vocational routes - appreciation of STEM and employability skills - uptake of STEM subjects at A levels 2 years after the programme - increased aspirations for STEM careers - improved motivation and engagement in lessons - raised achievements in relevant STEM subject(s) - % of participants reporting High/Medium impact - number of students impacted by the programme - number and range of STEM enrichment activities implemented per school - Sharing new knowledge and resources with colleagues - More links / engagements with employers, HEIs and STEM Ambassadors Increased use of industry and HEI examples in teaching - Rising quality of subject teaching - Improved career education in STEM subjects - Improved STEM career awareness among parents - Improved understanding of how to use formal and informal learning to inspire young people to wards STEM further education and careers - Better inform young people and parents about career pathways and employment opportunities available with STEM employers
Appendix Two (Internal data collection processes for project monitoring and evaluation) Information collected via STEM Learning s online impact toolkit, which participants use to plan and later on report on the completion of actions and achievement of outcomes and impacts, record the supporting evidence and feedback on the quality of the placement and CPD Case studies collected from former participants Focus groups and interviews with teachers during a follow-up CPD session Evidence from participating schools and STEM employers collected from the STEM Insight online community group or via social media