Stronger Communities Programme: Final Evaluation

Transcription

1 Stronger Communities Programme: Final Evaluation Executive Summary for SCDC and Foundation Scotland 28 November 2014 EKOS Limited, St. George s Studios, St. George s Road, Glasgow, G3 6JA Reg Telephone: Web: 1

2 Direct enquiries regarding this report should be submitted to: Regina Trenkler-Fraser, Associate Director, EKOS Tel: As part of our green office policy all EKOS reports are printed double sided on 100% sustainable paper 2

3 Executive Summary This report represents the final report of the ongoing evaluation of the Stronger Communities Programme undertaken from 2010 to The BIG Lottery Fund (BIG) supported three organisations and their respective initiatives: Future Balance 1 Sus It Out Plus; Scottish Community Development Centre - Achieving Community Empowerment (ACE); and Foundation Scotland 2 - Our Community Our Future (OCOF). The objective to provide capacity building support to community groups in Scotland was shared by all three programmes, but the approaches used and the levels of intensity of support provided differed. In this context, the Sus It Out Plus programme delivered a high volume, light touch and more conventional package of support including the delivery of an Action Plan and financial support, whereas the ACE and OCOF programmes offered more in-depth, long-term support delivering bespoke packages of assistance. Whilst OCOF included financial assistance and an interest to support new, evolving community groups, ACE focused on the delivery of capacity building. The management of the Stronger Communities Programme was shared between the three delivery partners with Future Balance responsible for website development, SCDC for evaluation; and Foundation Scotland for shared learning. Chapter 1 of the report presents a brief outline of the evaluation objectives and the four phases of its delivery. Phase 1 incorporated a policy review and the development of a Logic Model and monitoring & evaluation framework for each of the three delivery programmes. Phase 2 of the evaluation work examined the implementation of the Programme against targets in 2011 and 2012, including longitudinal case study work. This was concluded in August 2014 through Phase 3 and 4 which incorporated the assessment of outcomes and learning impacts of the overall delivery. The policy context of the Programme in terms of community capacity building is summarised in Chapter 2, building on work undertaken during Phase 1 and highlighting some of the key developments during the period from 2010 and The Summary Policy Overview presents a range of policies with particular reference made to the National Performance 1 Previously known as Forward Scotland. Future Balance has since ceased to operate. 2 Previously known as Scottish Community Foundation. 3

4 Framework emphasising the Programme s strong alignment with Outcome 11 We have strong, resilient and supportive communities where people take responsibility for their own actions and how they affect others. A further emphasis is placed on the Community Empowerment Scotland Bill which came on stream during the implementation time of the programme and which is designed to strengthen and increase the ability of community groups to exert influence and shape their own future and that of their local communities. The Programme activities supported the public consultation process of the Bill, thereby building capacity, confidence and engagement of participating groups to contribute to its development.. The policy review further outlines the wider context of the Stronger Communities Programme in light of the extensive austerity measures undertaken by national and local Governments. This coincides with the increasing encouragement of community groups to take on services which used to be delivered publicly, to become more self-sufficient in their funding, and to develop into social enterprises. The Programme therefore represented a timely intervention which was successful in bringing about increased sustainability, effectiveness, and capacities of participating community groups. Chapter 2 concludes that after four years of programme implementation, the strategic alignment of the Stronger Communities Programme can still be regarded as strong and relevant having contributed effectively to the overall policy aims and objectives in Scotland regarding capacity building, social enterprise development, and community empowerment. Furthermore, the Programme delivery bodies have effectively promoted new policy development in Scotland during the Programme implementation period thereby further supporting the engagement of community groups at the policy/decision-making level. In Chapter 3, the three delivery programmes are presented in greater detail drawing on the various annual evaluation reports delivered throughout the Programme period and summarising key findings and lessons learned. There have been both similarities and differences between the Sus it Out Plus, OCOF and ACE programmes. While there is strong evidence that all three programmes impacted positively on confidence levels, awareness raising, and capacities building of participants and beneficiary groups, the differences lay to some extent in the community groups targeted, availability of cash resources, and in the intensity and length of support provided. Although the light touch, but high volume Sus it Out Plus programme reached substantially more 4

5 community groups, the research suggests that the longevity of its positive impacts might be more limited. With a budget of around 617,000, Sus It Out Plus was delivered by Future Balance between July 2009 and March It was specifically targeted at working with established and growing community groups, charities and social enterprises to support them in embracing more sustainable behaviours and principles in all aspects of their operations. In the context of the programme, the term sustainability was interpreted in its wider sense, i.e. including organisational capacity, involvement in civic life, and organisational efficiency measures. The delivery of the programme included: a facilitated group session using an interactive board game to stimulate discussion around sustainable development issues. This was regarded as a very effective way of engaging groups; Future Balance used the information generated through the session to develop an Action Plan for the respective group including action points and training plans; and the third element of the programme was a Sustainable Communities Learning Account of 2,000 for each group (later reduced to 1,500 as not many groups required the full amount). The programme was very successful in attracting more participating groups (225) than anticipated at the outset (150). Findings from a number of surveys showed that the programme had a positive impact on the confidence levels of participating community groups (91% of groups stated this), and the Action Plans provided useful for identifying areas for further training. The availability of funding enabled the groups to implement some of the identified training. The majority reported positive impacts with improved skills in a range of areas, including sustainability and environmental issues, but mostly in communication, governance, and financial control. Over time the Action Plans were less used than immediately after the programme delivery and a final survey showed that the majority of groups did not use them anymore or have not updated them. In future, consideration should be given to arrange for follow-up support over a longer period of time which might help increase the potential for more lasting and sustained impact. 5

6 Delivered by SCDC, the Achieving Community Empowerment Programme (ACE) had a budget of around 524,000 and was delivered over five years from October 2009 to September ACE aimed at building the capacity of emerging community groups located in the bottom 20% of SIMD datazones or thematic groups demonstrating disability or exclusion issues. The delivery of the programme included the: provision of mentoring support via a series of structured and interactive reviewing and planning workshops over a period of approximately 12 months. This involved the assessment of needs, issues and problems faced by the group; introduction of an outcomes focused approach through workshops focusing on planning and supporting the groups in implementing their plans; facilitation of the sharing of knowledge, experience and learning between participating groups; and a final workshop reviewing progress and achievement. Against an original target of 50, later revised to 40, ACE supported a total of 37 community groups. The ACE programme did not have any cash resource available for the groups. Evaluation findings indicate that there was a significant increase in acquiring action planning skills through the ACE programme support. Whilst, at times, a clearer exit (a weaning off) from the programme support might have been appropriate. ACE further impacted positively on motivating participants, increasing their confidence, and validating their ambitions. In view of the specific target group of the programme (emerging groups in challenging socio-economic areas) a relevant and important achievement. Some of the key lessons learned by ACE were that the in-depth approach of supporting emerging groups required not only more time to build relationships and trust, but also a significant amount of flexibility to suit the individual circumstances of each group and to deliver a bespoke and effective programme of assistance. A small number of research findings show that the lack of a cash-resource for some of the ACE groups meant that identified training needs could not have been followed through. From an evaluation point of view, this can be regarded as a disadvantage for ACE and its beneficiaries. 6

7 Foundation Scotland delivered the Our Community Our Future Programme (OCOF) which was designed to support groups of individuals who shared a concern in their communities. The 18 months support aimed at building confidence and capacity to address particular issues/challenges the groups were facing, to mobilise wider community support, and to be better positioned to effectively engage with their community and influence decision-makers. From March 2010 to June 2014 and with a budget of around 466,000, OCOF worked with a small portfolio of seven early-stage community groups involving approximately 49 individuals over a period of approximately two years (one further group withdrew at an early stage). OCOF s delivery included the following distinctive features: initial and ongoing community capacity building support and action research; providing access to funding from a programme resource bank ( 20,000 per group; reduced to 5,000 for later groups); and connecting people, ideas and resources through facilitating learning exchanges. Whilst underachieving against its original targets of engaging with 96 individuals. 3 The evaluation findings indicate that OCOF was successful in increasing the social capital of the beneficiaries (better understanding of local issues and opportunities, increased capacities regarding commitments and mandates, improved connectivity of groups, etc.). The ability of OCOF to provide supervision over the use of the grant funding resource empowered a number of groups substantially although 20,000 proved too large an amount for most, the smaller amount of 5,000 was considered more manageable. Particularly noteworthy is that a number of asset transfers have been secured by some beneficiary groups that have built the groups credibility within their communities and has led to increased confidence and ability to take advantage of future opportunities. The key learning points of OCOF included the necessity of a flexible, responsive and longterm support provision particularly when working with fledgling community groups. This included the addressing of high membership turnover, the time-intensive process of setting up formal structures; and the need for providing exit strategies, or road maps for the future. 3 OCOF s outcomes were based on individuals not number of groups which might have been more appropriate for the programmes recruitment approach focusing on community groups. 7

8 The Stronger Communities Programme brought together the three sub-programmes through joined-up programme management procedures and a number of further mechanisms to support cross-programme exchanges which are discussed in Chapter 4. As part of the ongoing evaluation, programme development workshops (establishing the logic models for each sub-programme at the outset), and annual key informant workshops (reflecting on performance and potential improvements) supported joined learning between programme delivery staff. Similarly, yet at beneficiary level, the Stronger Communities Programme facilitated a number of learning exchanges bringing the beneficiary groups of the three programmes together. These events were highly regarded by the participating groups for the quality of organisation, content, and ability to share experience with other groups. A further joined-up concept was the shared area approach, whereby the initial idea was to bring groups together in workshop events who shared the same geographic location. Encountering a mixed success, a more thematic focus of the events was undertaken also with limited success. Finally, it was concluded that the most effective and beneficial sharing between community groups would need to take its steer much more from a highly bespoke, needs-driven perspective rather than a pre-defined geographic or thematic rationale. In addition, the Stronger Communities Programme should have been supported via a programme specific website from the outset. Although a dedicated site was designed and launched in the first year of the programme, it failed to attract participating groups and wider stakeholders to use it, therefore failing to achieve a profile or presence that would promote programme activities or community capacity building approaches 4. Only during the later stages of the programme was the Communities Chanel launched by SCDC as part of another programme which presenting a dedicated website space for beneficiaries and nonparticipants and offering access to manuals, documents, and case study material produced during the duration of the programme. The Programme outcomes and key evaluation findings are presented in Chapter 5 with a more detailed write up of findings available in the four appendices of the report. Overall, and across all three sub-programmes satisfaction levels with the delivery were very high. A consistent three-quarters of surveyed organisations felt that the programmes were important/very important in helping to make their group more successful, improving their 4 The Communities Channel is funded by Scottish Government and is available to all community groups in Scotland. A small underspend from the ACE website budget covered costs to ensure ACE groups were given a distinct profile on the site. 8

9 strategic and operational and policy documents. Here, the case study research of 13 beneficiary groups summarises the key findings as follows: participating groups appreciated the support as it provided them with time and a space to reflect and think on key issues of their groups, which usually are not addressed due to a pre-occupation with day-to-day delivery of services; groups felt that the support has enabled them to become more aware and have a more open mind not only towards their own group issues, but also with regard to their wider community and policy environments; and all case study groups felt benefits arising from participation particularly in terms of increased confidence levels, renewed focus of their group, improved communication and ability to deliver. A comparison between baseline values and scores provided at the final evaluation stage, findings suggest that substantial improvement has been achieved in group management, governance, group capabilities. However, the biggest increase in scores has been achieved in the sustainability of organisations confirming the validity and success of the Stronger Communities Programme in achieving its ultimate objective. Although less evident in the scoring, the majority of surveyed respondents (86%) reported that the programme had a positive impact on their personal skills and capacities, particularly their confidence levels. While the focus of the evaluation was to assess the extent to which the three programmes were effective in building the capacities of the participating community groups, the case study research findings indicate that the programmes also had a positive effect on the improved and increased delivery of the groups in their specific outcome areas. This is in line with the overall rationale of the Stronger Communities Programme that by building the capacities, communication skills, and know-how in implementation, community groups will be able to deliver better and increased services to their client base. The case study findings indicate the following achievements in this context: increased engagement and re-engagement of local citizens, including young people (Isle of Rum; Y-GRA; Bruce Road Community Flat; Reachout); improved ability to access funding, through which a number of outcomes were achieved, including landscape regeneration (Kyle Lochalsh); increased time for project delivery (through improved effectiveness of the groups); 9

10 community group members gaining confidence and moving on to enrol in college (WISH); and increased project activity, expansion and diversification of service provision (WISH, Building Bridges; Bruce Road Community Flat). Chapter 6 presents the key conclusions and recommendations of the evaluation. The main conclusion is that the Programme was a highly relevant and effective support mechanism in line with the current policy environment. Due to the specific characteristics of the target audience of ACE and OCOF (new and emerging groups) was at times difficult. Notwithstanding, geographically the reach of the three sub-programmes was considered fairly good with a clear commitment to engage with all areas in Scotland. Each of the three sub-programmes had a clear approach to their delivery including a focus on action planning, priority setting, action research and skills development in community engagement. Particularly in the ACE and OCOF programmes, there was a strong emphasis on bespoke support to the needs and stages of the individual groups supported which required more resources than originally anticipated. In terms of positive learning outcomes at participant level, the evaluation has identified good, progressive and well recognised achievement of outcomes throughout the programme implementation period for all three programmes. Considering the difference in approach and volume between Sus It Out Plus and ACE and OCOF programmes, the final evaluation findings indicate that the short and light-touch intervention whilst appreciated - did not necessarily lead to a longer-term change in capacities and skills. Whilst entirely useful and effective as a short term intervention, it might not be the most effective approach to create more long-term changes in capacities. It seems that the more intensive the support of OCOF and ACE has been, the more progress has been made by the groups (although monitoring data is not fully transparent in respect to the intensity levels of support provided to each group). In terms of Value for Money, the three sub-programmes present very different scenarios. The unit costs per group (at Programme level) 5 have been by far the lowest in the case of Sus It Out Plus, reflecting the high volume of groups assisted and the shortness of intervention delivered. The differences between the unit costs of ACE and OCOF are more concerning, as in principle the range of assistance delivered and type of community groups 5 excluding cash awards to community groups. 10

11 supported were similar, yet the unit costs of OCOF are five times higher than those of ACE. In both cases, the satisfaction levels of the community groups were very high and the impact and longevity of capacities built can be considered of similar success. The available monitoring data of the programmes are, however, insufficiently transparent to explore this topic in more detail. The following recommendations are made: 1. In view of the increasing expectations on community groups to deliver a broader spectrum of services in a more efficient, self-sustaining, and sustainable manner, consideration should be given to building on the experience of the Stronger Communities Programme and to continue with a similar support provision. 2. Consideration should be given in future to provide a coherent rationale for a joinedup programme delivery so that it is clear how the various sub-programmes complement each other in their type and range of support offered, and in their target groups to be reached. Clear demarcation and complementarity will allow for maximum learning across the sub-programmes as long as their overall strategic aim and objectives are of a common nature. 3. The combination of offering mentoring support and advice with the availability of a cash resource for participating groups to spend according to need was a substantially successful and effective concept in addressing some of the most urgent and relevant needs of the groups and to help grow their sense of responsibility, maturity, and sustainability. The experience of the Programme shows that these cash resources should be of a manageable size for participating groups according to their needs and capacities. 4. Consideration should be given to recognise the challenge and allocating more time for programmes to reach communities and emerging community groups that are not registered/engaged with the existing intermediary networks of support providers. 5. Consideration should be given to furnish wider capacity building support programmes in future with a cash resource as part of a packaged approach - not only to help build capacities etc., but to help promote the Programme and attract a wide range of community groups. 11

12 6. The highly bespoke yet clearly staged and structured approach to support provision has enabled the three programmes to create real impact in the vast majority of groups and the focus on action planning, and reflective development sessions with participant groups should become an integrative part of any future support programme. 7. Consideration should be taken to emphasise the importance of the exit stage of the support provision, so that each community group will be provided with a road map to guide them through the post-support period. 8. Consideration should be taken in future programmes to prescribe stricter monitoring requirements so that increased transparency can be assured and unit costs can be assessed. 9. A more distinctive Stronger Communities Programme website developed at the outset of the programme could have provided added value not only to capture and present learning achieved, but also to help create a stronger identity amongst the supported groups of being connected via this Programme. 10. The bringing together of Programme participants to facilitate learning, exchange, and networking opportunities at a Programme-wide level is widely regarded as good practice and was fully supported as such by the Programme s Learning Exchanges. However, the smaller these events become in terms of thematic topics, geographic area, or cohort specific characteristics, the more care needs to be taken that the focus of these smaller events is clearly based on the groups specific needs and interests in order to attract buy-in and participation. 12