Performance Accountability System The Santa Rosa County School District has a district strategic plan supported by individual school improvement plans. The strategic plan is a new document and not yet received an annual update regarding progress toward the established goals. Each area identified in the strategic plan contains a five-year goal with associated annual objectives designed to move the district toward accomplishment of the goal. Operational areas primarily function under vendor contracts and do not enjoy a systematic degree of evaluation and accountability, typically focusing on accounting aspects of the service but not on accountability for the service per se. With additional focus on the annual updating of existing plans and extending this planning to the operational areas, programs will become even more effective in assisting the district in recognizing exemplary performance areas and targeting areas for improvemen t. Conclusion The Santa Rosa County School District (the district) has several accountability procedures in place for its various educational programs. The strategic plan forms the basis for planning and accountability within the district. All program areas, especially areas associated with instructional and academic performance, are tied to the goals and objectives of this plan. However, the Santa Rosa County School District s strategic plan was only adopted in August of 2001. Thus, the plan has not been in place long enough to see an annual report of progress toward the goals that were established. While there is intent to execute the plan and to report on progress, the annual review aspects are not yet formalized. Until such time as this aspect of the cycle is accomplished accountability is not complete. In areas associated with the operations of the district such as transportation and food services, each program area has developed varying degrees of accountability systems. Since many of the functions in these areas are provided through contracts with outside vendors, the contracts contain performance criteria that are continuously monitored and reported quarterly to the Santa Rosa School Board. Generally speaking the operational areas do not have specific measurable goals and objectives in place. While the areas are accountable to their specific sections of the district strategic plan, true performance accountability is minimal at best. The individual chapters throughout the report reflect this lack of specific accountability and make specific recommendations for improvement in this area. Associating these yet to be developed goals and objectives with the district s strategic plan would provide a central document containing all aspects of accountability. SchoolMatch Consultants 4-1
Exhibit 4-1 The District Has a Notable Accomplishment in Performance Accountability Systems in the Last Three Years The district has developed a strategic plan outlining goals and objectives in major instructional and operational areas. Source: Santa Rosa County School District. Overview of Chapter Findings The SchoolMatch consultants reviewed the district s performance accountability system using the Best Financial Management Practices adopted by the Commissioner of Education and associated indicators. The consulting team reviewed district planning procedures and program documents, recent audits by outside entities, and contracts, as well as speaking directly with school and central office staff. An overview of chapter findings is presented below. Accountability of Programs 1. The district adopted a five-year strategic improvement plan with measurable long- and short-term goals and objectives. Individual school plans support the district plan. While accountability measures are contained within state and federally required plans for specific educational program areas such as exceptional student education, the district has not developed measurable objectives and performance measures for operational areas (page 4-4). 2. The district incorporates specific short- and long-term objectives in its strategic plan and specific program planning documents. Operational areas are primarily implemented through vendor contracts, which do not contain outcome or cost-effectiveness measures (page 4-8). 3. The district does not have a systematic process to review the cost /benefit of the various instructional programs and vendor contracts (page 4-9). 4. The district does not have formal structures to evaluate the performance and costs of major educational program areas or operational programs to improve program performance and costefficiency (page 4-12). 5. The district has several mechanisms to report educational performance, using school improvement plans and school advisory councils to communicate academic achievement to the public. The district s operational areas have not established methods to provide information to the public. The district provides the public with various venues to seek information, lodge complaints and makes its wishes known (page 4-14). Fiscal Impact of Recommendations There are no recommendations with a fiscal impact. 4-2 SchoolMatch Consultants
Background The Santa Rosa County School District board adopted a comprehensive strategic plan in August 2001. 1 The plan contains measurable long- and short-term goals for its major program areas. Each section of the strategic plan is organized around five-year goals with annual measurable objectives designed to lead to accomplishment of the goal in five years. The plan was developed with considerable community and staff input. Each program area manages day-to-day operations to support accomplishing the goals and objectives set forth in the plan. Academic achievement and provision of services to students are the foundation of the strategic plan. The strategic plan is managed through the office of the assistant superintendent for curriculum and instruction with the various other assistant superintendents contributing to the management function. Planning and accountability functions have been in place in the Santa Rosa County School District for many areas through program specific planning and accountability documents. For example, the technology area has been operating with a technology plan for many years. The degree to which these documents contain measurable objectives varies. Vocational education plans have been very specific, outlining various goals and objectives; special education and ESOL plans have specific goals related to program implementation and service to students. These plans are developed with community and stakeholder input and presented annually to the district s school board. While these plans have specific measurable objectives, they are often process oriented rather than outcomes oriented. Service areas such as food service, transportation and maintenance have been governed by contracts with outside vendors. These contracts provide a degree of management and accountability found in the private sector and afford the district considerable control with the vendor. However, there is presently no cost/benefit analysis nor peer comparison used related to contract development. As noted in the recommendations in the specific chapters related to the operational areas, there is a clear need for the district to implement a planned program of accountability related to the service areas provided through outside contracts. The district maintains the required federal and state planning documents in the areas of vocational education, instructional and administrative technology, programs for students with special needs and programs for English language learners. These documents contain goals and specific objectives for these program areas. Federal requirements in these areas related to planning are specific and require the district to file plans with the state in order to ensure the flow of federal dollars to the district. 2 1 Santa Rosa County School Board approved minutes, August 28, 2001. 2 District Technology Plan, District Vocational Education Plan, District Special Education Plan, District LEP Plan. SchoolMatch Consultants 4-3
Accountability of Programs 1 The district adopted a five year strategic improvement plan with measurable long- and short-term goals and objectives. Individual school improvement plans support the district plan. While accountability measures are contained within state and federally required plans for specific educational program areas such as exceptional student education, the district has not developed measurable objectives and performance measures for operational areas The Santa Rosa County School District adopted a formal strategic plan in August of 2001. 3, 4. The plan contains specific, measurable goals and objectives for the major program areas involved in providing educational services to the students of the district. Each school has a school improvement plan developed through a community process and delineating specific measurable goals and objectives that support the district s overall strategic plan. The strategic plan is new for Santa Rosa and had yet to be in place a year, thus no reporting of status on any aspect of the plan has been made. The 2001-2002 Santa Rosa School District strategic plan encompasses three school board initiatives: I. Improving Student Achievement: Santa Rosa students are expected to achieve at the highest levels of performance in reading, writing, mathematics, and science. The board is dedicated to supporting the development and implementation of challenging curricula at all grade levels and to providing effective intervention to ensure that all students read with fluency and comprehension, write effectively, solve mathematical problems, and apply critical thinking skills to real world tasks. II. Closing the Gap: Because most Santa Rosa students score higher than their counterparts across the state, the board will focus on increasing the achievement levels of those students in the middle and low achievement ranges in order to ensure equity in instruction for all. III. Enhancing the Learning Environment: The board will ensure that all students have a safe, nurturing, and technology-rich environment in order to enhance student achievement and promote literacy. The board supports these initiatives through ten goals that are in turn, supported by specific, measurable objectives found in the strategic plan. Those ten goals are: 1. Readiness to Start School: Communities and schools collaborate to ensure the success of all children in schools. 2. Graduation Rate and Readiness for Post-Secondary Education and Employment: Students who graduate are prepared to enter the workforce and/or post-secondary institutions of learning. 3. Student Performance: All stakeholders collaborate to ensure that student achievement is at the center of our focus and all students are provided with opportunities to perform at their highest potential. 4. Learning Environment: Schools and communities establish a nurturing and challenging environment for learning. 5. School Safety and Environment: Schools and communities provide an environment that promotes health, safety, and responsibility. 3 Santa Rosa County School District Strategic Improvement Plan 2001-2006. 4 Santa Rosa County School District approved board minutes, 8/28/01. 4-4 SchoolMatch Consultants
6. Teachers and Staff: Schools and communities promote professionalism and high expectations among teachers and staff with opportunities for professional growth. 7. Adult Literacy: Schools and communities promote literacy and good citizenship among adults and provide knowledge and skills necessary for a global economy. 8. Family and Community Involvement: Schools and communities are partners in achieving a quality educational system for all students. 9. Fiscal and Resource Responsibility: School district promotes efficient and effective use of resources to enhance educational opportunity. 10. Comprehensive Operational Planning and Program Accountability: School district implements systemic planning and accountability processes to enhance educational opportunity. The Santa Rosa strategic plan takes each of the ten goals adopted by the school board and delineates objectives and strategies designed to accomplish the goal. Each objective has a five-year plan stated in measurable terms and an annual objective related to it. For example, under Goal 1: Readiness to Start School, the plan contains the following: Objective 1.1 Kindergarten Readiness Five-year Plan: The percentage of students who meet the criteria for readiness to start kindergarten will increase 10%. Annual Objective: Establish baseline data by utilizing the required Kindergarten Readiness Assessment. Various strategies are listed to support the annual objectives. Goal 3: Student performance is a key goal in the plan and supports the first initiative of the board of Improving Student Performance. Objectives under this goal include: Objective 3.1 School Accountability Five-Year Plan: All district schools will maintain a grade of C or above, based upon Florida s A+ Plan Annual Objective: Increase the number of district schools achieving grades of A and B. Strategies: Analyze data on student performance to increase achievement. Provide professional development opportunities in reading, writing, math, and science. Implement curriculum-mapping project to ensure mastery of all standards. Objective 3.2 Student Achievement Five-Year Plan: The percentage of students achieving at a Level 3 or above on the FCAT (Florida Comprehensive Achievement Test) in math, reading, and writing will increase by 15%. Annual Objective: Decrease the number of students scoring at Level 1 and Level 2 on the FCAT math, reading, and writing by 3%. Strategies: Provide intervention and intensive instruction for students scoring at Level 1 and Level 2 on FCAT reading, writing, and math. Provide opportunities for professional development in best practices for instruction in reading, writing, and math. SchoolMatch Consultants 4-5
Implement Reading in the Content Area focus in secondary schools and Best Practices in Literacy Instruction in elementary schools. Provide support and materials to enhance literacy instruction at all grade levels. Objective 3.3 Curricular Alignment Five-Year Plan: Curriculum maps aligned with Sunshine State Standards and articulated within feeder patterns will have been published and implemented at all district schools. Annual Objective: Schools will constantly revise and refine their curriculum maps as they reach consensus across grade levels and feeder patterns. Strategies: Provide annual training of school-based leadership trams as additional schools joins the curriculum-mapping project. Support and facilitate revision and refinement of curriculum maps at individual schools Facilitate articulation of vertical alignment among feeder-pattern schools. Develop and provide software to enhance electronic versions of curriculum maps. The district operates with approved vocational education and special education plans. These plans meet local, state and federal requirements as evidenced by the various levels of approval and the flow to the district of funds that are contingent upon the plans approval. The plans contain measurable objectives and program plans and are reported to the school board and public on an annual basis. 5 As cited in Chapter 5- Educational Service Delivery, the Exceptional Student Education (ESE) program is evaluated annually by the district ESE staff and school-based administrators. Data used in this evaluation include graduation rates, inclusion rates, FCAT participation rates and performance, and a customer satisfaction survey. Two key indicators have been targeted for 2001-02: the dropout rate among students with disabilities and the disporportionality of student membership identified as gifted. As noted in the Educational Service Delivery Chapter 5 of the full report, the district s LEP (Language Enriched Pupils) Plan, Annual ESOL (English for Students of Other Languages) Report and Self Study, and Florida Department of Education ESOL waiver attest to adequate planning in this area. 6 These plans contain specific goals and objectives for each of the areas and are reported annually to the district board as well as the state. The district regularly assesses the status of this program despite the small number of students enrolled in it. This assessment has led to program improvements in the amount of time students are processed through the program and the implementation of on-line training to increase efficiency and convenience. The vocational education program has in place the Vocational Curriculum Frameworks and Students Performance Standards as updated by the Department of Education. These frameworks address the parity expected between vocational students and non-vocational students with respect to academic performance. Using the academic performance measures in place within the district, primarily those associated with FCAT, the academic performance of vocational students is compared to that of non-vocational students with the specific expectation that the performance of vocational students will be on par with that of their peers. Strategies are developed toward this end. The operational areas are primarily areas where services are provided on a contractual basis. This is the case for food service, transportation, various fiscal areas, and maintenance. These areas, and others, are specifically addressed in the district s Strategic Improvement Plan 2001-2006. 7 Their inclusion in the district s strategic improvement plan underscores their role in supporting the overall educational program. 5 District vocational education plan and district special education plan. 6 University of West Florida Evaluation of the ESE Dynamo/Liaison Pilot Program in Santa Rosa School District: Final Report, The Educational Research and Development Center, College of Professional Studies, The University of West Florida, Pensacola, Florida. 7 Strategic Improvement Plan 2001-2006, Goals 1, 2, 6, and 10. 4-6 SchoolMatch Consultants
Strategic plan goals and objectives for these areas reference the more comprehensive plans in place for each area. For example, in the area of building maintenance, the district has a comprehensive building maintenance plan containing initiatives slated for implementation. The strategic plan references the completion of these scheduled initiatives under Objective 10.2 and requires an evaluation of the degree to which these initiatives have been completed with a report of this to the local board. While these areas operate under contracts that are competitively bid and monitored by central office staff, best practice reports in the specific areas note several areas where improvements can be made. The various contracts contain program responsibilities and associated costs. However, there is not a coordinated, district-wide effort to establish goals and objectives for these areas. For example, Chapter 12, Transportation, indicates that there are measurable objectives for this area in the director s office but questions their dissemination to various stakeholders and the public. This report also indicates that such information should be in the provider s contract but is not currently include. Chapter 12, Transportation and Chapter 13, Food Service, both reflect the lack of accountability and cost effectiveness measures in these two areas. The contracts do not address cost effectiveness nor do they show evidence of benchmarking. Ancillary areas not operating with contracts, such as risk management and the various finance and infrastructure areas do not operate with specific, measurable objectives. Recommendations We recommend that the district establish a process to review the existing contracts and other support areas and establish specific goals and objectives for these various areas (see Action Plan 4-1). Action Plan 4-1 Increase Level of Accountability in the Performance Accountability System Strategy Establish a team to study existing contracts and establish district specific goals and objectives for the various operational areas; develop planning documents in areas not under contract Action Needed Step 1: Establish an office of primary responsibility (OPR) for contract review and subsequent development of recommendations Step 2: Appoint a team to review contracts and formulate recommendations as required. Step 3: Present recommendations to the superintendent and school board for approval Step 4: Amend contracts as necessary; develop planning documents where necessary Step 5: Periodically report findings and suggestions for improvement. Who Is Responsible Superintendent, school board Time Frame Establish OPR and team by December 2002 Report to superintendent by July1, 2003 Fiscal Impact This recommendation can be completed with existing resources and could ultimately result in cost savings. Source: SchooMatch Consultant. SchoolMatch Consultants 4-7
2 The district incorporates specific short- and long-term objectives in its strategic plan and specific program planning documents. Operational areas are primarily implemented through vendor contracts, which do not contain outcome or cost-effectiveness measures The strategic plan for the Santa Rosa County School District incorporates measurable objectives for both the short and long term. These objectives are designed to move the district toward the accomplishment of the goals established within the plan. Individual departments within the district have more detailed plans that support the more global goals and objectives in the strategic plan. For example, Objective 2.2 Workforce Development under Goal 2, Graduation Rate and Readiness for Post-Secondary Education and Employment, has a five-year goal of increasing the participation in career and technical education activities by 10%. The annual goal is a 2% increase, which, if met, will lead to success on the five-year goal. The vocational education plan for this district, which outlines more specific objectives and strategies, looks at the degree to which students are placed in these programs and their level of success in them. Educational areas such as vocational education, programs for exceptional students and English language learners are guided by program specific documents outlining goals and objectives. These plans are tied to specific areas of the strategic plan forming a vertical planning system in the district. There are significant state and federal planning requirements in all of these areas. Approvals of these plans from all levels attest to the adequacy of them in meeting these requirements. There is a clear emphasis upon not only providing services to students with varying and specific needs, but a focus upon their success in their programs. For example, both the plan for students with disabilities and for English language learners contains specific objectives designed to address the success of these students on the FCAT. While the district strategic plan focuses on the overall success of students on FCAT assessments, the specific program plans call for monitoring of their respective subpopulations on FCAT performance and delineate strategies for promoting a higher level of success. While the aforementioned documents contain long and short-term goals and objectives, there is no evidence that cost effectiveness is considered in these areas or specific benchmarking is present. The strategic plan and the various program plans contain budget information but do not reflect peer comparisons. While the goals and objectives are specific and measurable, the lack of inclusion of cost comparisons and benchmarking is noticeable. Vendors under contract provide the district s food, transportation and maintenance services. The chapters addressing the best practices for each of these operational areas indicate that the district needs to develop a more formal structure for assessing and comparing their services to those provided in peer school districts. The chapters addressing these operational areas discuss the district s failure to review the cost effectiveness of these services when compared to appropriate benchmarks. While the contracts in place reflect specific performance in terms of ridership and safety statistics in transportation and participation and food safety requirements in school food, they do no reflect a global attention to accountability. The specific criteria of linking program performance to program cost are not present. 4-8 SchoolMatch Consultants
Recommendations We recommend that the district develop a formal mechanism for comparing the levels of service provided in the various operational contracts with that of peer districts or appropriate private industry, state or federal standards. This comparison should include a measure of cost effectiveness (see Action Plan 4-2). Action Plan 4-2 Develop a Formal Mechanism for Comparing with Peer Districts the Levels of Service Provided and the Cost Effectiveness of Various Operational Contracts Strategy Appoint a multidisciplinary team to develop a plan to compare the levels of service in the various operational areas with those enjoyed in peer districts, private industry, state or federal agencies, as applicable. Action Needed Step 1: Identify an office of primary responsibility (OPR) for the project and appoint a team Step 2: Step 3: Step 4: Step 5: Step 6: Survey peer districts private industry, state or federal agencies, as applicable to gather information regarding service levels and costs Review the information from the peer districts private industry, state or federal agencies, as applicable compared to the current levels of service and costs in Santa Rosa. Develop recommendation as dictated by the data Establish action steps to accomplish recommendations, including a review cycle to assess the degree to which recommendations have been accomplished Present findings and plan to the superintendent and School Board for approval Who is Responsible Superintendent and/or designee Time Frame Plan presented to School Board by March 2003 Fiscal Impact None for planning aspect; potential cost savings resulting from studies. Source: SchooMatch Consultant. 3 The district does not have a systematic process to review the cost/benefit of the various instructional programs and vendor contracts. The district has a strategic plan with measurable objectives. However, the plan has not been in place for a year. While board minutes indicate intent to review the plan annually this has not yet occurred. Discussions with staff members verify the intent to report to the board regarding the status of the plan on an annual basis; however, there is no evidence of a specific plan to do so. School improvement plans are annually presented to the school board and reflect the progress of the schools toward meeting the goals and objectives in their plans. 8 While the district has heretofore had goals and objectives, this most recent plan represents its first comprehensive effort in this area. Individual school improvement plans have been in place for many years as required by the state. Schools within the district are required to develop school improvement plans that incorporate information from the various achievement tests and other information germane to the school s operation. These plans are developed in conjunction with school advisory committees and reflect a tight relationship to the 8 SRC approved school board minutes 6/5/01, 6/28/01, and 5/12/01. SchoolMatch Consultants 4-9
district s strategic plan. School improvement plans contain measurable goals and objectives and are disseminated widely. A review by the SchoolMatch consultant of a sample of the plans on the various school web sites, revealed a focus in academic achievement and support for staff development. 9 School improvement plans must be presented to the school board for approval on an annual basis. At that time the status of the school toward meeting the previous year s goals is also presented. 10 School improvement plans typically address the status of the school on student performance on the FCAT and other measures of student achievement. If specific areas presented problems on the FCAT, strategies are incorporated into the plan to provide students with extra support in meeting the state-prescribed performance goals. Schools within the district incorporate various strategies designed to move students toward improved academic performance and increase levels of service within the building. Schools develop these strategies as they are developing their plans. The district does not have in place a systematic process for evaluating various instructional programs being implemented within the district. Therefore, inclusion of specific strategies is often based on recommendations from teachers, principals and other staff. It should be noted that there are schools where programs are being more closely monitored but this is the exception rather than the rule. Schools also use programs that receive district-wide support for implementation. An example of this is the curriculum-mapping project being implemented within the district. The curriculum-mapping project incorporates the Sunshine State Standards into the daily plans of the school in a systematic and orderly fashion. The entire district is implementing the program. This implementation is found as an objective in the district strategic plan and in each school improvement plan. In visits to schools and in reviewing school web sites, the focus on the standards and the ready availability of the mapping information provides teachers, students, and parents access to instructional plans for Santa Rosa students. The district adopted curriculum mapping after reviewing its implementation in other localities. The emphasis placed on monitoring instruction on the state standards, inherent in curriculum mapping, is in concert with the student achievement goals of the Santa Rosa County School District. Operational services in the Santa Rosa County School District, such as food service, transportation and maintenance are delivered through vendor contracts. To varying degrees performance information was used to determine the need to move from district-provided services to contractually provided services. In the area of food service, the move to a vendor contract was made after several years of data indicating a need for a change in food service performance. As noted in the Food Service Chapter: the loss for food service kept spiraling due to maintenance of old equipment, need to increase participation, state audits by the Auditor General s Office and the Food and Nutrition Management Office revealed that accountability deficiencies required a need to enhance efficiency. Similarly, in the area of transportation, Santa Rosa moved to a vendor contract when data indicated the need to address pressing transportation issues. As noted in the Transportation chapter: The Santa Rosa County School District operated its own transportation system until 1998. The district had not purchased new buses on a regular schedule and was confronted with an aging bus fleet, prone to breakdowns and expensive repairs. In addition, the district was experiencing a high rate of worker s compensation claims from employees in the transportation department. As a result, the district decided to pursue a contract with a private vendor to manage the student 9 www.santarosa.k12.fl.us. 10 Santa Rosa County School Board approved minutes 6/5/01, 6/28/01, and 5/12/01. 4-10 SchoolMatch Consultants
transportation department. The district pursued a fixed price contract arrangement that assured costs would not exceed the department s current expenditures. The districts occupational therapist services are provided under contracts with local providers. Administrative computing services are provided in conjunction with the Gateway Consortium. Agreements with local educational institutions provide opportunities for students to engage in dual enrollment with Pensacola Junior College and the University of West Florida. Similar agreements provide teachers with opportunities for staff development. Technology services and maintenance is often achieved through contracts such as that for school wiring and for sharing of resource teachers. All of these contracts were entered into because evidence indicated that providing the service in this manner would be cost effective and increase the level of service within the district. As noted in the best practices section on technology, the district does not have a process to evaluate the cost effectiveness of various technology-related instructional programs. The same is true of instructional programs that are not technology based. While interviews with curriculum staff indicate the review of the programs as to their relationship to the instructional program and to the Sunshine State Standards, there is no district-wide process in place to evaluate the bang for the buck related to program adoption. As noted in the technology section, individual school and program initiatives have resulted in data gathering related to some program areas, these are not part of a district-wide process. The Santa Rosa School District has made extensive use of outside contracting as an option to providing services to students via directly employed staff. This has enabled the district to keep its staffing pattern low while still offering a full range of services. What is lacking in this scenario, whether it is in food service, transportation or occupational therapy is a consistent procedure to examine the cost effectiveness of the contractual arrangement and the use of benchmarking within that examination. As noted in the Management Structures, Exhibit 3-1, the district is recognized for frugal operations. This could be further substantiated with the incorporation of cost effectiveness and benchmarking examinations of all contracted services. Recommendations We recommend that the district review existing contracts for services and incorporate appropriate cost effectives and benchmarking measures into each one. Further, the strategic plan should be revised to incorporate cost effectiveness and benchmarking objectives in each service area. Action Plan 4-3 Develop a Protocol to Evaluate the Impact of Non-Technology Instructional Resources on Student Achievement Strategy Review each existing contract and develop cost effectiveness and benchmarking measures for each. Include the same in the district strategic plan. Action Needed Step 1: Determine the areas where contracting is employed and designate a lead person from the district and from the vendor to each of the areas. Step 2: Step 3: Step 4: Augment this committee as needed with additional school district personnel and/or community members Gather information regarding peer district, state federal or industry standards and benchmarks in the various areas. Develop objectives and associated strategies for incorporating cost effectives measures and benchmarking into each contractual area. SchoolMatch Consultants 4-11
Who is Responsible Time Frame Fiscal Impact Source: SchoolMatch Consultants. Step 5: Step 6: Step 7: Revise the district strategic plan to incorporate these additional objectives. Gather data required to establish the cost effectiveness of each area. Report the cost effectiveness data as part of the annual report on the strategic plan. Make associated program/contract adjustments as dictated by the data. Superintendent and/or designee Planning in the 2001-02 school year Implementation in the 2002-03 school year. Expected to be able to complete with existing resources 4 The district does not have formal structures to evaluate the performance and costs of major educational program areas or operational programs to improve program performance and costefficiency. The Santa Rosa County School District strategic plan represents the foundation upon which accountability rests in the district. The strategic plan is a commendable document presenting measurable goals and objectives in the major educational areas. As the strategic plan is implemented it is the intent for the board to receive an annual update on the accomplishment of the plan s goals and objectives. As the plan was only adopted in August of 2001, the annual cycle of review has not yet happened. Until such time as an annual review can take place there is not way to document the plan s evaluation and subsequent use of update information. Interviews with staff indicate intent to do so. A more formal level of accountability exists at the school level through the school improvement plan process. For many years the schools have been required to develop school improvement plans and submit them to the School Board for annual approval. Discussions with building staff, review of school web sites and board minutes indicate that this process is being followed. School improvement plans focus on academic areas and tie directly to the state requirements under the Sun shine State Standards as well as tie directly to the district s strategic plan. As noted above in several sections, the district has in place planning documents for instructional areas, but needs to establish more formal evaluation mechanisms. 11 There is no mechanism in place to identify programs for evaluative study or any district-wide protocol for executing such a study. This is true for both instructional programs and for the operational service areas. The district does not have in place formal procedures for assessing the cost-effectiveness of its operational programs outside those mechanisms contained in vendor contracts. Implementing such a program of review would provide the district with information upon which to make decisions regarding continuation of program, elimination of those not proving cost-effective and become a source of input for strategic and specific program plan revisions. 11 SRC BFMP Review, Chapter 4, Performance Accountability, Recommendations, Sections 1-3. 4-12 SchoolMatch Consultants
Recommendations We recommend that the district establish a systematic plan for program evaluation that uses performance data and risk assessments to target those areas most in need of formal evaluation. The district should then conduct or contract for evaluation of the targeted areas and use the results to improve performance or reduce costs. (see Action Plan 4-4). Action Plan 4-4 Develop a Plan to Disseminate Accountability Information Strategy Develop performance data and risk assessment techniques to target areas in need of evaluation and conduct formal evaluations of the targeted areas. Action Needed Step 1: Develop performance measures and risk assessment techniques to identify programs and strategies most in need of formal evaluation. Step 2: Establish a schedule of the evaluations to be conducted over the next fiscal year. Step 3: Contact schools of higher education for assistance in developing protocols for programs evaluation and explore the potential use of graduate student and/or faculty in the implementation of the evaluations. Step 4: Incorporate the results of existing evaluation studies into current evaluations. Step 5: Incorporate the evaluation process, including reporting results to the public, into the district strategic plan. Step 6: Present evaluation results to the Board and Superintendent for use in improving performance. Who is Responsible Time Frame Fiscal Impact Source: SchoolMatch consultants. Superintendent and/or designee Planning in the 2001-2002 school year Implementation in the 2002-03 school year. Assuming that schools of higher education provide in-kind services, an be completed with existing resources. SchoolMatch Consultants 4-13
Effective and Responsible Operation 5 The district and state have several mechanisms to report educational performance using school improvement plans and school advisory councils to communicate academic achievement to the public. The district s operational areas have not established methods to provide information to the public. The district provides the public with various venues to seek information, lodge complaints and makes its wishes known. The major assessment of academic achievement within the state of Florida is results on the FCAT tests and school grades. As evidenced in the Educational Services Chapter, Santa Rosa students and schools enjoy considerable success on these measures. Information regarding district and school scores and school grades issued by the Florida Department of Education is widely disseminated by the department itself. A major purpose of the entire program in Florida is to provide the public with a measure of accountability for public education that is consistent across the state and widely disseminated. In addition to the dissemination efforts of the Florida Department of Education, the Santa Rosa County School District makes wide use of its web site and those of the individual schools to further disseminate this information. 12 School newsletters also announce school grades issued by the state and further provide academic status information. 13 School Public Accountability Reports (SPAR) are mailed to every parent in this district. SPAR repots the student s academic performance on Florida state tests as well as school grades issued by the Department of Education. These reports are designed to provide information on student academic performance; they do not in any way reflect cost efficiency. In addition they do not disaggregate student performance by special populations, such as exceptional students, students who are not proficient in English, or disadvantaged students. Information contained in the various operational best practice sections does not reveal any consistent mechanisms for the provision of accountability information in these areas. While school lunches are advertised and bus routes known, there are no formal structures in place for public accountability in these and other contracts. The district conducts the business of educating its students in an open manner. In addition to the methods of communication cited previously, the board provides open sessions during each meeting whereby the public many address concerns to the board in any area. This open forum allows the public to address both academic and operational areas. There is also a formal process for registering complaints in the Santa Rosa Administrative Policies, Rules and Regulations, Chapter 2, Section 2.09. 12 www.santarosa.k12.fl.us 13 Samples of school/principal newsletters 2000-01, 2001-02. 4-14 SchoolMatch Consultants
Recommendations We recommend that the district develop a plan to disseminate to the public disaggregated student academic performance information and operational area performance information related to contract performance, service levels and cost effectiveness (see Action Plan 4-5). Action Plan 4-5 Develop a Plan to Disseminate Accountability Information Strategy Develop a dissemination plan for conveying disaggregated student academic performance data and data concerning the performance of operational programs to the public. Action Needed Step 1: Determine cost-effective, meaningful avenues for dissemination; explore the use of existing mechanisms to reduce any associated costs. Step 2: Incorporate the dissemination plans into the package presented to the Superintendent and Board. Step 3: Adopt the disseminations that are most cost effective. Who is Responsible Time Frame Fiscal Impact Source: SchoolMatch Consultants. Superintendent and/or designee Planning in the 2001-02 school year Implementation in the 2002-03 school year. Can be completed with existing resources SchoolMatch Consultants 4-15