Simplified Metrics
The Simplified Metrics The Simplified Metrics measure students progression along their educational journey from recruitment to completion, transfer, and the workforce. The metrics provide a holistic approach to our work on student success, across funding streams and initiatives. This focus will benefit college performance on the Funding Formula Metrics and system performance on Vision for Success goals.
Principles (1 of 7) 1. The focus will be strictly on students and their educational journeys from recruitment to completion. This approach will provide visibility on system-wide progress toward achieving the goals articulated in various initiatives including Guided Pathways, Doing What Matters for Jobs and the Economy, the Student Centered Funding Formula, Institutional Effectiveness, and the Vision for Success.
Principles (2-4) 2. The outcome will be ONE set of metrics drillable for details. This set of metrics will displace all other student-specific metrics including the Scorecard and the IEPI Student Outcomes, except for Accreditation metrics and Federal metrics such as Perkins. 3. Equity will not be viewed as a separate activity. Instead, it will be observed across all metrics. Therefore, each metric will be drillable by gender, age, ethnicity, first gen, SES, etc. 4. Metrics will be designed to align the progression of students along various initiatives, and the number of metrics will be limited to ensure that we focus only on critical measures.
Principles (5-7) 5. Each metric will be closely reviewed to ensure it is valid, drillable, measurable, and critical. Additionally, the definitions will be shared with colleges to ensure that the metrics can be replicated at the college, district, and regional level. 6. An implementation plan will be created for when each metric can be put into production and which ones need legislative work. Extensive analysis and modeling of data will be performed before the presentation of the implementation plan. 7. The process of simplification does not destroy current data sources or MIS processes which provide additional data for continuous quality improvement, research, and evaluation.
Steps Taken February 2, 2018 The first whitepaper was shared with the statewide working group February 12, 2018 first meeting with the statewide working group February 26, 2018 first statewide webinar March 5, 2018 second whitepaper was shared with the working group March 12, 2018 second meeting with the statewide working group March 26, 2018 second statewide webinar April 23, 2018 third meeting with the statewide working group April 30, 2018 third statewide webinar May 1, 2018 recommendations approved by the Chancellor s Cabinet May 16, 2018 third whitepaper was shared statewide Ma y July alignment with VCs on details of SM with various funding metrics August 6, 2018 data element dictionary (DED) completed, including alignment with SCFF
What is a data element dictionary (DED)? Describes exactly how each metric is constructed o Description: Description of the metric o Student Type: Description of which student type--adult education/esl, shortterm career education, degree/transfer--the metric applies to o Display: Whether the metric applies to the snapshot or cohort view o Data Source: The source or sources of information o Timeframe: The specific timeframe used to determine the data for the metric o Calculation: The specific calculations used to derive the data value o Value Type: The type of data value populated for the metric o Notes: General information or caveats about the data for the metric
Timeframe
Data Roadmap
CCCCO Data Roadmap 6/21/2018 Jun-18 Jul-18 Aug-18 Sep-18 Oct-18 Nov-18 Dec-18 Jan-19 Feb-19 Mar-19 Apr-19 May-19 Jun-19 Data Portal Survey of existing data dashboards, reports, extracts and portals - Phase 1 Dashboard /Report Survey - Phase 2 Combine/Eliminate/Refine Data Reports, extracts, Dashboards into CCC Data Portal PD, Roadshow, Socialization of new CCC Data Portal Legislative Work Legislative work to align funding streams and CCCApply with Simplified Metrics Data Definitions Build, clarify, combine Data Element Dictionary (DED) Simplified Metrics Simplified Metrics - V1 Integrates SM with Funding Formula and Vision Metrics Simplified Metrics - Production Development - V2 Simplified Metrics - Production Development - V3 Simplified Metrics - V4 (Full integration of Funding Formula and Vision metrics) CCC Data Lake Discuss, Evaluate, Design Data Lake - Build plan for build and migration of data Build and Populate CCC Data Lake Connect Data Portal Content to CCC Data Lake, Build cubes MOUs Build MOU Library, Clean up language, recreate as necessary, to move all MOUs accessible to CCCCO NOVA Build Remaining Funding Streams in NOVA and move storage to CCC Data Lake Build NOVA Dashboard in the new Data Portal Ed Planning Prepare CA Standards for Storage and Access for Program Plans and Student Ed Plans Prepare Data storage and establish Web Services/APIs Work with Ed Planning Partners to storage all program plans and ed plans in central storage in CCC Data Lake
Data Portal Currently being conceptualized Based on five tenants: 1. Grounded in users needs 2. Designed with the different user types in mind 3. Easy to navigate 4. Easy to understand 5. Enables the CCCCO, districts, and colleges to fulfill statutory data reporting requirements
Data Dashboard Will be renamed your input is needed! Will be implemented in phases to incorporate field input and conduct thorough testing
Alignment with Other Met rics
Metric Alignment The research team compared the Simplified Metrics to metrics associated with all major Chancellor s Office initiatives. In the tables that follow, items in italics address a similar measure found in both prior initiatives and the Simplified Metrics, but may be measured in a different fashion (such as looking at quarterly earnings versus annual earnings) Minor adjustments have been made where appropriate to align Simplified Metrics with the funding formula and the Vision Goals
Student Success Scorecard Scorecard Metrics Demographics Median Class Size Number of Credit/Noncredit Sections Full-Time Faculty/Student to Counselor Ratio Complete Remedial/Gateway Math & English Persistence Attain 30 Units Completion Rate/CTE Completion/CDCP Skills-Builder Earnings Gains Simplified Metrics Disaggregated Data Transfer Math & English Fall to Spring Retention Successful Unit Attainment in First Year Completion/Transfer Change in Earnings
IEPI IEPI Indicators Enrolled Adults 18-24 Median Time to Degree Annual FTES Served Accreditation Status/Compliance Salary/Fund/Cash/Financial Complete Remedial/Gateway Math & English Completion Rate/CTE Completion/CDCP Transfer Skills-Builder Earnings Gains Simplified Metrics Headcount Transfer English & Math Completion/Transfer Completion/Transfer Change in Earnings
SSSP SSSP Metrics Unduplicated Headcount Students Completing Orientation Students Completing Assessment Students Completing Abbreviated/ Comprehensive/Noncredit Ed Plans Students Receiving Advising/Services Students on Probation Simplified Metrics Headcount Enrolled After Application Transfer Math & English Student Type Skills Gains, 9+ CTE Units/Noncredit Workforce Milestones; Successful Unit Attainment in First Term; Successful Unit Attainment in First Year; Retention; Completion/Transfer Skills Gains, Course Success, Completion/Transfer
Student Equity Equity Metrics Population Compared to Community Successful Course Completion Complete Remedial/Gateway Math & English Completion Transfer Simplified Metrics Course Success Transfer Math & English Completion/Transfer Completion/Transfer
Guided Pathways Guided Pathways KPI College Course Success Rate Persisted from Term 1 to Term 2 Completed Transfer-Level Math/English/Both in Year 1 Earned 6+/12+/15+/Attempted 15+ Units in First Term Average Units/Average Degree Applicable Units/15+/24+/30+/Attempted 30+ Units in Year 1 Simplified Metrics Course Success Retention Transfer Math & English Successful Unit Attainment in First Term Successful Unit Attainment in First Year
Strong Workforce Program SWP Metrics CTE Course Enrollments Earned 12+ CTE Units/48+ CTE Contact Hours Completion Transfer Employed in Second/Fourth Fiscal Quarter Earnings in Second Fiscal Quarter Employment in Field of Study Change in Earnings Living Wage Attainment Simplified Metrics 9+ CTE Units, Noncredit Workforce Milestones Completion/Transfer Completion/Transfer Median Earnings Employment in Field of Study Change in Earnings Living Wage Attainment
AEBG AEBG Metrics Number of Adults Served Improved Literacy & Basic Skills Completion of High School Diploma/Equivalency Transition from ABE/ASE/ESL to Higher Level Programs Transition to Post-Secondary Completion of Post-Secondary Awards Employed in Second/Fourth Fiscal Quarter Earnings in Second Fiscal Quarter Employment in Field of Study Change in Earnings Living Wage Attainment Simplified Metrics Headcount Skills Gains Completion/Transfer Completed a Level Completion/Transfer Unemployed Students Who Became Employed Median Earnings Employment in Field of Study Change in Earnings Living Wage Attainment
CCCCO System Goals Vision Goals Completion & Skills Gains Transfers to UC & CSU Average Number of Units for Degree Earners Employment in Field of Study Equity Gaps Regional Achievement Gaps Simplified Metrics Completion/Transfer Completion/Transfer Average Number of Units for Degree Earners Employment in Field of Study Disaggregated Data Disaggregated Data
Alignment with Student Centered Funding Formula (SCFF) The Simplified Metrics are aligned with all the momentum points and outcomes of the Funding Formula. The data definitions are also the same for both. The variations between the sets are in terms of rates versus counts. For example, Simplified Metrics looks at students journeys along their educational pathways from recruitment to completion, therefore most metrics are based on rate. While SCFF metrics focus mostly on point-in-time data to generate counts for the purposes of funding allocations.
Alignment with Vision for Success The Simplified Metrics are aligned with the statewide metrics included in the Vision for Success goals. The data definitions are also aligned so that colleges and districts can: Identify where they can improve the student journeys in order to help the state achieve the Vision goals Set improvement goals aligned to the Vision as required by Budget Trailer Bill language
Alignment with IEPI Goals The Simplified Metrics are focused on students only. The Simplified Metrics displace all student outcomesrelated IEPI metrics.
Alignment with Strong Workforce Program Metrics The Simplified Metrics align with the concepts in the SWP metrics, but include slightly different definitions (such as examining annual earnings rather than only those in the second fiscal quarter after exit). Updated information will be released shortly about how the Strong Workforce Program metrics and incentive funding will be aligned with the Simplified Metrics.
Implementation Timeline
Next Steps Continue to meet with the Simplified Metrics Workgroup to collect detailed input from practitioners. Continue to hold webinars as another means for collecting input. Continue to review feedback on this work submitted by the field to research@cccco.edu Continue to explore potential for policy changes and reporting requirements with Scorecard and IEPI metrics.
Phase 1: October 15, 2018 Four student types adult ed/esl, short term career education, degree/transfer, undecided/other College level Academic years 2014-15, 2015-16, 2016-17 Snapshot of annual data Top -level metrics only, no disaggregations All 19 top-level metrics for learning progress, momentum, success, employment, earnings
Phase 1: October 15, 2018 Testing for this build includes: o August 27: initial build of first release metrics complete o First two weeks of September: validation testing by pilot colleges and initial interface development o Second two weeks of September: integrate any necessary changes to data and interface o First two weeks of October: Final testing o October 15 release
Phase 2: December 3, 2018 District level added Successful enrollment metric added Four types of drill down added: o Gender o Face/ethnicity o Age group o Financial aid status (College Promise & Pell grants for degree/transfer students, Perkins definition for all other student types)
Phase 3: February 4, 2019 All student types Macro-regions and statewide levels added Six types of drill down added: o First Generation o Foster Youth o Disabled Students o Ve t e ra n o LGBTQ o Homeless
Phase 4: May 6, 2019 Micro-regions level added Academic year 2007-2018 added Cohort view added Six types of drill-downs added: o Transfer English vs math (two drill-downs) o Retention at another college o Type of completion o Transfer with or without award o Transfer location
Simplified Metrics Q&A
Upcoming Webinars Webinars are hosted the last Wednesday of every month June 27 th at 10 a.m. Main Topic: CCCApply Redesign July 25 th at 10 a.m. Main Topic: NOVA Integrating Funding Streams August 29th 10 a.m. Main Topic: Simplified Metrics September 26 th 10 a.m. Main Topic: AB705/Multiple Measures Implementation Pipes
Metrics Details
Student Journey-Based Metrics Student progress and outcomes should be measured based on their informed educational goals: Adult education/esl Short-term career education Degree/transfer Undecided/other
Student Types Based on Goals Adult Education/ESL Short-term Career Education Degree/Transfer Undecided/Other Improve basic skills in English, reading or math Complete credits for high school diploma or GED Move from noncredit coursework to credit coursework Discover/formulate career interests, plans, goals Prepare for a new career (acquire job skills) Advance in current job/career (update job skills) Maintain certificate or license (e.g. Nursing, Real Estate) Earn a career technical certificate without transfer Obtain an associate degree and transfer to a baccalaureate granting institution Transfer to a baccalaureate granting institution without an associate degree Obtain a two-year associate degree without transfer Undecided on goal Pursue educational development (intellectual, cultural) University / 4-year college student taking courses to meet university / 4-year college requirements
Disaggregated Data Disaggregated information will be displayed for the following categories of students. Race/ethnicity Age group Gender Economically Disadvantaged Pell Grant Recipient College Promise Grant Recipient First generation Foster Youth Disabled Students Veterans Lesbian/Gay/Bisexual/Transgender In addition, colleges will be able to see time trends and information at the college, district, microregion, macroregion, and statewide level
Metrics Focus on Six Themes 1. Successful Enrollment 2. Learning Progress 3. Momentum 4. Success 5. Employment 6. Earnings
Enrolled After Application - Applies to All Four Student Types Percentage of applicants who enrolled, with disaggregated data on: enrolled in the same community college enrolled in a different community college enrolled in a four-year institution did not enroll in any college Metric will be based on CCC Apply, MIS, and the National Student Clearinghouse.
Learning Progress Applies to Adult Education/ESL & Undecided/Other Percentage of students who demonstrated a skills gain: as measured by the CASAS test by advancing one or more CB21 levels Metric will be based on CASAS and MIS submissions.
Learning Progress Applies to Short -Term Career Education & Undecided/Other Course success rate: Among enrollments Metric will be based on MIS submissions.
Learning Progress Applies to Degree/Transfer and Undecided/Other Percentage of students who completed both transfer-level math and English in their first year Courses must have been taken in the same district Metric will be based on MIS submissions.
Momentum - Applies to Adult Education/ESL & Undecided/Other Percentage of students who completed one or more levels of adult education Transitioned from adult basic ed/esl to adult secondary ed Metric will be based on MIS submissions.
Momentum - Applies to Short -Term Career Education and Undecided/Other Earned nine or more career education units in a single year Courses must be non-introductory Courses must have been taken in the same district Completed a noncredit workforce milestone Completed a noncredit workforce preparation course Completed a CTE course Had 48 or more contact hours in a career education course Metric will be based on MIS submissions.
Momentum - Applies to Degree/Transfer and Undecided/Other Percentage of students retained from fall to spring Percentage of students who successfully completed the following degree-applicable unit thresholds in the fall term: 0 units 1-5 units 6-8 units 9-11 units 12-14 units 15+ units Metric will be based on MIS submissions.
Momentum - Applies to Degree/Transfer and Undecided/Other Percentage of students who successfully completed the following degree-applicable unit thresholds in the academic year: 0 units 1-5 units 6-8 units 9-11 units 12-14 units 15-19 units 20-24 units 25-29 units 30+ units Metric will be based on MIS submissions.
Success Applies to Adult Education/ESL Percentage of students who earned an award and/or transferred to credit coursework, with disaggregated results showing the number who: Earned a high school diploma/ged/high school equivalency Earned a noncredit CDCP certificate or Chancellor s Office approved certificate Transitioned to any CTE course Transitioned from noncredit coursework to college credit coursework Metric will be based on MIS submissions and data match for high school equivalencies.
Success Applies to Short -Term Career Education Percentage of students who earned certificate and/or journey status, with disaggregated results showing the number who: Earned a noncredit certificate Earned a Chancellor s Office approved credit certificate Attained apprenticeship journey status Metric will be based on MIS submissions and a data match with the Department of Apprenticeship Standards.
Success Applies to Degree/Transfer Percentage of students who earned an award and/or transferred to a four-year institution, with disaggregated results showing the number who: Earned a Chancellor s Office approved credit certificate over 18 units Earned an associate degree Earned an associate degree for transfer Transferred without an award Transferred with an award Attained apprenticeship journey status Metric will be based on MIS submissions and a data match with the Department of Apprenticeship Standards.
Success Applies to Undecided Other Percentage of students who earned an award and/or transferred to a four-year institution, with disaggregated results showing the number who: Earned a high school diploma/ged/high school equivalency Earned a noncredit certificate Earned a Chancellor s Office approved credit certificate Earned an associate degree Earned an associate degree for transfer Transferred without an award Transferred with an award Attained apprenticeship journey status Metric will be based on MIS submissions and a data match with the Department of Apprenticeship Standards.
Employment Applies to Adult Education/ESL Percentage of unemployed students who became employed Metric will be based the Unemployment Insurance wage data.
Employment Applies to Short -Term Career Education, Degree/Transfer, and Undecided/Other Percentage of students who report they are working in a job very closely or closely related to their field of study Student had to respond to the CTE Outcomes Survey, which is only sent to students who earned a CTE award or took 9+ CTE units Metric will be based on the CTE Outcomes Survey.
Earnings Applies to Adult Education/ESL Median annual earnings of students Median change in earnings Metric will be based the Unemployment Insurance wage data.
Earnings Applies to Short -Term Career Education and Undecided/Other Students Median annual earnings of students Median change in earnings Percentage of students whose annual earnings met the regional living wage threshold Metric will be based the Unemployment Insurance wage data. Living wage will be based on regional figures for a single adult from the Insight Center for Community Economic Development. Students who transferred to a four-year institution will be excluded.
Earnings Applies to Awards/Transfer Median annual earnings of students Percentage of students whose annual earnings met the regional living wage threshold Metric will be based the Unemployment Insurance wage data. Living wage will be based on regional figures for a single adult from the Insight Center for Community Economic Development. Students who transferred to a four-year institution will be excluded.