Montana s Formative Science Repository Montana Data Use Conference (MDUC) Tuesday, September 26, 2017 Radisson Hotel in Helena, MT 59601 Ashley McGrath & Michelle McCarthy
Session Objectives Standards Context History of Science Standards in MT Assessment Context Summative Plans SCILLSS Partnership ACESSE Partnership Methods PAO Science Mission PAO Timeline Example of PAO Forms & Process Results PAO Phase I Findings Future Work Example of Repository Materials 2
Standards Context What students should KNOW and be able to DO Foundation Boxes 3
History of Science Standards in MT Last Science Standards adoption was in 2006. In 2009, the Essential Learning Expectations (ELE s) were written in an effort to provide guidance and support for teachers. 2012: Publication of A Framework for K 12 Science Education 2013: Publication of Next Generation Science Standards 2013: A critical look at NGSS for Montana and verification of gap analysis. 2014: NAEP Item Classification Study 2014: NRC Report Published on Assessments for NGSS identifies NAEP as a potential exemplar Note: Standards review period is approximately every five years.
NGSS Lead State 5
History of Science Standards in MT Cont 2016: Adopted NGSS-Aligned Standards 2017: Implementation of Standards July 1, 2017. 2017: PAO Science Workshop 2020: New Assessment 2015: OPI begins standard revision process
NGSS and Framework-Based State Adoptions Updated last as of April 2017 7
Assessment Context Montana Science Performance Standards (NGSSaligned) for college and career readiness Teacher resources for Formative Assessment Practices To improve instruction Summative Assessments Benchmarked to college And career readiness Teachers and schools have information and tools they need to improve teaching and learning and student learning All students leave high school college and career ready Interim Assessments Flexible, open, used for actionable feedback 8
Design Requirements Administered not less than one time during grades 3 through 5; grades 6 through 9; and grades 10 through 12. (ESSA Sec.1111 (b)(2)(b)(v)). Assessment and instruction are inseparable Worth Taking High-Quality Time-limited Fair and Supportive One of multiple measures Ties to improved learning 9
Summative Science Plans Since FY 2008 Montana has used the MontCAS CRT for Science. Two year contract extension for the MontCAS CRT-Science. Plan to measure the three-dimensional standards beginning in the spring of 2020. 10
Partnerships Montana is a partner state in the Strengthening Claims-Based Interpretations and Uses of Local and Large-Scale Science Assessments (SCILLSS) Educational Assistance Grant (EAG) funded by the US Department of Education. The Advancing Coherent and Equitable Systems of Science Education (ACESSE, or access ) project brings together partners from educational research and practice to improve equity by building coherence in science education. - Council of State Science Supervisors 11
What to Measure? Standards Assessment Assessment Standards Standards Assessment Assessment Items Standards
Montana s Theory of Action 13
What is Formative Assessment? Source: https://www.smarterbalanced.org/wp-content/uploads/2015/09/digital-library-factsheet.pdf 14
Balanced Assessment for Science Website: https://www.smarterbalancedlibrary.org/ 15
Why NAEP? Free and Accessible Variety of Formats Rigorous Benefits of NAEP Items Available scoring Guides Inclusion of science practices Available performance data
What is PAO Science? Last Row: Marshall Lagge, Bruce Dudek Sixth Row: Amanda Obery, Emily Currier, Karla Miller, Jessica Eilertson, John Deming, Jared Betz Fifth Row: Chris DeWald, Jennifer Stadum, Sue Mohr, Lily Haines, Nicole Kirschten Fourth Row: Ashley McGrath, Katie Burke, Melissa Johnson, Lindsay Manzo, Debbie Hanson Third Row: Maureen Karlin, Jacqueline Marshall, Michelle McCarthy, Brian Williams Second Row: Jodi Hall, Audrey Howard, Roni Sells, Monica Tomayer, Katherine Aune Front: Karen Pollari, Marcy Fortner, Yvonne Field, Molly Ward, Mary Williams, Summer Graber 17
PAO Formative Assessment 18
Methods
PAO Science Mission The following statements define the significance of this work: 1. PAO provides a model for taking existing Framework-aligned assessment materials and methods to examine them for formative purposes. 2. PAO provides a step-wise Understanding by Design (UbD) process using carefully crafted forms to consider resources and instructional strategies for instructional intention, clarity, evidence production, feedback, and closing the learning gap. 3. PAO offers guidance on accessibility through identifying areas of existing accessibility and areas where additional support or modification may be needed to meet the needs of all students. 4. PAO provides a state-specific form to increase the student engagement, authenticity, and connections to the community through the cultural relevance and significance section of this project. 5. PAO provides rubrics and processes for reviewing and considering materials for inclusion or exclusion in the online repository. 6. PAO Groups identify formative materials for inclusion on the OPI s online repository which will be freely shared with Montana educators across the state. 20
PAO Phase I - Pilot 21
PAO Data Collection Forms 22
Results
PAO Phase I Findings Facts: 25 teachers participated in this science formative development project. There were several teachers who participated on the team that wrote the Montana Science Standards (2016) Several informal science educators (e.g., educators form museums, science centers, etc. who work with teachers throughout the school year) were also participants. 24
PAO Phase I Findings Benefits to State: This workshop increased teacher s knowledge of the new science standards and around developing formative assessments that measure these new standards. 25
PAO Phase I Findings Benefits to Teachers: The course overall helped me grow. I appreciated the face-to-face experience the most, truly my best learning style. PAO Teacher Participant 26
PAO Phase I Findings Benefits to Students: Students of teachers who were a part of this workshop will have teachers who have more confidence teaching the new science standards. 27
Future Work 28
Goals for PAO Phase II Use the Hub evaluations and PAO in-person evaluation to revise the Hub Course materials for future re-launch. Use the evaluation feedback to revise PAO Forms, Review Rubric and Cover Profile for clarity and usability. Continue to expand our formative resource repository using the established training, development process, and review process. Continue to build statewide capacity through opportunities listed in Future Interest Survey.
PAO Repository Material 30
Repository Timeline Aug Nov Assemble PAO Recommended Resources for OPI Review Aug TBD Interim Holding Tank Aug Dec OPI Resource Review Oct Dec Publishing Review Dec Feb Publishing Materials for Teacher Access 31
Build Statewide Capacity What Next? For example: Resource in Practice - teachers use this work during the year and submit evidence of student work, personal reflections, and/or activity in action (video). Feedback on Resources pilot them and provide personal experience or modifications of resources. Contribute to the Pool submit more formative resources to the repository. See the Hub Catalog for ideas and additional courses!
Questions? Ashley McGrath NAEP State Coordinator Montana Office of Public Instruction Phone: 406.444.3450 E-mail: amcgrath@mt.gov Website: http://opi.mt.gov/leadership/assessment- Accountability/NAEP Michelle McCarthy Science Instructional Coordinator Montana Office of Public Instruction Phone: 406.444.3537 E-mail: MMcCarthy5@mt.gov Website: http://opi.mt.gov/educators/teaching-learning/k- 12-Content-Standards-Revision 33