ODU Certificate Program Guidelines for Assessment

Transcription

1 ODU Certificate Program Guidelines for Old Dominion University is committed to a continuous process of systematic assessment to ensure the highest possible quality for all university programs (ODU Mission). The focus of certificate assessment is on student learning outcomes, not on the courses within the certificate. Certificate Programs Certificate programs meet the same high-quality standards as its academic degree programs, while addressing the specific needs of students and professionals. Certificates may be pursued in conjunction with or independent from graduate or undergraduate degree programs (Faculty Handbook, Policy on Certificate Programs). of certificate programs is in compliance with the institutional effectiveness standard 8.2.A. This comes from ODU s regional accrediting body the Southern Association of Colleges and Schools Commission on Colleges (SACSCOC). Certificate Plan Guidelines Certificate programs in which student learning outcomes are not associated or aligned with an existing degree must be assessed. For example: Certificates that are created using primarily cross-disciplinary or stand-alone courses OR A certificate with re-packaged existing courses that are housed within a degree program but the student learning outcomes are significantly different from the degree program outcomes. of certificate programs is dependent upon student learning outcomes. Some certificate outcomes are directly aligned with existing degree programs outcomes while others are not. Certificate programs in which student learning outcomes are associated and directly aligned with an existing degree program do not require a separate assessment plan. Alignment of the outcomes and subsequent measures within the existing degree program needs to be made clear. With this, the certificate program is embedded into the existing degree program that is assessed annually and reported in WEAVE. Number of credit hours and enrollments do not determine whether or not a certificate needs an assessment plan. Associated licensures or professional certifications with a certificate program do not determine whether or not a certificate is assessed. Non-credit bearing courses or certificates are not included in the University s program at this time. Processes for Certificate of certificates should follow University assessment guidelines and reporting procedures (

2 3-5 student learning outcomes must be defined and program objectives should be created where appropriate o The focus of certificate assessment should be at the course and certificate program level. A certificate course may have students who are currently earning the certificate, students who are earning a degree, and/or students who may decide to pursue the certificate after taking the course. Because of this, the focus of assessment must be on the outcomes that all students will attain. o In many cases, we find that an eportfolio can be a very beneficial way to assess certificate students. Direct measures of student learning must be used. Indirect measures can only be used as a secondary measure (e.g. course grades, surveys, etc.). findings are reported yearly by September 30, in Weave o Findings need to be compared between learning environments if the certificate is offered fully online and fully face-to-face. o Certificate programs are asked to report assessment data every year. If there are no graduates in a given year, coordinators need to report this in Weave. They are also asked to enter their recruiting and marketing strategy for the upcoming year in an action plan. o Certificate programs are asked to analyze findings upon the cumulative graduation of 5 students or more. Processes for Embedded Certificate At least two certificate student learning outcomes should be identified and denoted to be alignment with the student learning outcomes of the existing degree program. At least two direct measures of student learning should be used to ensure the collection and analysis of artifacts from certificate courses/students. These can be existing measures or an additional measure may need to be added to the degree program s assessment plan. o The focus of certificate assessment should be at the course and certificate program level. A certificate course may have students who are currently earning the certificate, students who are earning a degree, and/or students who may decide to pursue the certificate after taking the course. Because of this, the focus of assessment must be on the outcomes that all students will attain. findings are reported yearly by September 30, in Weave o Findings need to be compared between learning environments if the certificate is offered fully online and fully face-to-face.

3 ODU Certificate Program Resources for INTRODUCTION What is assessment? An ongoing process aimed at understanding and improving performance. It involves: Making our expectations explicit Setting appropriate criteria and standards for quality Systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards A good assessment process can answer three d questions: 1. What are we trying to do? 2. How well are we doing it? 3. How are we using what we discover to improve what we will do in the future? Why assess? Because GOOD assessment informs the teaching and learning process. information can yield useful and accurate results specific to the learning outcomes so that improvements can be made. ASSESSMENT 101 plans include 5 parts; student learning outcomes, measures, targets, findings, and action plans. Student learning outcomes (SLO) specify what students will be able to do or know when they graduate from the program. Question: What three things do you want your students to be able to do or know as a result of completing the certificate? What knowledge, skills, or abilities have been gained as a result of the completing the certificate? Precise and concrete action verbs are used to indicate the specific behavior that will be demonstrated (e.g. Bloom s Taxonomy). Skills and abilities are typically assessed by direct observation of student performances (oral or visual presentations). Knowledge is typically assessed using appropriate written and oral examinations. Start here: Think critically, write well or proficiently, understand or be familiar with a particular concept, demonstrate knowledge, skill, proficiency or understanding, as well as appreciate a view point. End here: Identify and recognize, describe and explain, solve and apply, analyze and compare, create and design, as well as judge and critique.

4 Measures are the means by which evidence will be collected to determine the degree to which students have met the learning outcome. The conditions you want the learning to be performed are clearly stated. Question: What evidence can we provide to prove that students have met the SLO? Direct measures are tangible, visible, observable, and self----explanatory. The quality and quantity of student learning is concretely exhibited. Examples: test grades, rubrics, portfolios of student work, capstone projects, field supervisor ratings, employer ratings, and scores and pass rates on licensure exams. Indirect measures are signs, indicators, and less convincing (tell us about ) types of evidence. The results may be consistent with student learning, but do not solicit specifics. Examples: course grades, surveys, focus groups, course evaluations, admission to graduate school, student self----ratings, student/alumni satisfaction with learning, as well as honors, awards, and scholarships. Objective measures require no professional judgment to score correctly, usually the result of one correct answer. Examples: multiple choice test, matching items, and true/false questions. Subjective measures require professional judgment for scoring and allow for many possible answers of varying quality. Examples: essays, performances, and presentations. Targets describe the level/degree to which the learning outcome should be achieved. Question: At what level do you expect your students to perform? Examples % will pass each section of the licensure test % will score 70% or better on each section of the final exam % will Meet or Exceed standard for the outcome on the assignment rubric Findings are the analyzed results of each measure. They should be written in an executive summary with a link to the longer report with supporting artifacts (e.g., rubrics, scoring sheets, etc.) Question: What does this information tell us about the student s ability to meet the learning outcome? Action Plans discuss the results regarding student achievement of the learning outcomes, reflection on their implications, and plans to use the results to improve the certificate program. Question: With this information, how can we continue or improve on the teaching or structuring of this certificate?

5 Bloom s Taxonomy Action Verbs Level Definition Sample verbs Sample behaviors KNOWLEDGE Student recalls or recognizes information, ideas, and principles in the approximate form in which they were learned. arrange define describe duplicate identify label list match memorize name order outline recognize recall repeat reproduce select state define the 6 levels of Bloom's taxonomy of the cognitive domain. COMPREHENSION Student translates, comprehends, or interprets information based on prior learning. explain summarize paraphrase describe illustrate classify convert defend describe discuss distinguish estimate explain express extend generalized give example(s) identify indicate infer locate paraphrase predict Recognize rewrite review select summarize translate explain the purpose of Bloom's taxonomy of the cognitive domain. APPLICATION Student selects, transfers, and uses data and principles to complete a problem or task with a minimum of direction. use compute solve demonstrate apply construct apply change choose compute demonstrate discover dramatize employ illustrate interpret manipulate modify operate practice predict prepare produce schedule show sketch solve use write write an instructional objective for each level of Bloom's taxonomy. ANALYSIS Student distinguishes, classifies, and s the assumptions, hypotheses, evidence, or structure of a statement or question analyze categorize compare contrast separate apply change discover choose compute demonstrate dramatize employ illustrate interpret manipulate modify operate practice predict prepare produce schedule show sketch solve use write compare and contrast the cognitive and affective domains. SYNTHESIS Student originates, integrates, and combines ideas into a product, plan or proposal that is new to him or her. create design hypothesize invent develop arrange assemble categorize collect combine comply compose construct create design develop devise explain formulate generate plan prepare rearrange reconstruct reorganize revise rewrite set up summarize synthesize tell write design a classification scheme for writing educational objectives that combines the cognitive, affective, and psychomotor domains. EVALUATION Student appraises, assesses, or critiques on a basis of specific standards and criteria. Judge Recommend Critique Justify Appraise Argue Assess Attach Choose Compare Conclude Contrast Defend Describe Discriminate Estimate Evaluate Explain Judge Justify Interpret Relate Predict Rate Select Summarize Support Value judge the effectiveness of writing objectives using Bloom's taxonomy. Reference:

6 Learning Goal Grasp and persuade others of the strategic role procurement and contract management can play in governmental and nonprofit organizations. Promote, lead, and participate in the formation of contracts that provide for maximum practicable support of enterprise strategic goals. Make the case for, lead, and participate in contract administration that assures maximum support for enterprise strategic goals. Help assure that procurement and contract management decisions and action are legal and ethical. Example: Graduate Certificate in Public Procurement and Contract Management Outcome Create an argument for policymakers, senior managers, and others that effective procurement to support a public entity s strategic goals. Develop a strategic plan for procurements and form contracts. Describe and explain the scope and elements of contract administration. Evaluate the fundamental principles of public sector procurement law and ethics. Measure PPCM 726, Essay Question on Mid-Term or Final Exam PPCM 728, Final Project PPCM 718, Paper Assignment PPCM 731, Multiple Choice questions (5-10) on Mid-Term or Final Exam, collected using item analysis in Blackboard Cycle Fall Spring Fall Spring Target 90% of students will achieve at least 80% on the exam question 75% of students will achieve exemplary standards on the Final Project rubric categories % of students will achieve at least 80% on the assignment rubric, category 2 90% will achieve at least 80% on the exam question *This example was modified by the Office of Institutional Effectiveness and

7 Example Template: Certificate Mission and Goals: Educational purpose and goals are broad statements of purpose in philosophical terms often describing values and aspirations. Criteria: Mission and goals are comprehensive concise descriptions of the broad aspects covered within the certificate; Mission is clearly aligned with the department or college mission and the university mission; goals clearly identify the knowledge, skills, abilities, and transferable competencies student will acquire; goals are clearly aligned with mission. Student Learning Outcome Measure Target outcomes can also be called student learning outcomes. This specifies the knowledge, skills, and abilities that students will attain as a result of completing the certificate program. Strong outcomes use precise and concrete action verbs to indicate the specific behavior that will be demonstrated (e.g. Bloom s Taxonomy). Measures describe the evidence that will be collected to determine the degree to which students have met the assessment outcome. Describes the level or degree to which the student learning outcome should be achieved. Old Dominion University Office of Institutional Effectiveness and