Principles of Online Instructional Design RIDT 101 1DL

Transcription

1 Rabb Schl f Cntinuing Studies Curse Syllabus Summer 2015 Principles f Online Instructinal Design RIDT 101 1DL May 20, 2015 July 28, 2015; Online Curse Week: Tuesday thrugh Wednesday Instructr Infrmatin: I. Curse Infrmatin Brian L. Salern, M.S. Instructr; Directr f Online Learning & Instructinal Design; Prgram Chair; M.S. in Instructinal Design & Technlgy; Divisin f Graduate Prfessinal Studies, Rabb Schl bsalern@brandeis.edu Phne: Virtual Office Hurs: By Appintment, Mnday Friday 8:00am 4:00pm and evenings/weekends as needed. Via BlueJeans web-cnference: Via Cnference Call: ID: Curse Site Dcument Overview This syllabus cntains all relevant infrmatin abut the curse: its bjectives and utcmes, the grading criteria, the texts and ther materials f instructin, and f weekly tpics, utcmes, assignments, and due dates. Cnsider this yur radmap fr the curse. Please read thrugh the syllabus carefully and feel free t share any questins that yu may have. Please print a cpy f this syllabus fr reference. Curse Descriptin Catalg Descriptin This curse intrduces students t the fundatinal instructinal design methdlgies and mdels cmmnly utilized in the design and develpment f nline curses, training mdules, r prgrams. Students explre the applicatin f evidence-based learning science t nline curse develpment thrugh instructinal design systems such as ADDIE, utcmes-based backwards curse design, and mre cntemprary mdels, such as SAM (Successive Apprximatin Mdel). Participants examine the rles and respnsibilities f the instructinal designer as they relate t the nline learner, instructr, subject matter expert, and thers. Students explre best practices related t synchrnus and asynchrnus delivery f nline learning cntent. Thrughut the curse, students will apply macr and micr design principles and cllabrate t design and create nline lessn prttypes; write measurable learning utcmes and related

2 assessments; select and curate apprpriate learning resurces; integrate task analyses; and design activities that fster nline learning cmmunities and prmte cllabrative learning in asynchrnus envirnments. Instructr s Nte: Thank yu fr registering fr this curse! I am lking frward t wrking with yu this term as yur instructr, and helping yu t develp a slid understanding f nline instructinal design and the ability t apply thse skills tward the design f e-learning prjects. I have wrked t make the prcedures that we will use clear t everyne thrugh this syllabus and thrugh the materials psted in LATTE curse. Please familiarize yurself with these materials and feel free t ask me any questins that yu may have. I am happy t wrk with yu individually and as a class t help yu t learn and apply the new instructinal design skills and cncepts presented in the curse, and I encurage yu t ask questins when yu are unsure and respnd t yur classmates questins when yu believe yu have the answer; in explaining hw t apprach prblems, we learn mre urselves. We can all learn frm each ther, and I hpe that we ll have pen and enriching discussins as we mve frward! My full intrductin and bigraphy has been psted t ur discussin frum (Intrduce Yurself frum in Week 1). I lk frward t reading yur bigraphies and getting t knw yu as well. I sincerely hpe yu enjy this curse, and lk frward t yur cntributins. Relevant Prgrams: Required Curse in the M.S. in Online Instructinal Design & Technlgy Ptential Elective by petitin in ther Graduate Prfessinal Studies prgrams Prerequisites: Nne Materials f Instructin a. Required Texts Mrrisn, G.R., Rss, S.M., Kalman, H.K. & Kemp, J.E. (2011). Designing Effective Instructin (7th ed.). Jhn Wiley & Sns, Inc. ISBN-10: b. Required Sftware and Other Supplies : Nne c. Recmmended Text(s) / Jurnals d. Online Curse Cntent The Jurnal f Applied Instructinal Design: Recmmended Reading: Mager, Rbert F., Preparing Instructinal Objectives: A Critical Tl in the Develpment f Effective Instructin, 3 rd Editin, Center fr Effective Perfrmance, May 1997 Recmmended Reading: Dirksen, Julie, Design fr Hw Peple Learn, 1 st Editin, New Riders, Nvember 2011 This sectin f the curse will be cnducted cmpletely nline using Brandeis LATTE site, available at The site cntains the curse syllabus, assignments, discussin frums, links/resurces t curse-related prfessinal rganizatins and sites, and weekly checklists, bjectives, utcmes, tpic ntes, self-tests, and discussin questins. Access infrmatin is ed t enrlled students befre the start f the

3 Overall Curse Outcmes curse. T begin participating in the curse, review the Welcme Message and the materials fund in the Week 1 blck. Upn cmpletin f the curse, students will be able t: 1. Describe the value f systematic appraches t instructinal design and the rle f instructinal designer in nline learning envirnments. 2. Apply theries f learning and cgnitin t the design and develpment f nline learning cntent and curses. 3. Assess nline instructinal gals thrugh learner, cntextual, and task analyses. 4. Develp measurable instructinal utcmes and bjectives t supprt the effective assessment f nline learning. 5. Cmpare and apply the elements f current instructinal systems design mdels t the develpment e-learning prjects. 6. Develp instructinal strategies, including identificatin and sequencing f apprpriate learning resurces, assessment instruments, and pedaggical/andraggical appraches fr an nline learning envirnment. 7. Create an evaluatin plan, including frmative and summative evaluatins t assess the effectiveness f nline instructinal strategies. Curse Grading Criteria Percent Cmpnent 30% Discussins/Online participatin: 30% individual discussins (including riginal respnses and replies) 3% per week, 10 weeks 50% Instructinal Design Blueprint Assignments 1. Learning Outcmes/Objectives (15%) 2. Instructinal Gals & Needs Assessment (15%) 3. Learner, Cntext, & Task Analysis (20%) 20% Final Prject - Mdule Design & Assessment Plan Descriptin f Assignments Individual Discussins/ Online Participatin (30% f ttal grade/ 3% per week) Requirements: Pst an riginal respnse t the discussin questin in the Weekly Discussin frum n r befre Saturday (midnight EST) f the curse week. Pst at least tw substantive replies t the psts f thers n r befre Tuesday (midnight EST). These ther psts are respnses t the discussin tpic messages f thers. The assumptin is that yu will read thrugh the psts f yur classmates t enhance yur learning; respnd t thse f yur chice, based upn yur wn experiences and insights.

4 Pst t the discussins n at least three different days f the curse week. Viewing Frum Psts: In sme cases, I have set up the Discussins frums s that yu will nt be able t view the riginal respnses f thers until yu pst a respnse t my starter thread. In ther cases, the discussin is pen and visible withut the need t pst in advance. I will identify the nature f the discussin at the beginning f each week. Evaluatin Criteria All graded assignments and assessments will be evaluated n a 100 pint scale, and will be weighted tward yur final grade accrding t the Curse Grading Criteria sectin abve. All graded assignments and assessments will include rubrics cntaining detailed success criteria included with each assignment dcument n the curse hmepage. Each week, 100 pints may be earned tward the participatin cmpnent f the grade. Online Discussin Respnse Qualitative Rubric This rubric can serve as yur general guide fr the basic cmpnents f a well-crafted discussin respnse. Please als see the detailed discussin criteria and grading schema n the pages fllwing this rubric. Criteria A-level B-level C/D-Level F- Level Cnsideratin Psed a new riginal idea r develped an pinin in depth Opinin stated clearly Opinin nt clearly stated Little r n evidence f ruminatin Narrative Prvided vivid persnal r prfessinal examples r stry t give cntext t the tpic Prvided persnal/ prfessinal examples r stry related t the tpic N use f persnal/ prfessinal examples r stry Unrelated persnal examples r stry Off-tpic Reasning Justified reasning r use f metaphrical thinking that encuraged respnses Interesting idea r metaphr psed with sme justificatin Argument withut justified reasn Discnnected ideas r unriginal ideas duplicative f readings Citatin Apprpriately cited relevant ideas beynd the assigned readings Apprpriately referenced class ntes, material, r readings Inaccurate citatin r misapplied reference N citatin r references

5 Original Discussin Respnses (60% f Weekly Participatin Grade) Evaluatin Criteria: Includes yur wn insights int the tpics, sharing yur prfessinal experiences as apprpriate and yur wn cnclusins Includes references t weekly required readings and/r ther external surces, cited apprpriately. All riginal respnses must draw n external references which may include my tpic ntes, the additinal readings, the text, r ther surces that yu lcate t supprt and justify yur respnses. Answers the questin psed cmpletely Pses questins r pints f cnsideratin t elicit respnses frm classmates Cnsists f at least wrds Well written, with n spelling r grammatical errrs One day late: -10 ut f 30 pssible raw pints; mre than ne day late: n credit Substantive Discussin Replies (35% f Weekly Participatin Grade) Substantive (beynd an "I agree" r cmplimentary pst) with: Fllw-n pints frm yur related experiences and/r frm the readings Fllw-up questins f thers t extend the cnversatin (encuraged) Cnsists f at least 200 wrds Grammar/spelling/frmat/surces nted as apprpriate One r mre days late: n credit Psting Actively t Discussin Thrughut the Curse Week (5% f Participatin Grade) Pst n three days f the curse week: (full 5% credit) Pst n tw days f the curse week: (3% credit) Pst n ne day f the curse week: (1% credit) Thughts n Discussins Keep in mind that these pstings t the frums will be as rich as we make them; nt having a traditinal classrm in which t discuss tpics, we can have sme interesting discussins and share ur experiences during the 10 weeks. They are required t encurage yu t share yur knwledge and ideas while gaining frm the experiences f yur peers as well. Yu will quickly adjust t the weekly requirements and becme familiar with the review criteria, and I lk frward t sme rich discussins. Instructinal Design Blueprint Deliverables (50% f ttal grade) Students will develp a set f instructinal design deliverables leading up t the final prject, which will be discussed in detail in separate instructins psted t the Assignments sectins f each week in the curse site. Sme f the deliverables will be prduced individually; in sme cases students will wrk in small grups r pairs. The fllwing dcuments will be prduced: Learning Outcmes/Objectives (15%): Available Weds f Week 2, Due Tues f Week 3 Instructinal Gals & Needs Assessment (15%): Available Weds Week 4, Due Tues Week 5 Learner, Cntext & Task Analysis (20%): Available Weds Week 6, Due Tues Week 7 5

6 Final Prject Mdule Design & Assessment Plan (20% f ttal grade) The final prject will require students t leverage the Instructinal Design Blueprint Deliverables t create a cmplete instructinal design plan fr a mdule, e-learning unit, r shrt-curse, develp an assessment strategy fr the mdule, and an evaluatin plan t determine the effectiveness f their instructinal design. A prtin f the grade will be determined by students successful participatin in a peer-evaluatin and feedback prcess. A detailed descriptin and instructins fr the final prject will be prvided well in advance, and will be psted t the apprpriate Assignments sectin f the curse. Delicius Available Weds f Week 8, Prject Due Sunday f Week 10, Peer-Evaluatin due Tuesday f Week 10. II. Weekly Outline Week 1 (May 20-26) Instructinal Design: The Prcess and Prfessin Weekly Outcmes 1. Define the rle f the instructinal designer 2. Describe the key elements f the instructinal design prcess 3. Explain the value f a systematic apprach t the design f nline learning 4. Outline the relatinship between nline instructinal design and educatinal technlgy Discussins Intrduce Yurself Discussin Week 1 Discussin Respnse Week 1 & Intrduce Yurself Replies Readings & Resurces Week 1 Learning Resurces Week 1 Micr-Lessn Mrrisn Text: Chapter 1 Assignments / Assessments Week 1 Reflectin (Optinal) Week 2 (May 27 June 6) Learning Outcmes & Objectives: Design with Assessment In Mind Weekly Outcmes 1. Outline the purpse and functin f instructinal bjectives and utcmes 2. Demnstrate the prcess fr writing bjectives/utcmes fr each f the three majr dmains 3. Apply apprpriate taxnmies t the develpment f learning utcmes/bjectives 4. Describe the relatinship f bjectives/utcmes t the evaluatin f nline learning 6

7 Discussins Week 2 Discussin Respnse Week 2 Replies Readings & Resurces Week 2 Learning Resurces Week 2 Micr Lessn Mrrisn Text: Chapters 5 & 11 Assignments / Assessments Begin: Learning Outcmes/Objectives Assignment (Due Week 3) Week 3 (June 3-9) Week 2 Reflectin (Optinal) Learning Science & Instructinal Thery: An Overview Weekly Outcmes 1. Define learning science and instructinal thery 2. Identify the majr paradigms f learning thery 3. Describe hw instructinal designers leverage knwledge learning thery in the design f e-learning 4. Apply behaviral and cgnitive theretical appraches t instructinal design Discussins Week 3 Discussin Respnse Week 3 Replies Readings & Resurces Week 3 Learning Resurces Week 3 Micr Lessn Mrrisn Text: Chapter 14 Assignments / Assessments Submit: Learning Outcmes/Objectives Assignment (Due Tues) Week 4 (June 10 16) Week 3 Reflectin (Optinal) Instructinal Design Mdels: A Cmparisn Weekly Outcmes 1. Identify the majr elements f different cntemprary mdels f instructinal design 2. Cmpare the efficacy f varying instructinal design mdels and framewrks 3. Demnstrate the prcess fr selecting an apprpriate ID mdel r framewrk t address the needs f a specific e-learning prject Discussins Week 4 Discussin Respnse Week 4 Replies Readings & Resurces Week 4 Learning Resurces Week 4 Micr Lessn Mrrisn Text: Nne Assignments / Assessments Begin: Instructinal Gals & Needs Assessment (Due Week 5) Week 4 Reflectin (Optinal) 7

8 Week 5 (June 17-23) Needs Assessment & Online Instructinal Gals Weekly Outcmes 1. Describe the purpse f needs assessment, gal analysis, and perfrmance assessment in nline learning 2. Cnduct a fur phase needs/gals assessment t determine the necessity f an instructinal interventin 3. Develp instructinal gals that supprt the effective design f e- learning Discussins Week 5 Discussin Respnse Week 5 Replies Readings & Resurces Week 5 Learning Resurces Week 5 Micr Lessn Mrrisn Text: Chapter 2 Assignments / Assessments Submit: Instructinal Gals & Needs Assessment (Due Tues) Week 6 (June 24-30) Week 5 Reflectin (Optinal) Analyzing the Online Learner, Cntext, and Tasks Weekly Outcmes 1. Define Learner, Cntext, and Task Analysis 2. Describe the characteristics f a ppulatin f learners fr a prpsed e-learning mdule 3. Analyze the cntextual characteristics f the prpsed nline learning envirnment 4. Apply a task analysis methd t a prpsed e-learning mdule Discussins Week 6 Discussin Respnse Week 6 Replies Readings & Resurces Week 6 Learning Resurces Week 6 Micr Lessn Mrrisn Text: Chapters 3 & 4 Assignments / Assessments Begin: Learner, Cntext, & Task Analysis Dcument (Due Week 7) Week 7 (July 1-7) Week 6 Reflectin (Optinal) Designing Instructin: Sequencing & ID Strategies Weekly Outcmes Outline examples f learning-related, wrld-related, cnceptrelated, and elabratin thery instructinal sequencing Apply apprpriate sequencing strategies t a prpsed e-learning mdule Describe the elements f an nline instructinal strategy Develp an instructinal strategy fr a prpsed e-learning mdule Discussins Week 7 Discussin Respnse 8

9 Week 7 Replies Readings & Resurces Week 7 Learning Resurces Week 7 Micr Lessn Mrrisn Text: Chapters 6 & 7 Assignments / Assessments Submit: Learner, Cntext, & Task Analysis Dcument (Due Tues) Week 8 (July 8-14) Week 7 Reflectin (Optinal) Instructinal Materials & e-learning Cntent Design Weekly Outcmes 1. Cmpare and cntrast technlgy-driven instructinal delivery frmats cmmnly used in nline learning envirnments 2. Apply apprpriate instructinal delivery frmats t prpsed e- learning mdules 3. Design and curate nline instructinal materials t supprt an instructinal strategy Discussins Week 8 Discussin Respnse Week 8 Replies Readings & Resurces Week 8 Learning Resurces Week 8 Micr Lessn Mrrisn Text: Chapters 8, 9, & 10 Assignments / Assessments Begin: Final Prject Mdule Design & Assessment Plan (Due Week 10) Week 9 (July 15-21) Week 8 Reflectin (Optinal) Develping an Online Assessment Strategy & Instruments Weekly Outcmes 1. Describe hw instructinal bjectives drive the selectin f assessment and evaluatin methds 2. Evaluate appraches fr applying authentic assessments in nline learning envirnments 3. Cmpare varius assessment instrument types 4. Design apprpriate assessments fr a prpsed e-learning mdule Discussins Week 9 Discussin Respnse Week 9 Replies Readings & Resurces Week 9 Learning Resurces Week 9 Micr Lessn Mrrisn Text: Chapters 11 & 12 Assignments / Assessments Cntinue: Final Prject Mdule Design & Assessment Plan (Due Week 10) Week 9 Reflectin (Optinal) 9

10 Week 10 (July 21-28) Implementatin & Evaluatin f Online Learning Weekly Outcmes 1. Describe the elements and cnsideratins invlved in the develpment f an e-learning implementatin plan 2. Describe the purpse and benefits f frmative and summative evaluatins 3. Develp an evaluatin plan, including a frmative evaluatin, fr a prpsed e-learning mdule 4. Develp a cmprehensive Instructinal Design and Assessment Plan fr a prpsed e-learning mdule Discussins Week 10 Discussin Respnse Curse Wrap-Up Respnse Week 10 Replies Readings & Resurces Week 10 Learning Resurces Final Micr Lessn Mrrisn Text: Chapters 13 & 15 Assignments / Assessments Submit: Final Prject Mdule Design & Assessment Plan (Due Sunday) Cmplete: Peer-Evaluatin (Due Tues) Week 10 Reflectin (Optinal) III. Curse Plicies and Prcedures Orientatin All students wh are new t Graduate Prfessinal Studies r whm have nt previusly taken an nline curse are expected t have cmpleted the Student Online Orientatin curse accessible frm the GPS Resurces blck f the curse hmepage. This shuld be perfrmed befre the curse start date. Asynchrnus Wrk All required wrk fr the curse may be dne asynchrnusly; i.e., students can lgin t the curse, read/dwnlad materials, pst t the frums, and submit assignments thrughut the curse week. Please carefully fllw the syllabus and the weekly mdules t help manage yur time thrughut the curse week; nce we enter week 2 r 3, students typically becme much mre cmfrtable with the pace and flw f the curse. At pints thrughut the semester, I will make myself available fr synchrnus chat sessins using the BlueJeans vide cnferencing link nted n the first page f this syllabus. These sessins will be pen Q&A, and they are ptinal. I will pst a recrding f each chat sessin s that thse students wh did nt participate can view the recrded synchrnus discussins. Fr any grup prjects, students may use the asynchrnus grup frums available and, if they chse, they may use the synchrnus grup meeting rms available. 10

11 Wrk Expectatins Late Plicies Students are respnsible t explre each week's materials and submit required wrk by their due dates. On average, a student can expect t spend apprximately 3-5 hurs per week reading and apprximately 7-9 hurs per week cmpleting assignments and psting t discussins. The calendar f assignments and due dates is lcated at the end f this syllabus, and all assignments are due by the clse f the assciated week (Tuesday evenings). Late discussin psts are strngly discuraged as the success f class discussin is dependent upn the active engagement f all participants in the curse. Late plicies related t discussin psts can be fund in the Evaluatin Criteria sectin abve. Pints will be deducted fr late Instructinal Design Blueprint deliverables accrding t the fllwing scale: 1-2 days late -5 pints 3-4 days late -10 pints 5-6 days late -15 pints 7 r mre days late - nt accepted *Late Final Prjects cannt be accepted On rare ccasin, persnal r prfessinal issues d arise that may warrant an exceptin t the late plicy abve. Please ntify the instructr at least 24 hurs in advance f a due date if an issue arises that will make it impssible fr yu t meet a stated due date. Exceptins, althugh rare, will be cnsidered n a case-by-case basis. Cnfidentiality in the Classrm As we prceed thrughut ur Discussins, I'd like t highlight a pint abut cnfidentiality in ur nline classrm. We can draw n the wealth f examples frm ur rganizatins in class discussins and in ur written wrk. Hwever, it is imperative that we nt share infrmatin that is cnfidential, privileged, r prprietary in nature. We must be mindful f any cntracts we have agreed t with ur cmpanies. In additin, we shuld respect ur fellw classmates and wrk under the assumptin that what is discussed here (as it pertains t the wrkings f particular rganizatins) stays within the cnfines f the classrm. Finally, fr yur awareness, members f the University's technical staff have access t all curse sites t aid in curse setup and technical trubleshting. Prgram Chairs and a small number f Graduate Prfessinal Studies (GPS) staff have access t all GPS curses fr versight purpses. Students enrlled in GPS curses can expect that individuals ther than their fellw classmates and the curse instructr(s) may visit their curse fr varius purpses. Their intentins are t aid in technical trubleshting and t ensure that quality curse delivery standards are met. Strict cnfidentiality f student infrmatin is maintained. Grading Standards Students are graded n demnstratin f knwledge r cmpetence, rather than n effrt alne. Each student is expected t maintain high standards f hnesty and ethical behavir. All assignments are meant t represent yur wn wrk. I expect students t cnduct themselves curteusly nline. If in my judgment a student's cnduct is nt curteus, I reserve the right t reduce that student's grade. 11

12 Hw Pints and Percentages Equate t Grades A+/A C A C B D B D B D C+ 59 r < F Feedback Feedback will typically be prvided n assignments and the final prject within 10 days f the due date. Within 7 days f the clse f each week, feedback will be prvided n weekly participatin via the assciated Weekly Participatin Feedback links; yu d nt have t submit anything t these assignments. I will be recrding yur weekly participatin grades and prviding narratives describing yur discussin psts (participatin) fr that week. Calendar f Due Dates Assignment Available Date Due Date Learning Outcmes/Objectives Weds, Week 2 May 27 Tues, Week 3 June 2 Instructinal Gals & Needs Assess. Weds, Week 4 June 10 Tues, Week 5 June 16 Learner, Cntext & Task Analysis Weds, Week 6 June 24 Tues, Week 7 July 7 Final Mdule Design & Assess. Plan Weds, Week 8 July 8 Sun, Week 10 July 26 Respnse t Discussin Questin Other Substantive Psts (2 per week) by Sat each week 2 by Tues each week 12

13 IV. University and Divisin f Graduate Prfessinal Studies Standards {These are plicies and prcedures set at the University and Divisin level. An instructr may nt remve these sectins. All instructrs must make these plicies unquestinably clear t students by psting related annuncements t the nline classrm.} Please review the plicies and prcedures f Graduate Prfessinal Studies, fund at We wuld like t highlight the fllwing: Learning Disabilities If yu are a student with a dcumented disability n recrd at Brandeis University and wish t have a reasnable accmmdatin made fr yu in this curse, please cntact me immediately. Academic Hnesty and Student Integrity Academic hnesty and student integrity are f fundamental imprtance at Brandeis University and we want students t understand this clearly at the start f the term. As stated in the Brandeis Rights and Respnsibilities handbk, Every member f the University Cmmunity is expected t maintain the highest standards f academic hnesty. A student shall nt receive credit fr wrk that is nt the prduct f the student s wn effrt. A student's name n any written exercise cnstitutes a statement that the wrk is the result f the student's wn thught and study, stated in the students wn wrds, and prduced withut the assistance f thers, except in qutes, ftntes r references with apprpriate acknwledgement f the surce." In particular, students must be aware that material (including ideas, phrases, sentences, etc.) taken frm the Internet and ther surces MUST be apprpriately cited if quted, and ftnted in any written wrk turned in fr this, r any, Brandeis class. Als, students will nt be allwed t cllabrate n wrk except by the specific permissin f the instructr. Failure t cite resurces prperly may result in a referral being made t the Office f Student Develpment and Judicial Educatin. The utcme f this actin may invlve academic and disciplinary sanctins, which culd include (but are nt limited t) such penalties as receiving n credit fr the assignment in questin, receiving n credit fr the related curse, r suspensin r dismissal frm the University. Further infrmatin regarding academic integrity may be fund in the fllwing publicatins: "In Pursuit f Excellence - A Guide t Academic Integrity fr the Brandeis Cmmunity", "(Students') Rights and Respnsibilities Handbk", AND " Graduate Prfessinal Studies Student Handbk". Yu shuld read these publicatins, which all can be accessed frm the Graduate Prfessinal Studies Web site. A student that is in dubt abut standards f academic hnesty (regarding plagiarism, multiple submissins f written wrk, unacknwledged r unauthrized cllabrative effrt, false citatin r false data) shuld cnsult either the curse instructr r ther staff f the Rabb Schl Graduate Prfessinal Studies. University Caveat The abve schedule, cntent, and prcedures in this curse are subject t change in the event f extenuating circumstances. 13