Insiue of Saisics and Compuerized Informaion Sysems Faculy of Business Adminisraion Repor on assessmen of SICI oucomes (Augus December 21) Prepared by: Prof. Arnaldo I. Ramos Torres Revised by SICI ABET Commiee on February 23, 211: Dr. Kaherine Franceschi-Díaz Prof. María del R. Rodríguez-Orellana Dr. Rosario Sánchez-Morcilio February 22, 211
Repor on assessmen of SICI oucomes (Augus December 21) Prof. A. Ramos Inroducion During las semeser (Augus December 21) we assessed eigh of he en suden oucomes in our major. This achievemen was possible due o several facors, among which we wan o emphasize he use of course-embedded assessmen mehods, as well as he use of an auomaed ool o suppor he assessmen process. Oucomes number hree and six were no assessed because he courses which suppor hem (SICI 4278 and 4275, respecively) were no offered in ha semeser. Mos oucomes were assessed using course capsone projecs, following our educaional sraegy of Projec-based learning. Oher assessmen mehods used were shor essays, and exi inerviews. These inerviews were conduced by a commiee of a leas wo professors o several graduaing sudens individually. Excep for he exi inerviews, all oher mehods were assessed wihin courses, specifically in he following: SICI 3245, 3255, 415, 425, 4266, 4286, 4465, and 4998. The semeser was very paricular because classes were inerruped several imes, due o a suden srike. The semeser sared by mid Augus and finished by he end of January. Assessmen process for he semeser During he firs half of he semeser we worked wih he recommendaions made in he assessmen repor for he previous semeser (January May 21). Specifically, we worked wih he following: 1. In order o improve assessmen resuls for oucome #2, we emphasized logical sysems design in SICI 425 by changing some eaching sraegies, allocaing more ime o his opic, reviewing preliminary versions of he projecs and providing feedback o sudens. a. We are sill in he process of exposing sudens o some of he componens of a sysem (like screens, repors, and daabases) earlier in he course sequence, using a user-oriened daabase managemen sysem. This should be done in he very firs course ha our sudens ake (SICI 3211). 2. We addressed he dropou issue on SICI 3255 by incorporaing several quizzes hroughou he semeser, and also by promoing suden ineracion among hemselves. a. Addiional suppor o sudens hrough uoring was no possible due o lack of funds.
(A. Ramos) 2 3. The projec descripion for SICI 4998 was ranslaed o English, bu he projec descripions for SICI 425 and 4275 are sill in Spanish. 4. We emphasized in he assessmen insrucions emailed o professors ha he remarks column be used o ener suden feedback, paricularly when he suden did no ge he highes score. 5. Dr. Rosario Sánchez was assigned o keep he assessmen documenaion properly sored, including he assessmen repors, he rubrics (properly filled-ou), he descripion of he paricular assessmen projecs used, and evidence of he suden work. 6. We also emphasized in he assessmen insrucions emailed o professors ha sudens should orally presen heir projecs so ha he evaluaor can beer assess hem. Because projecs are ake-home, i is imporan for he evaluaor o validae he exen o which he suden really prepared he projec himself. Also, in he case of group projecs, presenaions will help he evaluaor assessing he individual conribuion of group members o he projec. 7. SICI 425 was assigned o suppor oucome #1 (Teamwork) insead of oucome #9 (Communicaion). This way he number of courses supporing hese oucomes will be beer balanced. Originally, oucome #1 (Teamwork) was suppored only by SICI 4278, while oucome #9 (Communicaion) was suppored by SICI 3245, SICI 425, SICI 4275 and SICI 4278. 8. SICI 4998 was assigned o suppor oucomes #9 (Communicaion) and #1 (Teamwork). The shor essay ha sudens are preparing a he end of he course provides for assessing boh oucomes. 9. The ables where suden oucomes are mapped o SICI courses, boh requisies and elecives, were updaed based on he recommendaions. On Ocober 28, we emailed he mos recen version of he corresponding rubrics, ogeher wih some insrucions and remarks regarding he assessmen process, o all professors involved in his process. The rubric for SICI 4465 was sen in a separae email on December 4. A memo was sen by he ABET Commiee o SICI professors on Ocober 28, furher emphasizing he insrucions and remarks emailed regarding he assessmen process. By mid February, I received all he rubrics already compleed by he professors, excep for one, ogeher wih remarks from some of he professors regarding heir experience wih he process. The resuls of he individual rubrics were inegraed ino a summary able by oucome (see Appendix 1), based on he oucomes suppored by each rubric. Finally, I analyzed he daa, prepared his repor and presened i o he ABET Commiee 1. 1 I wan o acknowledge he suppor and he conribuions received from he members of he ABET Commiee (Dr. Kaherine Franceschi-Diaz, Prof. María del R. Rodríguez-Orellana, and Dr. Rosario
(A. Ramos) 3 I consider appropriae o emphasize an imporan poin regarding faculy involvemen in our assessmen process. By he dae of his repor we sill have no received he compleed rubric for he assessmen in he course SICI 415. This is he hird semeser ha his course has o be lef ou of his repor: he firs ime (Augus December 29) we received he rubric more han a monh afer we had finished he assessmen repor, he second ime (January May 21) we did no receive he rubric and he hird ime (Augus December 21) we also did no receive he rubric. The ABET Commiee has provided follow-up o he professor in charge of he course, and he siuaion has been raised o he Deparmen Head, bu i has no been correced. Assessmen resuls The following remarks are derived from he daa in he summary able in Appendix 1, ogeher wih he feedback received from some of he professors involved in he assessmen. 1. Mos sudens did very well on he assessmen of suden oucomes. See Appendix 1 for approval percenages for each oucome assessed, boh by mehod and overall. a. The percenage of sudens ha obained a passing score ranged from 87.5% o 1% hroughou he individual oucomes. 2. Oucome #2 (To selec or design a sysem o solve he problems idenified in an operaion), which las semeser go he lowes percenage of suden performance, increased is percenage o 87.5. The wo conribuors o his percenage were SICI 425, wih 85.71, and SICI 4266, wih 1.. a. The previous semeser (January May 21), his oucome go he lowes percenage, wih 64.29, wih SICI 425 having a percenage of 42.86 and SICI 4266 wih 85.71. b. The professor eaching his course (Prof. María del R. Rodríguez Orellana) pu a lo of emphasis in reviewing preliminary versions of he logical design projecs and providing feedback o sudens, which helped hem undersand wha a logical design should accomplish. She also dedicaed more ime o logical design. 3. There was a high number of dropous in SICI 3255 (approximaely 5%). Bu he sudens ha sayed in he course did very well in he assessmen. Alhough we consider dropous o be ou of he scope of he assessmen process, since hese sudens are no included in he assessmen sample, i is an issue ha we need o address anyway. a. The fac ha he semeser had several inerrupions due o a suden srike mus have had an influence in he number of dropous. Sánchez-Morcilio), as well as he suppor and conribuions received from our Deparmen Head (Dr. Pedro Rodríguez) and he SICI professors.
(A. Ramos) 4 b. We addressed he dropou issue on SICI 3255 by incorporaing several quizzes hroughou he semeser, and also by promoing suden ineracion among hemselves. Alhough sudens agreed ha hese acions were very helpful, i was very difficul o find ou o wha exen his is so, because we faced anoher suden srike ha inerruped he semeser several imes. c. Addiional suppor o sudens hrough uoring was no possible due o lack of funds. d. We mus coninue analyzing his siuaion in order o find explanaions and o ake addiional correcive acions o improve i. 4. The number of dropous was also high in SICI 4266. Of he six sudens ha were originally regisered, hree disappeared and one did no deliver he final projec. This siuaion is unusual for his course, so probably he inerrupions due o he suden srike were he cause of his siuaion. Tradiionally in our major, once sudens pass he firs course, hey usually go hrough all SICI courses. Logisic consideraions 1. All professors used he remarks column in he rubrics o ener suden feedback regarding he score assigned o he corresponding characerisic, paricularly when he suden did no ge he highes score. 2. Alhough he logisics for he assessmen process improved, here are some issues ha we sill have o refine: a. Some projec descripions are sill in Spanish (SICI 425, and 4275). They have o be ranslaed o English. b. Two professors emailed he projecs, essays, ec., used for assessmen. This allowed me o elecronically sore he whole package (projecs, projec descripions and rubric) used in he assessmen process as evidence of he assessmen resuls. 3. As menioned before, he professor for SICI 415 did no deliver he rubric. This is he hird ime ha his course has o be lef ou of his repor. Recommendaions 1. Alhough he resuls obained in SICI 425 his semeser showed a significan improvemen over he previous semeser, we should coninue emphasizing logical sysems design in his course, hrough he measures previously menioned: a. Reviewing preliminary versions of he projecs and providing feedback o sudens.
(A. Ramos) 5 b. Allocaing more ime o his opic in SICI 425. c. Exposing sudens o some of he componens of a sysem (like screens, repors, and daabases) earlier in he course sequence, using a user-oriened daabase managemen sysem. This could be done in he very firs course ha our sudens ake (SICI 3211). 2. The dropou issue on SICI 3255 should be addressed o find ou is causes and o ake whaever acions are necessary. The following sraegies mus be iniiaed, or coninued, and we should measure heir effec on dropous: a. Coninue using quizzes. b. Promoe suden ineracion among hem. c. Provide more suppor o sudens hrough uoring. d. Analyze suden characerisics ha may be having an influence in he dropou rae, like suden habis and ime lag o acquire course maerials. 3. SICI 4266 has o be closely observed regarding dropous o see if he high dropou rae was due o he suden srike or o oher reasons. 4. SICI 4465 should be assigned o suppor oucome #9 (To communicae effecively wih a range of audiences.), besides supporing oucome #5 (To undersand he impac ha organizaional, local and global environmens have in he implemenaion and managemen of informaion sysems.) and oucome # 8 (To recognize he imporance of ehical values and inerpersonal relaionships in an informaion sysems professional.). In his course sudens prepare a research paper on legal issues of compuer echnology, which is very suiable o assess wrien communicaion. 5. The ABET Commiee should coninue promoing he following logisic maers: a. Tha projec descripions ha are sill in Spanish be ranslaed o English (SICI 425, and 4275). b. Tha all professors fill he remarks column in he rubrics. This column should be used o ener suden feedback regarding he score assigned o he corresponding characerisic, paricularly when he suden did no ge he highes score. c. Tha all he assessmen relaed documens be elecronically sored in a cenral locaion. This will allow o elecronically sore he whole package (projecs, projec descripions and rubric) used in he assessmen process as evidence of he assessmen resuls.
(A. Ramos) 6 Addiional remarks d. Tha all professors deliver he projec descripions, ogeher wih he corresponding rubric. e. Tha sudens orally presen heir projecs so ha he evaluaor can beer assess i. Because projecs are ake-home, i is imporan for he evaluaor o validae he exen o which he suden really prepared he projec himself. Also, in he case of group projecs, presenaions will help he evaluaor o assess he individual conribuion of group members o he projec. f. Managemen mus ake affirmaive acions o ensure ha all professors fulfill heir assessmen responsibiliies. 1. Rubrics coninue o be a useful ool o faciliae he assessmen process. They are easy o use, and reduce he probabiliy of daa enry and calculaion errors. They simplify he daa collecion, he calculaion of suden scores, as well as he calculaion of scores relaed o he specific oucomes being assessed. 2. The use of course-embedded assessmen allowed assessing suden oucomes wihou an addiional burden on sudens and professors. All ha professors had o do was o make sure ha he mehod requiremens suppor he characerisics conained in he corresponding rubric. 3. The main conribuors o he assessmen resuls obained are he conens of he courses and he educaional sraegies used by professors. Bu we believe hese resuls can also be aribued o oher facors, like he following: a. By he ime he assessmen was performed, mos sudens no performing well in he course had already abandoned i. b. Sudens knew beforehand all he characerisics o be assessed in he paricular mehod used. c. The mehods used were mosly ake-home projecs, wih reasonable ime for preparaion. Projecs usually raise suden ineres and performance. d. Projecs used resemble he work performed by IS professionals in indusry and governmen, which probably conribued addiionally o raise suden ineres. e. All sudens assessed belong o he SICI major, so i is reasonable o assume ha hey pu a special ineres in he courses, as well as he assessmen mehods.
(A. Ramos) 7 Mapping of suden oucomes o SICI courses, exi inerview and ABET oucomes As par of our coninuous improvemen process, Appendixes 2 and 3 presen a revised version of he ables mapping suden oucomes o SICI courses, exi inerview and ABET oucomes. The revision was based on he resuls obained from he assessmen process, as well as on he recommendaions we are making in his repor. Table mapping objecives, oucomes and performance crieria ( Consiuional able ) As par of our coninuous improvemen process, Appendix 4 conains a revised version of he SICI Consiuional Table, specifically in he column conaining he SICI courses ha will suppor each oucome. The revision was based on he recommendaions we are making in his repor. Graph of all oucomes (approved vs. no approved) Appendix 5 presens graphs showing he relaionship beween he number of sudens approving and no approving each oucome.
(A. Ramos) 8 Appendix 1 Summary of assessmen resuls
(A. Ramos) 9
(A. Ramos) 1
(A. Ramos) 11
(A. Ramos) 12
(A. Ramos) 13 Appendix 2 Mapping of suden oucomes o SICI (required) courses, exi inerview and ABET oucomes
(A. Ramos) 14
(A. Ramos) 15 Appendix 3 Mapping of suden oucomes o SICI (elecive) courses, exi inerview and ABET oucomes
(A. Ramos) 16
(A. Ramos) 17 Appendix 4 Table mapping objecives, oucomes and performance crieria ( SICI Consiuional Table )
(A. Ramos) 18 Universiy of Puero Rico Faculy of Business Adminisraion, Informaion Sysems Major SICI Consiuional Table Educaional objecives, suden oucomes and performance crieria (including modificaions) Prof. A. Ramos, February 211 Educaional objecives (Wha he graduae mus accomplish in he firs few years (3 o 5) of his professional career.) 1. To implemen and manage he developmen of informaion sysems in an organizaion. Suden oucomes (Wha he suden mus know, value, and be able o do, a he ime of his graduaion, which will enable him o achieve he educaional objecives.) 1. To analyze an operaion wihin an organizaion, idenify problems and make recommendaions o solve hese problems. Performance crieria (Wha he suden mus be able o do, or o produce, in order o show ha he complies wih he learning oucomes.) 1. To prepare funcional, echnical, and oher requiremens for an informaion sysem ha solves he problems idenified in an operaion. SICI Courses supporing he oucome ABET Oucomes suppored 425 A, B, J 2. To selec or design a sysem o solve he problems idenified in an operaion. 2. To design he componens of an informaion sysem based on he funcional requiremens prepared for ha sysem. 415,425, 4266, 445(E) A, C, J 3. To plan and supervise he implemenaion of a sysem ha solves he problems idenified in an operaion. 3. To develop a plan o implemen an informaion sysem, including he phases and aciviies ha his process requires. 4278 A, C, J 2. To apply echnological, analyical, and criical hinking skills in he soluion of problems relaed o informaion sysems in organizaions. 4. To use curren echniques, skills, ools and bes pracices o design, implemen and manage informaion sysems. 4. To idenify he hardware, sofware and daa communicaion componens needed o operae an informaion sysem, and o inegrae hem in a echnological soluion. 5. To code, es, and documen compuer programs o perform he auomaed processes ha compose a sysem, using modern programming ools. 3245 A, C, I 3255, 4266, 445(E), XXX2 (E) 6. To design a properly normalized daabase based on requiremens prepared by sysems analyss or by users. 415
(A. Ramos) 19 Universiy of Puero Rico Faculy of Business Adminisraion, Informaion Sysems Major SICI Consiuional Table Educaional objecives, suden oucomes and performance crieria (including modificaions) Prof. A. Ramos, February 211 Educaional objecives (Wha he graduae mus accomplish in he firs few years (3 o 5) of his professional career.) Suden oucomes (Wha he suden mus know, value, and be able o do, a he ime of his graduaion, which will enable him o achieve he educaional objecives.) Performance crieria (Wha he suden mus be able o do, or o produce, in order o show ha he complies wih he learning oucomes.) 7. To idenify he componens of a communicaions nework, menion he main characerisics of hese componens, and o presen he way hey inegrae ino a nework. SICI Courses supporing he oucome 4286, 4285(E) ABET Oucomes suppored 3. To ake ino consideraion he conex in which informaion sysems operae, when implemening and managing hese sysems. 5. To undersand he impac ha organizaional, local and global environmens have in he implemenaion and managemen of informaion sysems. 8. To analyze adminisraive, organizaional, local and global aspecs ha affec informaion sysems, and o define sraegies o deal wih hese aspecs. 4278, 4266, 445(E), 4465(E) G, J 6. To value he proecion of informaion sysem resources in an organizaion, and o idenify ways in which his proecion can be achieved. 9. To prepare a disaser recovery plan for he informaion sysem operaions of an organizaion. 4275 E, J 4. To mainain his professional experise by updaing his knowledge in echnology and informaion sysems. 7. To be aware of he high level of change in he Informaion Sysems field, and he need o use differen mechanisms o updae his knowledge. 1. To ge relaed wih professional associaions, publicaions and Coninuing Educaion alernaives in he geographic area where he lives or works. 4278, Exi H 5. To perform his funcions showing respec and appreciaion for ehical values, inerpersonal relaionships, communicaion, and eam 8. To recognize he imporance of ehical values and inerpersonal relaionships in an informaion sysems professional. 11. To idenify and evaluae ehical and inerpersonal relaionship aspecs relaed o informaion sysem professionals. 4275, Exi, 4465(E)499 8(E) E
(A. Ramos) 2 Universiy of Puero Rico Faculy of Business Adminisraion, Informaion Sysems Major SICI Consiuional Table Educaional objecives, suden oucomes and performance crieria (including modificaions) Prof. A. Ramos, February 211 Educaional objecives (Wha he graduae mus accomplish in he firs few years (3 o 5) of his professional career.) work. Suden oucomes (Wha he suden mus know, value, and be able o do, a he ime of his graduaion, which will enable him o achieve he educaional objecives.) 9. To communicae effecively wih a range of audiences. 1. To funcion effecively in eams seeking o accomplish a common goal. Performance crieria (Wha he suden mus be able o do, or o produce, in order o show ha he complies wih he learning oucomes.) 12. To prepare wrien repors and oral presenaions relaed o informaion sysem opics. 13. To demonsrae abiliy o work effecively in askoriened groups, like informaion sysem projec eams. SICI Courses supporing he oucome 3245, 4275, 4278, 4465(E), 4998(E) 425, 4278, Exi4998(E) ABET Oucomes suppored F D
(A. Ramos) 21 Appendix 5 Graph of oucome resuls
(A. Ramos) 22 Oucome resuls (All) Oucome 1 Oucome 9 S u d e n s 18 16 14 12 1 8 6 4 2 Approved No approved Oucome 8 Oucome 7 Oucome 6 Oucome 5 Oucome 4 Oucome 3 Oucome 2 Oucome 1 Analyze curren operaions (Oucome 1) S u d e n s 14 12 1 8 6 4 2 14 Approved No approved
(A. Ramos) 23 Selec or design a sysem (Oucome 2) S u d e n s 14 12 1 8 6 4 2 14 Approved 2 No approved Plan and supervise sysems implemenaion (Oucome 3) S u d e n s 1.8.6.4.2 Approved No approved
(A. Ramos) 24 Techniques, skills, ools and bes pracices (Oucome 4) S u d e n s 6 5 4 3 2 1 54 Approved 4 No approved Organizaional, local and global environmen (Oucome 5) S u d e n s 14 12 1 8 6 4 2 13 Approved No approved
(A. Ramos) 25 Proecion of IS resources (Oucome 6) S u d e n s 1.9.8.7.6.5.4.3.2.1 Approved No approved Lifelong learning (Oucome 7) S u d e n s 5 4.5 4 3.5 3 2.5 2 1.5 1.5 5 Approved No approved
(A. Ramos) 26 Ehical values/inerpersonal relaionships (Oucome 8) S u d e n s 16 14 12 1 8 6 4 2 16 2 Approved No approved Communicaion (Oucome 9) S u d e n s 35 3 25 2 15 1 5 32 Approved No approved
(A. Ramos) 27 Teamwork (Oucome 1) S u d e n s 2 18 16 14 12 1 8 6 4 2 2 Approved 1 No approved