RESEARCH ARTICLES A Quantitative Assessment of a Meicinal Chemistry Problem-base Learning Sequence Anrew A. Webster, PhD an Robert M. Riggs, PhD McWhorter School of Pharmacy, Samfor University Submitte August 29; accepte January 12, 2006; publishe August 15, 2006. Objectives. To establish an aggressive problem-base learning (PBL) format for the meicinal chemistry course an assess the outcomes of stuent learning. Methos. To assess learning in the new format, precourse an postcourse examinations were given to stuents enrolle before an after problem-base learning was implemente, an appropriate statistical analyses were conucte. Results. The PBL cohort i not learn the same amount of factual content yet performe the same on higher-orer thought questions as the non-pbl cohort. Conclusions. Problem-base learning may not be the ieal metho for teaching meicinal chemistry. This may be ue to several factors incluing: stuent learning type, the lack of a cognitive framework for learning in the basic sciences, an time constraints. Keywors: problem-base learning, meicinal chemistry, assessment INTRODUCTION Problem-base learning (PBL) was first implemente in a meical eucation curriculum by Toronto s McMaster University in the late 1960s. Numerous efinitions of problem-base learning aboun. Both Albanese an Anerson-Harper provie excellent reviews of PBL an PBL in pharmacy eucation respectively. 1,2 Over the last several years, the McWhorter School of Pharmacy has pursue a concerte course of action to implement active learning an problem-base learning strategies across the octor of pharmacy curriculum. As part of The Pew Charitable Trusts fune initiative in PBL at Samfor University, 13 core curriculum courses, covering all or part of 54 course creits an representing 48% of the pharmacy curriculum s iactic offerings, were reesigne into a PBL format an elivere as such. Beyon the Pew supporte initiatives, several aitional core an elective courses were evelope an elivere in a PBL-like format. The 2-semester meicinal chemistry sequence was part of this reesign initiative. Over the 5 years leaing up to the PBL initiative, the instructors incorporate increasing amounts of active learning within the course sequence. The out-of-class Corresponing Author: Anrew A. Webster. Aress: McWhorter School of Pharmacy, Samfor University, 800 Lakeshore Drive, Birmingham, AL 35229. Tel: 205-726- 2267. Fax: 205-726-2088. E-mail: aawebste@samfor.eu 1 problems require the application of the concepts central to meicinal chemistry in the esign of novel rug molecules. These exercises increase to 5 group exercises per semester in the year prior to the intentional restructuring of the course sequence. The main purpose ha been to reinforce existing course content rather than create portals for new learning. The pharmacy eucation literature is replete with examples of case-base teaching. Currie et al use patient case stuies in a meicinal chemistry course. 3 The cases were evelope an jointly facilitate with pharmacotherapeutics faculty members. The stuents worke in groups an were quizze at the en of the group session. The authors cite evience that the stuents learne meicinal chemistry better with this approach. No quantitative assessment of examination scores was given. Alsharif et al evelope a rubric as a means of teaching meicinal chemistry in a case-base manner. 4 The rubric facilitate the stuents approach to the problem. The rubric was use to solve problems on homework assignments an on examinations. A critique of this particular rubric is twofol: pharmacotherapeutics is emphasize over meicinal chemistry an elements of the rubric can not be aresse by the consieration of the rug structures in question. The closest example to the approach we use is that of Herrier et al who use groups to work through cases to teach pharmacology an meicinal chemistry. 5 The courses were separate iactic courses. Lectures
were use at the beginning of the course for introuctory purposes. The authors conclue that there was an improvement in knowlege of pharmacology but no real improvement was seen in meicinal chemistry knowlege. Overall the stuents felt that the course improve their self-learning skills. In summary, the use of case stuies in teaching meicinal chemistry has merit in encouraging stuents to apply their knowlege, thus reinforcing learning. Our goal was to enhance learning of meicinal chemistry content an principles that can be applie to solving pharmacotherapeutic issues in the therapeutics courses an in practice. Our central premise in the esign of the new syllabus was to incorporate group resolution of problems in the classroom to provie a founation for learning. This is a significantly ifferent approach than previously employe. To this en we esigne new problems along with utilizing resources alreay in print, especially the Meicinal Chemistry Case Stuy Workbook. 6 Stuent groups were require to maintain a course portfolio each semester which inclue copies of their group problems an presentations, along with peer an selfassessments. 2 METHODS Because the actual conversion to an active-learning format i not take place until the final year of the Pew grant, we were able to perform a pre/post comparative evaluation between a non-pbl cohort an the first PBL cohort. The emographics of both cohorts were similar. The non-pbl cohort (N 5 102) was 78% female, 84% Caucasian, with an average age of 24.4 years. The PBL cohort (N 5 95) was 69% female, 84% Caucasian, with an average age of 24.2 years. A 50-question multiplechoice examination covering the basic tenets of meicinal chemistry was prepare. The stuents were not require to participate in the evaluation. The content topics were taken from across the 2 semester sequence an range from electrolyte calculations an molecular weight, to utilization an application of structure activity relationships towar the resolution of therapeutic problems. Our goal was to assess the outcomes of stuent learning after the first iteration of this format change. Stuents were aske not to prepare for the examination an stuent anonymity was insure. The examination was aministere over a 2-hour perio. Pretests were given within 1 week of the start of the first semester an the posttests were aministere within 2 ays of the en of the secon semester of the meicinal chemistry course sequence. Statistical analyses of the examination scores were conucte using the SPSS software package. Comparisons of pretest versus posttest scores for each cohort were conucte using a epenent sample 2-taile t test with a 5 0.05. Comparisons were conucte across the cohorts for pretest versus pretest, posttest versus posttest, an posttest/pretest ifference versus posttest/pretest ifference using inepenent sample 2-taile t tests. A Bonferroni ajustment to a 5 0.01 was mae for each of the inepenent t tests since 3 tests woul be conucte on each of the cohorts. Fifteen questions that require extene thought were selecte an the percents of correct responses for each question from the pretest an posttest for both cohorts were tabulate. Extene thought questions were efine by those questions that require the stuents to use information in new situations or make inferences from that information to solve problems. These skills are efine at the application an analysis levels of Bloom s taxonomy. Analyses of the respective percentages of correct responses were conucte for the examination scores in the same manner as escribe above. RESULTS The posttest scores for the non-pbl cohort an PBL cohort, respectively, were significantly higher than the pretest scores base on the significant p values an the 95% confience interval of the ifference between the means (Table 1). A post hoc power analysis was not conucte because of the large sample size for each test. The comparison across the non-pbl cohort an PBL cohort, respectively, of the pretest scores reveale a statistically non-significant ifference between the scores base on the non-significant p value an the 99% confience interval between the means (Table 2). The comparison across the non-pbl cohort an PBL cohort, respectively, of the posttest scores reveale a statistically significant ifference between the scores base on the significant p value an the confience interval. This result was confirme by the posttest-pretest ifference of scores comparison across the cohorts. The p value an the confience interval for the test were significant (Table 2). A post-hoc power analysis was not conucte because of the large sample size for each test. The posttest percent of correct responses for the extene thought questions for non-pbl cohort an PBL cohort, respectively, were significantly higher than the pretest percent of correct responses base on the p value an the 95% confience interval (Table 3 an 4). Since n 5 15, a posthoc power analysis was conucte. 7 Cohen s was foun to be 3.42, the non-centrality parameter,, was 13.24, which gave a power.0.999 for the non- PBL epenent sample t test. For the PBL cohort: Cohen s
Table 1. Comparison of Examination Scores of Pharmacy Stuents Enrolle in a Meicinal Chemistry Course Before an After Problem-base Learning Was Implemente Cohort Pretest Score,* Mean (SD) Posttest Score, Mean (SD) Difference in Mean (SD) py 95% CI of the Difference Between the Means Non-PBL (n 5 94) 15.6 (3.8) 28.1 (5.8) 12.4 (7.4),0.05 10.9 to 13.9 PBL (n 5 79) 16.1 (3.5) 24.6 (4.7) 8.6 (6.3),0.05 7.2 to 10.0 *Out of a possible score of 50 ythe SPSS software returne a p value of 0 for the respective t values. The p values were estimate using a erivation of the t-istribution function for large egrees of freeom. 13 The erivation gave a more conservative p value when compare to using the z tables for a normal istribution. (a 5 0.05) PBL 5 problem-base learning 5 0.855, 5 3.31, power 5 0.74. The power of the aforementione test was slightly lower than the preferre power of 0.80. The pretest versus pretest, posttest versus posttest, an the posttest-pretest ifference results, respectively, were statistically nonsignificant base on the p values an confience intervals (Tables 3 an Table 4). The Levene s test for equality of variances results i not emonstrate a statistically significant ifference between the group variances within each of the respective inepenent sample t tests. This provies a quality assurance measure that each t test gave a statistically vali result. DISCUSSION For the non-pbl cohort the significant average increase in raw score for the pre to the post examination was 12.4. The PBL cohort emonstrate a significant increase of 8.6. There was also a significant increase of 18.3 of percent correct responses from pre to the post examination for the extene thought questions for the non-pbl cohort. The PBL cohort ha a significant increase of 16.7. These results suggest that learning occurre in both cohorts (Table 1, Table 3). Comparison of the pretest results shows no ifferences between the cohorts. Somewhat surprising was the statistical ifference observe in comparing the non- PBL cohort posttest results versus the PBL cohort posttest results (Table 2) an the lack of significant ifference in extene thought between the cohorts (Table 3). This suggests that the significant ifference in the entire examination seen between the cohorts exists in the basic content an fact portion of the sequence. A number of consierations may explain this result. The review by Albanese an Mitchell seems to support the ifference foun between the non-pbl/pbl cohort test results in Table 2 in that the PBL cohort appeare to learn less of the course content. The reason may be twofol: (1) insufficient evelopment of a cognitive framework for basic science; an (2) approximately 20% more time is require to cover content in a PBL course than when traitional course elivery methos are use. 1 The lack of a significant ifference foun in the extene thought questions (Table 3) is again supporte by Albanese et al. They conten that PBL promotes backwar reasoning, a term coine by Gilhooly, 8 which may interfere with efficient problem solving. This notion couple with an insufficient evelopment of Table 2. Pretest an Posttest Comparisons Across Cohorts Test Cohort n* Raw Scores,y Mean (SD) Mean Diff. p 99% CI of Difference Between Means Pretest vs. Pretest PBL 98 16.1 (3.5) 0.593 0.251 0.746 to 1.93 Non-PBL 100 15.6 (3.7) Posttest vs. Posttest PBL 79 24.6 (4.7) 3.42 0.0003 5.54 to 1.30 Non-PBL 94 28.1 (5.8) Posttest/Pretest Difference vs. Psttest/Petest Difference PBL 79 8.87 (6.0) 3.12 0.003 5.77 to 0.467 Non-PBL 94 12.0 (7.2) *Differences ue to voluntary stuent participation yout of a highest possible score of 50 PBL 5 problem-base learning 3
Table 3. Comparisons of Percentage of Correct Responses on Extene Thought Questions (n 5 15) Cohort Difference in Means (SD) P* 95% CI of the Difference Between the Means PBL 16.7 (19.5) 0.005 5.9 to 27.5 Non-PBL 18.3 (20.7) 0.004 6.8 to 29.8 PBL 5 problem-base learning a cognitive framework may explain the observe results in Table 3. Prior to the course sequence reesign, active learning ha alreay been inclue in about 20% of the meicinal chemistry course sequence. The course sequence previously containe many of the tenets of problem-base learning. These active-learning strategies were successfully employe to reinforce concepts an principles. Our successful utilization of active-learning strategies in the past encourage us to push the envelope in terms of their becoming the primary metho of elivery. To this en, well over 50% of the course sequence content was presente in a problem-base fashion, requiring the stuents to iscover information rather than reinforce existing knowlege. This may have been too much of a change for the majority of our stuents in light of the curriculum an their personalities. The profession of pharmacy attracts a certain type of personality. 9 In a recent McWhorter School of Pharmacy survey, of 4 possible subtypes ientifiable on the Myers- Briggs Personality Inicator, one, sensing/juging was ientifie in over 90% of the stuent population. 10 A number of broa generalizations about people with this attribute ( sensing/juging ) can be obtaine from Keirsey. 11 These iniviuals learn best by experiencing, practicing, an memorizing. Sensing/jugling iniviuals learn best when there is a focuse an structure learning environment. They prefer to listen an observe, eg, watch how other people o things, listen to a lecture or presentation, an take notes, especially when expectations, goals, an stanars are clearly presente. As learners, sensing/juging iniviuals are less intereste in abstract theories than in factual or practical information. Loose, unstructure teaching with unclear outcomes or with a high egree of experimentation, personal interaction, theory or play oes not work well for them. Therefore, the open-ene PBL format seems to be a less than optimal teaching methoology for this personality type. Along with meicinal chemistry, 2 other sequences in the secon-professional year were converte to problembase learning as part of the Pew grant. The 3 sequences ran concurrently with the stuents maintaining the same groups in each course sequence. This create an environment with a significant change in workloa an an increase nee for appropriate time management skills. The 3 sequences use varying styles, formats, an requirements, leaing to some stuent confusion an an inhibition of learning. Stuent focus group interviews an stuent-le quality teams (LEARN teams) were utilize to obtain this information. 12 In summary, the fining of ecrease content learning couple with the nees of the sensing/juging personality type for a structure-learning environment an a nee for a more evelope cognitive framework for learning preclues us from utilizing a PBL approach as the ominant process for classroom presentation. Arme with this information, substantial changes to the Meicinal Chemistry sequence from 50% pure PBL to a hybri approach has been evelope. The following are several key features of the hybri approach: To enhance builing a cognitive framework, early on in the course sequence the stuents are taught how to moel efficient problem solving. The steps are: s rea the problem; s comprehension of the terminology; s what information is explicitly state?; s what information is implicit? s review of knowlege base relevant to the problem; Table 4. Comparisons of Percentage of Correct Responses on Extene Thought Questions (n 5 15) Test Cohort Mean (SD) Mean Diff. P* 99% CI of Difference Between Means Pretest vs. Pretest PBL 34.7 (18.8) 1.93 0.789 17.9 to 21.7 Non-PBL 32.8 (20.4) Posttest vs. Posttest PBL 51.4 (20.4) 0.333 0.964 19.7 to 20.3 Non-PBL 51.1 (19.3) Posttest/Pretest Difference vs. Postest/Prestest Difference PBL 16.0 (18.9) 1.20 0.868 21.0 to 18.6 Non-PBL 17.2 (20.3) 4
s introuction to Plan, Do, Stuy, Act (PDSA) cycle; s an then cycle through PDSA until a satisfactory answer is achieve. Traitional course elivery methos intersperse with short active-learning sessions are use to eliver some of the more conceptually ifficult content areas to help buil on the cognitive framework. Short pre-lectures to introuce the stuents to the topic are given before the stuent groups engage in a PBL session. Short impromptu presentations by the groups an a wrap-up by the instructor provie the necessary closure to the process. Drug esign problems are use in both courses of the sequence to engage stuents at the synthesis an evaluation levels of Bloom s cognitive omains. Stuent groups give formal Microsoft Powerpoint presentations on the problems as a means to enhance an reinforce concepts an communication skills. CONCLUSION This approach to using problem-base learning in the Meicinal Chemistry course sequence seems to work as evience by positive feeback from course, instructor, an LEARN team evaluations. While this approach to problem-base learning is not pure PBL, it appears to be a viable metho of active learning for our stuent population. We believe that this metho provies both the content an higher-orer concepts we esire to instill in our stuents. We are currently investigating the stuentlearning outcomes of the current format of course elivery. REFERENCES 1. Albanese MA, Mitchell S. Problem-base Learning: A review of literature on its outcomes an implementation issues. Aca Me. 1993;68:52-81. 2. Cisneros RM, Salisbury-Glennon JD, Anerson-Harper HM. Status of problem-base learning in pharmacy eucation: A call for future research. Am J Pharm Euc. 2002;66:19-26. 3. Currie BL, Chapman R, Christoff J, Sikorski L. Patient-relate case stuies in meicinal chemistry. Am J Pharm Euc. 1994;58: 446-50. 4. Alsharif NZ, Theesen KA, Roche VF. Structurally base therapeutic evaluation: A therapeutic an practical approach to teaching meicinal chemistry. Am J Pharm Euc. 1997;61: 55-60. 5. Herrier RN, Jackson TR, Consroe PF. The use of stuent-centere, problem-base, clinical case iscussions to enhance learning in pharmacology an meicinal chemistry. Am J Pharm Euc. 1997;61:441-6. 6. Currie BL, Roche VF, Zito SW. Meicinal Chemistry Case Stuy Workbook. Baltimore, M: Williams an Wilkins; 1996. 7. Sheskin DJ. Hanbook of Parametric an Nonparametric Statistical Proceures. 2n e. Boca Raton, Fla: Chapman & Hall; 2004:447. 8. Gilhooly K. Cognitive psychology an meical iagnosis. Appl Cognitive Psychol. 1990;4:261-72. 9. Shuck AA, Phillips CR. Assessing pharmacy stuents learning styles an personality types: A ten-year analysis. Am J Pharm Euc. 1999;63:27-33. 10. Myers IB, McCaulley MH. Manual: A Guie to the Development an Use of the Myers Briggs Type Inicator. Palo Alto, Calif: Consulting Psychologists Press, Inc; 1985. 11. Keirsey D. Please Unerstan Me II, Temperament, Character, Intelligence. Del Mar, Cal: Prometheus Nemesis Book Company; 1998. 12. Webster A, Beasley J, McBrie H, Dean J. Utilization of total quality management tools in a meicinal chemistry course. Am J Pharm Euc. 1994;58:451-4. 13. von Mises R. Mathematical Theory of Probability an Statistics. New York, New York; Lonon, UK: Acaemic Press; 1964:412. 5
Appenix 1. Assessment instrument sample questions. 6