Mental Health Evaluations Used in Occupational Therapy

Mental Health Evaluations Used in Occupational Therapy (evaluations assessments surveys) Barbara J. Hemphill To teach students occupational therapy evaluation techniques (or methods) in psychiatry evaluations currently being used in the field were identified. Three consecutive surveys were sent to occupationaltherapy faculty clinzcal supervisors and clinicians to ascertain which evaluation techniques were being taught those which the students were expected to know about in their fieldwork experiences and those which were practiced in the field. On an average occupational therapy schools were teaching four evaluation tools. Affillating students were expected to know about 3 of 24 evaluation tools listed in the survey and none of the evaluation techniques were being practiced significantly by occupational therapists. Since 75 percent of the therapists reported that they developed their own evaluation tools it was suggested that instructors be familiar with the existing evaluation tools and teach students the methodology required for developing their own evaluation tools (or instruments). Barbara J. Hemphill M.S. OTR is an Assistant Professor at C leveland State University Cleveland Ohio. A ssessment and evaluation tools used in psychiatric occupational therapy that had been published or had been presented at workshops and conferences were identified for a O-week course given at Cleveland State University. n deciding which tools to include in the course the author first conducted two surveys to find out which tools were taught in other curricula and which ones the students were expected to know about or be proficient in using during their fieldwork experience. For the purpose of these surveys it was irrelevant to identify which tools were assessments and which were evaluations. Therefore for lack of a word that was encompassing the term evaluation was used. At the time this material was being identified for the course the newly formed Mental Health Specialty Section was meeting at the 1977 Annual Conference of the American Occupational Therapy Associa tion (AOTA). The most serious concern expressed at this meeting was the need for client evaluation methods or techniques. The specialty section steering committee then planned an institute for the 1978 AOTA conference designed to introduce three newly developed evaluation tools to instruct participants to their use and to begin the process of developing norms and standardization for these tools. Before the plan could be carried out tools traditionally used in practice had to be identified and judged on their need for further development and standardization (1). At the 1978 AOTA conference when the steering committee discovered that such a survey had been completed anq that another was in progress it recommended that do a third survey to find out where clinicians learned about evaluation tools or instruments-in school or through continuing education courses. The answers to this question could provide information not only about which tools needed to be developed and standardized but also about the effectiveness of introducing new evaluations through continuing education modes. Continuing education was defined as a method an individual chooses in order to further his or her knowledge or skill through any means such as reading and/or attending workshops conferences and institutes. The purpose of this article is to report the resul ts of three consecutive surveys designed to answer the following questions:. Which psychiatric are taught in occupational The Amerzcan Journal of Occupational Therapy 721

Figure 1 Frequency distribution to show the number of occupational therapy and psychological evaluations taught by anyone faculty member. 10 9 8 7 6 5 4 3 2 ".." /..../1.. Psych Evaluations OT Evaluations ~ ~ ~ ~ ~ ~.. to ".. o abc d e g h k m n 0 p q s u Faculty Member v w x y z aa bb cc dd therapy curricula; 2. Which psychiatric were the students expected to know about or be proficient in using upon arrival at the fieldwork center; and 3. Which psychiatric are used in clinical practice? Method For the first survey faculty at 50 professional occupational therapy schools were asked to identify which psychiatric occupational therapy evaluations were taught in the psychosocial component of the curriculum. The 50 instructors were asked to check which of 17 evaluation tools were taught and to add any that were not shown on the list. They were also asked to indicate whether they taught interviewing methods and/or group evaluation methods. Thirty of the 50 instructors or 64 percent completed the survey. The second survey contained the 15 evaluations listed on the first survey plus 9 others gathered from the results of the first survey. To identify the population a letter was sent to the clinical coordinator of 50 professional curricula requesting a list of psychiatric clinical centers. From the 26 lists returned 499 clinical supervisors were identified. Since a center may have more than one supervisor the supervisors were requested to complete the survey individually. One hundred and sev enty four supervisors or 35 percent returned the survey. The supervisors were asked:. to report whether students were expected to be proficient in the following evaluation methods-individual task group interview and chart review; 2. to add evaluations not listed and indicate whether they were developed by an occupational therapist; and 3. to iden tify their professional affiliation. Because some schools use nontraditional settings for fieldwork experience it was thought that if a significant number of students were supervised by nonoccupational therapists this would affect the results of the survey. However 97 percent or 169 of the returns were from registered occupational therapists. 722 November 1980 Volume 34 No. 11

Table 1 Psychiatric Evaluations Taught by Occupational Therapy Faculty Evaluation Percentage Using nstrument nterview 94' Group evaluation 61 Mosey Survey of living task skills 84' Group interaction skill survey 77 Activities of daily living 84 Child care survey 39 Work survey 74' Recreation survey 68 Activity configuration 71' Diagnostic Test Battery 58 Brayman and Kirby (Comprehension OT Evaluation) 10 Buck-Magazine Picture Collage 45 Lawn and O'Kane 42 Shoemyn 29 Fidler Battery 87' Azima Battery 74' Goodman Battery 52 Evaluationsare listed as they appeared in the first survey (N =30). 'Those evaluations that received higher than 70 percent agreement. The third survey was sent to occupational therapists who reported membership in the Mental Health Specialty Section on their 1978 registration. Of the 2793 questionnaires sent 241 or 12 percent were returned. Therapists were requested:. to indicate the evaluation methods they used-individual task group interview and chart review; 2. to check which of the 24 listed evaluations they used; 3. to report whether they learned about the evaluation through basic professional education or through continuing education; 4. to indicate whether a formal evaluation procedure other than the 24 on the list was used; and Table 2 The Frequencywith Which the Student Was Expected to Demonstrate Knowledge Proficiency or Neither by Clinical Supervisors (N =174) EXPECTED EVALUATON Proficient Knowledge Neither MOSEY Survey of task skill 44 71' 59 Group interaction 45 73' 56 Activities of daily living 45 66 g Child care survey 9 50 l~t Work survey 14 77. 2t Recreation survey 12 80 m ACTVTY CONFGURATON 24 73 77t DAGNOSTC TEST BATTERY 12 49 113t COMPREHENSON OT EVALUATON 19 71 84t BUCK-MAGAZNE PCTURE COLLAGE 19 35 120t LAWN AND O'KANE 3 26 145t SHOEMYN 7 37 130t FDLER BATTERY 7 70 97t AZMA BATTERY 7 64 102t GOODMAN BATTERY 14 52 108t NTEREST CHECKLST 57 61t 56 MOORHEAD OCCUPATONAL HSTORY 16 42 116' ADOLESCENT ROLE ASSESSMENT 9 50 115t LAFAYETTE CLNC BATTERY 3 30 141t OBJECT HSTORY 10 37 127t PLAY HSTORY 9 44 121t SOCAL ADAPTABLTY TEST 10 31 133t OCCUPATONAL BEHAVOR 9 42 123t RATNG SCALES SELF-PUZZLE 9 31 134t Note: 'p <.01 tp <.05 5. to add any evaluation tools that group evaluation methods. Mosey's were not shown on the list. Survey of Living Task Skills except for the Child Care and Recreation Results portions the Azima Battery. and the The results from the first survey are Fidler Battery were evaluations illustrated in Table. Of the 30 taught most often. The Compreinstructors responding 94 percent hensive Occupational Therapy were teaching interviewing meth Evaluation (COTE) was taught least ods and 61 percent were teaching often. The respondents added an The Ame1'1can Journal of Occupational Therapy 723

unexpected total of 72 psychological tests as being "other evaluations" they taught to occupational therapy students. A frequency distribution was done to compare the number of occupational therapy evaluations taught with the number of psychological tests taught at anyone curriculum (see Figure ). The mean number of taught by one faculty was 4. The mode or number of occupational therapy evaluations taught most often was 5. One faculty reported teaching none whereas two faculty reported teaching 7 evaluations. With regard to the number of psychological tests taught by anyone faculty the mean was 2 and the mode was zero. Seven faculty reported teaching none whereas one reported teaching 8 psychological tests. That no two faculty members were teaching the same psychological tests (except for the Draw-a Person and the House-Tree-Person) was striking. The results from the second survey show that clinical supervisors expected students to be proficien tin the following: individual task 70 percent; group evaluation 59 percent; interviewing 63 percent; and chart review 55 percent. All respondents expected students to know a formal evaluation procedure. The data in Table 2 record the frequency with which clinical supervisors responded according to the level of performance expected from students. A chi-square goodness of fit was calculated to ascertain whether the differences in responses resulted by chance. n general there was a higher significant number of supervisors who indicated neither knowledge nor proficiency in any of the evaluations listed. Also a higher significant number of supervisors expected students to know about Table 3 Frequency Distributions of Clinicians Using Evaluations in Practice (N =241) EVALUATON Learned About n Continuing Education or n Basic Professional Education Using in Continuing Practice Education Using School Using MOSEY Survey of task skill 32 46 22 103 12 Group interaction 31 34 17 121 14 Activities of daily living 24 39 17 99 7 Child care survey 7 37 5 36 2 Work survey 24 31 17 ~ 7 Recreation survey 26 35 ~ J.1 7 ACTVTY CONFGURATON 31 26 12 91 19 DAGNOSTC TEST BATTERY 5 7 o 45 5 COMPREHENSON OT EVALUATON 39 62 29 41 10 BUCK-MAGAZNE PCTURE COLLAGE 31 48 24 43 7 LAWN AND O'KANE o 5 o 14 o SHOEMYN BATTERY 12 10 5 31 7 FDLER BATTERY 10 22 5 118 5 AZMA BATTERY 7 16 2 87 5 GOODMAN BATTERY 12 29 7 65 5 NTEREST CHECKLST 53 55 24 128 29 MOORHEAD: OCCUPATONAL HSTORY 14 22 12 38 2 ADOLESCENT ROLE ASSESSMENT 19 27 12 29 7 LAFAYETTE CLNC BATTERY 2 9 2 12 o OBJECT HSTORY 4 12 2 26 2 PLAY HSTORY 10 19 5 29 5 SOCAL ADAPTABLTY TEST 2 2 2 9 o OCCUPATONAL BEHAVOR RATNG SCALES 5 7 5 17 o SELF-PUZZLE 4 17 2 19 2 of evaluating patients; 43 percent used group evaluation and 35 per cent used observation. All respon dents reported using a formal evalua tion procedure. The same 72 psy chological tests identified through the first survey as being taught to students were added by the respon dents. Again no two therapists used the same psychological tests in prac tice. Mosey's Survey of Living Task Skills and Mosey's Group nteraction as well as Matsutsuyu's nterest Checklist. There were no differences in the responses of supervisors to Mosey's Activities of Daily Living Survey. The data from the third survey show that 75 percent of the 241 respondents used only individual task or interview methods as a means 724 November 1980 Volume 34 No. 11

Table 3 represents the frequency with which these evaluations were used in the field. The data demonstrate that the 24 evaluations listed in the survey are not being significantly used. Table 3 also lists the frequency of use of those evaluations learned about through either con tinuing education or school. Again a chi-square goodness of fit statistical test was applied. None of the evauations were used in significant numbers. There was no significant difference between the method of learning about the evaluations and their use. Although 20 evaluations first introduced to the therapist in school were used to some degree the Lawn and O'Kane Lafayette Clinic Battery Social Adaptability Test and the Occupational Behavior Rating Scale were not used. The nterest Checklist was most often introduced in school and was used more often than any other occupational therapy psychiatric evaluation. The results regarding those evaluations first introduced to the therapist through continuing education show that both the Diagnostic Test Battery and the Lawn and O'Kane Evaluation were not used and that the COTE was used the most. Discussion A comparison of the three surveys reveals a discrepancy between that which is taught in basic professional education and that which is used in practice. Evaluations taught to students in school and about which they were expected to be knowledgeable at their field centers were not used significantly by therapists in the field. On the other hand standardized psychological tests and interviewing methods were tools taught in school and were also used by practitioners. n addition the data from the third survey appeared to reflect that which was expected from students in the second survey in that none of the evaluations were used significantly. Since clinical supervisors are practicing therapists perhaps the respondents in the third survey are representative of the population tapped in the second survey. When examining the mode of acquiring knowledge about various evaluations neither the basic professional education nor the continuing education modes decisively influenced the useof evaluation tools. And nei ther the type of evaluation nor whether it had been published presented at a conference or learned in school made a difference-none of the psychiatric occupational therapy evaluations were being used. The reasons the clinicians were not using the evaluations listed in the survey may be many. For example:. The majority of respondents were new graduates and did not use the traditional instruments. Since the curriculum could not influence the use of an instrument perhaps the faculty chose not to teach the use of these traditional instruments. This is supported by the fact that few were teaching the use of specific eval uations tools and 94 percent of the faculty respondents were teaching students the interviewing methods. 2. The clinicians surveyed worked with a population for which the evaluations listed were not appropriate. Since instruments listed in the survey were designed to be used with adults it would be inappropriate for respondents working with children to use any of them. 3. A difference in philosophy regarding the use of evaluation tools may exist among therapists. The number of therapists reporting the use of a formal evaluatior procedure was 24; however 75 >ercent used interviewing methods S their only means of evaluation. Therapists who believe that an ir terview is adequate for planning patient treatment might not feel the need to use additional data-gathering methods. 4. The evaluations listed in the surveys that had been presented at workshops and conferences were unfamiliar to the majority of the therapists. This may be because the proceedings of these meetings were not published nor distributed to other than the few who attended these meetings. 5. The methods for introducing new instruments perhaps merely inform rather than instruct the clinician in administering and using the results of the evaluation. The information given in published articles is mean t to inform the reader about a tool not to train the reader in its use.~ he data demonstrated that the DiagnosticTest Battery the COTE the Buck-Magazine Picture Collage the Lawn and O'Kane the Shoemyn Battery the nterest Checklist Moorhead's Occupational History and the Adolescent Role Assessment all published in The American Journal of Occupational Therapy do not train therapists in their use. Because of limi ted space it is impossible to publish an entire assessment (evaluation) protocol that would include the rationale the literature review the definition of terms the rating scale the administration procedures the research studies the limitations and its application. fan individual is not taught how to apply and interpret an evaluation the likelihood of its being used is limited. 6. The therapists are dissatisfied with the present occupational therapy evaluations and have turned to The American Journal of Occupational Therapy 725

standardized psychological tests. This was supported by the fact that 72 standardized psychological tests were reported as being used by clinicians. None of the evaluations listed in the surveys is standardized and therapist respondents expressed the need for standardized. Perhaps without normative data to compare patients' pathology therapists resort to tests that give reliable information. Recommenda tions Efforts toward planning continuing education need to be carefully considered in view of the present methods of disseminating information about evaluations. Since many therapists reported using evaluation instruments of their own design the specialty sections and other groups can continue to provide institutes to teach the procedures necessary to develop an evaluation tool. Other institutes can be designed to teach practitioners how to standardize or improve existing evaluation tools. The participants can seek advice present their work for a critique and when introducing a new tool can teach the skills required for its administration interpretation and use. n order to reach a larger audience the proceedings of a workshop institute or conference could be published in their entirety and made available to practicing therapists at cost. n these cases the material published would depend upon the presenters extending their permission to record and publish their material and where necessary signing copyright release forms. Some therapists fear that their eval uations or assessments might become compromised because clinicians might adapt parts of them or change their appearance in some way. Publication of new evaluations would keep faculty abreast of new developments and it is recommended that they become familiar with the traditional evaluations as well as teach them to their studen ts including their assets and limitations. To encourage the use of evaluations standardization procedures employed when developing or improving the eval uations should be included in this instruction. By learning traditional evaluations and new evaluation tools students will enter the profession with a repertoire of methods for evaluating patient needs to complement the interview mode. Summary This study shows that existing psychiatric occupationa therapy evaluations were not being used by practicing therapists. Possible causes for this were lack of standardization inadequate dissemination failure of the faculty to teach the evaluations and failure of continuing education programs to teach therapists in the use of eval uation tools. Recommended solutions to these problems were: Continue to provide research institutes provide more avenues for publishing and presenting evaluation material and teach existing evaluations pointing out their strengths and weaknesses. These solutions would not ensure the use of an evaluation; however when therapists are confident that evaluations measure what they are stated to measure are assured of reliable data that can be compared with norms are effectively trained in the use of the evaluations and are better informed about their existence the probability is greater that these specific evaluations will be used. Acknowledgments The author extends gratitude to Beth Moyer M.S. FAOTA; to the Mental Health Section for financing the third survey; and to the educators clinical supervisors and the therapists who responded to the surveys. This article was based in part on a presentation given at the Mental Health nstitute Annual Conference American Occupational Therapy Association Detroit Michigan April 1979. REFERENCE 1. Critical issues identified. Mental Health Specialty Section Newsletter 2: 1 1979 RELATED READNGS Andoes L Dreyfus E Bloesch M: Diagnostic test battery for occupational therapy. Am J Occup Ther 19: 53-59 1965 Ayres A: A form to evaluate the work behavior of patients. Am J Occup Ther 8: 73-74 1954 Azima H: Dynamic occupational therapy. Dis Nerv System 22: 1-5 1961 Black M: Adolescent role assessment. Am J Occup Ther 30: 73-791976 Brayman S Kirby T Misenheimer A Short M: Comprehensive occupational therapy evaluation scale. Am J Occup Ther 30: 95-1001976 Casenova J: Comprehensive evaluation of basic living skills. Am J Occup Ther 30: 101-1051976 Fidler J Fidler G: A diagnostic and evaluative process. n Occupational Therapy: A Communicative Process in Psychiatry New York: MacmiHan Co 1963 Lerner C Ross G: The magazine picture collage. Am J Occup Ther 31: 156-161 1977 Matsutsuyu J: The nterest Check List. Am J Occup Ther 23: 323-328 1969 Moorhead L: The occupational history. Am J Occup Ther 23: 329-334 1969 Mosey A: Activities Therapy New York: Raven Press 1973 Shoemyn C: Occupational therapy orientation and evaluation. Am J Occup Ther 24: 276-279 1970 Wolff R: A behavior rating scale. Am J Occup Ther 15: 13-16 1961 The Lafayette Clinic Battery Object History Play History Social Adaptability Test Occupational Behavior Rating Scales and Self Puzzle have not been published. 726 November 1980 Volume 34 No. 11