Effects of Trainer Expressiveness, Organization, and Trainee Goal Orientation on Training Outcomes

Transcription

1 Journal of Applied Psychology 200). Vol. 86. No Copyright 2001 by the American Psychological Association, Inc /01/S5.00 DOI: // Effects of Trainer Expressiveness, Organization, and Trainee Goal Orientation on Training Outcomes Annette J. Towler and Robert L. Dipboye Rice University This laboratory experiment investigated the effects of trainer expressiveness, lecture organization, and trainee goal orientation on training outcomes. Participants (N = 135) listened to lectures that differed in organization and trainer expressiveness. Participants completed recall and problem-solving tests immediately and 2 days later. The results indicated that participants had the highest recall after an expressive and organized lecture. The findings for problem-solving performance were more complex. Participants with a high mastery orientation had their poorest problem-solving performance after listening to an organized and inexpressive lecture, whereas participants with a low mastery orientation did not respond to the effects of organization or expressiveness. Each year, corporations invest millions of dollars in training their personnel in the hope of obtaining a sustained competitive advantage (Barney, 1991; Filipczak, 1994). To this end, researchers have focused on evaluating alternative training techniques (e.g., Burke & Day, 1986) and have investigated how trainee characteristics, such as motivation, affect the success or failure of these techniques (e.g., Ford & Noe, 1987; Quinones, 1995). Despite a large and growing research literature, the role of the trainer has been largely ignored. This neglect is surprising when one considers the potential importance of the trainer in influencing the information processing, motivation, and learning of the trainee (Stammers, 1987). In the present research, we took an initial step in the direction of correcting this neglect by examining the effects of a trainer's delivery in the context of a lecture. In this study, we explored two aspects of the trainer's delivery of a lecture that seem to be especially important: the expressiveness of the verbal presentation (Ware & Williams, 1975) and the organization of the textual content of the lecture (van Dijk & Kintsch, 1983). Effective trainers are often thought to be both expressive and organized. An expressive trainer is one who shows appropriate vocal intonations and is generally fluent, whereas an inexpressive trainer is one who conveys the text in a monotone and hesitant voice (Abrami, Dickens, Perry, & Leventhal, 1980; Meier & Feldhusen, 1979; Williams & Ware, 1976). An organized trainer provides clarifying and elaborative content that makes the lecture easy to follow, whereas a less organized trainer requires the trainees to impose their own structure on a lecture (McNamara, Kintsch, Songer, & Kintsch, 1996). Annette J. Towler and Robert L. Dipboye, Department of Psychology, Rice University. We thank Matthew Howell for delivering the lecture and Loan Lam, Jane Shin, and Amy Rees for their assistance in data collection. We also thank Mickey Quinones, Steve Currall, and David Lane for their comments on an earlier version of this article. Correspondence concerning this article should be addressed to Annette J. Towler, who is now at the Department of Psychology, University of Colorado at Denver, Campus Box 173, P.O. Box , Denver, Colorado Previous research on the effects of these two components of a lecture has been simplistic to the extent that "more" of each is seen to improve learning and the effects of these components are assumed to be additive. Implicit in this thinking is the assumption that trainees react more positively to an expressive and organized lecture and are better able to recall and apply what they learn than if they are presented with an inexpressive or less organized lecture. Despite the widespread acceptance of this approach, there has been surprisingly little research directly examining the effects of the trainer's delivery. There is reason to question whether expressiveness and organization influence training outcomes in the manner typically assumed. Trainees' subjective reactions to training programs are not highly related to measures of trainee learning (Alliger & Janak, 1989). Also, some interventions that facilitate the acquisition of material and engender positive reactions in a training program appear to hinder the application and transfer of what is learned in training (Ghodsian, Bjork, & Benjamin, 1995). It is our contention that the effects of a trainer's organization and expressiveness on training outcomes are more complex than typically assumed in two respects. First, the effects of organization depend on the type of training outcome in that organization facilitates recall but may actually harm problem solving (Mannes & Kintsch, 1987; McNamara et al, 1996). Second, we would suggest that the effects of organization on training outcomes are moderated by the expressiveness of the trainer and the dispositional motivation of the trainee. Specifically, organization is only likely to benefit recall and a lower level of organization is only likely to benefit problem solving to the extent that trainees are positive toward the lecture as a result of their dispositional motivation and the expressiveness of the trainer. The present experiment was designed to examine these boundary conditions. The Effects of Organization Depend on the Outcomes An increasing amount of research and theory suggests that "good" organization may not always facilitate deeper processing of material and the application of knowledge (Mannes & Kintsch, 1987; McNamara et al., 1996). To the extent that gaps and inconsistencies in instructional material encourage trainees to actively 664

2 EXPRESSIVENESS, ORGANIZATION, AND GOAL ORIENTATION 665 process information, trainees confronted with a somewhat disorganized presentation may be able to use what they have learned to a greater extent than trainees who are presented with wellorganized material. Several studies have substantiated this nonobvious hypothesis, but two seem particularly relevant to our concerns. McNamara et al. (1996) found that students who read an organized text performed better on recall tasks than those who read less organized text. However, students who read less organized material performed better on tasks that required an application of knowledge to problem solving than those given the organized material. In a similar manner, Mannes and Kintsch (1987) found that participants who were provided with a consistent outline performed better on recall tasks than inconsistent-outline students. However, on problem tasks that required an application of acquired knowledge, those provided with an inconsistent outline performed best. These findings are intriguing and suggest that presenting subtle barriers to trainees that require them to more actively process information may enhance learning. Before this research is used as a justification for "bad teaching," however, there are two caveats that should be mentioned. First, it is important to recognize that these studies have been concerned with cognitive influences and no attention has been given to attitudes or motivation of the trainee with regard to the material conveyed. Although less organized material appears to encourage more active processing of the material, the organization of the material in these studies never fell to the level of incoherence that one would expect to adversely affect trainee motivation. We propose that a subtle manipulation of low coherency, such as that used by McNamara et al. (1996), might result in a deeper processing of the material, but this benefit occurs only to the extent that trainees react positively to the delivery of the material. If the material is delivered in a way that adversely affects the motivation of the trainees (e.g., dull or boring) or if the trainees approach the task with a low level of motivation, high organization seems less likely to benefit recall, and a low level of organization seems less likely to benefit problem solving. A second caveat is that previous research has been concerned with written material and has not been concerned with the oral delivery of training material such as might occur in a lecture. There are important differences between the two modalities in the cognitive and motivational factors that can potentially influence performance. For instance, letter identification and iconic processing are more important in reading written material, whereas speech perceptions and echoic processing are more important when one is listening to an oral delivery (Eysenck, 1988, p. 211). These differences in processing modalities suggest that there might be differences in the amount of attention that trainees pay to verbal versus written material that might affect their performance. There are also important differences in the potential impact on motivation as a result of the affect and paralinguistic behaviors associated with an oral delivery. In our study, participants listened to an audiotaped lecture in which levels of organization were manipulated in a manner similar to those used in previous studies (Mannes & Kintsch, 1987; McNamara et al., 1996). We examined two potential moderators of the effects of organization on recall and problem solving: the expressiveness of the trainer's delivery and the dispositional mastery orientation of the trainee. Following training, trainees are expected to recall and apply the knowledge they gained through training at later time periods. Therefore, in our study, we were also concerned with the stability of these effects and measured recall and problem solving both immediately and 2 days later. The Moderating Effects of Trainer Expressiveness and Mastery Orientation Two outcomes of training programs are the amount of information that trainees can recall and the extent to which they can apply what they learn to problem solving (Kraiger, Ford, & Salas, 1993). In this study, we hypothesized that the manipulated expressiveness of a trainer and the measured mastery orientation of the trainee would moderate the effects of organization on both of these outcomes. The small amount of research that has been conducted on the effects of trainer delivery on recall suggests that the more organized and expressive the instructor, the more trainees remember (e.g., Land, 1979; Ramagli & Greenwood, 1980). A lack of clarity in the delivery of a lecture has been shown to negatively affect students' achievement on recall tasks, whereas lecturer expressivity has been shown to positively affect students' learning (Murray, Rushton, & Paunonen, 1990). On the basis of these previous findings, we expected that the effects typically found for organization on recall would be diminished when trainers delivered the material with a low level of expressiveness. In addition to considering the effects of the trainer's delivery, in the present research we also explored the goal orientation of the trainees as a potential moderator of the effects of organization on recall and problem solving. Researchers have recognized the importance of accounting for the individual differences among trainees (e.g., Cronbach & Snow, 1977; Noe, 1986). Goal orientation has been identified as a stable dispositional trait that can moderate the effects of training (e.g., Button, Mathieu, & Zajac, 1996; Dweck & Leggett, 1988). Viewed from this dispositional perspective, trainees who are less motivated to master material are less inclined to actively process material that is low on organization. Conversely, trainees who are motivated to learn for the sake of learning seem to be more likely to actively process the material and impose their own structure (Ames, 1984; Diener & Dweck, 1978; Dweck & Leggett, 1988; Nicholls, 1984). Therefore, we expected that the facilitative effects of organization on recall shown in previous research (Mannes & Kintsch, 1987; McNamara et al., 1996) would be diminished among trainees with a low level of mastery orientation. Hypothesis /: Trainees who listen to a trainer whose presentation is relatively high on organization will show more accurate recall from the lecture than trainees who listen to a less organized presentation. However, a difference will be shown only when Hypothesis la: the trainer is high on expressiveness than when the trainer is low on expressiveness or Hypothesis Ib: the trainee is high on mastery orientation than when the trainee is low on mastery orientation. The second hypothesis was for the moderating influence of manipulated expressiveness and measured mastery orientation on the effects of organization on problem solving. As shown by McNamara et al. (1996) and Mannes and Kintsch (1987), lower levels of organization may hinder recall but may benefit problem

3 666 TOWLER AND DIPBOYE solving. Again, we expected that the trainer expressiveness and trainee goal orientation would moderate these effects such that this effect would occur among trainees who heard an expressive delivery and who had a high mastery orientation. Hypothesis 2: Trainees who listen to a trainer whose presentation is relatively low on organization will perform better on problem solving than trainees who listen to a more organized lecture. However, a difference will be shown only when Hypothesis 2a: the trainer is high on expressiveness than when the trainer is low on expressiveness or Hypothesis 2b: the trainee is high on mastery orientation than when the trainee is low on mastery orientation. Additional Effects on Training Outcomes Other important training outcomes are the reactions of trainees (Kraiger et al., 1993) to an expressive and organized lecture in the form of self-efficacy, motivation to learn, and evaluations of the lecture and the lecturer. We expected that the subtle manipulation of organization included in this experiment would not influence reactions but that the manipulation of expressiveness and measured mastery orientation would be related to trainee reactions. As a check on this assumption, we included self-efficacy, evaluations of the lecture and the trainer, and motivation to learn as dependent variables. There is considerable support for the contention that expressiveness of the trainer enhances trainee self-efficacy expectations. This evidence is in the form of research on Pygmalion effects (Eden & Ravid, 1982), leadership (Howell & Frost, 1989; Kirkpatrick & Locke, 1996), and lecturer delivery (Abrami et al., 1980; Meier & Feldhusen, 1979; Williams & Ware, 1976). To the extent that expressive trainers generate interest in a topic because they are exciting and intellectually stimulating (e.g., Perkins, Schenk, Stephan, & Vrungos, 1995), trainees should be motivated to learn material conveyed on that topic. Recent evidence also links a mastery orientation to self-efficacy and motivation to learn (e.g.. Ford, Smith, Weissbein, Gully, & Salas, 1998; Martocchio & Hertenstein, 1999; Phillips & Gully, 1997). However, there is little research investigating the joint effects of mastery orientation and trainer expressiveness on motivation to learn, self-efficacy, and evaluations of the lecture and lecturer. Therefore, we conducted an exploratory analysis to investigate the joint effects on these other training outcomes. Participants Method In exchange for course credit, 142 undergraduates from a private university in the southern United States participated in this experiment. Data from 1 participant were not analyzed because of suspicion, and data from another 6 participants were discarded because of a large amount of missing data. Of the 135 undergraduates, 34 were male, 99 were female, and 2 did not state their gender. Their ages ranged from 18 to 22 years. Of the participants, 66% were White, 20% were Asian, 8% were Hispanic, and 6% were Black. Design We used a three-factor experimental design containing two betweensubjects variables that were manipulated (trainer expressiveness and lecture organization), a third factor consisting of a measured variable (mastery orientation), and a covariate consisting of another measured variable (performance orientation). Procedure Participants were randomly assigned to one of the lecture organization conditions (organized vs. less organized) and to one of the delivery style conditions (expressive vs. inexpressive). Participants were told that they would hear a lecture and then would be tested on what they had heard. Participants listened to one of the four audiotaped lectures, all of which were approximately 10 min in length. Immediately following the lecture, they completed a motivation to learn scale, rating scales, and self-efficacy scales. After completing these measures, they were given the recall and problem tasks to perform. All participants returned 2 days later and completed the self-efficacy scales, followed by a second recall and problem test. At the end of the second session, participants completed the goal orientation inventory. Operationalization of Variables Manipulation of lecture organization. Two scripts about employment discrimination legislation were used in the manipulation of organization: a maximally organized version and a less organized version. These two scripts were taken from passages contained in the Society of Industrial/ Organizational Psychologist's Report on Affirmative Action (Kravitz et al., 1997) and from a practical text on diversity in the workplace (Powell, 1994). Following suggestions of McNamara et al. (1996), text was added and deleted to manipulate coherence at the local and global levels. The trainer delivered the lecture by omitting or including phrases that enhanced or reduced coherency. For example, to maximize global coherence, the trainer stated a topic header before delivering a section of the lecture pertaining to that heading. Four types of text revision were used to maximize local coherence (McNamara et al., 1996): (a) replacing pronouns with noun phrases when the referent was potentially ambiguous (e.g., replacing "others" with "women and minorities"), (b) replacing acronyms with noun phrases to decrease ambiguity (e.g., replacing "AA" with "affirmative action"), (c) adding sentence connectives (e.g., "whereas," "nonetheless," "however") to specify the relation between sentences or ideas, and (d) adding descriptive elaborations that expanded on an unfamiliar concept (e.g., for "perform utilization analyses," include "determine number of women and minorities at different levels within organization"). To maximize global coherence, we used the following devices: (a) adding topic headers and (b) adding macro propositions that linked each paragraph to the rest of the text. To test the effectiveness of the manipulation, we asked participants how organized they found the lecture on a scale ranging from 1 (not at all organized) to 7 (very organized). We included this item along with items used in evaluating reactions to the lecture and the trainer. Manipulation of expressiveness. A drama student with several years of acting experience was trained to deliver the lecture. In the expressive condition, the speaker used effective linguistic devices, as suggested by Holladay and Coombs (1993). These linguistic devices were an animated and enthusiastic voice and the maintenance of vocal fluency (i.e., the avoidance of undue hesitations). In the inexpressive condition, the actor was instructed to use a monotone voice and occasional vocal disfluencies (i.e., "ers," "urns," "and" hesitancies). To test the effectiveness of the manipulation, we used a bipolar adjective (inexpressive-expressive) on a scale from 1 to 7, with a higher score showing high expressiveness. We also asked participants to indicate the amount of vocal fillers they found in the trainer's delivery on a scale ranging from 1 (few vocal fillers) to 7 (high amount of vocal fillers). We included these items in trainee reactions toward the lecture and the trainer. Evaluation of trainer and lecture. A series of 14 bipolar adjectives (on a 7-point scale) were used to measure perceptions of the delivery (Holladay

4 EXPRESSIVENESS, ORGANIZATION, AND GOAL ORIENTATION 667 & Coombs, 1993). Examples include incompetent-competent, weakstrong, and unfriendly-friendly. Scale reliability (a) was.88 for the Stimulation factor and.87 for the Knowledge factor (see factor derivation in the Results section). Participants also rated the lecture (on a 7-point scale) on five items measuring coherency, usefulness, enjoyment, interest, and attention grabbing. Scale reliability of the five-item scale was.90. The higher the score, the more positive the rating. Measure of goal orientation. The 17-item Goals Inventory was used to measure mastery (12 items) and performance (5 items) orientation (Roedel- Debacker, Schraw, & Plake, 1994). Typical items for the mastery orientation scale include "I persevere even when I am frustrated by a task" and for the performance orientation scale include "I like others to think I know a lot." We chose mastery orientation as the main variable of interest because mastery orientation influences the extent to which trainees attempt to deeply process material from a lecture and is a potential moderator of the effects of expressiveness and organization on learning. Individuals with a performance goal orientation seek to perform well to demonstrate their ability to others. We included performance orientation as a covariate because performance goals have typically been associated with an avoidance of challenges and deterioration in performance in the face of obstacles (Diener & Dweck, 1978). Therefore, it was possible that performance goal orientation would be associated with the dependent measures, although we had no strong a priori basis for making such predictions in the context of this experimental context. Also, given that performance orientation was measured with the same self-report instrument, at the same time as mastery orientation, we could not entirely rule out the possibility that the two goal orientation scores would be related. However, we found that performance goal orientation was unrelated to the dependent variables and mastery goal orientation. Whether we included performance goal orientation or not had minimal effects on the findings. Participants indicated their level of agreement with each item by using a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). The scale reliability for both mastery orientation and performance orientation was.76. We performed a median split on the mastery orientation variable. Participants scoring higher than the median (4.09) were classified as having a high mastery orientation, and those scoring lower than the median were classified as having a low mastery orientation. An independent sample t test revealed the mean for the high mastery orientation was 4.39 and the mean for the low mastery orientation was 3.70,?(133) = 15.40, p <.001. We chose to administer the goal orientation measure at the end of the second session for two reasons. First, similar to Button et al. (1996) and Miller, Green, Montalvo, Ravindran, and Nichols (1996), we viewed goal orientation as a relatively stable disposition. Previous research with the scale used in this experiment has supported the assumption that scores on this measure reflect a stable disposition and that scores on goal orientation demonstrated after the experiment should reflect differences among participants prior to the experiment. Second, we did not wish to administer the scale prior to the experiment to avoid priming participants and thereby confounding or interfering with the measurement and manipulation of the other variables. Recall and problem tests. The order in which these tests were presented was counterbalanced so that half of the participants took the recall first and the problem test second, whereas the other half took the tests in the reverse order. Twenty questions were used to measure participants' recall of the lecture material. These 20 questions were divided into two sets of 10 questions that were matched as nearly as possible for difficulty and content matter. Half of the participants received one set of questions as the immediate recall test and the other set as the delayed recall test. The remaining participants were given the tests in the reverse order. In the recall test, participants were given 10 incomplete sentences that were statements from the lecture and were asked to fill in the blanks (there were 18 blanks in total). Scores on each of the recall tests could range from 0 to 18. Six questions were derived to measure participants' problem-solving performance, using the same procedure as for the recall questions. For problem solving, they were given three situations involving employment discrimination issues and were asked questions regarding each of these situations that required applying what they had learned. An example question was "The Chief Executive at Johnston Inc. is worried that most of the managers are predominantly White. He asks you to recommend some practical steps the organization can do to increase diversity. What do you recommend?" Another question asked participants to evaluate the effects a diversity policy might have on fictional job incumbents. Scores on the problem-solving tests ranged from 0 to 14. Objective score keys were developed in advance. Annette J. Towler, who was naive to the experimental conditions, scored the tests. To ensure reliability, an independent coder who was unaware of the experimental hypotheses scored a random sample of 10 problem tests. Interrater reliability was.96 for the immediate problems and.97 for the delayed problems. Motivation to learn scale. A 10-item scale adapted from Quinones (1995) was used as a dependent measure to assess the effects on participants' motivation to learn the material presented in the lecture. Participants indicated their level of agreement with each item by using a 5-point Likert scale. The higher the score, the greater the motivation to learn. Sample items include "I put forth a lot of effort during this lecture." Scale reliability was.91. Typically, other studies have measured motivation to learn prior to training (e.g., Quinones, 1995). Similar to Baldwin, Magjuka, and Loher (1991), we measured motivation to learn as a dependent variable. Self-efficacy for recall and problem solving. These scales were derived from Wood and Locke's (1987) academic self-efficacy scale. In the recall self-efficacy measure, participants were told that they would take a test that would require them to recall material from the lecture. They then were asked to indicate how confident they felt about taking the test. They were asked to indicate with "yes" or "no" whether they could recall a specific number of facts and concepts (range from 60% to 100%, in increments of 10%) and for each level of recall indicated on a 10-point scale their confidence that they could perform at that level. In the problem selfefficacy measure, they were told that they would be asked to apply the material in practical situations. The self-efficacy score was computed by adding the confidence ratings of those levels of performance that they said they "can do." Scale reliabilities ranged from.77 to.88. Results Manipulation Check on Expressiveness In support of the manipulation, participants in the expressive condition perceived the trainer to speak in a less monotone voice (M = 5.00) than did those in the inexpressive condition (M = 6.58), f(133) = -6.90, p <.001. Furthermore, in the expressive condition, participants perceived the trainer's speech to have fewer vocal fillers (M = 2.98) than did those in the inexpressive condition (M = 4.94), /(133) = -5.80, p <.001. Manipulation Check on Lecture Organization In support of the manipulation, participants who listened to an organized lecture perceived the lecture to be better organized (M = 4.91) than did those in the less organized lecture (M = 3.82), r(133) = 4.42, p <.05. Derivation of Factor Scores for Trainer and Lecture Ratings A principal-components analysis with varimax rotation performed on the 14 trainer rating adjectives revealed two interpret-

5 668 TOWLER AND DIPBOYE Table 1 Means, Standard Deviations, and Zero-Order Correlations for All Variables (N = 135) Variable Expressiveness Organization Mastery orientation Performance orientation Immediate recall Delayed recall Immediate problem solving Delayed problem solving Immediate recall SE Delayed recall SE Immediate problem SE Delayed problem SE Motivation to learn Reactions: Stimulation Reactions: Knowledge Reactions to lecture M SD Note. Coding for lecture organization: 0 = organized, 1 = less organized. Coding for expressiveness: 0 = expressive, 1 coefficients: r >.22, p <.01, r >.17, p <.05. SE = self-efficacy. = inexpressive. Correlation able factors that accounted for 59% of the variance. Factor 1, which accounted for 46% of the variance, was labeled Stimulation and consisted of the following items: leadenanimated, boring-exciting, ineffective-motivating, inattentiveattentive, uncharismatic-charismatic, unfriendly-friendly, weakpowerful, and closed-open. Factor 2, which accounted for 13% of the variance was labeled Knowledge and consisted of the following items: unqualified-qualified, unreliable-reliable, unintelligentintelligent, unskilled-skilled, foolish-wise, and incompetentcompetent. A principal-components analysis performed on the five Likert-type lecture rating items (usefulness, enjoyment, practical application, attention grabbing, and interest) revealed one factor, which accounted for 72% of the variance. The lecture items were summed to create a single score of evaluation of the lecture. Descriptive Data Table 1 presents the means, standard deviation, and intercorrelations for all variables. Tests of the Hypotheses Three multivariate analyses of covariance were performed on the recall and problem-solving variables,' the motivation variables (the four self-efficacy scales and the motivation to learn scale) and the trainee evaluations of the lecture and the trainer, respectively. We used standardized scores for the motivation variables and the recall and problem-solving variables to perform these analyses. To control for Type I errors resulting from the use of correlated dependent measures, we conducted multivariate analyses. We used performance orientation as a covariate in all the analyses. Results of tests for homogeneity of regression and homogeneity of variance-covariance matrices were satisfactory (Tabachnik & Fidell, 1996). Tukey a post hoc comparison tests were used to test two-way and three-way interactions. The means reported here are the unstandardized 2 adjusted means. Hypotheses 1 and 2: Expressiveness and mastery orientation as moderators of the effects of organization on recall and problem solving. We hypothesized that expressiveness and mastery orientation would moderate the effects of organization on recall and problem solving. To test these hypotheses, we performed a 2 (mastery orientation) X 2 (expressiveness) X 2 (organization) multivariate analysis of covariance on the four standardized recall 1 Dichotomization of mastery orientation results in a loss of information. We performed a series of hierarchical regression analyses on the recall and problem-solving scores and the motivation scales, entering the three-way interaction of Expressiveness X Organization X Mastery Orientation after the main effects and two-way interactions. We obtained the same results as for the multivariate analyses of covariance. 2 We present the unstandardized means in the Results section to aid the reader's clarity. However, all relevant analyses were conducted using the standardized means.

6 EXPRESSIVENESS, ORGANIZATION, AND GOAL ORIENTATION 669 and problem-solving measures. This analysis revealed a statistically significant Expressiveness X Organization interaction, Wilks's A =.90, F(4, 123) = 3.30, p <.01, rf =.10, and a significant three-way interaction of expressiveness, lecture organization, and mastery orientation, Wilks's A =.90, F(4, 123) = 3.47, p<.01, if =.10. Univariate analyses revealed statistically significant interaction effects, including Expressiveness X Organization interactions on immediate recall, F(l, 126) = 10.62, p <.001, if =.09, and delayed recall, F(l, 126) = 6.37, p <.01, rj 2 =.09. Consistent with the hypothesis, the facilitative effects of organization on recall were enhanced when the trainer was expressive. Tukey a post hoc comparisons revealed that those in the expressive and organized lecture had higher immediate recall (M = 8.42) than those in the expressive and less organized lecture (M = 5.90), the inexpressive and organized lecture (M = 5.61), and the inexpressive and less organized lecture (M = 5.64). Similar findings were found for delayed recall, with those in the expressive and organized lecture scoring higher on the delayed recall (M = 8.20) than those in all other conditionsthe expressive and less organized lecture (M = 5.85), the inexpressive and organized lecture (M = 5.88), and the inexpressive and less organized lecture (M = 5.36). Figure 1 shows the interaction effect for delayed recall. Univariate analyses also revealed a statistically significant Expressiveness X Organization interaction effect for delayed problem solving, F(l, 126) = 5.23, p <.05, rj 2 =.09, but Tukey a post hoc comparisons revealed a pattern that was contrary to the hypothesis. Those who listened to the inexpressive and less organized lecture scored higher (M = 2.66) on delayed problem solving than those in the inexpressive and organized condition (M = 1.56). There were no other statistically significant differences between conditions (for the expressive and organized lecture, M = 2.43, and for the expressive and less organized lecture, M = 2.24). Univariate analyses of the Expressiveness X Organization X Mastery Orientation interaction revealed a statistically significant effect on immediate problem solving, F(l, 126) = 8.53,p <.004, n 2 =.10, and delayed problem solving, F(l, 126) = 7.26, p<.01, T) 2 =.10. The effects of expressiveness and organization on problem solving depended on the level of mastery orientation of the participant. For immediate problem solving, there was a statistically significant two-way interaction of expressiveness and organization for high mastery orientation participants, F(3, 60) = 4.79, p <.01, T) 2 =.23. The interaction of expressiveness and organization was not statistically significant for low mastery orientation participants, F(3, 65) = 0.74, p =.53. Tukey a analyses revealed that among the high mastery orientation participants, those in the inexpressive and less organized condition scored higher on immediate problem solving (M = 3.06) than those in the inexpressive and organized (M = 1.47) condition. There were no other statistically significant differences between the lecture conditions (for the expressive and organized lecture, M = 3.66, and for the expressive and less organized lecture, M = 2.61). Similar results were found for the delayed problem solving. There was a statistically significant two-way interaction of expressiveness and organization for high mastery orientation participants, F(3, 60) = 4.02, p <.01, if =.23. The interaction of expressiveness and organization was not statistically significant for low mastery orientation participants, F(3, 65) = 0.95, p =.42. Tukey a post hoc comparisons showed that among the high mastery orientation participants, those in the inexpressive and less organized condition scored higher on delayed problem solving (M = 3.04) than those in the inexpressive and organized (M = 1.35) condition. There were no other statistically significant differences between the lecture conditions (for the expressive and organized lecture, M = 2.81, and for the expressive and less organized lecture, M = 1.64). Figure 2 shows the interaction effect for delayed problem solving. Effects of trainer expressiveness, organization, and mastery orientation on motivation variables. We performed a 2 (mastery orientation) X 2 (expressiveness) X 2 (organization) multivariate analysis of covariance on the motivation to learn scale and the four o V T DOrgcrlzed DLess orgcnlzed l*5- I 4. 9, a, " ' b b b n - Expressive Expressiveness Inexpressive Figure 1. Mean delayed recall test score as a function of trainer expressiveness and organization. Conditions not sharing a common letter denote those means are significantly different from each other at p <.05.

7 670 TOWLER AND DIPBOYE High mastery E I 2.5! 2 U ab ab '-i'r-!-- 1 a - : '; - Expresslv^xpress venes lnexpresslve DC 13 Organized DL o organized I 1 a, 3.5 -] ; : ;>' a 13 ^.... Low mastery a a Expressive^^^lnexpresslve QQ DU organized Figure 2. Mean delayed problem-solving test score as a function of trainer expressiveness and organization and trainee mastery orientation. Conditions not sharing a common letter denote those means are significantly different from each other at p <.05. self-efficacy scales (all scores were standardized). Our hypothesis could not be supported because of a statistically nonsignificant interaction effect of mastery orientation and expressiveness. However, in support of previous findings, there were statistically significant main effects of expressiveness, Wilks's A =.89, F(5, 122) = 3.10, p <.01, ff =.11, and mastery orientation, Wilks's A =.89, F(5, 122) = 3.06, p <.01, r, 2 =.11. Univariate analyses revealed that those who listened to the expressive trainer experienced higher immediate recall self-efficacy (M = 12.64) than those who listened to the inexpressive trainer (M = 8.51), F(l, 126) = 8.43, p <.01, -r/ 2 =.06. Univariate analyses also revealed that expressiveness had a positive impact on immediate problem self-efficacy (M = vs. M = 10.23), F(l, 126) = 8.75, p <.01, Tj 2 =.06; delayed problem self-efficacy (M = 9.07 vs. M = 6.01), F(l, 126) = 3.96, p <.05, rf =.03; and motivation to learn (M = 3.22 vs. M = 2.85), F(l, 126) = 0.84, p <.01, rf =.06. Univariate analyses also revealed the positive impact of mastery orientation on motivation variables. Mastery orientation had a positive impact on immediate recall self-efficacy (M = vs. M = 8.28), F(l, 126) = 12.05, p <.001, if =.09; delayed recall self-efficacy (M = 5.35 vs. M = 3.41), F(l, 126) = 12.05, p <.001, -rf =.09; delayed problem self-efficacy (M = 9.35 vs. M = 5.97), F(l, 126) = 3.96, p <.05, ff =.09; and motivation to learn (M = 3.17 vs. M = 2.91), F(l, 126) = 3.96, p <.05, -r/ 2 =.04. There was also a significant two-way interaction of expressiveness and organization, Wilks's A =.92, F(5, 122) = 2.26, p <.05, Tj 2 =.09. Univariate analyses revealed that participants who listened to an expressive and organized trainer experienced higher self-efficacy (M = 7.14) than those in the expressive and less organized (M = 2.81), the inexpressive and less organized (M = 4.01), and the inexpressive and organized (M = 2.44) conditions, F(3, 130) = 4.65, p <.01, tf =.10. There were no other statistically significant findings. Effects of trainer expressiveness, organization, and mastery orientation on evaluations of trainer and lecture. We performed a multivariate analysis of covariance (Mastery Orientation X Ex-

8 EXPRESSIVENESS, ORGANIZATION, AND GOAL ORIENTATION 671 pressiveness X Organization) on the two trainer rating factors and the lecture rating factor. No effects were found for organization on the three rating factors (consistent with our intent to keep this manipulation within the narrow band used in previous research). However, as we expected, there was a statistically significant main effect of expressiveness, Wilks's A =.71, F(3, 124) = 11.15, p <.0001, Tj 2 =.29, and a statistically significant effect of mastery orientation, Wilks's A =.92, F(3, 124) = 7.63, p <.01, rj 2 =.08. We conducted univariate analyses on the three evaluation measures to interpret the significant multivariate analysis of covariance effects. A main effect for mastery orientation indicated that highmastery-oriented participants evaluated the lecture more positively (M = 3.72) than low-mastery-oriented participants (M = 3.22), F(l, 126) = 1.63, p <.01, tf =.06. The analyses on the other two reactions measures showed statistically nonsignificant effects. A statistically significant effect for expressiveness was found for all three reactions measures. Compared with participants who heard an inexpressive delivery, those who heard an expressive delivery evaluated the trainer as being more stimulating (M = 3.94 vs. 3.08), F(l, 126) = 33.24, p <.001, rj 2 =.21, and more knowledgeable (M = 4.49 vs. 3.55), F(l, 126) = 40.92, p <.001, T) 2 =.25, and evaluated the lecture more positively (M = 3.96 vs. 2.96), F(l, 125) = 27.95, p <.001, if =.19. Discussion In this study, we investigated the impact of trainer expressiveness and organization and trainee goal orientation on the recall, problem solving, and motivation of the trainee. Our hypotheses for recall received substantial support and provided support for the contention that the trainer's delivery has important effects on the performance and motivation of the trainee and trainee reactions to the lecture and trainer. A high level of organization of the text facilitated immediate and delayed recall but only when the trainer was expressive. The findings were more complex for problem solving. Although an Expressiveness X Organization interaction was found in the analyses of problem solving, the means for this interaction revealed a pattern that was the opposite of what we hypothesized. Contrary to this hypothesis, expressiveness lessened rather than enhanced the effects of organization. Indeed, the only condition in which higher problem solving was found for a low level of organization than a high level of organization was when the trainer was inexpressive. When the trainer was expressive, the difference between the organized and less organized lectures was statistically nonsignificant. A three-way interaction revealed that this Expressiveness X Organization interaction occurred only for the high-mastery-oriented participants. Although the findings for problem solving were not as we had predicted, they clearly cast doubt on the traditional view that there is a simple relation between the delivery of the trainer and the extent to which trainees can apply what they learn to problem solving. The level of problem solving achieved with an inexpressive and low organization lecture and the expressive and low organization lecture was high, and neither condition differed statistically from the condition that is often thought to be ideal (i.e., the high-high lecture). These findings suggest that several of the theories that have been used to understand training would benefit from incorporating the trainer's behavior into the process in addition to cognitive and motivational constructs. In particular, this study confirms and extends research on text comprehension (Mannes & Kintsch, 1987), self-efficacy (Bandura, 1982), and mastery orientation (Dweck & Leggett, 1988). The effects that we found for text organization on recall and problem solving partially replicated what Kintsch and his colleagues (Mannes & Kintsch, 1987; Mc- Namara et al., 1996) found with written text. The findings of our research suggest, however, that the cognitive interpretations that have dominated the research on text comprehension may be quite limited in situations in which trainees become bored or excited with a presentation and draw inferences for the characteristics of the trainer. Organization enhanced recall only when the trainer was expressive and was detrimental to problem solving only when the trainer was inexpressive and the trainees were high in mastery orientation. The importance of the trainer's delivery and the trainee's motivation may be more apparent in an oral delivery than when participants read written textual material without being able to see or hear the author of the text. In fact, the trainer's expressiveness substantially enhanced the motivation of the trainee. These findings also have implications for goal orientation theory. The findings that mastery orientation was unrelated to recall but was positively related to problem solving support previous evidence that mastery goal orientation is associated with an active cognitive engagement in the task (e.g., Stevens & Gist, 1997). Consistent with previous research (e.g., Ford et al., 1998; Martocchio & Hertenstein, 1999; Phillips & Gully, 1997), the highmastery trainees had higher self-efficacy for both recall and problem solving. Also, high-mastery trainees expressed a stronger motivation to learn and reacted more positively to the lecture and trainer. The lowest level of problem-solving performance was exhibited by the high-mastery-oriented participants who were presented with an inexpressive and organized lecture. One possible interpretation is that this condition was the least challenging (high organization with little in the delivery to motivate the trainee) and, as a consequence, was most detrimental to the high-mastery trainees who seek and prefer challenge. The low-mastery participants, by contrast, did not appear to be as influenced by either expressiveness or organization. As in the case of text comprehension and self-efficacy, these findings suggest that the trainer's delivery of a program needs to be considered to understand fully the relation of mastery orientation and outcomes of a training program. In addition to the implications for previous theories of training, the findings also have some practical implications. The findings clearly show that if the objective is to either enhance memorization or induce trainee motivation, trainers should strive to be both organized and expressive in their delivery. To the extent that the objective is a deeper level of understanding than mere recall, however, we should not assume that the expressive and organized trainer is effective. Although we would not conclude from any aspect of these findings that trainers should strive to be disorganized or nonexpressive, the reliance on trainee reactions as measures of trainer effectiveness seems ill-advised. At the same time that trainers seek to arouse the interest and motivation of trainees, perhaps they should seek to organize the material in a manner that pushes the limits of their understanding. Instructional materials that are "too close to the reader's knowledge are redundant" whereas those that are "too far removed are too difficult" (Kintsch, 1994, p. 297). The trainer needs to seek what Vygotsky (1986)

9 672 TOWLER AND DIPBOYE referred to as "the proximal zone of development" to ensure future growth and learning. There are several potential limitations in our research that not only may impose constraints on generalizing these findings but also may present interesting opportunities for future research. The materials, stimuli, and measures that we used are potential barriers to generalizing these results. The most obvious constraint is our use of only one audiotaped lecture. Research is needed to determine whether these findings generalize to repeated lectures in which there are visual as well as audio cues. A final limitation that is also a potential opportunity for additional research involves the intrinsic interest of material for trainees. Ratings of reactions to the lecture material (M = 3.48) indicated that the material in this study was of moderate to low interest to trainees. An important question for future research is whether the effects of trainer delivery differ as a function of the intrinsic interest of trainees in the material conveyed in the program. Given the lack of previous research on the role of the trainer, the present findings must be viewed with caution. Obviously, we do not know the extent to which these findings will generalize to other types of subjects, presentation modalities, textual content, or performance measures. Nevertheless, the limitations that we have identified are not, in our opinion, fatal problems. For instance, the use of an audiotaped presentation is a potential limitation, but the effects demonstrated here may be even stronger when participants have visual information such as posture, gesturing, and eye contact to reinforce the vocal cues and when they have repeated experience with the trainer. Although this study does not provide the final word on these issues, the findings are intriguing in their practical and theoretical implications. The strange neglect of the trainer clearly needs to end, and this study provides some possible directions for future research on this crucial variable in the training process. We would further conclude that a fixation on the traditional model of training is ill-advised. The linkage between attitudes of trainees and how much they learn is not as simple as is often assumed. Happy trainees do not always learn more. And to the extent that it is important for trainees to apply what they have learned to problems outside the session, trainers need to be encouraged to take risks in challenging their trainees even at the cost of dissatisfying them. References Abrami, P. C, Dickens, W. J., Perry, R. P., & Leventhal, L. (1980). Do teacher standards for assigning grades affect student evaluations of instruction? Journal of Educational Psychology, 72, Alliger, G. M., & Janak, E. A. (1989). Kirkpatrick's levels of training criteria: Thirty years later. Personnel Psychology, 42, Ames, C. (1984). Achievement attributions and self-instructions under competitive and individualistic goal structures. Journal of Educational Psychology, 76, Baldwin, T. T., Magjuka, R. J., & Loher, B. T. (1991). The perils of participation: Effects of choice of training on trainee motivation and learning. Personnel Psychology, 44, Bandura, A. (1982). Self-efficacy mechanism in human agency. American Psychologist, 33, Barney, J. B. (1991). Firm resources and sustained competitive advantage. Journal of Management, 17, Burke, M. J., & Day, R. R. (1986). A cumulative study of the effectiveness of managerial training. Journal of Applied Psychology, 71, Button, S. B., Mathieu, J. E., & Zajac, D. M. (1996). Goal orientation in organizational research: A conceptual and empirical foundation. Organizational Behavior and Human Decision Processes, 67, Cronbach, L. J., & Snow, R. E. (1977). Aptitudes and instructional methods. New York: Irvington. Diener, C. I., & Dweck, C. S. (1978). An analysis of learned helplessness: Continuous changes in performance, strategy, and achievement cognitions following failure. Journal of Personality and Social Psychology, 36, Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and personality. Psychological Review, 95, Eden, D., & Ravid, G. (1982). Pygmalion versus self-expectancy: Effects of instructor and self-expectancy on trainee performance. Organizational Behavior and Human Performance, 30, Eysenck, M. W. (1988). A handbook of cognitive psychology. London: Erlbaum. Filipczak, B. (1994, October). Looking past the numbers. Training, 31, Ford, J. K., & Noe, R. A. (1987). Self-assessed training needs: The effects of attitudes toward training, managerial level, and function. Personnel Psychology, 40, Ford, J. K., Smith, E. M., Weissbein, D. A., Gully, S. M., & Salas, E. (1998). Relationships of goal orientation, metacognitive activity, and practice strategies with learning outcomes and transfer. Journal of Applied Psychology, 83, Ghodsian, D., Bjork, R. A., & Benjamin, A. S. (1995). Evaluating training during training: Obstacles and opportunities. In M. A. Quinones & A. Ehrenstein (Eds.), Training for a rapidly changing workplace: Applications of psychological research (pp ). Washington, DC: American Psychological Association. Holladay, S. J., & Coombs, W. T. (1993). Communicating visions: An exploration of the role of delivery in the creation of leader charisma. Management Communication Quarterly, 6, Howell, J. M., & Frost, P. J. (1989). A laboratory study of charismatic leadership. Organizational Behavior and Human Decision Processes, 43, Kintsch, W. (1994). Text comprehension, memory, and learning. American Psychologist, 49, Kirkpatrick, S. A., & Locke, E. A. (1996). Direct and indirect effects of three core charismatic leadership components on performance and attitudes. Journal of Applied Psychology, 81, Kraiger, K. J., Ford, K. J., & Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78, Kravitz, D. A., Harrison, D. A., Turner, M. E., Levine, E. L., Chaves, W., Brannick, M. T., Denning, D. L., Russell, C. J., & Conard, M. A. (1997). Affirmative action: A review of psychological and behavioral research. Bowling Green, OH: Society for Industrial and Organizational Psychology. Land, M. L. (1979). Low-interference variables of teacher clarity: Effects on student concept learning. Journal of Educational Psychology, 71, Mannes, S., & Kintsch, W. (1987). Knowledge organization and text organization. Cognition and Instruction, 4, Martocchio, J. J., & Hertenstein, E. J. (1999, April). Learning orientation and goal orientation context: Relationships with cognitive and affective learning outcomes. Paper presented at the 14th Annual Conference of the Society for Industrial and Organizational Psychology, Atlanta, GA. McNamara, D. S., Kintsch, W., Songer, N. B., & Kintsch, W. (1996). Are good texts always better? Interactions of text coherence, background knowledge, and levels of understanding in learning from text. Cognition and Instruction, 14, Meier, R. S., & Feldhusen, J. F. (1979). Another look at Dr. Fox: Effect of stated purpose for evaluation, lecturer expressiveness, and density of

10 EXPRESSIVENESS, ORGANIZATION, AND GOAL ORIENTATION 673 lecture content on student ratings. Journal of Educational Psychology, 71, Miller, R. B., Green, B. A., Montalvo, G. P., Ravindran, B., & Nichols, J. D. (1996). Engagement in academic work: The role of learning goals, future consequences, pleasing others, and perceived ability. Contemporary Educational Psychology, 21, Murray, H. G., Rushton, J. P., & Paunonen, S. V. (1990). Teacher personality traits and student instructional ratings in six types of university courses. Journal of Educational Psychology, 82, Nicholls, J. G. (1984). Achievement motivation: Conceptions of ability, subjective experience, task choice, and performance. Psychological Review, 91, Noe, R. A. (1986). Trainees' attributes and attitudes: Neglected influences on training effectiveness. Academy of Management Review, 11, Perkins, D., Schenk, T. A., Stephan, L., & Vrungos, S. (1995). Effects of rapport, intellectual excitement, and learning on students' perceived ratings of college instructors. Psychological Reports, 76, Phillips, J. M., & Gully, S. M. (1997). Role of goal orientation, ability, need for achievement, and locus of control in the self-efficacy and goal-setting process. Journal of Applied Psychology, 82, Powell, G. N. (1994). Gender and diversity in the workplace. Thousand Oaks, California: Sage. Quinones, M. A. (1995). Pretraining context effects: Training assignment as feedback. Journal of Applied Psychology, 80, Ramagli, H. J., Jr., & Greenwood, G. E. (1980, April). The Doctor Fox effect: A paired lecture comparison of lecturer expressiveness and lecture content. Paper presented at the meeting of the American Educational Research Association, Boston, MA. Roedel-Debacker, T., Schraw, G., & Plake, B. S. (1994). Validation of a measure of learning and performance goal orientations. Educational and Psychological Measurement, 54, Stammers, R. B. (1987). Training and the acquisition of knowledge and skill. In P. Warr (Ed.), Psychology at work (pp ). London: Penguin Books. Stevens, C. K., & Gist, M. E. (1997). Effects of self-efficacy and goal orientation training on negotiation skills maintenance: What are the mechanisms? Personnel Psychology, 50, Tabachnik, B. G., & Fidell, L. S. (1996). Using multivariate statistics. New York: Harper Collins. van Dijk, T. A., & Kintsch, W. (1983). Strategies of discourse comprehension. New York: Academic Press. Vygotsky, L. S. (1986). Thought and language. Cambridge, MA: MIT Press. Ware, J. E., & Williams, R. G. (1975). The Dr. Fox effect: A study of lecturer expressiveness and ratings of instruction. Journal of Medical Education, 50, Williams, R. G., & Ware, J. E. (1976). Validity of student ratings of instruction under different incentive conditions: A further study of the Dr. Fox effect. Journal of Educational Psychology, 68, Wood, R. E., & Locke, E. A. (1987). The relation to self-efficacy and grade goals to academic performance. Educational and Psychological Measurement, 47, Received September 9, 1999 Revision received July 27, 2000 Accepted August 7, 2000 ORDER FORM Start my 2001 subscription to Journal of Applied Psychology! ISSN: $76.00, APA Member/Affiliate $153.00, Individual Non-Member $362.00, Institution In DC add 5.75% sales tax / In MD add 5% sales lax TOTAL AMOUNT ENCLOSED $ Subscription orders must be prepaid. (Subscriptions are on a calendar basis only.) Allow 4-6 weeks for delivery of the first issue. Call for international subscription rates. SEND THIS ORDER FORM TO: American Psychological Association Subscriptions 750 First Street, NE Washington, DC Or call (800) , fax (202) TDD/TTY (202) subscriptions@apa.org Send me a Free Sample Issue Q Q Check Enclosed (make payable to APA) Charge my: Q VIS A Q MasterCard Q American Express Cardholder Name Card No. Exp. date Signature (Required for Charge) Credit Card Billing Address City State Zip Daytime Phone SHIP TO: Name Address City State.Zip APA Customer # PLEASE DO NOT REMOVE - A PHOTOCOPY MAY BE USED GADOl