Use of a Virtual Environment in the Geowall to Increase Student Confidence and Performance During Field Mapping: An Example from an Introductory-Level Field Class Michael M. Kelly MMKAA, Inc., 3354 N Crest, Flagstaff, AZ 86001, design@mmkaa.com Nancy R. Riggs Department of Geology, Northern Arizona University, Flagstaff, AZ 86011, nancy.riggs@nau.edu ABSTRACT Geology students often have difficulty learning the baseline terrain-analysis skills required for success in introductory field geology. Students in the Introductory Field Methods class at Northern Arizona University are prepared, in part, for field-mapping experiences through exercises with stereo photographs and topographic maps. To improve spatial skills and enhance confidence, we added a computer-based virtual environment (VE) to this early training. Using the GeoWall, we developed a VE in which students navigate and transfer location information and geologic contacts from the VE to a paper topographic map. Following this exercise, students go into the field to produce a geologic map of the field area. Using a Wilcoxon rank sum test we examined scoring differences between the experimental group from 2003/2004 (n=27, Median = 80) and those in a control group from previous years without the visualization exercise (n=35, Median = 60). At an alpha level of 0.05 the mean ranks of the control and experimental groups are statistically different (z = 3.67). These results, together with student narratives and attitude surveys, suggest that the virtual environment had an effect on student mapping performance that is coupled with an increase in spatial survey knowledge and increased confidence in the field. INTRODUCTION Students in their first field class often seem bewildered by aspects of field techniques that eventually become second nature to most geologists. Concerns are often heard about self-location, translating scale (i.e., how to gauge distance on the ground from looking at a map), and how geology seen in the field in three dimensions can be reasonably represented on a map in two dimensions. The GeoWall is visualization hardware that employs stereopsis (binocular vision) to enhance the understanding of spatial relationships between physical objects. It is used primarily in introductory-geology laboratory activities to illustrate the 3-dimensional nature of earth science data (see www.geowall.org). Students continuing in geosciences clearly need to begin to achieve comfort with thinking in three dimensions. One objective of this study, therefore, was to begin to test the application of the GeoWall technology to student learning through field experiences. This paper describes the addition of a virtual environment (VE) visualization exercise in the GeoWall as a component of an established field exercise at Northern Arizona University. Our goal in adding this new component was to enhance student confidence in the field, which our empirical observations would suggest leads to more accuracy and efficiency. Assessment of performance and attitudes suggests that virtual-environment activities, such as that introduced in the course, are valuable to student learning when applied to field settings. Virtual Environment - "A Virtual Environment (VE) is a synthetic, spatial (usually 3-dimensional) world seen from a first-person point of view" (Bowman et al., 2003, p. 82). VEs are often described as immersive (MacEachren et al., 1999) to some degree and generally allow for user navigation through the world. While VEs can be viewed on a range of displays from desktop computers to completely immersive "caves", the Geology Department at NAU uses a low-cost GeoWall for this purpose. The GeoWall is a computer-driven, two-source projection system that allows the display of stereo 3-dimensional data. This arrangement allows groups of students wearing clear polarized glasses to work together in groups around a large projection screen (Figure 1). The visualization software program employed in this exercise is ROMA, which supports stereopsis and is compatible with the GeoWall. ROMA allows flight-like navigation of 1:1 scale representations of real terrains. Co-registered digital elevation maps and aerial photographs are combined within ROMA to create a land surface, with lighting and atmospheric effects added for increased realism. Markers and objects can be added as visual signposts or cues. Navigation of the VE is accomplished using a commonly available Logitech Wingman wireless game controller, which allows natural mobility in all directions, and easy transfer of navigation duties between students. TEST CLASS Introduction to Field Methods (GLG240) is a sophomore-level field class at Northern Arizona University (NAU) required for Geology majors and Environmental Sciences/Applied Geology-emphasis majors. Students are taught basic field skills such as how to describe rocks, self locate on a topographic map, use a Brunton compass, and map and interpret simple geologic structures. Prerequisites are Physical and Historical Geology. Overall, 60-75% of students take the prerequisite classes at NAU and 25-40% are transfer students from state community colleges or from out-of-state community colleges or universities. Several different professors rotate through teaching GLG240, each with his/her own exercises and pedagogical styles. This report discusses a subset of these classes, all taught in the same style, with the same exercises and rubrics, by the same instructor. The overarching goal of the course is to provide experiences in the field that will give the student self-confidence in understanding and using the basic tools required to complete work- or thesis-related problems in field geology. The goal is met through a series of exercises 158 Journal of Geoscience Education, v. 54, n. 2, March, 2006, p. 158-164
Figure 1. Students using the SP Crater Virtual Environment. Program being navigated by young man with ponytail and game controller, center of left-hand bench. that provide experiences in rock description, simple stratigraphic analysis, and basic mapping. SP Crater Exercise - The VE component was added to the first major field exercise undertaken by the students. Prior to this exercise, students have spent several afternoons in and around Flagstaff, learning self location, local stratigraphy (late Paleozoic and early Mesozoic sedimentary rocks, late Tertiary - Quaternary volcanic rocks), and how to recognize and measure faults and joints. SP Crater is a mid-pleistocene (Baksi, 1974, in Ulrich, 1987) cinder cone in the northeastern San Francisco Volcanic Field (Figure 2). The field area is on open-access private property, and comprises Paleozoic limestone, several Pliocene or early(?) Pleistocene basaltic cinder cones and flows, the SP cone and flow, and alluvium. A graben is present in the area, but bounding structures are not exposed. Bedded Paleozoic rocks allow opportunity for measuring strikes and dips, and lava flows that are readily distinguishable based on erosional morphology and on phenocryst size and percentage provide experience in hand-lens use and mineral identification. The objectives of the SP Crater exercise are to 1) introduce students to different media available for mapping (specifically aerial photographs, see below, and topographic maps), 2) increase student confidence in self location by having students map in an area with significant topographic features that can aid in sighting or location, 3) teach students to understand the differences between scales on different maps and how to translate distance on the ground to distance on a map, and 4) teach students to follow and map clear contacts between the units in the field area. Student performance is assessed through a map and a written report. The SP Crater exercise comprises three parts, the third of which is the VE component. For all years the exercise has been taught, the exercise has begun in the laboratory with examination of a 1:30,000 stereo triplet aerial photographs of the SP Crater area. Students use stereoviewers and tracing paper to make a map that represents a ~ 50 km2 area. On the following weekend the class goes to the field, where students map a ~1 km2 part of the air photo area. During this part of the exercise, the students are introduced to the rock types, given a Kelly and Riggs - Use of a Virtual Environment in Field Mapping Figure 2. A. Eastern part of the San Francisco Volcanic Field. Flagstaff lies ~15 km south of San Francisco Mountain. B. SP Crater viewed from the northwest. Older flow in middle ground covered by bushes, older cinder cone to right (west) of SP crater; SP flow in front of and on middle skyline behind bush-covered older flow. three-hour introduction to basic mapping, and then are sent to work in groups without direct supervision. Instructor and teaching assistant map as well, and interact with students whom they meet in the field. The Experiment - We began the experiment in the Fall of 2003 and have continued it through Fall of 2004. Twenty-seven students have participated in the experimental phase. Research Goals - Our main research goal with the introduction of the VE component into the exercise was to affect the student's survey knowledge (Montello et al., 2004) of the SP Crater field area through student navigation of a VE modeled on SP Crater. Montello et al. (2004, p. 270) define survey knowledge as: Often conceived as a map in the head, survey representations allow direct access to quantitative spatial relationships such as distances and directions between arbitrary locations in an environment - not solely those locations between which one has traveled. Our objective was for students to gain survey knowledge and go into the field with a cognitive map (Johns, 2003) of the SP Crater field area. The visualization exercise was designed to increase spatial 159
Figure 3. VE as produced in ROMA software. Students navigate to each numbered box (e.g., 3 in foreground), describe the rock units beneath the box, mark location of the box on a color photocopy of the SP Crater 7.5 topographic sheet, and transfer contact information to the map. View of SP Crater from the east. understanding through a series of tasks to be completed within the VE. We also wanted to learn about student attitudes toward the technology, which were recorded through survey questions and narrative. Using the SP Crater Virtual Environment - Students completed the air-photo analysis working in one classroom while the VE of SP Crater was set up in another room. As students finished the air-photo exercise they moved into the VE room. An instructor (Kelly) introduced the students to the equipment and insured that students shared navigation duties when operating the VE, but offered only minor commentary during the exercise. Written instructions directed students to navigate to numbered markers floating above key locations on the landscape (Figure 3). The students were directed to locate each marker on a color reproduction of the topographic map, and to describe and record the observable rock units and the nature of any geologic contacts beneath the marker. Students were asked to draw any contacts they observed on the topographic map. Each student took a turn "driving" through the VE while the observing students offered unsolicited suggestions about navigation goals and steering techniques (Figure 1). For both years of the experiment, during the exercise the number of students using the VE gradually increased as more of the class finished the standard materials and 160 moved on to the VE lab room. Early arriving students generally stayed in the VE lab the longest. This resulted eventually in a large group actively using the VE. The group nature of the exercise meant cooperation among students was required for successful completion of the exercise. The instructor noted active verbal exchange of ideas, theories, and data. At the end of the exercise the color topographic maps and observation sheets were collected. The maps and descriptions were graded: two points were allotted for correctly locating the point on the topographic map, and one point for providing a correct geologic description Measuring Student Performance - The rubric used to score student geologic maps from the field is divided into a portion generally called "logistics", i.e., scale, north arrow, map explanation, etc., a portion for strikes and dips, and a section on "coverage and accuracy of contacts". The first two categories are objective and simply a matter of assessing if the required feature is on the map. Scoring in the third category follows a list (not provided to the student) that comprises overall coverage and general accuracy in distribution of units location of the SP flow margin older flow outcrop general shape and presence of small outcrops in the graben Journal of Geoscience Education, v. 54, n. 2, March, 2006, p. 158-164
Figure 4. Frequency distribution of scores for control group (classes in Fall 1998 and Spring 2002). location of the contact between Paleozoic limestone and "old" flow in west of graben general shape of Paleozoic limestone outcrop location of alluvium in selected areas An unexpected requirement of our study was the detailed articulation of a map-scoring rubric. In order to effectively compare different years of classes, we needed confidence that maps in all years had been scored similarly. Prior to 2002 this scoring rubric was not explicitly articulated, and we accomplished this before proceeding with the experimental phase. Intuitively, the problem in grading maps is that mapping is a subjective activity. Although certain contacts can very definitively been considered "correct" in only one place and configuration, most field geologists find that each visit to a field area yields new insights and new interpretations, clearly not based on new rocks present. In the case of a field class, perspective can also change about the importance of different aspects of the mapping area. The recorded map rubric for years 1995-2002 stated that points were allotted for "logistics", as defined above, and for "geologic reality": "doesn't need to be exactly correct, but should not have extraneous lines, should show some correspondence with reality, evidence of thought and work". Clearly the issue, in terms of comparing rubrics, was how the instructor (Riggs) scored "should show some correspondence with reality". Very careful reflection, using examples of maps scored with this method, yielded the rubric detailed above. The classes described in this paper were taught and graded by a single instructor. This instructor articulated the rubric and applied it, and found no places where a part of the rubric should be added to or changed. This inspired confidence that the articulation process reflected the original thought process used in the earlier years. Measuring Student Attitudes - Student attitudes about the VE were measured using a survey administered to students on their way to the actual field exercise. The survey queried the students on (1) their assessment of the importance of VE technology to their academic career; (2) their assessment of the importance of VE technology to their working career; (3) how effective they found the group experience using the VE; (4) if they would prefer to use the VE alone; and (5) their expectations for exposure to VEs in the college science classroom. Narrative Evaluation - As part of the final report students were required to write a one-paragraph narrative evaluating the use of the VE in the exercise. RESULTS The Lab Topographic Map - We combined the scores (n=27) from the two classes (2003, 2004) that used the VE. The mean score on the VE component "in-lab" maps and descriptions was 83.3 with a standard deviation of 12.3. Geologic Map Scoring Statistical Analysis - We wanted to test the effect, if any, of the VE component on student performance on the geologic map. The student groups who took the class before the use of the VE component are referred to as the control group and the students who took part in the VE component are referred to as the experimental group. Our goal in using a statistical analysis was look for differences between geologic map scores for these groups. We choose the non-parametric Wilcoxon rank sum test (Ott, 1993) because of non-gaussian distributions in class score sets. No data were excluded in our analysis and no students occurred both in the control and experimental groups, supporting the assumption that they are derived from independent samples. Control Group - We combined two years of GLG-240 classes prior to the use of the SP Crater VE into a single control group for this experiment. The discussion above indicates why we are confident that the scoring rubric has remained consistent between these two classes (Fall 1998 and Spring 2002) and the two classes that experienced the SP Crater VE (Fall 2003 and Fall 2004). We combined the two classes into one control group based on the instructor's knowledge of the classes and a non-parametric test (Wilcoxon rank sum) that suggest there is no significant difference between their ranked scores. Figure 4 shows the frequency distribution of scores of the control group (N=35). The median for this group is 60.0 while the mean is 59.9 and the variance is 360. Kelly and Riggs - Use of a Virtual Environment in Field Mapping 161
Figure 5. Frequency distribution of scores for experimental group (classes in Fall 2003 and Fall 2004). Survey Statement Positive Score (%) Importance of the VE technology in their academic career 96% Importance of VE technology in their future working career 92% The effectiveness of the VE group exercise 72% Importance of single-person access to the VE technology 92% Expectation of VE-technology abailability at NAU 48% Table 1. Student attitudes toward the SP Crater Virtual Environment component. Two separate class response sets (Fall 2003 and Fall 2004) were combined (N=25). Two students did not participate in this survey portion. Responses were measured on a 5 point Likert scale: 1 = not very important, not very effective, or no real expectation, while 5 = very important, very effective, or high expectation. A score of 3 was interpreted as neutral. Score counts of 4 and 5 were summed and divided by the total number of students to derive a percentage of positive scores Experimental Group - For the last two years (Fall 2003 and Fall 2004) that this exercise was taught, the VE component has been employed as described above. We have combined the two years into a single experimental group based on the instructor's knowledge of the students and a non-parametric test (Wilcoxon rank sum) that suggests there is no significant difference between their ranked scores. Figure 5 shows the frequency distribution of scores in the experimental group (N=27). The median for this group is 80.0, the mean is 75.4 with a variance equal to 184. We formulated the null and alternative hypotheses: Ho: There is no relationship between the control group and experimental group and their test score rank. If we accept this hypothesis then the VE probably had no effect. Ha: There is a relationship between the groups and their test score rank. If we accept this hypothesis then we suggest that there is an effect. A Wilcoxon rank sum test compared the ranked scores of the experimental group with the control group ranked scores. For an alpha level of 0.05, the mean ranks of 41.09 for the experimental group and 24.10 for control group were found to be statistically different, with a z-statistic = 3.67. This result suggests that we should reject the null and accept the alternative hypothesis. Student Attitudes and Narrative - Table 1 summarizes the student responses to questions regarding expectations toward the VE component. Informally, most students said that they had never experienced technology like the VE. Student narratives provided an extremely positive evaluation of the usefulness and effectiveness of the VE component. Two major themes that stood out in the narratives were the desire to have more structured, smaller group use of the VE, and the importance of the VE in their visualization of the SP crater terrain. These latter "visualization" comments are included in Table 2. Some students noted that they were already thinking of extensions of VE technology to other scenarios like "exploring mid-ocean ridges, abyssal plains" or "magma chambers". DISCUSSION The SP Crater VE component was designed to familiarize the students with the landscape, landforms, and surface geology of SP Crater from multiple perspectives with an anticipated effect of increased survey knowledge (Montello et al., 2004). Concomitant with this effect we anticipated an increase in student confidence and efficiency measurable through changes in accuracy in certain key parts of the geologic map made in the field and in narrative accounts of preparedness for fieldwork. As noted above, in the first test of the VE component students marked contacts on a topographic map and wrote preliminary descriptions at a series of locations in the laboratory. Student scores for this preliminary test were moderately high (mean = 83). This suggests that seeing a feature in the VE is useful in understanding its location on a two-dimensional map, an idea supported by student narrative. We surveyed the students and discovered they had a very high expectation for VE technology in both their academic careers and in later work careers (see Table 1). The observation that only about 48% thought they would 162 Journal of Geoscience Education, v. 54, n. 2, March, 2006, p. 158-164
The geowall was helpful for me because it allowed me to see that topography of the area and to further understand the characteristics of the lava flows. For instance, why the flows generally flowed to the north was because the elevation slightly decreased to the north, thus allowing the flows to head north. Through geowall I was able to see this concept at hand. It also put into perspective the size of the area we were covering. I was able to grasp the distances between flows or cinder cones much easier. The Geowall gave me a more realistic approach than the stereoscope because it was like driving around the area. The topography on the GeoWall was easy to see and easier to use compared to the Stereoscope. Its ability to map areas in three dimensions and allow the viewer to see them.. greatly decreased my amazing ability to get lost in the field. After viewing it I had a better understanding of the topography of the area. I think that the ROMA exercise was very helpful to be able to visualize the area and to see the surroundings before we actually went out there. I don't think that it helped me much in the actually mapping of the area but it was fun and interesting to see it that way. As far as the use of the Geowall, I believe it was a great new way to give a different or more complete perspective on the topography of the SP crater area. It was extremely helpful in visualizing the topography and structure of SP area. It was relatively simple to use with the controller once you played with it for a while. The exercise completely using GeoWall definitely helped me gain a better visual image of the field area before I went out to map it. I knew more of what to expect and what not to expect. The 3-D images give the students a different perspective on what they are researching both inside and outside the classroom. It didn't help me map the area any better bit it gave me a better understanding of what we were studying. I found it very helpful in picturing the layout of our map and where things might be found. I had never been to SP Carter and believe without looking at it first in 3-D I would have been a little more lost than I already was. As it has been said, it is not a replacement for actually going out in the field and seeing it yourself. But it was a big help in seeing how steep slopes were and how tall other structures were compared to SP Cater. The best use of the Geo-wall, however, was to check over field work. It made for great clarification of area I had trouble seeing through the stereoscope. It is very versatile in respect to the different vantage points of observation; both close up and zoomed way out. The stereoscopes were really hard to use and you couldn't just span an area without readjusting the photos and such. With the GeoWall we were able to see different viewpoints and zoom in or out of areas that the air photos couldn't. The stereoscopes were helpful but with Geowall you can move in and out of area without having to realign photographs. Table 2. Selected student narratives about the VE; transcribed verbatim. The major theme in the student narratives was about the power of visualizing the SP Crater terrain using the VE. Note that students refer to the VE as either Roma, or the Geowall. Complete narratives are available by contacting the authors. be exposed to VE technology at NAU suggests that this generation of students is "primed" for this technology, but may be prepared to be disappointed if instructors don't use it. Student narratives from the experimental group (Table 2) indicate that students found the VE a very helpful way for them to visualize SP Crater. Many students describe the importance of seeing different perspectives, being able to navigate and zoom in and out, and the superiority of the VE to the stereo photograph analysis. The vocabulary of these narratives supports the idea that survey knowledge was acquired, and that students built a cognitive map of SP Crater before they visited the field. Many students noted that after the VE component they knew what to expect at SP Crater, which along with the visualization comments we interpret as an increase in confidence. Using the non-parametric Wilcoxon rank sum test we detected a significant difference between the control group and the experimental group, suggesting that students using the VE perform better on geologic mapping than students who prepare for fieldwork without it. In the narratives, a few students (see Table 2) indicated that they found the VE useful in gaining a better visual image of SP but they didn't think the experience helped them map better. Although only a few students (2 out of 27) articulated this idea, the statistical analysis suggests that the VE component had an effect on performance, whether recognized or not. Jones (2003) suggests that navigation of a VE or any real environment provides for a subconscious mapping of the place. As noted earlier, our intent was to increase student confidence, and we suggest that the VE component accomplished this by putting "a map in the students head" before they went into the field. Student narratives indicate that they gained perspective; comments about location, heights, distances, and steepness of slopes all suggest an increase in survey knowledge. This performance difference between the control and experimental groups must be interpreted through the lens of the experimental constraints. First, the test group can be characterized as amateurs - at the very beginning of their field-geology careers. The effect of the VE component may be unusually large on this uninitiated group relative to students or professionals in later stages of geologic expertise. In addition we are constrained by small sample sizes. Ideally, this experiment should be reproduced at larger institutions and in varied populations. Second, although we carefully analyzed the rubric used for map scoring by the instructor in the control group, these same criteria were not as explicitly listed on the scoring rubrics in 1998 or 2002. We do have confidence that scoring remained constant through the four classes we discuss in this paper. Our suggestions for further work in this area include increasing the availability of VEs to students in field geology. Individual students should be able to spend more time manipulating the VE, but we turn to our Kelly and Riggs - Use of a Virtual Environment in Field Mapping 163
student surveys to suggest that the group exercise was also valuable (see Table 1). One innovation that we intend to implement is to allow students to put their own geologic field map into the ROMA software as a layer, so they can observe their own field mapping results in it. ACKNOWLEGDEMENTS We acknowledge the able assistance of teaching assistants Karen Vanaman, Luke Martin, Christian de Fontaine, Caroline Harris, Erin Todd, and Kelly Dilliard. Babbitt Ranches (CO Bar) are gratefully acknowledged for allowing access to their land. We would like to thank the Geology Department at NAU and in particular Abe Springer for maintaining the GeoWall Equipment, Brian Davis at SAIC/USGS EROS Data Center for his help with georeferenced imagery, and all the attendees of the 2003 Annual GeoWall Developers meeting for review of the VE. REFERENCES Baksi, A.K., 1974, K-Ar study of the S.P. flow, Canadian Journal of Earth Science, v. 11, p. 1350-1356. Bowman, D., North, C., Chen, J., Polys, N., Pyla, P., and Yilmaz, U., 2003, Information-rich virtual environments: Theory, tools, and research agenda, Proceedings of ACM Virtual Reality Software and Technology, p. 81-90. Jones, C., 2003, Spatial learning: cognitive mapping in abstract virtual environments, Proceedings of the 2nd international conference on computer graphics, virtual reality, visualisation and interaction in Africa, p. 7-16 MacEachren, A.M., Edsall, R., Haug, D., Baxter, R., Otto, G., Masters, R., Fuhrmann, S., and Qian, L., 1999, Virtual environments for geographic visualization: potential and challenges, Proceedings of the 1999 workshop on new paradigms in information visualization and manipulation in conjunction with the eighth ACM international conference on Information and knowledge management. Kansas City, Missouri, United States, p. 35-40 Montello, D. R., Waller, D., Hegarty, M., and Richardson, A. E., 2004, Spatial memory of real environments, virtual environments, and maps, in Allen, G.L., editor, Human spatial memory: remembering where, Mahwah, NJ, Lawrence Erlbaum Associates, p. 251-285 Ott, R. L., 1993, An Introduction to Statistical Methods and Data Analysis, Duxbury Press, CA, 1051 p. Ulrich, G.E., 1987, SP Mountain cinder cone and lava flow, northern Arizona, Geological Society of America Centennial Field Guide, Rocky Mountain Section, p. 385-388. 164 Journal of Geoscience Education, v. 54, n. 2, March, 2006, p. 158-164
Membership Application or Renewal Form Name: Phone: Mailing Address: Fax: Email: City: State: Zip: College/University Professor @ Precollege Teacher @ Other @ Checks, MasterCard, or VISA (US funds only) are payable to:national Association of Geoscience Teachers. Mail to: NAGT, PO Box 5443, Bellingham, WA 98227-5443 Membership Regular USA Outside USA Student USA Student outside USA Retired NAGT member Library Subscriptions Regular USA Outside USA New Rates (US funds) $35 $47 $20 $32 $30 $55 $67 Renewal Check Credit card: MC/VISA (circle one) Number: Signature: Exp. Date The Journal and membership year runs from January to December. Subscriptions received after June 1 will begin receiving the Journal in January of the following year. Back issues are available for $15 (foreign $18) each. *To qualify for student rate, indicate and obtain verification from a NAGT member: Undergraduate Graduate Signature of NAGT member School