This article was downloaded by: [Andrew Paney] On: 17 November 2014, At: 14:01 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK Music Education Research Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/cmue20 Singing video games may help improve pitch-matching accuracy Andrew S. Paney a a Department of Music, University of Mississippi, 164 Music Building, Oxford, MS 38655, USA Published online: 14 Nov 2014. To cite this article: Andrew S. Paney (2014): Singing video games may help improve pitch-matching accuracy, Music Education Research, DOI: 10.1080/14613808.2014.969218 To link to this article: http://dx.doi.org/10.1080/14613808.2014.969218 PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the Content ) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/termsand-conditions
Music Education Research, 2014 http://dx.doi.org/10.1080/14613808.2014.969218 Singing video games may help improve pitch-matching accuracy Andrew S. Paney* Department of Music, University of Mississippi, 164 Music Building, Oxford, MS 38655, USA (Received 19 May 2013; accepted 18 September 2014) The purpose of this study was to investigate the effect of singing video games on the pitch-matching skills of undergraduate students. Popular games like Rock Band and Karaoke Revolutions rate players singing based on the correctness of the frequency of their sung response. Players are motivated to improve their scores each time they play. In this experiment, I tested undergraduate nonmusicians (N = 33) pitch-matching ability before and after playing one of these games for 10 minutes. Acoustical analysis revealed a significant difference between pre- and post-test scores. These data suggest that playing the video game for 10 minutes may have produced an immediate, observable change in pitch-matching scores. Music teachers regularly teach students how to develop pitch-matching skills. Games like the one used in this study may add to the tools used in that process. Keywords: feedback; pitch-matching; video games; independent learning One of the most basic musical skills is to match pitch. Music teachers, especially those who teach beginning musicians, regularly teach students to match pitch in order to develop their singing, instrument playing and other musical skills. This can be a tedious process, particularly when classes include both students who match pitch well and those who do not. Generally, teachers approach this problem by addressing it immediately after students have sung: the teacher gives verbal feedback or models a correct response and students work to emulate it and improve their singing. The real-time visual feedback provided by a video game may be able to reduce the delay between a student s response and constructive feedback. Computer programs and singing growth Technology that gives feedback on singing may be more easily interpreted by students and leads to quicker resolution of pitch problems by offering more direct, applicable information, especially when used in conjunction with traditional instruction. Hoppe et al. 2006 examined the benefits of using four real-time visual feedback programs designed specifically for use in teaching singing: SINGAD, ALBERT, Sing& See TM and WinSingad. These programs test pitch, pronunciation, vowel identity, timbre and combinations of these. They were designed for both novice and skilled singers from children through adults. The authors emphasised *Email: apaney@olemiss.edu 2014 Taylor & Francis
2 A.S. Paney that, though these programs can be helpful in lessons, the feedback may be more effective when it is interpreted with a singing teacher: It is noteworthy that VFB [Visual Feedback] serves well as a tool for assistance, rather than a replacement of the singing teacher (315). They also suggested that computer feedback is more helpful when it focuses on an external result rather than an internal behaviour. For instance, seeing spectral information on pitch accuracy is more helpful than seeing how to adjust the vocal tract. Singers, particularly novice singers, preferred feedback that displayed spectrogrammes (contour) to models showing only a piano keyboard. Overall, the researchers saw consistent benefits of using visual feedback both with and without supervision. Welch et al. (1989) used SINGAD software to test the singing development of seven-year-old children (n = 32) over seven weeks. They used a pre-/post-test design with three matched groups to investigate the effects of using a software program with interaction from a teacher, the software program alone and a control group that used a more traditional singing approach with no computer software. The computer program served in both the evaluation (pre- and post-tests) and the treatment of the two experimental groups. The experimental portion involved having students see the contour of their voice on a computer screen, with higher pitches near the top of the screen. One of the activities available on SINGAD asked participants to adjust their sung pitch to hit target icons (rockets, horses, flowers, etc.) displayed at various levels on the screen. All three groups improved over the seven-week period, but the experimental groups, the ones who used the software, showed a significantly higher improvement than the control group. The experimental interactive group showed the most improvement. Welch et al. concluded that the traditional method of giving feedback after a performance may not be as effective as real-time visual feedback (that can only be given reliably by a computer) and that the most effective use of the technology is in a classroom with both computer- and teacher-based feedback. Wilson, Thorpe, and Callaghan (2005) found that pitch accuracy improved with use of the visual feedback computer program Sing&See. They tested the pitch accuracy of three groups of adults using the program. All participants received a single one-hour voice lesson, though their activities varied depending on their assigned group. Two treatment groups received visual feedback in different forms on a computer screen as they sang scales, single notes and songs. The third group, the control, sang the same material, but did not receive feedback. Researchers tested their pitch accuracy before, during and at the end of the treatment using the same five-note pattern (the pattern changed based on the level of experience of each singer). Both treatment groups scored lower than the control on the test conducted during the treatment. On the final test, however, the two treatment groups scored significantly higher than the control. The researchers concluded that, although using concurrent, visual feedback initially caused scores to worsen, those who used it ultimately improved their pitch accuracy more than those who did not. Welch et al. (2005) tested the reliability of data provided by WinSingad, a computer program designed for use with adults in private voice lessons. They investigated the use of the program over 12 months in two voice studios in England, each with four adult voice students ranging from skilled amateur to advanced professional. Data included journals kept by the teachers and students and a secondby-second time log of the lesson. Researchers tracked the amount of time spent using the program in lessons and the change in the type of instruction in a lesson once the
Music Education Research 3 technology was introduced. Teachers were free to use the program as much or as little as they liked. One of the teachers used it about one-third of the lesson time, while the other used it about two-thirds of the time. Both teachers reported that they were able to successfully integrate the technology into their lessons. The teachers evaluated and discussed the feedback provided by the program with students, both during and after singing. Use of the program increased the use of teacher demonstration for one teacher and the use of teacher instruction and student performance for the other. Both teachers and students reported clear benefits of using the program. One of the teachers insisted on using it with all of his students because he felt it would be unethical not to, based on the significant benefits it provided for those who were using it. Music learning and video games The computer programs described are not widely available (except Sing&See, which can be purchased online) and require some expertise, either for using the software or for interpreting the feedback. Similar work has not yet been completed using commercially available video games, though there are published papers suggesting the benefits of using video games in teaching music (Denis and Jouvelot 2004; Lee et al. 2005; Gower and McDowall 2012; Nardo 2010). Video games may help develop some musical skills. Gower and McDowall s (2012) qualitative study involved interviews of nine children aged 9 11 and two music teachers. Participants responded to semi-structured interview questions about their experience with video games, their views about interactive music games and about their own musical backgrounds. The authors concluded that games are important to students and more research is necessary to see how they can be best used in the classroom. Though they generally do not use traditional notation, video games may be a stepping stone to more advanced musical concepts. Peppler et al. (2011) tested the effects of using the video game Rock Band with 26 youth at an after school club. Data were collected from the video game itself, from surveys and tests, and from researchers observations. They found that students traditional rhythmic skills improved over the nine months of observations. They concluded that using games like Rock Band may provide an entry point to inexperienced students to develop traditional musical skills. Though studies have found computer programs to improve some singing skills and papers have suggested the possibilities of musical growth through video games, no study was found that tested participants singing improvement through the use of real-time-visual-feedback singing video games. Teachers of singing regularly give students verbal feedback and suggest strategies for improving singing, but it is difficult for them to give objective, continuous feedback while students are singing. The purpose of this experimental study was to test the effect of using commercially available singing video games on college age, non-music majors pitch-matching ability.
4 A.S. Paney Method Participants (N = 33) were non-music students at a large American university who responded to a sign requesting volunteers for a research study in music and video games. The sample included men (n = 16) and women (n = 16) aged 18 22 (one participant did not report gender). Self-reported experience playing video games ranged from beginners to experts and 13 reported having played singing video games, though only one had played the game used in the study. The most reported singing video game was Rock Band (n = 10), a game whose format is similar to that of the game used in this study (both were developed by Harmonix Music Systems). Five participants were not included in the final sample: One changed octaves in the post-test, one did not finish singing the pre-test, and data were unusable for three others due to problems with microphone interference. Treatment This experiment used a pre-/post-test design with a treatment. For the treatment, participants sang a song on the video game Karaoke Revolution Presents: American Idol Encore (2008) on the PlayStation 2 as many times as possible in 10 minutes. Participants chose a song from the 37 pop songs available on the game. Participants who did not recognise the song titles listened to previews of the songs before choosing one that was familiar to them. All were able to find a song they knew. The game provides a visual representation of the contour of the melody and scores participants by how close their sung response is to that melody. Participants sing into a microphone while hearing a sung melody and an accompaniment equivalent to the standard radio edition of the song. In contrast to other singing games that highlight participants responses immediately after they sing them, this game uses a moving arrow that follows the contour in real time and gives participants concurrent feedback on each pitch they sing. The arrow tells them the degree to which their pitch is too low or too high. If they are matching the pitch, the arrow sparkles and follows the contour on the screen like a train on railroad tracks. The game was set to the easiest, most forgiving level for all participants. The game provides a score between zero and 30,000 for each song a player completes. How the score is calculated is proprietary information, but the score increases when more notes are sung correctly. Participants sang their chosen song as many times as possible during the 10 minutes. Then they recorded their score on each attempt and were asked to try to improve their scores. This task was intended to encourage participants to work for high scores. Their reported scores were not included as data in this experiment. Participants saw the game on a 46-inch Sharp AQUOS LC-46D64U TV and heard the songs through the built-in speakers. Pre- and post-tests The pre- and post-tests were identical. Participants sang Happy Birthday to You while hearing a recording of a flute playing the melody and a piano playing a simple, chordal accompaniment. The song was recorded at 80 bpm in F-Major so that the octave range fell between C and high C. This design was chosen to be similar to the game in which players hear the sung melody and an accompaniment, but cannot adjust the range or the key. Volume levels were set so that the accompaniment could
Music Education Research 5 be heard clearly while a person was singing. I created the recording using Finale 2009 and converted it into Audacity 1.2.5. The song was chosen for its familiarity it was expected that every participant would have sung it before (see Mang 2006; Price 2000). Participants could read the text from a printed sheet on a music stand in front of them. Participants heard the melody on JVC HA-G11 headphones, sang into a Logitech USB hand-held microphone (the model sold with the video game) and were recorded using Audacity software on a MacBook Pro computer. Procedure Participants began the experiment by familiarising themselves with the controls for the game system and by choosing a song. Next, they entered a private room, put on headphones, held a microphone and sang Happy Birthday to You. They then moved to another room where the video game was set up and sang the song they had chosen as many times as possible in the allotted time of 10 minutes. Finally, participants returned to the first room and again sang Happy Birthday to You with the same accompaniment. Analysis Participants vocal responses were analysed using a procedure similar to that of Demorest and Clements (2007). Target tones were the four notes after the anacruses in each phrase ( birth-day to you ) a total of 16 notes per participant. Target tones were analysed, omitting the attack and release, for estimation of fundamental frequency by the frequency analysis spectrum component of the Audacity software program. The resulting fundamental frequencies for each sample were compared to that of the reference pitch of the flute. The distances between the reference pitches and the participants pitches were calculated using a hertz-to-cents conversion macro in Microsoft Excel 2008 for Mac. The number of out-of-tune notes for each subject was calculated by counting the number of responses that were more than 50 cents from the target pitch. This range was intended to give a report of the number of notes that, though possibly out-of-tune, would essentially be considered a match (Demorest and Clements 2007). Reliability Two months after the initial analysis, I re-examined a random sample of 25% of participants fundamental frequency scores. Test/re-test reliability coefficients of r = 0.95 were obtained for all samples. Results This project examined participants pitch-matching scores before and after playing a singing video game. Participants mean scores and the number of notes they sang outside of a 50 cent range are shown in Table 1. A correlated-samples t-test found a significant difference in participants (n = 28) pre- and post-test mean scores (t(27) = +2.41, p = 0.023). A comparison of only participants who averaged more than 50 cents from the target pitches on the pre-test (n = 19) also showed a significant difference (t(18) = +2.58, p = 0.019).
6 A.S. Paney Table 1. A summary of participants pre- and post-tests. Mean cent deviation Number of pitches > 50 cents Participants Pre-test Post-test Pre-test Post-test 1 16.31 21.91 1 3 2 20.37 31.44 1 1 3 21.22 21.32 0 1 4 23.33 29.01 2 3 5 24.63 28.28 0 3 6 26.32 19.8 3 1 7 27.75 32.47 2 3 8 27.87 25.01 2 1 9 32.6 29.61 4 3 10 52.01 26.62 4 2 11 53.04 138.07 7 10 12 58.64 42.88 5 1 13 58.73 60.08 9 9 14 71.44 31.01 11 3 15 79.59 53.7 10 6 16 81.86 124.44 9 10 17 92.81 119.17 10 5 18 96.07 64.97 10 7 19 139.2 72.9 11 9 20 160.78 94.51 11 10 21 202.83 207.73 14 15 22 302.43 55.8 13 6 23 324.47 376.93 13 15 24 365.83 248.54 16 10 25 368.7 228.97 14 12 26 415.55 40.31 12 5 27 480.51 293.28 12 15 28 523.98 369.43 16 15 Note: Participants are ordered by their pre-test mean cent deviation. To compare less accurate singers, those whose average deviation was greater than 50 cents on the pre-test were compared (participants 10 28, n=19). Those with more than four inaccurate pitches (pitches more than 50 cents from the target) on the pre-test were also compared (participants 11 28, n=18). The number of notes outside of a 50 cent range from the target pitches for each participant was compared for the pre- and post-tests. Participants showed a significant decrease in notes considered out-of-tune (t(27) = +2.60, p = 0.032). A comparison of only inaccurate participants (n = 18), those who had more than 4 notes of the 16 (25%) outside of the 50 cent range on the pre-test, yielded a similar result (t(17) = +2.69, p = 0.015). Playing a singing video game for 10 minutes appeared to improve participants overall pitch-matching scores, particularly those of inaccurate singers (participants who averaged more than 50 cents from the target on the pre-test or who had more than 25% inaccurate pitches on the pre-test).
Music Education Research 7 Discussion and implications This study found an increase in pitch-matching scores for untrained singers at an American university after using a commercially available singing video game. Participants showed a significant improvement after playing the video game for 10 minutes. Providing visual feedback may help students improve their singing. Welch et al. (1989) found that singers who received computer feedback improved significantly more over a seven-week period than those who did not. Their suggestion that realtime visual feedback given by a computer may be more effective than verbal instruction alone is supported by the current study. The format of the video game feedback used in this study, the contour on the screen juxtaposed with the sung response, may be an effective way to provide feedback for inexperienced singers. Hoppe et al. (2006) found a preference for spectrogramme contour representations in their investigation with novice singers. A visual representation of pitch was also found to be effective in improving pitchmatching with children using high and low visual aids for ages 6 9 (Persellin 1993) and using gestures for ages 5 6 (Liao 2008). Using multiple modalities, particularly visual feedback, appears to be an effective approach to teaching pitch-matching to multiple age groups. It is possible that simply singing for 10 minutes improved participants singing scores, and not the influence of the concurrent feedback or the game scores. A design using a control group that did not see feedback while they sang could, perhaps, answer this question. A game that encourages singing for several minutes at a time, however, may have instructional value. Inaccurate singers, those who sang outside of a 50 cent range of the target pitch on the pre-test, experienced greater growth than those who sang accurately. Working towards pitch-matching with new singers is often a time-intensive task. A video game that encourages singing independently and provides visual feedback may be able to help reduce time needed with a voice instructor. It may also help inexperienced singers to sing more frequently and may help with motivation, especially in early stages of vocal learning. Though scores on the video game were not part of the analysed data, I noted that most singers improved their scores on the video game, only three showed a decrease, and seven scored perfectly on each attempt. The game was set at the easiest level for all participants. Each level is harder than the prior level and some participants may have benefited more from using a harder level. Selecting a harder game level or adjusting the level to match participants level of singing achievement could mitigate this. Asking participants to sing the same song multiple times with or without visual feedback could also yield valuable results. Participants appeared to enjoy the game s feedback and many expressed a desire to play the game again. Giving students early successes may encourage them to continue working on developing singing skills. Games may also be able to address some of the psychological issues (Karpinski 2000) related to singing (like previous bad experiences) by allowing students to see their own growth. Further quantitative and qualitative research could explore novice singers growth and their perceptions of video games in their musical growth over a longer period of time.
8 A.S. Paney Previous studies that found a positive effect of using visual feedback used a treatment of 1 hour, 7 weeks and 12 months (Welch et al. 2005; Welch et al. 1989; Wilson, Thorpe, and Callaghan 2005; ). The current study, however, demonstrated a difference after a 10-minute treatment. Though it is encouraging to see scores improve quickly, it is unclear whether the learning would persist. This study could serve as a basis for a similar study with a longer treatment. Asking participants to take the game home and play it the way video games are normally played (i.e. repeated attempts until goals are met) could provide additional information. Would people who played a singing game at home over a month see a lasting improvement in their pitch-matching ability? The effects of using the game with coaching rather than alone, an approach that showed success for Welch et al. (1989), might also be beneficial in understanding the role a teacher could serve and how a teacher could make better use of available technologies. Studies could also examine two separate groups: one that received concurrent visual feedback and one that did not. Studies testing the pitch-matching ability of instrumentalists could follow a similar approach. Conclusion This study found a significant improvement in pitch-matching for subjects who played a singing video game for 10 minutes. This suggests that inexperienced singers may be able to improve their pitch-matching using singing programs that give realtime, objective, visual feedback. Few teachers at any level see their students as often as they would like. Because of this, ways to improve musicianship outside of the classroom have great value. Games like the one used in this study may help students begin to hear and to feel what pitch-matching is. As they see their success charted on the game, their confidence in their skills may improve and they may begin to grow towards consistent pitch-matching. Though teachers expertise as musicians and pedagogues are essential for student learning, using a program like the one used in this study could prove valuable for students use in class or outside of class for additional practice. Notes on contributors Andrew S. Paney is Assistant Professor of Music Education at the University of Mississippi. He holds degrees from Wheaton College in Illinois and Texas Tech University. He taught public school music in Illinois and Texas and worked with children s choirs in Lubbock, Texas and Oxford, Mississippi. His research interests include aural skills acquisition, pitch-matching and best practices in music education. References Demorest, S. M., and A. Clements 2007. Factors Influencing the Pitch-matching of Junior High Boys. Journal of Research in Music Education 55 (3): 190 203. doi:10.1177/002242 940705500302. Denis, G., and P. Jouvelot. 2004. Building the Case for Video Games in Music Education. In Second International Computer Game and Technology Workshop 156 161. http:// citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.4.7822 Denis, G., and P. Jouvelot. 2005. Motivation-driven Educational Game Design: Applying Best Practices to Music Education. In Proceedings of the 2005 ACM SIGCHI International
Music Education Research 9 Conference on Advances in Computer Entertainment Technology, edited by Newton Lee, 462 465. New York: ACM. Gower, L., and J. McDowall 2012. Interactive Music Video Games and Children s musical Development. British Journal of Music Education 29 (01): 91 105. doi:10.1017/s02650 51711000398. Hoppe, D., M. Sadakata, and P. Desain 2006. Development of Real-time Visual Feedback Assistance in Singing Training: A Review Computer-assisted Singing Training. Journal of Computer Assisted Learning 22 (4): 308 316. doi:10.1111/j.1365-2729.2006.00178.x. Karaoke Revolution Presents: American Idol Encore [PlayStation 2 game]. 2008. Tokyo: Konami. Karpinski, G. S. 2000. Aural Skills Acquisition: The Development of Listening, Reading, and Performing Skills in College-level Musicians. Oxford and New York: Oxford University Press. Liao, M.-Y. 2008. The Effects of Gesture Use on Young Children s Pitch Accuracy for Singing Tonal Patterns. International Journal of Music Education 26 (3): 197 211. doi:10. 1177/0255761408092525. Mang, E. 2006. The Effects of Age, Gender and Language on Children s Singing Competency. British Journal of Music Education 23 (2): 161 174. doi:10.1017/s02650517 06006905. Nardo, R. 2010. Got Rhythm? Get RockBand! General Music Today 23 (2): 38 39. doi:10.1177/1048371309352957. Peppler, K., M. Downton, E. Lindsay, and K. Hay 2011. The Nirvana Effect: Tapping Video Games to Mediate Music Learning and Interest. International Journal of Learning 3 (1): 41 59. doi:10.1111/1467-9280.00345. Persellin, D. C. 1993. Influence of Young Children s Learning Modalities on Their Pitch Matching. Perceptual and Motor Skills 76: 313 314. Price, H. E. (2000). Interval Matching by Undergraduate Non Music Majors. Journal of Research in Music Education 48: 360 372. doi:10.2307/3345369. Welch, G. F., D. M. Howard, E. Himonides, and J. Brereton 2005. Real-time Feedback in the Singing Studio: An Innovatory Action-research Project Using New Voice Technology. Music Education Research 7 (2): 225 249. doi:10.1080/14613800500169779. Welch, G. F., D. M. Howard, and C. Rush 1989. Real-time Visual Feedback in the Development of Vocal Pitch Accuracy in Singing. Psychology of Music 17 (2): 146 157. doi:10.1177/0305735689172005. Wilson, P. H., C. W. Thorpe, and J. Callaghan. 2005. Looking at Singing: Does Real-time Visual Feedback Improve the Way we Learn to Sing? Paper presented at the Second APSCOM Conference: Asia-Pacific Society for the Cognitive Sciences of Music, Seoul, South Korea, August 4.