Voice Recognition based on vote-som

Voice Recognition based on vote-som Cesar Estrebou, Waldo Hasperue, Laura Lanzarini III-LIDI (Institute of Research in Computer Science LIDI) Faculty of Computer Science, National University of La Plata La Plata, Buenos Aires, Argentina {cesarest, whasperue, laural}@lidi.info.unlp.edu.ar Abstract Voice signal processing with the purpose of recognizing the person speaking is a very interesting problem in the area of security that can be solved based on biometric techniques. This paper presents a new method, vote-som, to obtain a model that allows recognizing speakers based on their voice, independently of the text used. The model is built from adapted dynamic selforganizing maps to suitably represent the Cepstral coefficients of the corresponding audio signals. Speakers are recognized by means of a voting system. The results obtained have been better than those obtained with other existing methods. Index Terms Voice Recognition; Dynamic Self-Organizing Maps; Cepstral coefficients. I. INTRODUCTION Voice recognition to identify the person speaking is a highly useful tool in the area of security, since it allows validating that the person in question is who they claim to be or when, for some particular reason, the person speaking needs to be recognized. Audio signal digital treatment is one of the biometric techniques that can be used to perform the tasks of identification and verification [7] [13] [16]. For several years, researchers have developed methods and techniques that allow recognizing people through their speech. Since voice is produced through the convolution of an excitation and the impulsive response of the vocal tract model, it is sometimes useful to isolate one of the two components for subsequent digital treatment. One of the simplest techniques to separate the components of an audio signal is filtering. Currently, voice frequency analysis is the most widely used technique for capturing the typical components of this type of signals. The spectral parameters that are most frequently used for audio identification are mel scale Cepstral coefficients, or MFCC (Mel-Frequency Cepstral Coefficients) [6]. This analysis method with Cepstral coefficients is based on the human perception system, establishing a logarithmic relation between the real frequency scale (Hz) and the perceptual frequency scale (mels). These coefficients, by allowing the extraction of specific characteristics from an audio signal, are commonly used for building voice models for different people. Cepstral components in themselves cannot recognize speakers by means of their audio signals; however, a set of consecutive components in the signal are helpful for the identification and validation task. For this reason, it is useful to have intelligent strategies that allow building models that can represent a set of components and, at the same time, represent an audio signal. Several methods have been developed to achieve a modeling that solves the recognition problem [10] [11] [14] [15]. Selforganizing maps (SOM) [9] have proven to be very efficient tools for building models. Their learning capability allows them organizing their internal structure respecting the topology of input data. Dynamic self-organizing maps are an advance in these architectures, since they solve the main problem of SOMs the need to define beforehand the dimension of the structure to use. To carry out this task, they add, remove and reconnect elements within their own architecture during the learning stage. This dynamism allows improving the capacity to represent input data and thus yields better results. For this paper, a three-stage algorithm, called vote-som, was developed to recognize several people by analyzing their voices. During the first stage, the signal is pre-treated and the characteristic components are extracted; the second stage consists in the treatment of a dynamic SOM network with [8]; and the third and final stage establishes the correspondence between each element of the network and the person it represents. Once the model is built, the identification and validation tasks can be carried out to identify people. The remaining sections of this paper are organized as follows: In Section 2, a brief description of the operation of dynamic self-organizing maps is included. Section 3 describes the representation used for voice recognition, whereas Section 4 describes in detail how to use a neural network with these characteristics for solving voice recognition problems. Section 5 explains the mechanism used by the vote SOM method to determine who the speaker is. The results obtained are presented in Section 6, and Section 7 presents the conclusions and future lines of work. II. DYNAMIC SELF-ORGANIZING MAPS Competitive neural networks with non-supervised training are one of the most widely used tools to solve clustering problems, since they do not require knowledge of isolated solutions to the problem to carry out their learning process. In this category, self-organizing maps (SOMs) [9] have proven to be capable of learning the organization of input data, which allows obtaining a structure that respects their topology. However, SOMs, and other similar networks, have two major limitations. In the first place, the dimension and the structure of the network must be defined beforehand, before the training stage begins, which conditions the results and the efficiency of 89

the response obtained. Secondly, the capacity of the network is given both by the number of nodes it contains and its learning parameters. Dynamic self-organizing maps are used in an attempt to solve these problems. Among the different existing methods proposed to define the architecture, it can be seen that the incorporation of elements is varied, with neural networks that add them in an isolated fashion, while others add entire layers [1] [4] [5]. For this paper, the AVGSOM [8] method was used as the tool to generate voice models corresponding to different people for their subsequent validation and identification, taking advantage of the capability that these networks have to group in a suitable and dynamic manner the information received as input. Thus, each person s speech characteristics will be grouped in a same neighborhood within the structure of the network. III. VOICE SIGNAL REPRESENTATION The first stage of the algorithm to generate the model consists in carrying out an analysis of the voice signal for all speakers for whom the model is to be built. From this analysis, the Cepstrum coefficients that will be used in the second stage of the algorithm are obtained. The voice signal is sampled by applying a pre-emphasis filter to flatten its spectrum. Then, samples are separated in 20-ms frames or fragments, with a 10-ms overlap between adjacent frames to ensure stationality both in each frame as well as in overlapping intervals. This division of the signal generates high-frequency noise on the edges of each frame because there are sudden changes that go from zero to the signal and from the signal to zero. To reduce this effect, a Hamming window is applied to smooth down the signal towards the edges of each frame. Then, the fast Fourier transform (FFT) is applied to obtain the spectrum corresponding to the frequency of the signal. To imitate the mechanism of the human ear and its sensibility, a series of triangular band-pass filters with great overlapping is applied based on the mel scale. These mel filters, linearly distributed at low frequencies and logarithmically distributed at high frequencies, capture the most important phonetic characteristics of the voice signal. Then, the log values of the signal after obtaining the mel filters are calculated to obtain the logarithmic spectrum. Finally, the discrete cosine transform (DCT) is applied to the signal to obtain mel frequency Cepstral coefficients (MFCC). These coefficients are 13-dimensional vectors. IV. USE OF SOMS FOR SPEECH MODELING Once the process of extracting the MFCC of the voice signal finishes, a set of vectors representative of each speaker to be incorporated to the model is obtained. This set of vectors is divided into two parts: one for training and another for testing. Thus, the training set used for the SOM is formed by all 13-dimensional vectors that were obtained as a result of the analysis of the voice signal of all speakers that were used to build the model. Figure 1 shows all the processes through Fig. 1. Processes of the analysis of an audio signal. Fig. 2. Final structure of an AVGSOM network after being trained with voice signals from three subjects. Each ellipse identifies the competitive neuron region associated with each speaker which an audio signal goes through until becoming the voice model of a speaker. For the second stage, the MFCC are used to train a dynamic SOM network by means of the AVGSOM method. As self-organizing maps use the concept of neighborhood for their training, during this stage, a structure with its inner elements organized as well-defined regions is generated; that is, the elements located at close-by positions within the structure correspond to vectors with similar characteristics in the input space (Figure 2). Once the training stage is finished, each neuron in the network may represent one or more speakers. This is determined from the vectors used for training that belong to the receptive field of the neuron. If the vectors within the receptive field belong to the same speaker then the neuron represents only this speaker, otherwise the neuron represents more than one. For the neurons that represent more than one speaker, the level of representation of each speaker must be established. 90

Therefore, for each neuron that has a set of vectors V i within its receptive field, corresponding to a set of speakers L i, an ordered list of elements of L i is generated. The order of this list is defined by the rate of occurrence of each speaker of L i among the vectors of the set V i. Thus, a neuron with 50 vectors in its receptive field, 5 of whom belong to the speaker A, 30 to B and 15 to C, will have the following ordered list: B, C, A. After the training stage and determining the level of representation of each speaker for each neuron, a voice model that allows validating or identifying the person speaking is obtained. V. IDENTIFICATION MECHANISM The response mechanism to decide who the speaker is consists in a voting system. Since any vector on its own does not allow identifying any given person, several consecutive vectors in the audio signal are taken to determine who the speaker is. The idea of this mechanism is that a given sequence of vectors in the voice signal is enough to determine to whom the signal belongs. The voting system uses n consecutive vectors belonging to the audio signal of the speaker to identify. Upon being input to the network, each vector yields a winning neuron. A neuron is considered to be the winner if it best represents the input vector for a given similarity measure. As already mentioned, each neuron in the network can represent one or several speakers. The ordered list that is associated to each neuron allows not only identifying these speakers, but also specifying a relative significance level for them. This level takes its maximum value at the top of the list and decreases towards the end. In order to quantify this significance level, the following decreasing function is used: f(x) = K log 2 (K) + x where K is the number of speakers represented in the network and x is the position of the speaker on the list. The function described by equation (1) will assign the highest values to the first entries on the list, and lower values as it goes down the list. Thus, the speakers that have a higher representation within the neuron will have a greater weight. Therefore, after the network vector is input and the corresponding winning neuron is determined, the frequencies of the values obtained with function (1) for each speaker on the list are accumulated in a vector. The frequency vector has as many elements as different speakers are represented in the network. Therefore, the position that corresponds to the maximum value in this structure will allow, after all vectors have been processed, identifying who the speaker is. In Figure 3, the pseudo-code of the voting system is shown (1) V ects = Vectors uses for identifying K = number of speakers using for training T = frequency vector T[s] = 0,s = 1..K for all each element v from Vects do N = winner neuron for v L = ordered list of speaker belongs N x = 1 for all each speaker s in L do T[s] = T[s] + K/(log 2 (K) + x) x = x + 1 end for end for B = b, where T[b] = max(t[s]), s = 1..K B is the speaker indentified by the model Fig. 3. Pseudo-code for the voting system TABLE I FOR EACH TEST, THE PERCENTAGE OF SUCCESSFUL VOICE IDENTIFICATIONS IS SHOWN. Window Recognition Rate Test Length Shift 1 seg. 2 seg. 5 seg. 8 seg. 1 15ms 5ms 73.3% 79.7% 85.0% 88.7% 2 20ms 5ms 77.6% 85.1% 91.2% 93.8% 3 20ms 10ms 86.1% 91.0% 94.5% 95.9% 4 25ms 5ms 81.6% 87.1% 92.4% 94.0% 5 25ms 10ms 75.6% 84.2% 88.9% 91.9% 6 30ms 5ms 84.1% 89.7% 92.5% 94.7% 7 30ms 10ms 82, 9% 89.3% 92.9% 94.4% 8 30ms 10ms 78, 4% 87.0% 90.9% 93.8% VI. RESULTS The vote-som method has been tested using the voices of 30 speakers. The text read by each speaker has an approximate duration of 1 minute. In all cases, the first 20 seconds were used for training, and the remaining time was used for testing. All audio signals were processed with the Praat software [12] to transform the signal into 13-dimensional vectors, or Cepstral coefficients. With the purpose of comparing the results obtained with the recognition method proposed in this paper to those presented in [3], eight tests with various window sizes and audio signal spacing were carried out. In all cases, neural network training consisted in 200 iterations over the entire set of test vectors. Even though satisfactory results have been obtained with this number, it can be modified by a termination condition based on the stabilization of the number of network neurons. The average time required to complete this stage was 2 hours. The results obtained are summarized in Table I and Figure 4. As it can be observed, even with small window sizes, the recognition rate obtained is acceptable. The fact that the best results were obtained with larger window spacing is a positive aspect, since this reduces the number of vectors generated from the audio signal. For instance, Table I shows that a window whose size is 30 yields better recognition rates than a window whose size is 91

97,0 92,0 ) 1 (% 87,0 a te 2 R n 3 itio g n 4 co 82,0 5 e R 6 77,0 72,0 Fig. 4. 1 sec. 2 secs. 5 secs. 8 secs. Representation of the values in Table I # Test TABLE II RECOGNITION RATES FOR FIVE METHODS. THESE VALUES WERE TAKEN FROM [3]. Recognition Rate Method 2 seg. 5 seg. 8 seg. VQ 78.1% 83.5% 84.6% GMM 83.3% 88.4% 90.4% SVM 84.5% 91.7% 92.5% LS-SVM 85.1% 92.1% 92.9% Vote-SOM 91.0% 94.5% 95.9% 15, despite the fact that the former requires half the training vectors than the latter. This considerably reduces the response time of vote-som. As regards other existing methods, it can be said that the vote-som method offers higher recognition rates, as shown in Table 2 and Figure 5. The values included in this table for the other methods are taken from [3]. 95,0 90,0 85,0 80,0 75,0 Fig. 5. 2 seconds 5 seconds 8 seconds VQ GMM SVM LS-SVM Vote-SOM Representation of the values in Table II 8 VII. CONCLUSIONS AND FUTURE WORK A novel efficient recognition method, called vote-som, has been developed. This method can be used as a solution to certain real-life problems. The model obtained is based on building a dynamic competitive network trained with the AVGSOM method. Each element of the network has been used to represent a portion of the space defined by the Cepstral coefficients of the voice signal. At present, work is being done to define new strategies that allow entering information to the network without having to re-train it. Competitive networks adapt through a non-supervised organizing process whose result is strongly dependent on the initial weights. It would be interesting to consider strategies that, using the concept of bias, add new knowledge without losing what has already been learnt. The method described in this paper does not model the rejection area. This can be modified by using, during the training stage, audio signals that are supplementary to the expected signals in order to generate, within the network, competitive neurons with this information. Even though this would improve the response of the model, it would also increase training time considerably. Certainly, the identification of a person cannot be carried out based on isolated biometric signals; it rather requires a process that combines several of these signals. Therefore, this proposal may be useful as part of a larger identification system. REFERENCES [1] Alahakoon, D., Halgamuge, S.K., Srinivasan, B: Dynamic Self- Organizing Maps with Controlled Growth for Knowledge Discovery. IEEE Transactions On Neural Networks 11, 601 614 (2000) [2] Chen, W., Peng, C., Zhu, X., Wan, B., Wei, D.: SVM-based Identification of Pathological Voices. In: 29th Annual International Conference of the IEEE, Engineering in Medicine and Biology Society, EMBS, pp. 3786 3789 (2007) [3] Dan, Z., Zheng, S., Sun, S; Dong, R: Speaker Recognition based on LS-SVM. In: 3rd International Conference on Innovative Computing Information and Control, pp. 25-28 (2008). [4] Fritzke, B.: Growing cell structures: A self organizing network for supervised and un-supervised learning. Neural networks 7, 1441 1460 (1994) [5] Fritzke, B: A growing neural Gas Network Learns Topologies. In: Tesauro, G., Touretzky, D. S., Leen, T. K. (eds.) Advances in Neural Information Processing Systems, vol. 7, pp. 625 632. MIT Press, Cambridge MA (1995) [6] Gopalan, K., Anderson, T.R., Cupples, E.J.: A comparison of speaker identification results using features based on cepstrum and Fourier-Bessel expansion. IEEE Transactions on Speech and Audio Processing 7, 289 294 (1999). [7] Gudnason, J., Brookes, M.: Voice source cepstrum coefficients for speaker identification. IEEE International Conference on Acoustics, Speech and Signal Processing. pp. 4821 4824 (2008) [8] Hasperu, W., Lanzarini, L.: Dynamic Self-Organizing Maps. A new strategy to upgrade topology preservation. XXXI Congreso Latinoamericano de Informtica. pp.1081 1087 (2005). [9] Kohonen, T.: Self-Organizing Maps. 2nd Edition. Springer. ISSN 0720-678X (1997) [10] Maeran, O., Piuri, V., Storti Gajani, G.: Speech recognition through phoneme segmentation and neural classification. IEEE Instrumentation and Measurement Technology Conference, vol. 2, pp. 1215 1220 (1997) [11] New, T.L., Li, H.: On fusion of timbre-motivated features for singing voice detection and singer identification. IEEE International Conference on Acoustics, Speech and Signal Processing.. pp. 2225 2228 (2008) 92

[12] Praat. Weenink, D., Boersma, P. Institute of Phonetic Sciences. http://www.praat.org [13] Rashid, R.A., Mahalin, N.H., Sarijari, M.A., Abdul Aziz, A.A.; Security system using biometric technology: Design and implementation of Voice Recognition System (VRS). International Conference on Computer and Communication Engineering. pp. 898 902 (2008) [14] Zhang, X, Sun, Y., Zhang, X.: A Fuzzy Neural Network Based on Particle Swarm Optimization Applied in the Speech Recognition System. In: Eighth International Conference on Intelligent Systems Design and Applications. pp. 693 697 (2008) [15] Zhang, X., Wang, P., Li, G., Hou, W.: A Fuzzy Neural Network Applied in the Speech Recognition System. In: Fourth International Conference on Natural Computation. pp. 14 18 (2008). [16] Zhang, Y., Abdulla, W.H.: Voice as a Robust Biometrics. Second International Conference on Future Generation Communication and Networking. vol. 3, pp. 41 46 (2008) 93