BUILDING CONTEXT-DEPENDENT DNN ACOUSTIC MODELS USING KULLBACK-LEIBLER DIVERGENCE-BASED STATE TYING

Save this PDF as:
Size: px
Start display at page:

Download "BUILDING CONTEXT-DEPENDENT DNN ACOUSTIC MODELS USING KULLBACK-LEIBLER DIVERGENCE-BASED STATE TYING"

Transcription

1 BUILDING CONTEXT-DEPENDENT DNN ACOUSTIC MODELS USING KULLBACK-LEIBLER DIVERGENCE-BASED STATE TYING Gábor Gosztolya 1, Tamás Grósz 1, László Tóth 1, David Imseng 2 1 MTA-SZTE Research Group on Artificial Intelligence, Szeged, Hungary 2 Idiap Research Institute, Martigny, Switzerland { ggabor, groszt, tothl inf.u-szeged.hu, ABSTRACT Deep neural network (DNN) based speech recognizers have recently replaced Gaussian mixture (GMM) based systems as the state-of-the-art. HMM/DNN systems have kept many refinements of the HMM/GMM framework, even though some of these may be suboptimal for them. One such example is the creation of context-dependent tied states, for which an efficient decision tree state tying method exists. The tied states used to train DNNs are usually obtained using the same tying algorithm, even though it is based on likelihoods of Gaussians. In this paper, we investigate an alternative state clustering method that uses the Kullback-Leibler (KL) divergence of DNN output vectors to build the decision tree. It has already been successfully applied within the framework of KL- HMM systems, and here we show that it is also beneficial for HMM/DNN hybrids. In a large vocabulary recognition task we report a 4% relative word error rate reduction using this state clustering method. Index Terms Speech recognition, deep neural networks, state tying, Kullback-Leibler divergence 1. INTRODUCTION Deep neural network (DNN) based hybrid speech recognizers are nowadays regarded as the state-of-the-art and have replaced conventional Gaussian mixture modeling (GMM) based hidden Markov models (HMMs). Since the introduction of HMMs, the speech community developed many techniques to optimize the process of the training of GMM-based acoustic models. HMM/DNN hybrid systems have inherited most of these methods, even though some of these may be inappropriate for them. Two such examples are the flat start training scheme and the creation of context-dependent (CD) phone models, which are vital components of conventional HMM/GMM systems. This publication is supported by the European Union and co-funded by the European Social Fund. Project title: Telemedicine-oriented research activities in the fields of mathematics, informatics and medical sciences. Project number: TÁMOP A-11/1/KONV More specifically, HMM/GMM systems are usually trained by an iterative re-estimation and re-aligment of the models, also known as flat start training. Since it is not obvious how to perform such a flat start training with HMM/DNN-based acoustic models, most HMM/DNN systems are trained on frame-level labels that were obtained from a previously trained HMM/GMM system using forced alignment. Although sequence-based training strategies have begun to emerge, these still give better results when initialized with frame-level training [1]. Quite recently, it was shown by several researchers that, if done with proper caution, flat start training can also be performed with DNNs [2, 3]. While hybrid models applied only context-independent (CI) phone models for a long time [4], there is now common agreement that HMM/DNN systems also greatly benefit from using context-dependent tied states [5, 6]. Thus, it is necessary to find an approach for efficiently creating contextdependent tied states in DNN systems. However, this seems to be a more challenging task than flat start training. Currently, the dominant solution is the decision tree-based state tying method [7]. This technique fits Gaussians on the distribution of the states, and uses the likelihood gain to govern the state-splitting process. Thanks to the Gaussian assumption and the decision tree representation, this approach is computationally very efficient. However, as already mentioned, sometimes it may be inappropriate to just impose the common HMM/GMM-based techniques on the HMM/DNN training procedure. For several reasons, such as the usage of different features and the fact that the objective functions during training are completely different, this may be so for the state tying approach. GMM-based methods assume that the Gaussian components have diagonal covariance matrices, and thus require decorrelated features like cepstral coefficients (MFCCs). However, it was observed that HMM/DNN hybrids work better on more primitive features like mel filter bank energies [8]. Since conventional HMM/GMM systems cannot be efficiently trained on these features, one would have to train a HMM/GMM system on a standard feature set like MFCCs, create the tied state inventory and alignment, and then discard

2 the feature set. Therefore, it may be better to perform the state clustering not on the raw features, but on the output of a DNN. This approach was investigated by Senior et al [2]. A simple modification of this method is to use the activations on the last hidden layer of the DNN instead of the outputs of the final softmax layer [9]. In a similar study, Zhang et al. derived formulas for converting the output of the DNN softmax layer or a hidden network layer into class-conditional Gaussian distributions [3]. Note that all these studies manipulated only the input of the clustering algorithm, but for the clustering they used the same standard Gaussian-based decision tree clustering method. A second argument is that, intuitively, the state clustering algorithm should split those states where the splitting would be beneficial for the respective classifier. Since the objective functions during GMM and DNN training fundamentally differ, measuring how a Gaussian models a given class may be unrelated to the difficulty of modeling that class by a DNN that is able to represent much more complex decision boundaries. This suggests that some metric other than the likelihood of Gaussians should be used by the state clustering process. Recently, a variant of the decision tree clustering algorithm was proposed that also works on DNN output vectors [10]. In contrast to the earlier cited studies which converted the DNN outputs into class conditional distributions and fit Gaussians on these, this algorithm exploits the fact that the DNN output vectors form discrete probability distributions. A natural distance function for such distributions is the Kullback-Leibler divergence [11]. Hence, it is reasonable to modify the state clustering algorithm so that it works with the Kullback-Leibler divergence instead of Gaussian likelihoods. Imseng et al. successfully used this KL-divergence based state tying routine in the framework of Kullback-Leibler divergence-based HMMs (KL-HMM) [12]. In this paper, we investigate the applicability of this algorithm for creating tied states in a HMM/DNN hybrid. The evaluation will be carried out on a large vocabulary speech recognition task of 28 hours of Hungarian broadcast news data. As a baseline, a context dependent HMM/DNN hybrid that applies conventional GMM-based state tying is used. Then we repeat the same experiments by training a context independent (CI) auxiliary neural network, and then create context-dependent (CD) states by applying the modified, KLdivergence based clustering code on the network output. 2. DECISION TREE BASED STATE TYING The decision tree-based state tying algorithm was introduced by Young et al. [7], and evolved into a vital component of training large vocabulary speech recognizers. The main idea is to pool all context variants of a state, and then build a decision tree by successively splitting this set into two. For each step, the algorithm chooses one of the pre-defined questions in such a way that the resulting two non-overlapping sub-sets of the original state set S differs maximally. The algorithm measures this difference by using a likelihood-based decision criterion. Although minor improvements to the algorithm like the automatic generation of the questions via clustering were proposed [13], the main scheme of the method proved so successful that it has remained unaltered ever since Likelihood based decision criterion Odell formulated a maximum likelihood-based decision criteria [14] and proposed a computationally efficient algorithm by approximating the splitting criterion as L(S) 1 ( log[(2π) K Σ(S) ] + K ) N(s), (1) 2 s S where s S are the individual states, Σ(S) is the variance of data in S, and N(s) is the number of examples (frames) in the training data which belong to state s. Using this formula, we should choose the question q which maximizes the likelihood difference L(q S) L(q S) = ( L(S y (q)) + L(S n (q)) ) L(S), (2) where S y (q) and S n (q) are the two subsets of S formed based on the answer to the question q. It can be seen that the likelihood values do not depend on the training observations themselves, but only on the variance over training data corresponding to the states, and the raw number of frames belonging to each state. Although this assumption (regarding the variance of the feature vectors) fits well to a system employing GMMs, in a HMM/DNN hybrid speech recognizer framework some other decision criterion might result in a more suitable set of tied states Kullback-Leibler divergence based decision criterion This decision criterion was introduced by Imseng et al., who successfully applied it in their KL-HMM framework [15]. Next, we will give a brief description of this algorithm, based on articles [10] and [12]. Although the Kullback-Leibler divergence is known to be asymmetric, unfortunately there is no closed form of the symmetric KL-divergence based cost function. Therefore we will apply the asymmetric KL-divergence between two posterior vectors z t and y s, defined as D KL (y s z t ) = k=1 y s (k) log y s(k) z t (k), (3) where k {1,..., K} is the dimensionality index of the posterior distribution vector [11]. The KL-divergence is always non-negative and zero if and only if the two posterior vectors are equal. So instead of maximizing the likelihood, we will

3 minimize the KL-divergence D KL (S) = s S f F (s) k=1 y S (k) log y S(k) z f (k), (4) where S is a set of states s, and F (s) is the set of training vectors corresponding to state s. The posterior vector associated with the set S (y S ) can be calculated as the normalized geometrical mean of the example vectors belonging to the elements of S, i.e. ( s S f F (s) z f (k) y S (k) = K k=1 ỹs(k) After expanding and simplifying, we get [10] D KL (S) = s S N(s) log ) 1 N(S). (5) ỹ S (k), (6) k=1 so the KL divergence of a set of states S can be calculated based on the statistics y s and N(s) of the individual states. For the splitting of a set of states S, the straightforward option is to choose the question that maximizes the KLdivergence difference D KL (q S): D KL (q S) = D KL (S) ( D KL (S y (q)) + D KL (S n (q)) ). 3. APPLYING KL-BASED STATE TYING FOR HMM/DNN HYBRIDS In our baseline system context-dependent HMM/GMM phone models are trained first, which are then used in force aligment mode to generate CD training labels for the DNN. This system operates on MFCC features, and was implemented in HTK [16]. It applies the standard, Gaussian-based state tying process as part of the training process of the HMM/GMM CD phone models. Having obtained the clustered states using the HMM/GMM, a DNN is trained using these tied states as the training labels. This DNN is used during the decoding process, which is preformed by applying a modified version of the HTK Hdecode routine [16]. The KL divergence-based clustering algorithm requires CI state label posterior estimates as its input. To get these, we trained an auxiliary neural network with one hidden layer (ANN) on the CI labels got from the HMM/GMM system. Next, we applied the KL-divergence based clustering algorithm on the output of this ANN. Then, having obtained the clustered states, we trained the DNN using these tied states as the training labels. Similar to the baseline system, the decoder used this DNN during recognition. 4. EXPERIMENTAL SETUP As the DNN component of our hybrid recognizer, we applied a deep network consisting of rectified linear units as hidden neurons [17]. The main advantage of deep rectifier nets is that they can be efficiently trained with the standard backpropagation algorithm, without any tedious pre-training [18]. We used our custom implementation, which achieved the best accuracy known to us on the TIMIT database with a phone error rate of 16.7% on the core test set [19]. For the actual task we employed a DNN with 5 hidden layers, each containing 1000 rectified neurons. In the output layer, we applied the softmax function. We used 40 mel filter bank energies as features along with their first and second order derivatives; following the HTK notation, we will refer to this feature set as the FBANK feature set. Decoding and evaluation was performed by a modified version of HTK [16]. The speech corpus of Hungarian broadcast news was collected from eight TV channels. From the 28 hours of recordings, 22 hours were used as the train set, 2 hours for development and 4 hours for testing. The total number of different triphone occurrences was 13,467, resulting in 40,401 initial CD phone models. We built a trigram language model from a corpus of about 50 million words taken from the news portal, using the language modelling tools of HTK [16]. As Hungarian is an agglutinative language with a lot of word forms, the recognition dictionary consisted of 486,982 words. Though the DNN was always trained on the FBANK feature set, we investigated two variants of the auxiliary ANN. First, we trained it on the MFCC feature set that was also used by the HMM/GMM system. The second version was trained on the FBANK feature set that was utilized by the DNN. Note that for the baseline system we had to use different feature sets to construct the tied states and for learning them by the DNN (MFCCs vs. FBANK), as training GMMs on FBANK features would have produced unusable results. As the auxiliary ANN was thrown away after state tying, it contained only one hidden layer. We will examine the relevance of the size of this network later on. For both clustering algorithms, we varied the state tying stopping threshold to get roughly 600, 1200, 1800, 2400, 3000 and 3600 tied states. 5. RESULTS As shown in figures 1 and 2, the KL divergence-based clustering method performed consistently and significantly better than the conventional GMM/HMM clustering on both sets. The standard algorithm gives the best performance with 600 tied states, though the results are roughly the same across all state values. The KL-divergence based system has a clear optimum at 1200 states, and it yielded a 4% relative error rate reduction compared to the best score of the conventional system. Among the two variants of the auxiliary ANN, the one trained on FBANK features that is, the same feature set that the final DNN was trained on led to somewhat better scores, though the difference was not significant. An obvious drawback of the KL state tying approach over

4 Word Error Rate (%) Word Error Rate (%) HTK KL using fbank ANN KL using MFCC ANN No. of tied states 16.5 HTK KL using fbank ANN KL using MFCC ANN No. of tied states Fig. 1. Word error rates as a function of the number of tied states on the development set. Fig. 2. Word error rates as a function of the number of tied states on the test set. the conventional algorithm is that we first need to train the auxiliary ANN to obtain the input of clustering. The GMMbased method uses Gaussians for the same purpose, and fitting these on the data is much faster than training an ANN. While it is obvious that the result of the KL divergence-based method depends on the auxiliary ANN, it is not at all clear how accurate this network should be. Perhaps a much smaller network could also lead to similar results, while the training time could be considerably reduced. To discover if this is so, we repeated the experiment by varying the size of the hidden layer of the auxiliary ANN. The clustering step and the training of the DNN was the same as in the previous experiments. The state clustering algorithm was configured so as to get roughly 1200 tied states, as this value gave the best performance earlier. No. of hidden WER % neurons Dev. set Test set % 16.76% % 16.54% % 16.44% Table 1. Word error rates as a function of hidden layer size in the auxiliary ANN. The word error rates for different network sizes are given in Table 1. Although the size of the hidden layer of the ANN affected the WER scores, the difference is minimal, and even the worst scores are much better than the ones obtained via the GMM-based state tying method. This fact tells us that the KL-clustering algorithm can give good results even when the auxiliary network is much smaller than the final DNN. Besides keeping the auxiliary ANN as small as possible, there is a further option available to reduce the training time. Here the idea is to keep the weights of the auxiliary ANN, and use them to initialize the lowest hidden layer of the DNN. This might reduce the training time on one hand, and yield slightly better results on the other. Of course, in this case the ANN must have the same number of hidden units as the final DNN, which in our case was set to 1000 units. The results are listed in Table 2 below. Unfortunately, the accuracy of the system trained this way was no better than the previous scores. Further studies are required to see whether this could be improved if the auxiliary ANN contained more hidden layers. State tying method WER % Dev. set Test set KL with MFCC ANN 17.35% 16.64% KL with fbank ANN 17.12% 16.54% KL with fbank ANN + ANN init % 16.79% GMM/HMM clustering 17.83% 17.26% Table 2. Word error rates for the different training strategies. 6. CONCLUSIONS We evaluated a state clustering algorithm that is based on the KL-divergence of posterior probability distributions. Compared to the standard method that uses the likelihood of Gaussians, this algorithm seems to be more plausible and appropriate when the input data to be clustered are ANN output vectors. Indeed, there is experimental evidence that the KL-based algorithm to create the CD targets of a HMM/DNN hybrid yields slightly better recognition scores with the same number of tied states. In a large vocabulary recognition task we reported a 4% relative word error rate reduction compared to that for the standard state clustering method.

5 7. REFERENCES [1] K. Veselý, A. Ghoshal, L. Burget, and D. Povey, Sequence-discriminative training of deep neural networks, in Proceedings of Interspeech, 2013, pp [2] A. Senior, G. Heigold, M. Bacchiani, and H. Liao, GMM-free DNN training, in Proceedings of ICASSP, [3] C. Zhang and P. Woodland, Standalone training of context-dependent Deep Neural Network acoustic models, in Proceedings of ICASSP, 2014, pp [4] H. Bourlard and N. Morgan, Connectionist Speech Recognition A Hybrid Approach, Kluwer Academic, [5] D. Yu, L. Deng, and G. Dahl, Roles of pretraining and fine-tuning in context-dependent DNN-HMMs for real-world speech recognition, in Proceedings of NIPS Workshop on Deep Learning and Unsupervised Feature Learning, [6] G. Dahl, D. Yu, L. Deng, and A. Acero, Contextdependent pre-trained Deep Neural Networks for large vocabulary speech recognition, IEEE Trans. ASLP, vol. 20, no. 1, pp , [14] J.J. Odell, The Use of Context in Large Vocabulary Speech Recognition, Ph.D. thesis, University of Cambridge, [15] M. Razavi, R. Rasipuram, and M. Magimai-Doss, On modeling context-dependent clustered states: Comparing HMM/GMM, hybrid HMM/ANN and KL-HMM approaches, in Proceedings of ICASSP, [16] S. Young, G. Evermann, M. J. F. Gales, T. Hain, D. Kershaw, G. Moore, J. Odell, D. Ollason, D. Povey, V. Valtchev, and P.C. Woodland, The HTK Book, Cambridge University Engineering Department, Cambridge, UK, [17] X. Glorot, A. Bordes, and Y. Bengio, Deep sparse rectifier networks, in Proceedings of AISTATS, 2011, pp [18] L. Tóth, Phone recognition with deep sparse rectifier neural networks, in Proceedings of ICASSP, 2013, pp [19] L. Tóth, Combining time- and frequency-domain convolution in convolutional neural network-based phone recognition, in Proceedings of ICASSP, 2014, pp [7] S. J. Young, J. J. Odell, and P. C. Woodland, Treebased state tying for high accuracy acoustic modelling, in Proceedings of HLT, 1994, pp [8] A. Mohamed, G. E. Dahl, and G. Hinton, Acoustic modeling using deep belief networks, IEEE Trans. ASLP, vol. 20, no. 1, pp , [9] M. Bacchiani and D. Rybach, Context dependent state tying for speech recognition using deep neural network acoustic models, in Proceedings of ICASSP, 2014, pp [10] D. Imseng and J. Dines, Decision tree clustering for KL-HMM, Tech. Rep. Idiap-Com , Idiap Research Institute, [11] S. Kullback and R.A. Leibler, On information and sufficiency, Ann. Math. Statist., vol. 22, no. 1, pp , [12] D. Imseng, J. Dines, P. Motlicek, P.N. Garner, and H. Bourlard, Comparing different acoustic modeling techniques for multilingual boosting, in Proceedings of Interspeech, [13] K. Beulen and H. Ney, Automatic question generation for decision tree based state tying, in Proceedings of ICASSP, 1998, pp

A Sequence Training Method for Deep Rectifier Neural Networks in Speech Recognition

A Sequence Training Method for Deep Rectifier Neural Networks in Speech Recognition A Sequence Training Method for Deep Rectifier Neural Networks in Speech Recognition Tamás Grósz, Gábor Gosztolya, and László Tóth MTA-SZTE Research Group on Artificial Intelligence of the Hungarian Academy

More information

Convolutional Deep Maxout Networks for Phone Recognition

Convolutional Deep Maxout Networks for Phone Recognition INTERSPEECH 2014 Convolutional Deep Maxout Networks for Phone Recognition László Tóth MTA-SZTE Research Group on Artificial Intelligence Hungarian Academy of Sciences and University of Szeged, Hungary

More information

FACTORIZED DEEP NEURAL NETWORKS FOR ADAPTIVE SPEECH RECOGNITION.

FACTORIZED DEEP NEURAL NETWORKS FOR ADAPTIVE SPEECH RECOGNITION. FACTORIZED DEEP NEURAL NETWORKS FOR ADAPTIVE SPEECH RECOGNITION Dong Yu 1, Xin Chen 2, Li Deng 1 1 Speech Research Group, Microsoft Research, Redmond, WA, USA 2 Department of Computer Science, University

More information

Comparison and Combination of Multilayer Perceptrons and Deep Belief Networks in Hybrid Automatic Speech Recognition Systems

Comparison and Combination of Multilayer Perceptrons and Deep Belief Networks in Hybrid Automatic Speech Recognition Systems APSIPA ASC 2011 Xi an Comparison and Combination of Multilayer Perceptrons and Deep Belief Networks in Hybrid Automatic Speech Recognition Systems Van Hai Do, Xiong Xiao, Eng Siong Chng School of Computer

More information

Learning Small-Size DNN with Output-Distribution-Based Criteria

Learning Small-Size DNN with Output-Distribution-Based Criteria INTERSPEECH 2014 Learning Small-Size DNN with Output-Distribution-Based Criteria Jinyu Li 1, Rui Zhao 2, Jui-Ting Huang 1, and Yifan Gong 1 1 Microsoft Corporation, One Microsoft Way, Redmond, WA 98052

More information

Speaker Adaptation. Steve Renals. Automatic Speech Recognition ASR Lecture 14 3 March ASR Lecture 14 Speaker Adaptation 1

Speaker Adaptation. Steve Renals. Automatic Speech Recognition ASR Lecture 14 3 March ASR Lecture 14 Speaker Adaptation 1 Speaker Adaptation Steve Renals Automatic Speech Recognition ASR Lecture 14 3 March 2016 ASR Lecture 14 Speaker Adaptation 1 Speaker independent / dependent / adaptive Speaker independent (SI) systems

More information

MINIMUM RISK ACOUSTIC CLUSTERING FOR MULTILINGUAL ACOUSTIC MODEL COMBINATION

MINIMUM RISK ACOUSTIC CLUSTERING FOR MULTILINGUAL ACOUSTIC MODEL COMBINATION MINIMUM RISK ACOUSTIC CLUSTERING FOR MULTILINGUAL ACOUSTIC MODEL COMBINATION Dimitra Vergyri Stavros Tsakalidis William Byrne Center for Language and Speech Processing Johns Hopkins University, Baltimore,

More information

CS224 Final Project. Re Alignment Improvements for Deep Neural Networks on Speech Recognition Systems. Firas Abuzaid

CS224 Final Project. Re Alignment Improvements for Deep Neural Networks on Speech Recognition Systems. Firas Abuzaid Abstract CS224 Final Project Re Alignment Improvements for Deep Neural Networks on Speech Recognition Systems Firas Abuzaid The task of automatic speech recognition has traditionally been accomplished

More information

Deep Neural Network Training Emphasizing Central Frames

Deep Neural Network Training Emphasizing Central Frames INTERSPEECH 2015 Deep Neural Network Training Emphasizing Central Frames Gakuto Kurata 1, Daniel Willett 2 1 IBM Research 2 Nuance Communications gakuto@jp.ibm.com, Daniel.Willett@nuance.com Abstract It

More information

Improved feature processing for Deep Neural Networks

Improved feature processing for Deep Neural Networks Improved feature processing for Deep Neural Networks Shakti P. Rath 1,2, Daniel Povey 3, Karel Veselý 1 and Jan Honza Černocký 1 1 Brno University of Technology, Speech@FIT, Božetěchova 2, Brno, Czech

More information

I D I A P. Using more informative posterior probabilities for speech recognition R E S E A R C H R E P O R T. Jithendra Vepa a,b Herve Bourlard a,b

I D I A P. Using more informative posterior probabilities for speech recognition R E S E A R C H R E P O R T. Jithendra Vepa a,b Herve Bourlard a,b R E S E A R C H R E P O R T I D I A P Using more informative posterior probabilities for speech recognition Hamed Ketabdar a,b Samy Bengio a,b IDIAP RR 05-91 December 2005 published in ICASSP 06 Jithendra

More information

Mispronunciation Detection and Diagnosis in L2 English Speech Using Multi-Distribution Deep Neural Networks

Mispronunciation Detection and Diagnosis in L2 English Speech Using Multi-Distribution Deep Neural Networks Mispronunciation Detection and Diagnosis in L2 English Speech Using Multi-Distribution Deep Neural Networks Kun Li and Helen Meng Human-Computer Communications Laboratory Department of System Engineering

More information

Asynchronous, Online, GMM-free Training of a Context Dependent Acoustic Model for Speech Recognition

Asynchronous, Online, GMM-free Training of a Context Dependent Acoustic Model for Speech Recognition Asynchronous, Online, GMM-free Training of a Context Dependent Acoustic Model for Speech Recognition Michiel Bacchiani, Andrew Senior, Georg Heigold Google Inc. {michiel,andrewsenior,heigold}@google.com

More information

Towards Lower Error Rates in Phoneme Recognition

Towards Lower Error Rates in Phoneme Recognition Towards Lower Error Rates in Phoneme Recognition Petr Schwarz, Pavel Matějka, and Jan Černocký Brno University of Technology, Czech Republic schwarzp matejkap cernocky@fit.vutbr.cz Abstract. We investigate

More information

Improved Neural Network Initialization by Grouping Context-Dependent Targets for Acoustic Modeling

Improved Neural Network Initialization by Grouping Context-Dependent Targets for Acoustic Modeling INTERSPEECH 2016 September 8 12, 2016, San Francisco, USA Improved Neural Network Initialization by Grouping Context-Dependent Targets for Acoustic Modeling Gakuto Kurata, Brian Kingsbury IBM Watson gakuto@jp.ibm.com,

More information

Resource Optimized Speech Recognition using Kullback-Leibler Divergence based HMM

Resource Optimized Speech Recognition using Kullback-Leibler Divergence based HMM Resource Optimized Speech Recognition using Kullback-Leibler Divergence based HMM Ramya Rasipuram David Imseng, Marzieh Razavi, Mathew Magimai Doss, Herve Bourlard 24 October 2014 1/23 Automatic Speech

More information

Word Recognition with Conditional Random Fields

Word Recognition with Conditional Random Fields Outline ord Recognition with Conditional Random Fields Jeremy Morris 2/05/2010 ord Recognition CRF Pilot System - TIDIGITS Larger Vocabulary - SJ Future ork 1 2 Conditional Random Fields (CRFs) Discriminative

More information

Joint Decoding for Phoneme-Grapheme Continuous Speech Recognition Mathew Magimai.-Doss a b Samy Bengio a Hervé Bourlard a b IDIAP RR 03-52

Joint Decoding for Phoneme-Grapheme Continuous Speech Recognition Mathew Magimai.-Doss a b Samy Bengio a Hervé Bourlard a b IDIAP RR 03-52 R E S E A R C H R E P O R T I D I A P Joint Decoding for Phoneme-Grapheme Continuous Speech Recognition Mathew Magimai.-Doss a b Samy Bengio a Hervé Bourlard a b IDIAP RR 03-52 October 2003 submitted for

More information

An Acoustic Model Based on Kullback-Leibler Divergence for Posterior Features

An Acoustic Model Based on Kullback-Leibler Divergence for Posterior Features R E S E A R C H R E P O R T I D I A P An Acoustic Model Based on Kullback-Leibler Divergence for Posterior Features Guillermo Aradilla a b Jithendra Vepa b Hervé Bourlard a b IDIAP RR 06-60 January 2007

More information

INVESTIGATION OF UNSUPERVISED ADAPTATION OF DNN ACOUSTIC MODELS WITH FILTER BANK INPUT

INVESTIGATION OF UNSUPERVISED ADAPTATION OF DNN ACOUSTIC MODELS WITH FILTER BANK INPUT INVESTIGATION OF UNSUPERVISED ADAPTATION OF DNN ACOUSTIC MODELS WITH FILTER BANK INPUT Takuya Yoshioka,, Anton Ragni, Mark J. F. Gales Cambridge University Engineering Department, Cambridge, UK NTT Communication

More information

Speaker Independent Speaker Dependent. % Word Error. Supervised Adapted Unsupervised Adapted No. Adaptation Utterances

Speaker Independent Speaker Dependent. % Word Error. Supervised Adapted Unsupervised Adapted No. Adaptation Utterances FLEXIBLE SPEAKER ADAPTATION USING MAXIMUM LIKELIHOOD LINEAR REGRESSION C.J. Leggetter & P.C. Woodland Cambridge University Engineering Department Trumpington Street, Cambridge CB2 1PZ. UK. ABSTRACT The

More information

Maximum Likelihood and Maximum Mutual Information Training in Gender and Age Recognition System

Maximum Likelihood and Maximum Mutual Information Training in Gender and Age Recognition System Maximum Likelihood and Maximum Mutual Information Training in Gender and Age Recognition System Valiantsina Hubeika, Igor Szöke, Lukáš Burget, Jan Černocký Speech@FIT, Brno University of Technology, Czech

More information

Convolutional Neural Networks for Speech Recognition

Convolutional Neural Networks for Speech Recognition IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL 22, NO 10, OCTOBER 2014 1533 Convolutional Neural Networks for Speech Recognition Ossama Abdel-Hamid, Abdel-rahman Mohamed, Hui Jiang,

More information

Language dependence in multilingual speaker verification

Language dependence in multilingual speaker verification Language dependence in multilingual speaker verification Neil T. Kleynhans, Etienne Barnard Human Language Technologies Research Group, University of Pretoria / Meraka Institute, Pretoria, South Africa

More information

Classification with Deep Belief Networks. HussamHebbo Jae Won Kim

Classification with Deep Belief Networks. HussamHebbo Jae Won Kim Classification with Deep Belief Networks HussamHebbo Jae Won Kim Table of Contents Introduction... 3 Neural Networks... 3 Perceptron... 3 Backpropagation... 4 Deep Belief Networks (RBM, Sigmoid Belief

More information

Learning Methods in Multilingual Speech Recognition

Learning Methods in Multilingual Speech Recognition Learning Methods in Multilingual Speech Recognition Hui Lin Department of Electrical Engineering University of Washington Seattle, WA 98125 linhui@u.washington.edu Li Deng, Jasha Droppo, Dong Yu, and Alex

More information

Word Recognition with Conditional Random Fields. Jeremy Morris 2/05/2010

Word Recognition with Conditional Random Fields. Jeremy Morris 2/05/2010 ord Recognition with Conditional Random Fields Jeremy Morris 2/05/2010 1 Outline Background ord Recognition CRF Model Pilot System - TIDIGITS Larger Vocabulary - SJ Future ork 2 Background Conditional

More information

PROBABILISTIC LEXICAL MODELING AND UNSUPERVISED TRAINING FOR ZERO-RESOURCED ASR

PROBABILISTIC LEXICAL MODELING AND UNSUPERVISED TRAINING FOR ZERO-RESOURCED ASR PROBABILISTIC LEXICAL MODELING AND UNSUPERVISED TRAINING FOR ZERO-RESOURCED ASR Ramya Rasipuram 1,2, Marzieh Razavi 1,2, Mathew Magimai-Doss 1 1 Idiap Research Institute, CH-1920 Martigny, Switzerland

More information

I D I A P. Phoneme-Grapheme Based Speech Recognition System R E S E A R C H R E P O R T

I D I A P. Phoneme-Grapheme Based Speech Recognition System R E S E A R C H R E P O R T R E S E A R C H R E P O R T I D I A P Phoneme-Grapheme Based Speech Recognition System Mathew Magimai.-Doss a b Todd A. Stephenson a b Hervé Bourlard a b Samy Bengio a IDIAP RR 03-37 August 2003 submitted

More information

PHONEME-GRAPHEME BASED SPEECH RECOGNITION SYSTEM

PHONEME-GRAPHEME BASED SPEECH RECOGNITION SYSTEM PHONEME-GRAPHEME BASED SPEECH RECOGNITION SYSTEM Mathew Magimai.-Doss, Todd A. Stephenson, Hervé Bourlard, and Samy Bengio Dalle Molle Institute for Artificial Intelligence CH-1920, Martigny, Switzerland

More information

CS229 Final Project. Re-Alignment Improvements for Deep Neural Networks on Speech Recognition Systems. Firas Abuzaid. Abstract.

CS229 Final Project. Re-Alignment Improvements for Deep Neural Networks on Speech Recognition Systems. Firas Abuzaid. Abstract. CS229 Final Project Re-Alignment Improvements for Deep Neural Networks on Speech Recognition Systems Abstract The task of automatic speech recognition has traditionally been accomplished by using Hidden

More information

INVESTIGATION ON CROSS- AND MULTILINGUAL MLP FEATURES UNDER MATCHED AND MISMATCHED ACOUSTICAL CONDITIONS

INVESTIGATION ON CROSS- AND MULTILINGUAL MLP FEATURES UNDER MATCHED AND MISMATCHED ACOUSTICAL CONDITIONS INVESTIGATION ON CROSS- AND MULTILINGUAL MLP FEATURES UNDER MATCHED AND MISMATCHED ACOUSTICAL CONDITIONS Zoltán Tüske 1, Joel Pinto 2, Daniel Willett 2, Ralf Schlüter 1 1 Human Language Technology and

More information

A Tonotopic Artificial Neural Network Architecture For Phoneme Probability Estimation

A Tonotopic Artificial Neural Network Architecture For Phoneme Probability Estimation A Tonotopic Artificial Neural Network Architecture For Phoneme Probability Estimation Nikko Ström Department of Speech, Music and Hearing, Centre for Speech Technology, KTH (Royal Institute of Technology),

More information

Sphinx Benchmark Report

Sphinx Benchmark Report Sphinx Benchmark Report Long Qin Language Technologies Institute School of Computer Science Carnegie Mellon University Overview! uate general training and testing schemes! LDA-MLLT, VTLN, MMI, SAT, MLLR,

More information

Gaussian Free Cluster Tree Construction using Deep Neural Network

Gaussian Free Cluster Tree Construction using Deep Neural Network Gaussian Free Cluster Tree Construction using Deep Neural Network Linchen Zhu, Kevin Kilgour, Sebastian Stüker, Alex Waibel International Center for Advanced Communication Technologies - InterACT, Institute

More information

Enabling Controllability for Continuous Expression Space

Enabling Controllability for Continuous Expression Space INTERSPEECH 2014 Enabling Controllability for Continuous Expression Space Langzhou Chen, Norbert Braunschweiler Toshiba Research Europe Ltd., Cambridge, UK langzhou.chen,norbert.braunschweiler@crl.toshiba.co.uk

More information

Modeling function word errors in DNN-HMM based LVCSR systems

Modeling function word errors in DNN-HMM based LVCSR systems Modeling function word errors in DNN-HMM based LVCSR systems Melvin Jose Johnson Premkumar, Ankur Bapna and Sree Avinash Parchuri Department of Computer Science Department of Electrical Engineering Stanford

More information

SPEECH RECOGNITION WITH PREDICTION-ADAPTATION-CORRECTION RECURRENT NEURAL NETWORKS

SPEECH RECOGNITION WITH PREDICTION-ADAPTATION-CORRECTION RECURRENT NEURAL NETWORKS SPEECH RECOGNITION WITH PREDICTION-ADAPTATION-CORRECTION RECURRENT NEURAL NETWORKS Yu Zhang MIT CSAIL Cambridge, MA, USA yzhang87@csail.mit.edu Dong Yu, Michael L. Seltzer, Jasha Droppo Microsoft Research

More information

Speaker Adaptation. Steve Renals. Automatic Speech Recognition ASR Lectures 13&14 10, 13 March ASR Lectures 13&14 Speaker Adaptation 1

Speaker Adaptation. Steve Renals. Automatic Speech Recognition ASR Lectures 13&14 10, 13 March ASR Lectures 13&14 Speaker Adaptation 1 Speaker Adaptation Steve Renals Automatic Speech Recognition ASR Lectures 13&14 10, 13 March 2014 ASR Lectures 13&14 Speaker Adaptation 1 Overview Speaker Adaptation Introduction: speaker-specific variation,

More information

CHAPTER 4 IMPROVING THE PERFORMANCE OF A CLASSIFIER USING UNIQUE FEATURES

CHAPTER 4 IMPROVING THE PERFORMANCE OF A CLASSIFIER USING UNIQUE FEATURES 38 CHAPTER 4 IMPROVING THE PERFORMANCE OF A CLASSIFIER USING UNIQUE FEATURES 4.1 INTRODUCTION In classification tasks, the error rate is proportional to the commonality among classes. Conventional GMM

More information

A New Perspective on Combining GMM and DNN Frameworks for Speaker Adaptation

A New Perspective on Combining GMM and DNN Frameworks for Speaker Adaptation A New Perspective on Combining GMM and DNN Frameworks for Speaker Adaptation SLSP-2016 October 11-12 Natalia Tomashenko 1,2,3 natalia.tomashenko@univ-lemans.fr Yuri Khokhlov 3 khokhlov@speechpro.com Yannick

More information

ADAPTIVE training [1], [2] has become increasingly popular

ADAPTIVE training [1], [2] has become increasingly popular 1932 IEEE TRANSACTIONS ON AUDIO, SPEECH, AND LANGUAGE PROCESSING, VOL. 15, NO. 6, AUGUST 2007 Bayesian Adaptive Inference and Adaptive Training Kai Yu, Member, IEEE, and Mark J. F. Gales, Member, IEEE

More information

A CLUSTER-BASED MULTIPLE DEEP NEURAL NETWORKS METHOD FOR LARGE VOCABULARY CONTINUOUS SPEECH RECOGNITION

A CLUSTER-BASED MULTIPLE DEEP NEURAL NETWORKS METHOD FOR LARGE VOCABULARY CONTINUOUS SPEECH RECOGNITION A CLUSTER-BASED MULTIPLE DEEP NEURAL NETWORKS METHOD FOR LARGE VOCABULARY CONTINUOUS SPEECH RECOGNITION Pan Zhou 1, Cong Liu 2, Qingfeng Liu 2, Lirong Dai 1, Hui Jiang 3 1 National Engineering Laboratory

More information

Sequence Discriminative Training;Robust Speech Recognition1

Sequence Discriminative Training;Robust Speech Recognition1 Sequence Discriminative Training; Robust Speech Recognition Steve Renals Automatic Speech Recognition 16 March 2017 Sequence Discriminative Training;Robust Speech Recognition1 Recall: Maximum likelihood

More information

LEARNING LINEARLY SEPARABLE FEATURES FOR SPEECH RECOGNITION USING CONVOLUTIONAL NEURAL NETWORKS

LEARNING LINEARLY SEPARABLE FEATURES FOR SPEECH RECOGNITION USING CONVOLUTIONAL NEURAL NETWORKS RESEARCH REPORT IDIAP LEARNING LINEARLY SEPARABLE FEATURES FOR SPEECH RECOGNITION USING CONVOLUTIONAL NEURAL NETWORKS Dimitri Palaz Mathew Magimai.-Doss Ronan Collobert Idiap-RR-24-2015 JUNE 2015 Centre

More information

Recurrent Neural Networks for Signal Denoising in Robust ASR

Recurrent Neural Networks for Signal Denoising in Robust ASR Recurrent Neural Networks for Signal Denoising in Robust ASR Andrew L. Maas 1, Quoc V. Le 1, Tyler M. O Neil 1, Oriol Vinyals 2, Patrick Nguyen 3, Andrew Y. Ng 1 1 Computer Science Department, Stanford

More information

Semi-Supervised GMM and DNN Acoustic Model Training with Multi-system Combination and Confidence Re-calibration

Semi-Supervised GMM and DNN Acoustic Model Training with Multi-system Combination and Confidence Re-calibration INTERSPEECH 2013 Semi-Supervised GMM and DNN Acoustic Model Training with Multi-system Combination and Confidence Re-calibration Yan Huang, Dong Yu, Yifan Gong, and Chaojun Liu Microsoft Corporation, One

More information

Autoregressive product of multi-frame predictions can improve the accuracy of hybrid models

Autoregressive product of multi-frame predictions can improve the accuracy of hybrid models Autoregressive product of multi-frame predictions can improve the accuracy of hybrid models Navdeep Jaitly 1, Vincent Vanhoucke 2, Geoffrey Hinton 1,2 1 University of Toronto 2 Google Inc. ndjaitly@cs.toronto.edu,

More information

DEEP HIERARCHICAL BOTTLENECK MRASTA FEATURES FOR LVCSR

DEEP HIERARCHICAL BOTTLENECK MRASTA FEATURES FOR LVCSR DEEP HIERARCHICAL BOTTLENECK MRASTA FEATURES FOR LVCSR Zoltán Tüske a, Ralf Schlüter a, Hermann Ney a,b a Human Language Technology and Pattern Recognition, Computer Science Department, RWTH Aachen University,

More information

I D I A P R E S E A R C H R E P O R T. July submitted for publication

I D I A P R E S E A R C H R E P O R T. July submitted for publication R E S E A R C H R E P O R T I D I A P Analysis of Confusion Matrix to Combine Evidence for Phoneme Recognition S. R. Mahadeva Prasanna a B. Yegnanarayana b Joel Praveen Pinto and Hynek Hermansky c d IDIAP

More information

Inter-Ing INTERDISCIPLINARITY IN ENGINEERING SCIENTIFIC INTERNATIONAL CONFERENCE, TG. MUREŞ ROMÂNIA, November 2007.

Inter-Ing INTERDISCIPLINARITY IN ENGINEERING SCIENTIFIC INTERNATIONAL CONFERENCE, TG. MUREŞ ROMÂNIA, November 2007. Inter-Ing 2007 INTERDISCIPLINARITY IN ENGINEERING SCIENTIFIC INTERNATIONAL CONFERENCE, TG. MUREŞ ROMÂNIA, 15-16 November 2007. FRAME-BY-FRAME PHONEME CLASSIFICATION USING MLP DOMOKOS JÓZSEF, SAPIENTIA

More information

RECOGNITION OF ACOUSTIC EVENTS USING DEEP NEURAL NETWORKS. Oguzhan Gencoglu, Tuomas Virtanen, Heikki Huttunen

RECOGNITION OF ACOUSTIC EVENTS USING DEEP NEURAL NETWORKS. Oguzhan Gencoglu, Tuomas Virtanen, Heikki Huttunen RECOGNITION OF ACOUSTIC EVENTS USING DEEP NEURAL NETWORKS Oguzhan Gencoglu, Tuomas Virtanen, Heikki Huttunen Department of Signal Processing, Tampere University of Technology, 337 Tampere, Finland ABSTRACT

More information

FLAT START TRAINING OF CD-CTC-SMBR LSTM RNN ACOUSTIC MODELS. Kanishka Rao, Andrew Senior, Haşim Sak. Google

FLAT START TRAINING OF CD-CTC-SMBR LSTM RNN ACOUSTIC MODELS. Kanishka Rao, Andrew Senior, Haşim Sak. Google FLAT START TRAINING OF CD-CTC-SMBR LSTM RNN ACOUSTIC MODELS Kanishka Rao, Andrew Senior, Haşim Sak Google {kanishkarao,andrewsenior,hasim}@google.com ABSTRACT We present a recipe for training acoustic

More information

Phoneme Recognition Using Deep Neural Networks

Phoneme Recognition Using Deep Neural Networks CS229 Final Project Report, Stanford University Phoneme Recognition Using Deep Neural Networks John Labiak December 16, 2011 1 Introduction Deep architectures, such as multilayer neural networks, can be

More information

Using Posterior-Based Features in Template Matching for Speech Recognition Guillermo Aradilla a Jithendra Vepa a Hervé Bourlard a IDIAP RR 06-23

Using Posterior-Based Features in Template Matching for Speech Recognition Guillermo Aradilla a Jithendra Vepa a Hervé Bourlard a IDIAP RR 06-23 R E S E A R C H R E P O R T I D I A P Using Posterior-Based Features in Template Matching for Speech Recognition Guillermo Aradilla a Jithendra Vepa a Hervé Bourlard a IDIAP RR 06-23 June 2006 published

More information

Discriminative Phonetic Recognition with Conditional Random Fields

Discriminative Phonetic Recognition with Conditional Random Fields Discriminative Phonetic Recognition with Conditional Random Fields Jeremy Morris & Eric Fosler-Lussier Dept. of Computer Science and Engineering The Ohio State University Columbus, OH 43210 {morrijer,fosler}@cse.ohio-state.edu

More information

Automatic Speech Segmentation Based on HMM

Automatic Speech Segmentation Based on HMM 6 M. KROUL, AUTOMATIC SPEECH SEGMENTATION BASED ON HMM Automatic Speech Segmentation Based on HMM Martin Kroul Inst. of Information Technology and Electronics, Technical University of Liberec, Hálkova

More information

Modeling function word errors in DNN-HMM based LVCSR systems

Modeling function word errors in DNN-HMM based LVCSR systems Modeling function word errors in DNN-HMM based LVCSR systems Melvin Jose Johnson Premkumar, Ankur Bapna and Sree Avinash Parchuri Department of Computer Science Department of Electrical Engineering Stanford

More information

FILTER BANK FEATURE EXTRACTION FOR GAUSSIAN MIXTURE MODEL SPEAKER RECOGNITION

FILTER BANK FEATURE EXTRACTION FOR GAUSSIAN MIXTURE MODEL SPEAKER RECOGNITION FILTER BANK FEATURE EXTRACTION FOR GAUSSIAN MIXTURE MODEL SPEAKER RECOGNITION James H. Nealand, Alan B. Bradley, & Margaret Lech School of Electrical and Computer Systems Engineering, RMIT University,

More information

Joint Sequence Training of Phone and Grapheme Acoustic Model based on Multi-task Learning Deep Neural Networks

Joint Sequence Training of Phone and Grapheme Acoustic Model based on Multi-task Learning Deep Neural Networks Joint Sequence Training of Phone and Grapheme Acoustic Model based on Multi-task Learning Deep Neural Networks Dongpeng Chen 1, Brian Mak 1, Sunil Sivadas 2 1 Department of Computer Science & Engineering

More information

An Improvement of robustness to speech loudness change for an ASR system based on LC-RC features

An Improvement of robustness to speech loudness change for an ASR system based on LC-RC features An Improvement of robustness to speech loudness change for an ASR system based on LC-RC features Pavel Yurkov, Maxim Korenevsky, Kirill Levin Speech Technology Center, St. Petersburg, Russia Abstract This

More information

DEEP LEARNING FOR MONAURAL SPEECH SEPARATION

DEEP LEARNING FOR MONAURAL SPEECH SEPARATION DEEP LEARNING FOR MONAURAL SPEECH SEPARATION Po-Sen Huang, Minje Kim, Mark Hasegawa-Johnson, Paris Smaragdis Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign,

More information

Multilingual Exemplar-Based Acoustic Model for the NIST Open KWS 2015 Evaluation

Multilingual Exemplar-Based Acoustic Model for the NIST Open KWS 2015 Evaluation Multilingual Exemplar-Based Acoustic Model for the NIST Open KWS 2015 Evaluation Van Hai Do, Xiong Xiao, Haihua Xu, Eng Siong Chng and Haizhou Li School of Computer Engineering, Nanyang Technological University,

More information

Environmental Noise Embeddings For Robust Speech Recognition

Environmental Noise Embeddings For Robust Speech Recognition Environmental Noise Embeddings For Robust Speech Recognition Suyoun Kim 1, Bhiksha Raj 1, Ian Lane 1 1 Electrical Computer Engineering Carnegie Mellon University suyoun@cmu.edu, bhiksha@cs.cmu.edu, lane@cmu.edu

More information

SEQUENCE TRAINING OF MULTIPLE DEEP NEURAL NETWORKS FOR BETTER PERFORMANCE AND FASTER TRAINING SPEED

SEQUENCE TRAINING OF MULTIPLE DEEP NEURAL NETWORKS FOR BETTER PERFORMANCE AND FASTER TRAINING SPEED 2014 IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP) SEQUENCE TRAINING OF MULTIPLE DEEP NEURAL NETWORKS FOR BETTER PERFORMANCE AND FASTER TRAINING SPEED Pan Zhou 1, Lirong

More information

Optimizing Deep Bottleneck Feature Extraction

Optimizing Deep Bottleneck Feature Extraction Optimizing Deep Bottleneck Feature Extraction Quoc Bao Nguyen, Jonas Gehring, Kevin Kilgour and Alex Waibel International Center for Advanced Communication Technologies - InterACT, Institute for Anthropomatics,

More information

Unsupervised Methods for Speaker Diarization: An Integrated and Iterative Approach!

Unsupervised Methods for Speaker Diarization: An Integrated and Iterative Approach! Unsupervised Methods for Speaker Diarization: An Integrated and Iterative Approach! Stephen Shum, Najim Dehak, and Jim Glass!! *With help from Reda Dehak, Ekapol Chuangsuwanich, and Douglas Reynolds November

More information

Pavel Král and Václav Matoušek University of West Bohemia in Plzeň (Pilsen), Czech Republic pkral

Pavel Král and Václav Matoušek University of West Bohemia in Plzeň (Pilsen), Czech Republic pkral EVALUATION OF AUTOMATIC SPEAKER RECOGNITION APPROACHES Pavel Král and Václav Matoušek University of West Bohemia in Plzeň (Pilsen), Czech Republic pkral matousek@kiv.zcu.cz Abstract: This paper deals with

More information

Automatic Segmentation of Speech at the Phonetic Level

Automatic Segmentation of Speech at the Phonetic Level Automatic Segmentation of Speech at the Phonetic Level Jon Ander Gómez and María José Castro Departamento de Sistemas Informáticos y Computación Universidad Politécnica de Valencia, Valencia (Spain) jon@dsic.upv.es

More information

Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition

Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition Paul Hensch 21.01.2014 Seminar aus maschinellem Lernen 1 Large-Vocabulary Speech Recognition Complications 21.01.2014

More information

Usable Speech Assignment for Speaker Identification under Co-Channel Situation

Usable Speech Assignment for Speaker Identification under Co-Channel Situation Usable Speech Assignment for Speaker Identification under Co-Channel Situation Wajdi Ghezaiel CEREP-Ecole Sup. des Sciences et Techniques de Tunis, Tunisia Amel Ben Slimane Ecole Nationale des Sciences

More information

The 1997 CMU Sphinx-3 English Broadcast News Transcription System

The 1997 CMU Sphinx-3 English Broadcast News Transcription System The 1997 CMU Sphinx-3 English Broadcast News Transcription System K. Seymore, S. Chen, S. Doh, M. Eskenazi, E. Gouvêa, B. Raj, M. Ravishankar, R. Rosenfeld, M. Siegler, R. Stern, and E. Thayer Carnegie

More information

A New DNN-based High Quality Pronunciation Evaluation for. in Computer-Aided Language Learning (CALL) to

A New DNN-based High Quality Pronunciation Evaluation for. in Computer-Aided Language Learning (CALL) to INTERSPEECH 2013 A New DNN-based High Quality Pronunciation Evaluation for Computer-Aided Language Learning (CALL) Wenping Hu 1,2, Yao Qian 1, Frank K. Soong 1 1 Microsoft Research Asia, Beijing, P.R.C.

More information

Gender Classification Based on FeedForward Backpropagation Neural Network

Gender Classification Based on FeedForward Backpropagation Neural Network Gender Classification Based on FeedForward Backpropagation Neural Network S. Mostafa Rahimi Azghadi 1, M. Reza Bonyadi 1 and Hamed Shahhosseini 2 1 Department of Electrical and Computer Engineering, Shahid

More information

THIRD-ORDER MOMENTS OF FILTERED SPEECH SIGNALS FOR ROBUST SPEECH RECOGNITION

THIRD-ORDER MOMENTS OF FILTERED SPEECH SIGNALS FOR ROBUST SPEECH RECOGNITION THIRD-ORDER MOMENTS OF FILTERED SPEECH SIGNALS FOR ROBUST SPEECH RECOGNITION Kevin M. Indrebo, Richard J. Povinelli, and Michael T. Johnson Dept. of Electrical and Computer Engineering, Marquette University

More information

Deep Neural Networks for Acoustic Modelling. Bajibabu Bollepalli Hieu Nguyen Rakshith Shetty Pieter Smit (Mentor)

Deep Neural Networks for Acoustic Modelling. Bajibabu Bollepalli Hieu Nguyen Rakshith Shetty Pieter Smit (Mentor) Deep Neural Networks for Acoustic Modelling Bajibabu Bollepalli Hieu Nguyen Rakshith Shetty Pieter Smit (Mentor) Introduction Automatic speech recognition Speech signal Feature Extraction Acoustic Modelling

More information

Feature-based Robust Techniques For Speech Recognition

Feature-based Robust Techniques For Speech Recognition Feature-based Robust Techniques For Speech Recognition presented by Nguyen Duc Hoang Ha Supervisors Assoc. Prof. Chng Eng Siong Prof. Li Haizhou 08-Mar-2017 Outline An of Robust ASR The 1st proposed method

More information

CS 545 Lecture XI: Speech (some slides courtesy Jurafsky&Martin)

CS 545 Lecture XI: Speech (some slides courtesy Jurafsky&Martin) CS 545 Lecture XI: Speech (some slides courtesy Jurafsky&Martin) brownies_choco81@yahoo.com brownies_choco81@yahoo.com Benjamin Snyder Announcements Office hours change for today and next week: 1pm - 1:45pm

More information

Active and Semi-Supervised Learning in ASR: Benefits on the Acoustic and Language Models

Active and Semi-Supervised Learning in ASR: Benefits on the Acoustic and Language Models INTERSPEECH 2016 September 8 12, 2016, San Francisco, USA Active and Semi-Supervised Learning in ASR: Benefits on the Acoustic and Language Models Thomas Drugman, Janne Pylkkönen, Reinhard Kneser Amazon

More information

MODIFIED WEIGHTED LEVENSHTEIN DISTANCE IN AUTOMATIC SPEECH RECOGNITION

MODIFIED WEIGHTED LEVENSHTEIN DISTANCE IN AUTOMATIC SPEECH RECOGNITION Krynica, 14 th 18 th September 2010 MODIFIED WEIGHTED LEVENSHTEIN DISTANCE IN AUTOMATIC SPEECH RECOGNITION Bartosz Ziółko, Jakub Gałka, Dawid Skurzok, Tomasz Jadczyk 1 Department of Electronics, AGH University

More information

I D I A P R E S E A R C H R E P O R T. 26th April 2004

I D I A P R E S E A R C H R E P O R T. 26th April 2004 R E S E A R C H R E P O R T I D I A P Posteriori Probabilities and Likelihoods Combination for Speech and Speaker Recognition Mohamed Faouzi BenZeghiba a,b Hervé Bourlard a,b IDIAP RR 04-23 26th April

More information

Efficient Methods to Train Multilingual Bottleneck Feature Extractors for Low Resource Keyword Search

Efficient Methods to Train Multilingual Bottleneck Feature Extractors for Low Resource Keyword Search Efficient Methods to Train Multilingual Bottleneck Feature Extractors for Low Resource Keyword Search Chongjia Ni, Cheung Chi Leung, Lei Wang, Nancy Chen and Bin Ma 9 March 2017 ICASSP 2017, New Orleans

More information

A Hybrid Neural Network/Hidden Markov Model

A Hybrid Neural Network/Hidden Markov Model A Hybrid Neural Network/Hidden Markov Model Method for Automatic Speech Recognition Hongbing Hu Advisor: Stephen A. Zahorian Department of Electrical and Computer Engineering, Binghamton University 03/18/2008

More information

Robust Speech Recognition using DNN-HMM Acoustic Model Combining Noise-aware training with Spectral Subtraction

Robust Speech Recognition using DNN-HMM Acoustic Model Combining Noise-aware training with Spectral Subtraction INTERSPEECH 2015 Robust Speech Recognition using DNN-HMM Acoustic Model Combining Noise-aware training with Spectral Subtraction Akihiro Abe, Kazumasa Yamamoto, Seiichi Nakagawa Department of Computer

More information

I D I A P R E S E A R C H R E P O R T. Sriram Ganapathy a b. May to appear in EUSIPCO 2008

I D I A P R E S E A R C H R E P O R T. Sriram Ganapathy a b. May to appear in EUSIPCO 2008 R E S E A R C H R E P O R T I D I A P Spectro-Temporal Features for Automatic Speech Recognition using Linear Prediction in Spectral Domain Samuel Thomas a b Hynek Hermansky a b IDIAP RR 08-05 May 2008

More information

Discriminative Learning of Feature Functions of Generative Type in Speech Translation

Discriminative Learning of Feature Functions of Generative Type in Speech Translation Discriminative Learning of Feature Functions of Generative Type in Speech Translation Xiaodong He Microsoft Research, One Microsoft Way, Redmond, WA 98052 USA Li Deng Microsoft Research, One Microsoft

More information

Spoken Language Identification with Artificial Neural Network. CS W Professor Torresani

Spoken Language Identification with Artificial Neural Network. CS W Professor Torresani Spoken Language Identification with Artificial Neural Network CS74 2013W Professor Torresani Jing Wei Pan, Chuanqi Sun March 8, 2013 1 1. Introduction 1.1 Problem Statement Spoken Language Identification(SLiD)

More information

Towards Speaker Adaptive Training of Deep Neural Network Acoustic Models

Towards Speaker Adaptive Training of Deep Neural Network Acoustic Models Towards Speaker Adaptive Training of Deep Neural Network Acoustic Models Yajie Miao Hao Zhang Florian Metze Language Technologies Institute School of Computer Science Carnegie Mellon University 1 / 23

More information

Investigating the Learning Effect of Multilingual Bottle-Neck Features for ASR

Investigating the Learning Effect of Multilingual Bottle-Neck Features for ASR Investigating the Learning Effect of Multilingual Bottle-Neck Features for ASR Ngoc Thang Vu, Jochen Weiner, Tanja Schultz Karlsruhe Institute of Technology, Germany {thang.vu, jochen.weiner, tanja.schultz}@kit.edu

More information

Context-Dependent Connectionist Probability Estimation in a Hybrid HMM-Neural Net Speech Recognition System

Context-Dependent Connectionist Probability Estimation in a Hybrid HMM-Neural Net Speech Recognition System Context-Dependent Connectionist Probability Estimation in a Hybrid HMM-Neural Net Speech Recognition System Horacio Franco, Michael Cohen, Nelson Morgan, David Rumelhart and Victor Abrash SRI International,

More information

Integration of Diverse Recognition Methodologies Through Reevaluation of N-Best Sentence Hypotheses

Integration of Diverse Recognition Methodologies Through Reevaluation of N-Best Sentence Hypotheses Integration of Diverse Recognition Methodologies Through Reevaluation of N-Best Sentence Hypotheses M. Ostendor~ A. Kannan~ S. Auagin$ O. Kimballt R. Schwartz.]: J.R. Rohlieek~: t Boston University 44

More information

Automatic speech recognition using context-dependent syllables

Automatic speech recognition using context-dependent syllables Automatic speech recognition using context-dependent syllables Jan Hejtmánek and Tomáš Pavelka Abstract In this work, we deal with advanced contextdependent automatic speech recognition (ASR) of Czech

More information

Adaptive Mixtures of Local Experts

Adaptive Mixtures of Local Experts In Neural Computation, 3, pages 79-87. Adaptive Mixtures of Local Experts Robert A. Jacobs Michael I. Jordan Department of Brain & Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA

More information

Convolutive Bottleneck Network Features for LVCSR

Convolutive Bottleneck Network Features for LVCSR Convolutive Bottleneck Network Features for LVCSR Karel Veselý 1, Martin Karafiát 2, František Grézl 3 Speech@FIT, Brno University of Technology Božetěchova 2, 612 66 Brno, Czech Republic 1 iveselyk@fit.vutbr.cz

More information

GRAPHEME AND MULTILINGUAL POSTERIOR FEATURES FOR UNDER-RESOURCED SPEECH RECOGNITION: A STUDY ON SCOTTISH GAELIC

GRAPHEME AND MULTILINGUAL POSTERIOR FEATURES FOR UNDER-RESOURCED SPEECH RECOGNITION: A STUDY ON SCOTTISH GAELIC GRAPHEME AND MULTILINGUAL POSTERIOR FEATURES FOR UNDER-RESOURCED SPEECH RECOGNITION: A STUDY ON SCOTTISH GAELIC Ramya Rasipuram 1,2, Peter Bell 3 and Mathew Magimai.-Doss 1 1 Idiap Research Institute,

More information

Enhancing the TED-LIUM Corpus with Selected Data for Language Modeling and More TED Talks

Enhancing the TED-LIUM Corpus with Selected Data for Language Modeling and More TED Talks Enhancing the TED-LIUM with Selected Data for Language Modeling and More TED Talks Anthony Rousseau, Paul Deléglise, Yannick Estève Laboratoire Informatique de l Université du Maine (LIUM) University of

More information

ACCENT ADAPTATION USING SUBSPACE GAUSSIAN MIXTURE MODELS

ACCENT ADAPTATION USING SUBSPACE GAUSSIAN MIXTURE MODELS ACCENT ADAPTATION USING SUBSPACE GAUSSIAN MIXTURE MODELS Petr Motlicek, Philip N. Garner Idiap Research Institute Martigny, Switzerland {motlicek,garner}@idiap.ch Namhoon Kim, Jeongmi Cho Samsung Electronics

More information

UNSUPERVISED NEURAL NETWORK BASED FEATURE EXTRACTION USING WEAK TOP-DOWN CONSTRAINTS

UNSUPERVISED NEURAL NETWORK BASED FEATURE EXTRACTION USING WEAK TOP-DOWN CONSTRAINTS UNSUPERVISED NEURAL NETWORK BASED FEATURE EXTRACTION USING WEAK TOP-DOWN CONSTRAINTS Herman Kamper 1,2, Micha Elsner 3, Aren Jansen 4, Sharon Goldwater 2 1 CSTR and 2 ILCC, School of Informatics, University

More information

Neural Network Language Models

Neural Network Language Models Neural Network Language Models Steve Renals Automatic Speech Recognition ASR Lecture 12 6 March 2014 ASR Lecture 12 Neural Network Language Models 1 Neural networks for speech recognition Introduction

More information

21-23 September 2009, Beijing, China. Evaluation of Automatic Speaker Recognition Approaches

21-23 September 2009, Beijing, China. Evaluation of Automatic Speaker Recognition Approaches 21-23 September 2009, Beijing, China Evaluation of Automatic Speaker Recognition Approaches Pavel Kral, Kamil Jezek, Petr Jedlicka a University of West Bohemia, Dept. of Computer Science and Engineering,

More information