A Simple Segmentation Approach for Unconstrained Cursive Handwritten Words in Conjunction with the Neural Network. Amjad Rehman Khan PhD researcher Department of Computer Graphics and Multimedia University Technology Malaysia Skudai, 81310, Malaysia Dr. Zulkifli Mohammad Associate Professor Department of Computer Graphics and Multimedia University Technology Malaysia Skudai, 81310, Malaysia amjadbzu2003@yahoo.com dzullkilfli@utm.my Abstract This paper presents a new, simple and fast approach for character segmentation of unconstrained handwritten words. The developed segmentation algorithm over-segments in some cases due to the inherent nature of the cursive words. However the over segmentation is minimum. To increase the efficiency of the algorithm an Artificial Neural Network is trained with significant amount of valid segmentation points for cursive words manually. Trained neural network extracts incorrect segmented points efficiently with high speed. For fair comparison benchmark database IAM is used. The experimental results are encouraging. Keywords: Image analysis, Segmentation, Neural Network, Preprocessing, Pattern matching. 1. INTRODUCTION An extensive research has been done in the field of handwriting recognition in the last few decades [1]. It seems that the research has been reached to its maturity for the recognition of isolated characters recognition, hand printed words recognition, automatic address processing and bank check reading (holistic approaches) [2-4]. In contrast for the analytical approaches where the word is segmented into its component characters, recognition results for unconstrained handwriting is still low due to the poorly segmented words. Segmentation errors mislead classifier during character recognition [5-7]. In fact segmentation problem has persisted for nearly as long as handwriting recognition problem itself. In literature, segmentation algorithms for unconstrained handwritten words can be generalized into two categories.[7] External segmentation and Holistic segmentation. In the former category letter boundaries are determined prior to recognition while in the latter, segmentation and recognition are carried out at the same time and the final character boundaries are determined dynamically by semantic analysis and classification performance.[8-10]. Higher the segmentation accuracy, the more beneficial it is to the recognition rates [11]. Hence the segmentation is the backbone of the recognition process and is still active research topic. Researchers have acknowledged the important role that segmentation plays in handwriting recognition process [7, 12-13]. That is why more innovative, accurate and fast methods need to be employed and compared to the work of other researchers using benchmark databases. International Journal of Image Processing, volume (2) issue (3) 29
In most of the existing segmentation algorithms, human writing is evaluated empirically to deduce rules [15]. Sometimes the rules derived are satisfactory but there is no guarantee for their optimum results in all style of writing. Because human writing varies from person to person and even for the same depending on mood, speed, environment etc. On the other hand researchers have employed artificial neural networks, hidden Morkov models, statistical classifiers etc to extract rules based on numerical data [16-21, 36-37] This research attempts to integrate, rule based segmentation approach and intelligent method for the character segmentation of unconstrained handwritten words. A simple but efficient, rule based segmentation algorithm is presented that performs character segmentation very well with high speed but some characters are over-segmented. Therefore an ANN is integrated with the proposed approach as artificial neural networks have been successfully used in the field of pattern recognition [14, 20-23]. To verify the segmentation points marked by proposed algorithm, an artificial neural network is trained with correct and incorrect segmentation points for the words images taken from benchmark database [24]. The rest of the paper is organized in four sections. Section 2 presents proposed segmentation algorithm along with segmentation results. In section 3, neural based experimentation is performed and results are discussed. Finally, conclusion and future work is drawn in section 4. 2. PROPOSED SEGMENTATION APPROACH In this section segmentation algorithm and preprocessing steps are presented. Artificial neural network is trained manually for the correct and incorrect segmentation points obtained from the proposed segmentation technique. MATLAB 7.0 is used for all experiments performed on system of 1.6 GHz processor and 1 GB DDR RAM. 2.1 Preprocessing and proposed segmentation algorithm The original grey scaled image is binarized using Otsu algorithm by selecting automatically a threshold value for a given image [22]. If required, following binarization, slant correction is performed [23]. Finally, image is converted to skeleton format to allow users verity of writing device, pen tilt and to suppress extra data. The proposed segmentation algorithm is explained in the figure 1. Step 1. Take word image from database. Step 2. Perform pre-processing. Step 3. Calculate sum of foreground pixels (white pixels) for each column. Save those columns as candidate segment column (CSC) for which sum is 0 or 1 only. Step 4. By previous step, we have more candidate segmentation columns than actual required. Hence threshold (approximate character width) is selected empirically from candidate segment columns to come out with actual segment columns. FIGURE 1: Proposed Segmentation Algorithm Due to the simplicity of the proposed segmentation technique, it is very fast and performs well in most of the cases. For few characters such as m, n, u, v and w over segmentation occurs and this technique fails to find accurate character boundaries. Segmentation results by the proposed segmentation technique are shown in figure 2. International Journal of Image Processing, volume (2) issue (3) 30
FIGURE 2: Segmentation Results by Proposed Segmentation Approach. It can be seen from the results that segmentation is good except for few characters, where over segmentation occurs. Therefore it is required to integrate this technique with some intelligent method to increase its performance. In this regard a trained neural network is employed. It is mention worthy that over segmentation is minimum and occurs for few characters only. Hence it lessened burden of the classifier used and therefore processing speed increased. 2.2 Handwriting database For the fair comparison, patterns are selected from IAM V3.0 benchmark database [24]. A few samples for segmentation, training and testing of the ANN are shown in figure 3. The reason for selecting this database is that, it is freely available for researchers to comparing their results. FIGURE 3: Samples of Word Images from IAM Database 3. EXPERIMENT AND RESULTS 3.1 Training Artificial Neural Network. A simple program in MATLAB 7.0 is developed to detect co-ordinates of all segmentation points given birth by the proposed segmentation technique for each pattern. These segmentation points are divided into correct and incorrect categories manually and stored in a training file. Data is preprocessed prior to use for ANN training. For training, ANN with standard back propagation algorithm is used. A number of experiments with different structures, weights, epochs, momentum and learning rate are performed to enable ANN to distinguish between correct and incorrect segmentation points. The ANN trained with 25072 training patterns (segmentation points) taken from 2678 words. The optimal structure of ANN thus found contained 235 to 310 inputs, 25 to 38 hidden units and one output (correct or incorrect segmentation point) with 300 epochs. Learning rate and momentum was set to 0.2 and 0.6 respectively. MATLAB 7.0 is used for implementation. Trained neural network operates in figure 4. FIGURE 4: Incorrect Segmentation Points are rejected by Trained Neural Network. International Journal of Image Processing, volume (2) issue (3) 31
3.2 Performance of the neural rule-based Segmentation Technique. After training, the testing phase occurs. For testing phase 2936 samples are selected from the IAM database. These new patterns are segmented by the proposed algorithm. Segmentation points thus obtained are fed to the train ANN for their classification into correct and incorrect categories. Finally, correct are left and incorrect are rejected by trained ANN as shown in figure 4. Segmentation results for test set are presented in table 1. Correct segmentation rate. 2678/2936 % Correct segmentation rate. 91.21 % % Miss segmentation rate. 5.38 % % Over segmentation rate. 3.20 % TABLE 1: Segmentation Rates 3.3 Analysis and discussion of results The neuro rule-based segmentation algorithm achieved recognition rate of 91.21% for valid identification of 2936 segmentation points pattern. Two problems are found during the analysis of the results. Firstly, noisy characters, so some additional preprocessing is done before training ANN. Secondly, touched/ overlapped characters. This type of problem is very hard to deal with. When two characters are tight together, ligatures can t be found and therefore they can t be segmented. Hence overall correct segmentation results decreased. It is very hard to compare segmentation results with the other researcher because segmentation is intermediate process of recognition. In addition to that many researchers report recognition rates only. Moreover different researcher used different database and report results under some constrains. Segmentation results are reported in literature is presented in table 2 for fair comparison. Author Segmentation Segmentation Database used Comments method rate Tappert et al [25] Feature based + Rule based 81.08% CEDAR number of words not mentioned Srihari [26] ANN 83% Handwritten zip No alphabetic codes Han and Sethi [27] Heuristic algorithm 85.7 % Latin handwritten Words on 50 real mail envelopes Only 50 mail envelopes are taken. Lee et al [28] ANN 90% Printed latin alphanumeric characters Eastwood et al [29] Blumenstein and Verma [30] Yanikoglu and Sandon [31] Nicchiotti and Scagliola. [32] Verma and Gader [33] ANN 75.9 % Cursive latin handwritten from CEDAR database Printed alphanumeric characters used 100,000 training pattern used ANN + conventional method 81.21 % 2568 words from CEDAR Linear Programming 97 % 750 words No bench mark database used Rule-based 86.9 % CEDAR 850 words used only. Feature based + 91 CEDAR words number not ANN mentioned Blumenstein and Verma [34] Feature based+ ANN 78.85% CEDAR words number not mentioned Verma[35] Feature based + ANN 84.87 CEDAR 300 words only Cheng et al [36] Feature based + ANN 95.27 CEDAR 317 words only Cheng et al [37] Enhanced feature based+ ANN 84.19 CEDAR 317 words only International Journal of Image Processing, volume (2) issue (3) 32
TABLE 2: Comparison of Segmentation Results in the Literature. 4. CONCLUSION & FUTURE WORK Segmentation is the important step of analytical approaches employed to handwritten word recognition. Hence it is the base of most modern approaches. It is admitted fact that no segmentation method can directly locate character location accurately without an intelligent method. In this paper, proposed segmentation algorithm is integrated with neural network using standard back propagation. Initial experiments exhibit very encouraging results with segmentation accuracy up to 91.21%. Speed is another important factor, overlooked in many past researches. Due to the minimum over segmentation, neural network is least burdened and therefore speed is optimum. This paper has briefly described one stage on our progress towards final goal of unconstrained cursive words recognition with higher recognition rate and speed. ACKONOLOGEMENT This research work is fully supported by the Ministry of Science, Technology and Innovation Malaysia under grant vote 79258. The author is also thankful to colleague researcher Fajri Kurniawan for his help in experimentation. 5. REFERENCES 1. H.Bunke. Recognition of cursive roman handwriting, past, present and future. In Proceeding of 7th International Conference on Document Analysis and Recognition, 448-461, 2003. 2. C.Y.Suen, R.Legault, C. Nadal, M.Cheriet and L.Lam. Building a new generation of handwriting recognition systems. Pattern Recognition Letters, 14: 305-315, 1993. 3. S.W. Lee. Multilayer cluster neural network for totally unconstrained handwritten numerical recognition. Neural Networks, 8: 783-792, 1995 4. H. I. Avi-Itzhak, T. A. Diep and H.Garland. High accuracy optical character recognition using neural networks. IEEE Trans. Pattern Analysis and Machine Intelligence, 18: 648-652, 1996. 5. S.B. Cho. Neural networks classifiers for recognition totally unconstrained handwritten numericals. IEEE Trans. On Neural Networks, 8. 6. S.W. Lee. Off-line recognition of totally unconstrained handwritten numericals using multilayer cluster neural network. IEEE Trans Pattern Analysis and Machine Intelligence, 18: 648-652, 1996. 7. R.G. Casey, E. Lecolinet. A survey of methods and strategies in character segmentation. IEEE Trans. Pattern Analysis and Machine Intelligence, 18: 690-706, 1996. 8. R. Farag. Word level recognition of cursive script. IEEE Trans. Computing 28: 172-175, 1979. 9. J. M. Bertille, M. E. Yacoubi. Global cursive postal code recognition using hidden Marko models. In Proceeding of the First European Conference Postal Technology, France, 129-138, 1993. 10. J. Wang, J. Jean. Segmentation of merged characters by neural networks and shortest path Pattern Recognition 27(5): 649-658, 1994. International Journal of Image Processing, volume (2) issue (3) 33
11. C. K. Cheng, M. Blumenstein. "The neural-based segmentation of cursive words using enhanced heuristics". In Proceedings of Eighth International Conference on Document Analysis and Recognition (ICDAR'05), 650-654, 2005. 12. S.N. Srihari. Recognition of handwritten and machine printed text for postal address interpretation. Pattern recognition letters, 14: 291-302, 1993. 13. M. Gilloux. Research into the new generation of character and mailing address recognition systems at the French post office research center. Pattern Recognition Letters.14: 267-276 1993. 14. M. Blumenstein, B. K. Verma. An artificial neural network based segmentation algorithm for off-line handwriting recognition. In proceedings of International Conference on Computational Intelligence and Multimedia Applications (ICCIMA 98), Gippsland, Australia, 1997. 15. X. Xiao, G. Leedham. Knowledge based English cursive script segmentation. Pattern recognition letters 21: 945-954, 2000. 16. M. Giloux. Hidden Markov Models in Handwriting Recognition. Fundamentals in Handwriting Recognition, S.Impedovo ed., NATO ASI Series F: Computer and System Science, 24: Spinger Verlang, 1994. 17. M. EI. Yacoub, M. Gilloux and J. M. Bertille. A statistical approach for phrase location and recognition with in a text Line. An application to Street Name Recognition, IEEE Trans. Pattern Analysis and Machine Intelligence, 24(2): 172-188, 2002. 18. A. Khotanzad, J. Lu. Shape and texture recognition by a neural network. Artificial Neural Networks in Pattern Recognition, Elsevier Science Publishers B.V., Amsterdam, Netherlands, 109-131, 1991. 19. B. Zheng, W. Qian and L.Clarke. Multistage neural network for pattern recognition in mammogram screening. IEEE ICNN, Orlando, 3437-3448, 1994. 20. A.D. Kulkarni. Artifical neural networks for image understanding. Van Nostrand Reinhold, New York. 154-270, 1994. 21. K. Han, I. K. Sethi. Handwriting signature retrieval and Identification. Pattern Recognition Letters, 17, 83-90, 1996. 22. N. Otsu. A threshold selection method from gray level histograms. IEEE transactions on systems, Man and Cybernetics, 9(1): 62-66, 1979. 23. S. Knerr, E. Augustin. A neural network hidden markov model hybrid for cursive word recognition. In Proceedings of International Conference on Pattern Recognition, Brisbane, 2: 1518-1520, 1998. 24. U. Marti, H. Bunke. The IAM database: An English sentence database for off-line handwriting recognition. International Journal of Document Analysis and Recognition, 15: 65-90, 2002. 25. C. C. Tappert., C. Y. Suen and T. Wakahara. The state of the art in on-line handwriting recognition. IEEE Trans. Pattern Analysis. Machine. Intelligence. 12: 787-808, (1990) International Journal of Image Processing, volume (2) issue (3) 34
26. S. N. Srihari. Recognition of handwritten and machine-printed text for postal address Interpretation. Pattern Recognition Letters 291-302, 1993. 27. K. Han, I. K. Sethi. Off-line cursive handwriting segmentation. ICDAR 95, Montreal, Canada, 894-897, 1995 28. S-W. Lee, D-J. Lee and H-S Park. A new methodology for gray scale character segmentation and recognition. IEEE Transaction on Pattern Analysis and Machine Intelligence, 1045-1051, 1996. 29. B. Eastwood, A. Jennings and A. Harvey. A feature based neural network segmenter for handwritten words. ICCIMA 97 Australia, 286-290. 1997 30. M. Blumenstein, B. Verma. A segmentation algorithm used in conjunction with artificial neural networks for the recognition of real-word postal addresses. In Proceeding of International Conference on Computational Intelligence and Multimedia Applications (ICCIMA 97), Gold Coast, Australia. 155-160, 1997. 31. B. Yanikoglu, P.A.Sandon. Segmentation of off-line cursive handwriting using linear programming. Pattern Recognition, 31: 1825-1833, 1998. 32. G. Nicchiotti, C.Scagliola. Generalized projections: a tool for cursive handwriting normalisation. In Proceedings of 5th International Conference on Document Analysis and Recognition, Bangalore, 729-733, 1999. 33. B. Verma, P. Gader. "Fusion of multiple handwritten word recognition techniques". Neural Networks for Signal Processing X, 2000. In Proceedings of the IEEE Signal Processing Society Workshop, 2: 926-934, 2000. 34. M. Blumenstein, B. Verma. "Analysis of segmentation performance on the CEDAR benchmark database". In Proceedings of Sixth International Conference on Document Analysis and Recognition (ICDAR'01), 1142, 2001. 35. B. Verma. "A contour character extraction approach in conjunction with a neural confidence fusion technique for the segmentation of handwriting recognition. In Proceedings of the 9th International Conference on Neural Information Processing. 5: 18-22, 2002. 36. C. K. Cheng., X. Y. Liu., M. Blumenstein and V. Muthukkumarasamy. Enhancing neural confidence-based segmentation for cursive handwriting recognition. In Proceeding of 5th International Conference on Simulated Evolution and Learning (SEAL '04), Busan, Korea, SWA-8, 2004. 37. C. K. Cheng, M. Blumenstein. Improving the segmentation of cursive handwritten words using ligature detection and neural validation. In Proceedings of the 4th Asia Pacific International Symposium on Information Technology (APIS 2005), Gold Coast, Australia, 56-59, 2005. International Journal of Image Processing, volume (2) issue (3) 35