Multi-lingual text identification and recognition: a deep learning approach

Abstract

In this thesis, an attempt has been made to solve the problem of text detection and identification for multi-lingual text on natural images and applied a pre-trained model, Tesseract OCR to recognize these texts. A method for detecting and classification of the multilingual text using You Look Only Once (YOLO), a Deep Learning approach and recognition of the text using Tesseract OCR from natural scene images is proposed. To accomplish this task, YOLO version-3 has been used which is immensely fast and give accurate result. The YOLOV3 uses the Darknet-53 and has an overall 53 conventional layers. This model not only predict the location and make bounding boxes over the texts but also predict the class level of the bounding box. The convolutional networks are first trained on Custom dataset with smaller number of images. Then a standard dataset, MITDI, containing 400 train natural scene images and 400 train born digital images, is used to effectively train the model. Using the trained model, the text regions are successfully extracted from the input image and the proposals obtained are labelled with the language class of the text within it. Then, Optical Character Recognition (OCR) engines are used to recognize the text in a specific language. Tesseract which has been implemented using a Long Short-term Memory (LSTM) based recognition engine, performed well on the natural scene images. Multiple applications can be realized, ranging from human computer interaction for visually impaired person, context extraction from image, autonomous machines etc.