Computer Sciences and Information Technology
Title : | Digitization of ancient manuscripts in Thigalari and Grantha scripts using multimodal recognition of speech and online handwritten transcript of an expert in the loop |
Area of research : | Computer Sciences and Information Technology |
Focus area : | Digitization of ancient manuscripts |
Principal Investigator : | Prof. Ramakrishnan AngaraiGanesan, Indian Institute Of Science, Bangalore, Karnataka |
Timeline Start Year : | 2019 |
Timeline End Year : | 2022 |
Contact info : | agr@iisc.ac.in |
Details
Executive Summary : | Objective: To create editable digital storage of content in the ancient scripts of Thigalari and Grantha from sources such as palm leaf manuscripts. Precious manuscripts in scripts known only to a few experts in the country are targeted in this project for digital capture as e-text. Background: Valuable knowledge related to different fields such as science, engineering, arts and medicine exists in ancient manuscripts on media such as palm leaf, paper, special cloth and stone inscriptions. Given that no commercial solution is available even for unstructured, offline handwritten text in English, it is a herculean task to develop successful recognition technologies for each of the ancient scripts present in these manuscripts. Thus, an innovative solution needs to be found for effective digitization (conversion to e-text) of these, which can then be easily accessed over the internet and the information hidden in them extracted.
Process: The digitization starts with the availability of an expert, who can read the script of the manuscript. Investigators record the expert’s speech, as he/she reads the manuscript. They also ask the expert or a scribe (under the supervision of the expert) to transcribe the same on a Tablet (PC), in Tamil, Kannada or Devanagari script. They already have engines that recognize online handwriting in these three scripts. They will recognize the speech as a sequence of universal phonemes (that support all the languages under consideration), transcribe it into the same language as the handwriting and store it as a Unicode string. The recognition ambiguities in one modality (say, speech) will be corrected based on the information available from the other modality (say handwriting) and vice versa. Thus, investigators develop a vocabulary independent, bimodal recognition system, that can store the recognized text in a script-independent representation, similar to the ISCII codes. The recognized text will finally be converted to the UNICODE text in the desired script and displayed with the font exclusively designed for that script. Fonts (graphemes) will be developed by CDAC Pune for both Thigalari and Grantha. By-products are two vocabulary-independent, recognition engines: (i) a language independent recognition of speech as a sequence of phonemes represented using a generic code and (ii) recognition of free vocabulary text in any language written online using specific scripts. This is a must, since the content of many manuscripts if from a language, distinct from that of the script.
Deliverable: A prototype solution that can convert manuscript in an ancient script to an editable document using bimodal recognition of inputs from a human expert in the loop. This is a software that runs on a desktop or tablet PC, complete with GUI for acquiring the read speech and handwritten transcript from the human expert. The e-text will be displayed in the original, ancient script and can also be read by Kannada TTS, to the extent the phonemes are supported by it. |
Total Budget (INR): | 77,81,400 |
Achievements : | A recognizer has been developed for Devanagari handwritten character recognition.Vocabulary independent recognizers have been developed for Tamil, Kannada and Devanagari online handwriting. Speech recognizers with finite performance have also been developed for Tamil, Kannada and Sanskrit. |
Patents : | 1 |
Organizations involved