Handwritten Notes Digitization and Multilingual Summarization Using OCR and Generative AI

Suchetha N V; Anisha Upadhayaya H S; Anushree U Rao; H N Swati; Mallana  Gowda G S

doi:10.55524/ijircst.2026.14.1.2

Abstract

This research proposes a fully client-side system for digitising and summarising handwritten notes that integrate optical character recognition (OCR) with advanced natural language generation models. Initial experimentation with a Convolutional Neural Network (CNN)– based handwritten character recognition system revealed significant challenges, including low recognition accuracy, strong dependence on dataset quality, and poor generalisation across diverse handwriting styles. A subsequent implementation using Gemini 2.5 Pro provided high-quality English summaries but failed to deliver equivalent performance for multilingual academic content. To overcome these issues, this study adopts a hybrid methodology combining Tesseract.js OCR for multilingual text extraction with Gemini 2.5 Flash for fast, context-aware, and language-flexible summarization. The system is able to take handwritten notes, convert them into clear and readable text, and produce well-structured academic summaries across different subjects and languages. Our tests show that it works reliably, adapts well to multiple languages, and is easy to use, making it a strong and practical tool for educational digitization.

Keywords

Handwritten Note Digitization, OCR, CNN, NLP, Multilingual Processing, Summarization, Deep Learning

References

M. Gupta and K. Soeny, “Algorithms for rapid digitalization of prescriptions,” Visual Informatics, vol. 5, no. 3, 2021. Available from: https://doi.org/10.1016/j.visinf.2021.07.002
S. L. Chen ShanWei and D. A. Ng Theam Foo, “A CNN based handwritten numeral recognition model for four arithmetic operations,” Procedia Computer Science, vol. 192, 2021. Available from: https://doi.org/10.1016/j.procs.2021.09.218
C. Lehenmeier, M. Burghardt, and B. Mischka, “Layout detection and table recognition—Recent challenges in digitizing historical documents and handwritten tabular data,” Lecture Notes in Computer Science, 2020. Available from: https://doi.org/10.1007/978-3-030-54956-5_17
P. Bannigidad and C. Gudada, “Digitization and recognition of historical Kannada handwritten manuscripts using text line segmentation with LBP features,” Journal of Emerging Technologies and Innovative Research, vol. 6, no. 4, 2019. Available from: https://tinyurl.com/mrydhfym
K. Amulya et al., “A survey on digitization of handwritten notes in Kannada,” International Journal of Innovative Technology and Exploring Engineering, vol. 12, no. 1, 2022. Available from: https://tinyurl.com/37fujfe4
R. P. Ram Kumar et al., “Automated handwritten text recognition,” E3S Web of Conferences, vol. 430, 2023. Available from: https://doi.org/10.1051/e3sconf/202343001022
S. Rakesh et al., “Handwritten text recognition using deep learning techniques: A survey,” MATEC Web of Conferences, vol. 392, 2024. Available from: https://doi.org/10.1051/matecconf/202439201126
P. Bala, “Handwritten character recognition and evaluation,” Journal of Artificial Intelligence Practice, vol. 4, 2025.
M. K. Habibi Dehkordi et al., “Improving large language models’ summarization accuracy by adding highlights to discharge notes: Comparative evaluation,” JMIR Medical Informatics, vol. 13, 2025. Available from: https://medinform.jmir.org/2025/1/e66476
A. Chiney et al., “Handwritten data digitization using an anchor-based multi-channel CNN (MCCNN) trained on hybrid dataset (h-EH),” Procedia Computer Science, vol. 189, 2021. Available from: https://doi.org/10.1016/j.procs.2021.05.095
L. G. Singh and S. E. Middleton, “Tabular context-aware optical character recognition and tabular data reconstruction for historical records,” International Journal on Document Analysis and Recognition, vol. 28, no. 3, 2025. Available from: https://doi.org/10.1007/s10032-025-00543-9
N. Iskandar, W. J. Chew, and S. K. Phang, “The application of image processing for conversion of handwritten mathematical expression,” Journal of Physics: Conference Series, vol. 2523, 2023. Available from: https://doi.org/10.1088/1742-6596/2523/1/012014
J. Käser et al., “Multilingual text summarization in healthcare using pre-trained transformer-based language models,” Computers, Materials and Continua, vol. 83, no. 1, 2025. Available from: https://doi.org/10.32604/cmc.2025.061527
M. C. S. Menezes et al., “The potential of generative pre-trained transformer-4 (GPT-4) to analyse medical notes in three different languages: A retrospective model-evaluation study,” The Lancet Digital Health, vol. 7, no. 1, 2025. Available from: https://doi.org/10.1016/S2589-7500(24)00246-2
A. Supriyono et al., “A survey of text summarization: Techniques, evaluation and challenges,” Natural Language Processing Journal, vol. 7, 2024. Available from: https://doi.org/10.1016/j.nlp.2024.100070

Cites this article as

S. N. V, A. U. H. S, A. U. Rao, H. N. Swati, M. . G. G. S, "Handwritten Notes Digitization and Multilingual Summarization Using OCR and Generative AI", International Journal of Innovative Research in Computer Science and Technology (IJIRCST), vol. 14, no. 1, pp. 9-16, 2026. doi: https://doi.org/10.55524/ijircst.2026.14.1.2

Corresponding Author

Suchetha N V

Associate Professor, Department of Computer Science & Engineering, Sri Dharmasthala Manjunatheshwara Institute of Technology, Ujire, Karnataka, India

Download Full Paper

Download PDF

No. of Downloads: 62 | No. of Views: 5187

Handwritten Notes Digitization and Multilingual Summarization Using OCR and Generative AI

Citations

Total View 5187

Total Download 62