Indian Sign Language Recognition Using CNN and Mediapipe

Abstract

Deep learning has significantly advanced the field of image classification, yet deploying such models in lightweight, real-time applications remains a challenge. This study presents a complete end-to-end system, titled Real-Time ISL Recognition Using CNN and MediaPipe, which integrates a Convolutional Neural Network (CNN) model with a Flask-based web application for real-time image classification. A custom-built dataset was collected and augmented using a self-defined image transformation pipeline. The model was trained and fine-tuned using categorical cross-entropy loss and Adam optimizer, and the final architecture was saved in H5 format for fast deployment. Grad-CAM visualizations were used to interpret the network’s predictions. The entire system achieves a classification accuracy of 98.32% and demonstrates a minimal-latency deployment pipeline suitable for real-time web-based applications. This lightweight and interpretable framework bridges the gap between academic modeling and practical deployment, especially in resource-constrained environments..