Development Of A CNN Model For Recognizing The Indonesian Sign Language (BISINDO) Alphabet
DOI:
https://doi.org/10.35194/mji.v17i2.5936Keywords:
Indonesian Sign Language (BISINDO), Convolution Neural Network (CNN), Deep Learning, Computer Vision, Sign Language InterpreterAbstract
Deaf people in Indonesia face communication barriers due to the limited understanding of Indonesian Sign Language (BISINDO) among the general public. This results in limited social interaction between deaf people and their surroundings. This study aims to develop a Deep Learning and Computer Vision-based BISINDO alphabet translator model using the Convolutional Neural Network (CNN) method, addressing the limited availability of publicly documented BISINDO datasets for alphabet recognition . The method used involves training the model with a dataset of 3,120 BISINDO alphabet images, covering the letters A to Z. The dataset was divided into 80% for training and 20% for testing. The training process included model architecture design, parameter tuning, selection of the best model based on accuracy, and performance evaluation. The evaluation results showed that the developed CNN model achieved an accuracy of 99.84% in classifying BISINDO letters; however, challenges remain in generalizing the model to variations in lighting, hand orientation, and user differences. Nevertheless, the high accuracy achieved indicates the model’s potential to support effective BISINDO translation and improve communication accessibility. This research also opens up opportunities for further development towards comprehensive translation of gestures or sentences in BISINDO.References
[1] N. Haliza, E. Kuntarto, and A. Kusmana, “Language acquisition in children with special needs (deaf) in understanding language,” Jurnal Genre (Bahasa, Sastra, dan Pembelajarannya), vol. 2, no. 1, pp. 5–11, Jul. 2020, doi: 10.26555/jg.v2i1.2051.
[2] J I. Sari, Fivrenodi, E. Altiarika, and Sarwindah, “Sign Language Development System for Communicating with Persons with Disabilities (Deaf),” Journal of Information Technology and society, vol. 1, no. 1, pp. 20–25, Jun. 2023, doi: 10.35438/jits.v1i1.21.
[3] T. Handhika, R. I. M. Zen, Murni, D. P. Lestari, and I. Sari, “Gesture recognition for Indonesian Sign Language (BISINDO),” J Phys Conf Ser, vol. 1028, p. 012173, Jun. 2018, doi: 10.1088/1742-6596/1028/1/012173.
[4] R. K. Pathan, M. Biswas, S. Yasmin, M. U. Khandaker, M. Salman, and A. A. F. Youssef, “Sign language recognition using the fusion of image and hand landmarks through multi-headed convolutional neural network,” Sci Rep, vol. 13, no. 1, Dec. 2023, doi: 10.1038/s41598-023-43852-x.
[5] Iffat Zafar, Richard Burton, Leonardo Araujo, and Giounona Tzanidou, Hands-On Convolutional Neural Networks with TensorFlow: Solve computer vision problems with modeling in TensorFlow and Python. Packt Publishing Ltd, 2018.
[6] R. Z. Fadillah, A. Irawan, M. Susanty, and I. Artikel, “Data Augmentation to Overcome Data Limitations in the Indonesian Sign Language Translation Model (BISINDO),” JURNAL INFORMATIKA, vol. 8, no. 2, 2021, [Online]. Available: http://ejournal.bsi.ac.id/ejurnal/index.php/ji
[7] M. Hassaballah and A. I. Awad, Deep Learning in Computer Vision, 1st ed. CRC Press, 2020.
[8] C. Nebauer, “Evaluation of convolutional neural networks for visual recognition,” IEEE Trans Neural Netw, vol. 9, no. 4, pp. 685–696, Jul. 1998, doi: 10.1109/72.701181.
[9] S. Indolia, A. K. Goswami, S. P. Mishra, and P. Asopa, “Conceptual Understanding of Convolutional Neural Network- A Deep Learning Approach,” Procedia Comput Sci, vol. 132, pp. 679–688, 2018, doi: 10.1016/j.procs.2018.05.069.
[10] I. Arel, D. C. Rose, and T. P. Karnowski, “Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier],” IEEE Comput Intell Mag, vol. 5, no. 4, pp. 13–18, Nov. 2010, doi: 10.1109/MCI.2010.938364.
[11] E. A. Smirnov, D. M. Timoshenko, and S. N. Andrianov, “Comparison of Regularization Methods for ImageNet Classification with Deep Convolutional Neural Networks,” AASRI Procedia, vol. 6, pp. 89–94, 2014, doi: 10.1016/j.aasri.2014.05.013.
[12] A. ANHAR and R. A. PUTRA, “Design and Implementation of a Self-Checkout System in Retail Stores using Convolutional Neural Networks (CNN),” ELKOMIKA: Jurnal Teknik Energi Elektrik, Teknik Telekomunikasi, & Teknik Elektronika, vol. 11, no. 2, p. 466, Apr. 2023, doi: 10.26760/elkomika.v11i2.466.
[13] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998, doi: 10.1109/5.726791.
[14] Seth Weidman, Deep Learning from Scratch Building with Python from First Principles. y O’Reilly Media, Inc., 2019.
[15] H. Liu, B. Li, X. Lv, and Y. Huang, “Image Retrieval Using Fused Deep Convolutional Features,” Procedia Comput Sci, vol. 107, pp. 749–754, 2017, doi: 10.1016/j.procs.2017.03.159.
[16] Y. Zhou, H. Wang, F. Xu, and Y.-Q. Jin, “Polarimetric SAR Image Classification Using Deep Convolutional Neural Networks,” IEEE Geoscience and Remote Sensing Letters, vol. 13, no. 12, pp. 1935–1939, Dec. 2016, doi: 10.1109/LGRS.2016.2618840.
[17] Giuseppe Ciaburro and Balaji Venkateswaran, Neural Networks with R: Smart models using CNN, RNN, deep learning, and artificial intelligence principles. 1BDLU1VCMJTIJOH-UE, 2017.
[18] J. Fieres, J. Schemmel, and K. Meier, “Training convolutional networks of threshold neurons suited for low-power hardware implementation,” in The 2006 IEEE International Joint Conference on Neural Network Proceedings, IEEE, 2006, pp. 21–28. doi: 10.1109/IJCNN.2006.246654.
[2] J I. Sari, Fivrenodi, E. Altiarika, and Sarwindah, “Sign Language Development System for Communicating with Persons with Disabilities (Deaf),” Journal of Information Technology and society, vol. 1, no. 1, pp. 20–25, Jun. 2023, doi: 10.35438/jits.v1i1.21.
[3] T. Handhika, R. I. M. Zen, Murni, D. P. Lestari, and I. Sari, “Gesture recognition for Indonesian Sign Language (BISINDO),” J Phys Conf Ser, vol. 1028, p. 012173, Jun. 2018, doi: 10.1088/1742-6596/1028/1/012173.
[4] R. K. Pathan, M. Biswas, S. Yasmin, M. U. Khandaker, M. Salman, and A. A. F. Youssef, “Sign language recognition using the fusion of image and hand landmarks through multi-headed convolutional neural network,” Sci Rep, vol. 13, no. 1, Dec. 2023, doi: 10.1038/s41598-023-43852-x.
[5] Iffat Zafar, Richard Burton, Leonardo Araujo, and Giounona Tzanidou, Hands-On Convolutional Neural Networks with TensorFlow: Solve computer vision problems with modeling in TensorFlow and Python. Packt Publishing Ltd, 2018.
[6] R. Z. Fadillah, A. Irawan, M. Susanty, and I. Artikel, “Data Augmentation to Overcome Data Limitations in the Indonesian Sign Language Translation Model (BISINDO),” JURNAL INFORMATIKA, vol. 8, no. 2, 2021, [Online]. Available: http://ejournal.bsi.ac.id/ejurnal/index.php/ji
[7] M. Hassaballah and A. I. Awad, Deep Learning in Computer Vision, 1st ed. CRC Press, 2020.
[8] C. Nebauer, “Evaluation of convolutional neural networks for visual recognition,” IEEE Trans Neural Netw, vol. 9, no. 4, pp. 685–696, Jul. 1998, doi: 10.1109/72.701181.
[9] S. Indolia, A. K. Goswami, S. P. Mishra, and P. Asopa, “Conceptual Understanding of Convolutional Neural Network- A Deep Learning Approach,” Procedia Comput Sci, vol. 132, pp. 679–688, 2018, doi: 10.1016/j.procs.2018.05.069.
[10] I. Arel, D. C. Rose, and T. P. Karnowski, “Deep Machine Learning - A New Frontier in Artificial Intelligence Research [Research Frontier],” IEEE Comput Intell Mag, vol. 5, no. 4, pp. 13–18, Nov. 2010, doi: 10.1109/MCI.2010.938364.
[11] E. A. Smirnov, D. M. Timoshenko, and S. N. Andrianov, “Comparison of Regularization Methods for ImageNet Classification with Deep Convolutional Neural Networks,” AASRI Procedia, vol. 6, pp. 89–94, 2014, doi: 10.1016/j.aasri.2014.05.013.
[12] A. ANHAR and R. A. PUTRA, “Design and Implementation of a Self-Checkout System in Retail Stores using Convolutional Neural Networks (CNN),” ELKOMIKA: Jurnal Teknik Energi Elektrik, Teknik Telekomunikasi, & Teknik Elektronika, vol. 11, no. 2, p. 466, Apr. 2023, doi: 10.26760/elkomika.v11i2.466.
[13] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, 1998, doi: 10.1109/5.726791.
[14] Seth Weidman, Deep Learning from Scratch Building with Python from First Principles. y O’Reilly Media, Inc., 2019.
[15] H. Liu, B. Li, X. Lv, and Y. Huang, “Image Retrieval Using Fused Deep Convolutional Features,” Procedia Comput Sci, vol. 107, pp. 749–754, 2017, doi: 10.1016/j.procs.2017.03.159.
[16] Y. Zhou, H. Wang, F. Xu, and Y.-Q. Jin, “Polarimetric SAR Image Classification Using Deep Convolutional Neural Networks,” IEEE Geoscience and Remote Sensing Letters, vol. 13, no. 12, pp. 1935–1939, Dec. 2016, doi: 10.1109/LGRS.2016.2618840.
[17] Giuseppe Ciaburro and Balaji Venkateswaran, Neural Networks with R: Smart models using CNN, RNN, deep learning, and artificial intelligence principles. 1BDLU1VCMJTIJOH-UE, 2017.
[18] J. Fieres, J. Schemmel, and K. Meier, “Training convolutional networks of threshold neurons suited for low-power hardware implementation,” in The 2006 IEEE International Joint Conference on Neural Network Proceedings, IEEE, 2006, pp. 21–28. doi: 10.1109/IJCNN.2006.246654.
Downloads
Published
2026-01-03
Issue
Section
Articles
