Herramienta computacional para la traducción de la lengua de señas colombiana a texto

Abstract

According to the most recent study by the National Institute for the Deaf (INSOR), Colombia does not have enough political and institutional tools to guarantee adequate inclusion of people with hearing impairments in both academic and job-related fields. The ongoing development of new technologies plays an important role in social inclusion of people with different impairments; As a result, this paper aims to design a computational tool for the translation of sign language into text, based on the evaluation of different models built on the deep learning concept with the ability to perform classifications of Colombian Sign Language (CSL) video expressions through the implementation of four different pre-trained architectures (DenseNet, MobileNet, Inception and Efficiencies), MobileNet, Inception and EfficienNet), and a customized one with three-dimensional convolutional layers (Conv 3d), in order to define which of the models provides the best results when identifying the different LSC signs representing the 12 different months of the year, and to use it as the core of the translation tool. The performance evaluation showed that the best model was the personalized one, as it managed to recognize most of the signs with a 79% accuracy in the validation set.