Development of a neural network for automatic recognition of sign language and its translation into text
Students Name: Huk Maksym Rostyslavovych
Qualification Level: magister
Speciality: Informatively-Instrumentation Technologies in Roboticotekhnic
Institute: Institute of Computer Technologies, Automation and Metrology
Mode of Study: full
Academic Year: 2024-2025 н.р.
Language of Defence: ukrainian
Abstract: This master’s thesis is devoted to the development of neural networks for automatic recognition and translation of sign language into text. The importance of this work lies in the creation of innovative tools to overcome communication barriers between people with hearing and speech impairments and other members of society. This paper reviews modern methods and technologies for sign language recognition and analyzes existing translation systems. Particular attention was paid to the use of deep neural networks, computer vision algorithms, and video data processing. In this paper, we review the architecture of models that are effectively used to analyze sequential behavior, such as recurrent neural networks (RNNs), convolutional neural networks (CNNs), and transformers. Based on the theoretical knowledge gained, a system was developed and implemented that includes a neural network for gesture recognition and a module for converting recognized gestures into text. To train the model, a unique sign language dataset was created, including videos using different categories of gestures. The data was pre-processed: segmentation of movements, selection of key points of movement, and normalization of parameters. Experimental studies have shown high accuracy of gesture recognition and real-time speed of the model. A comparison with existing solutions confirmed the competitiveness of the proposed approach. This process involves creating software that can be integrated into web applications. A comparison with existing solutions confirmed the competitiveness of the proposed approach. This process involves the creation of software that can be integrated into web applications, which ensures ease of use in various fields, including education, medicine, services, and social integration. The proposed methods and developed systems facilitate communication for people with hearing impairments, expand opportunities for integration into social life, and are the basis for further research in the field of sign language.