Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
6.8
3.9
Journals
Sensors
Special Issues
Towards Sign Language Recognition: Achievements and Challenges
sensors-logo
Submit to Sensors Review for Sensors Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Towards Sign Language Recognition: Achievements and Challenges

A special issue of Sensors (ISSN 1424-8220). This special issue belongs to the section "Physical Sensors".

Deadline for manuscript submissions: closed (31 August 2022) | Viewed by 19880

Special Issue Editors

Dr. Tomasz Kapuscinski

E-Mail Website
Guest Editor
Department of Computer and Control Engineering, Rzeszow University of Technology, Rzeszow, Poland
Interests: image processing and recognition; human–computer interaction; recognition of gestures and signed expressions; hierarchical temporal memory and its applications in vision; point cloud processing
Dr. Marian Wysocki

E-Mail Website
Guest Editor
Department of Computer and Control Engineering, Rzeszow University of Technology, 35-959 Rzeszow, Poland
Interests: Image processing and recognition; Machine Learning; Human–computer interaction; Hand gesture and sign language recognition; Computational intelligence; Optimization and game theory; Control theory and automation
Dr. Kosmas Dimitropoulos

E-Mail Website1 Website2
Guest Editor
Information Technologies Institute, Centre for Research and Technology Hellas, 57001 Thessaloniki, Greece
Interests: artificial intelligence; computer vision, human-computer interaction
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

There are about 70 million deaf people in the world. They face barriers in social contacts because they cannot articulate words, have difficulty understanding written content and expressing thoughts in writing. Their primary means of communication is sign language. Unfortunately,  sign language skills in the hearing community are negligible, and most existing communication systems use written or spoken language. Therefore, works on automatic recognition of sign language are of great social importance and meet the obligations arising from international and national legal acts. Despite many years of research and noticeable progress in this field, researchers still face many difficult challenges. Interdisciplinary approaches focused on real-world applications, greater involvement of deaf people, and the preparation of benchmark datasets with uniform annotation standards are welcome.

We are inviting original research works covering all aspects related to automatic sign language recognition.

Dr. Tomasz Kapuscinski
Dr. Marian Wysocki
Dr. Kosmas Dimitropoulos
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Sensors is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2600 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • sign language recognition
  • finger alphabet recognition
  • fingerspelling recognition
  • hand gesture recognition
  • gesture spotting
  • continuous sign language recognition
  • sign language datasets
  • data annotation
  • data augmentation
  • human–computer interaction

Published Papers (3 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

17 pages, 1410 KiB  
Article
Context-Aware Automatic Sign Language Video Transcription in Psychiatric Interviews
by Erion-Vasilis Pikoulis, Aristeidis Bifis, Maria Trigka, Constantinos Constantinopoulos and Dimitrios Kosmopoulos
Sensors 2022, 22(7), 2656; https://doi.org/10.3390/s22072656 - 30 Mar 2022
Cited by 2 | Viewed by 1605
Abstract
Sign language (SL) translation constitutes an extremely challenging task when undertaken in a general unconstrained setup, especially in the absence of vast training datasets that enable the use of end-to-end solutions employing deep architectures. In such cases, the ability to incorporate prior information [...] Read more.
Sign language (SL) translation constitutes an extremely challenging task when undertaken in a general unconstrained setup, especially in the absence of vast training datasets that enable the use of end-to-end solutions employing deep architectures. In such cases, the ability to incorporate prior information can yield a significant improvement in the translation results by greatly restricting the search space of the potential solutions. In this work, we treat the translation problem in the limited confines of psychiatric interviews involving doctor-patient diagnostic sessions for deaf and hard of hearing patients with mental health problems.To overcome the lack of extensive training data and be able to improve the obtained translation performance, we follow a domain-specific approach combining data-driven feature extraction with the incorporation of prior information drawn from the available domain knowledge. This knowledge enables us to model the context of the interviews by using an appropriately defined hierarchical ontology for the contained dialogue, allowing for the classification of the current state of the interview, based on the doctor’s question. Utilizing this information, video transcription is treated as a sentence retrieval problem. The goal is predicting the patient’s sentence that has been signed in the SL video based on the available pool of possible responses, given the context of the current exchange. Our experimental evaluation using simulated scenarios of psychiatric interviews demonstrate the significant gains of incorporating context awareness in the system’s decisions. Full article
(This article belongs to the Special Issue Towards Sign Language Recognition: Achievements and Challenges)
►▼ Show Figures

Figure 1

25 pages, 555 KiB  
Article
Artificial Intelligence Technologies for Sign Language
by Ilias Papastratis, Christos Chatzikonstantinou, Dimitrios Konstantinidis, Kosmas Dimitropoulos and Petros Daras
Sensors 2021, 21(17), 5843; https://doi.org/10.3390/s21175843 - 30 Aug 2021
Cited by 36 | Viewed by 9809
Abstract
AI technologies can play an important role in breaking down the communication barriers of deaf or hearing-impaired people with other communities, contributing significantly to their social inclusion. Recent advances in both sensing technologies and AI algorithms have paved the way for the development [...] Read more.
AI technologies can play an important role in breaking down the communication barriers of deaf or hearing-impaired people with other communities, contributing significantly to their social inclusion. Recent advances in both sensing technologies and AI algorithms have paved the way for the development of various applications aiming at fulfilling the needs of deaf and hearing-impaired communities. To this end, this survey aims to provide a comprehensive review of state-of-the-art methods in sign language capturing, recognition, translation and representation, pinpointing their advantages and limitations. In addition, the survey presents a number of applications, while it discusses the main challenges in the field of sign language technologies. Future research direction are also proposed in order to assist prospective researchers towards further advancing the field. Full article
(This article belongs to the Special Issue Towards Sign Language Recognition: Achievements and Challenges)
►▼ Show Figures

Figure 1

20 pages, 2915 KiB  
Article
Continuous Sign Language Recognition through a Context-Aware Generative Adversarial Network
by Ilias Papastratis, Kosmas Dimitropoulos and Petros Daras
Sensors 2021, 21(7), 2437; https://doi.org/10.3390/s21072437 - 01 Apr 2021
Cited by 31 | Viewed by 5611
Abstract
Continuous sign language recognition is a weakly supervised task dealing with the identification of continuous sign gestures from video sequences, without any prior knowledge about the temporal boundaries between consecutive signs. Most of the existing methods focus mainly on the extraction of spatio-temporal [...] Read more.
Continuous sign language recognition is a weakly supervised task dealing with the identification of continuous sign gestures from video sequences, without any prior knowledge about the temporal boundaries between consecutive signs. Most of the existing methods focus mainly on the extraction of spatio-temporal visual features without exploiting text or contextual information to further improve the recognition accuracy. Moreover, the ability of deep generative models to effectively model data distribution has not been investigated yet in the field of sign language recognition. To this end, a novel approach for context-aware continuous sign language recognition using a generative adversarial network architecture, named as Sign Language Recognition Generative Adversarial Network (SLRGAN), is introduced. The proposed network architecture consists of a generator that recognizes sign language glosses by extracting spatial and temporal features from video sequences, as well as a discriminator that evaluates the quality of the generator’s predictions by modeling text information at the sentence and gloss levels. The paper also investigates the importance of contextual information on sign language conversations for both Deaf-to-Deaf and Deaf-to-hearing communication. Contextual information, in the form of hidden states extracted from the previous sentence, is fed into the bidirectional long short-term memory module of the generator to improve the recognition accuracy of the network. At the final stage, sign language translation is performed by a transformer network, which converts sign language glosses to natural language text. Our proposed method achieved word error rates of 23.4%, 2.1% and 2.26% on the RWTH-Phoenix-Weather-2014 and the Chinese Sign Language (CSL) and Greek Sign Language (GSL) Signer Independent (SI) datasets, respectively. Full article
(This article belongs to the Special Issue Towards Sign Language Recognition: Achievements and Challenges)
►▼ Show Figures

Figure 1

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-3
Back to TopTop