Gloss-free Argentinian Sign Language Translation with pose-based deep learning models

Abstract

The main challenge of automatic Sign Language Translation (SLT) is obtaining data to train models. For Argentinian Sign Language (LSA), the only dataset available for SLT is LSA-T, which contains extracts of a news channel in LSA and the corresponding Spanish subtitles provided by the authors. LSA-T contains a wide variety of signers, scenarios, and lightnings that could bias a model trained on it. We propose a model for Argentinian gloss-free SLT, since LSA-T does not contain gloss representations of the signs. The model is also pose-based to improve performance on low resource devices. Different versions of the model are also tested in two other well-known datasets to compare the results: GSL and RWTH Phoenix Weather 2014T. Our model stablished the new SoTA over LSA-T, which proved to be the most challenging due to the variety of topics covered that result in a vast vocabulary with many words appearing few times.