Content uploaded by Abdelmoty M. Ahmed
Author content
All content in this area was uploaded by Abdelmoty M. Ahmed on Nov 05, 2017
Content may be subject to copyright.
Towards the design of Automatic Translation System from Arabic
Sign Language to Arabic Text
Abdelmoty M.Ahmed 1
1Department of Systems and Computer Engineering
Faculty of Engineering, Al Azhar University
Cairo, Egypt
Reda Abo Alez2
2Department of Systems and Computer Engineering
Faculty of Engineering, Al Azhar University
Cairo, Egypt
Gamal Tharwat3
3Department of Systems and Computer Engineering
Faculty of Engineering, Al Azhar University
Cairo, Egypt
Muhammad Taha4
4Department of Mathematics
Faculty of Science, Al Azhar University
Cairo, Egypt
Wade Ghribi5
5Department Computer Engineering
College of Computer Science, King Khalid University
Abha, Saudi Arabia
Ahmed Said Badawy6
6Department Computer Engineering
College of Computer Science, King Khalid University
Abha, Saudi Arabia
Suresh Babu Changalasetty7
7 Department Computer Engineering
College of Computer Science, King Khalid University
Abha, Saudi Arabia
J.Subash Chandra Bose8
8 Department Computer Engineering
College of Computer Science, King Khalid University
Abha, Saudi Arabia
Abstract— Automatic translation from or to sign language by
recognizing patterns to produce a written text by using an
annotated sign language corpus or any other tools of
grammatical structure, syntactic rules, synthesis specifically for
Arabic Sign Language. Arabic Sign Language still lack
exhaustive scientific studies of their grammatical structure,
morphology and syntactic rules, as well as the rules that govern
the construction of sentences in this language. Such studies are
necessary for the development and evolution of any sign
language. In addition, the lack of tools that help researchers in
studying Arabic Sign Language is another obstacle. In this paper
we will show the importance of a system that would help to
represent and translate Sign Language to a written text. In this
context, we present a new proposal for the way towards the
establishment of a system for the automated translation of the
Arabic sign language into written text based on creation an
annotated Arabic Sign Language corpus. This new proposed
scheme can be useful for learning, teaching or studying for
people who suffer from hearing difficulty. We also propose some
rules which help developing Sign Language automatic translation
systems.
Keywords—Arabic Sign Language, Annotation System, Sign
language automatic translation systems.
I. INTRODUCTION
Sign Language (SL) is the prominent means of
communication among the deaf and the hearing-disabled
people [1]. It is a native language for them, and about 18
million of this community is only in Arabic world. Despite
availability of very important number of deaf people, Arabic
Sign Language (ArSL) suffers from the lack of rules and
grammatical structure that is required for the construction of an
ArSL sentence.
which makes it different from spoken language but serve the
same function [2]
There are some initiatives that support the idea of
unification of SL in some Arabic countries. These initiatives
comes because the SL from the same spoken language which is
Arabic. The difference between sign language is due the dialect
and not to the mother tongue. Many papers and dictionaries
had been published toward a unified ArSL [3].
From another direction, there are some other trends that
refuse the idea of unification because the sign language is
related to the environment and most signs are derived from it.
So, for this reason, Arabian deaf person can share and
communication between them easily without creation new
signs which is needed if we talk about a unified SL [4][5].
After discussions with many Arabic deaf persons, and
experts translator for sign language, we found there has been a
lack of linguistic studies in ArSL and local Arab SLs. We also
found that the main problem is the lack of an annotation or
transcription system. The annotation writing system is a
solution to write specific properties of ArSL in text or symbols
like hand shape, direction, movement, location and non-manual
gestures like facial expressions.
In this context, we propose the first initiative for a new
automatic translation system based on signs corpus database in
this research paper. This system aims to design database of
signs that interpret any input sign to Arabic text language using
transfer automatic translation for SL.
II. MOTIVATION AND ISSUES
Nowadays, it can be time in order to be capable to create
more ArSL regardless the particular problem associated with
unification. Within additional, we would like to create a new
Automatic translation system in order to hard of hearing and
non-deaf from Arabic countries in order to write, share, discuss
understand this SL.
Since the eighties of the last century, the researchers began
the work in the analysis and design of automatic translation
systems for SL process to develop routines and adapt the
software for communication intra-deaf and between hearing
and deaf people. Many Automatic translation systems are
proposed for American Sign Language (ASL), British sign
Language (BSL), etc but few of them are focused on Arabic
Sign Language (ArSL) [6][7].
The lack of standard SL system limits the development of
ArSL in general. For example, until now, there isn't research
about grammar or syntax of ArSL like American Sign
Language (ASL) or British Sign Language (BSL)
In addition, software application or web application aren't
rich or advanced, due to the absence of standard or convention
for writing and processing ArSL data for programming
language. From a linguistic point of view, the number of
publications (articles, dictionaries, signs corpus and books) for
ArSL is very limited. For example, until now, we cannot find
an exact rule for building a sentence in Egyptian Sign
Language (ESL) or any other ArSL. As a result, there
rules for signing or interpreting. Each interpret strive to
construct statement when we compare to ASL or BSL, we
found that this SLs are very developed and advanced, thanks to
the automatic translation system based on a Arab corpus gloss
of ArSL. In this context, Machine Translation Systems
(MTS's) proposal is based on a corpus database for the
meanings of which are similar to the Arabic spoken will be a
solution for learning and understanding of ArSL, because the
learning and understanding of ArSL is not easy at this time
because we cannot write discourse. So, teachers encounter
difficulties when teaching Arabic courses for deaf people. In
the next section, we will talk about guiding rules of ArSL.
III. ARABIC SIGN LANGUAGE SPECIFICATIONS
Some believe that the Arabic sign language has no rules or
regulations for freedom from letters and links which is about
symbols for words help the deaf communicate with their
community. While others believe that there are rules for sign
language but have not been searched or attempt be highlighted
or modularization.
The guiding rules described here reflect the annotation
guidelines that we used for the transcription of Arabic texts.
We will present the Automatic Translation of Arabic Sign to
Arabic Text (ATASAT) System proposed for syntax and
morphology rules. This work is based on the Arabic and
Quatarian Sign Language Rules [4] and we will refer to all
guidelines outlined in the agreements Liddell applied to ArSL
.
A. ArSL phonology and Morphology
SL is composed of basic elements of gesture and location,
- optical phoneme, the main articulators in
SLs are the fingers, hands and arms (phonemes). There are
two categories of phoneme in SL Manual Features (MFs)and
Non-Manual Features (NMFs). The optical phoneme is the
basic unit of sign, as it contains four elements: hand shape,
orientation and position of the hand, and the movement of
direction of the hand (MFs), as well as the head eyes and face
(NMFs)[8].
SL employs the space around the signer for
communication, and the signer can use a combination of MFs
and NMFs. These are called `multichannel signs', MFs are
basic components of any sign, whereas NMFs play an
important role in composing signs in conjunction with MFs[9].
The figure below shows how similar forms of symbols of
the alphabet ArSL with the forms of the characters in the
Arabic language.
Fig. 1. Alphabet forms in ArSL (spelling fingers)[10]
The letters of the Arabic language are the main voices that
make up the word at the spoken language, as well as
considered fingers spelling as sounds that make up the
indicative symbols and their meanings show the support of the
other hand or move it or put it on the part of the body.
Fingers spelling is one of the ArSL supplements might be a
supporting element in the use of language and to support its
deployment. It is used at the number of things including the
following:-
Used by the deaf, teachers and translators at the Spelling
names.
Used as a substitute for the words that do not have the
word or symbol indicative known among the deaf in the
language.
Used in educational activities as a picture frame of mind
to teach the deaf child to read and write by Arabic
Language.
Some forms of vocalizations formation or symbols
indicative result combined the form of one or more
characters from the alphabets fingers.
Fig. 2. Representation of the symbols or word indicative[4]
We described the optical phoneme, which is the basic unit
of sign, as it contains four elements: hand shape, orientation
and position of the hand, and the movement of direction of the
hand, as well as the NMFs.
TABLE I. Matching between the ArSL alphabets and alphabets in
Arabic Language
We also provide a set of fields and values specifically
designed for the annotation of data for ArSL. Each gloss is
represented by an Arabic word. This annotation, although
simplified, does not reflect the morphological richness of
ArSL signs only, but also the very important grammatical
function of facial expressions. The non-manual information
provided by the face can occur simultaneously with the
manual components. It is represented by a line above signs.
For our translation, it annotates all the important features of
grammar in ArSL. One Arabic word identifies one ArSL sign,
for example the sign of a Telephone ( ) is " " .
Arabic words separated by dashes are also a single sign where
more than one English word is required to translate. The -"
" (Illegal) sign is a transcription of the Arabic phrase
without dashsymbol In addition, reversing the hand shape to
a different position gives an opposite meaning such as the
signs " " (back) and "" "front", Also, a new sign can
be delivered from compounding two different signs, such as
the sign "devotion" which contains two signs, "fear" and
"God".
Fig. 3. Transcription of the sign " " (Telephone in english)
Fig. 4. Transcription of the signs " " (back) and "" "front"
Fig. 5. Transcription of the new sign Compound word " devotion "
() "" "fear" and "God"
The ArSL sign can belong to the following groups
Anto-Signs: two signs with the same MFs except that the
movements are different
Compounds: a combination of two signs to deliver a new
meaning.
Hamo-Signs: two similar signs used to express different
meanings, which are known from the context.
Syno-Signs: two signs with the same meaning. This type
exists in case of shifting from one sign to another, one
dominates and the other disappears.
B. Grammar and Structure
Grammar is the rules and principles followed by the users
of sign language by communicating them to explain the
components and sections Language. Compared with the
Arabic language, the parts of speech in ArSL consist of noun,
verb and Preposition
1. Noun
A noun denotes either tangible or intangible identities.
Nouns are independent of other words in indicating their
Face Matching
Arabic
Language
/ArSL
Face Matching
Arabic
Language
/ArSL
Approximate at the
Format with a note
raising the thumb to an
indication of a point
above letter""
ض
match with
alphabetical letter
ا
Approximate at the
Format with the letter
"" and raising the index
finger to the top
ط
indication to the
presence of one
point below the
shaped ""
ب
Convergence of form
with a note raising
thumb over the middle
for an indication of a
point above letter ""
ظ
indication to the
presence of two
points above the
shaped ""
ت
match with alphabetical
letter
ع
indication to the
presence of three
points above the
shaped ""
ث
Match with the form of
the alphabetical letter
and raise the thumb
above the palm of the
hand to indicate the
presence of point above
letter""
غ
Similar form with
a thumb
somewhere dot
character ""
ج
Match with the form of
alphabetical letter and
put the index finger
above the thumb to
indicate a point above
letter""
ف
Similar form with
a thumb along the
other fingers for
lack a point to a
character ""
ح
Match form with the
alphabetical letter with a
index and middle fingers
over the thumb to
indicate a point above
letter""
ق
Similar form with
a thumbs higher
other fingers,
referring to the
presence of a point
above letter ""
خ
Match with the form of
alphabetical letter and
put the thumb at the
middle of the palm of
the hand to indicate the
presence of Hamza "
for a character""
ك
match with
alphabetical letter
د
match with alphabetical
letter
ل
Similar form with
the middle put up
the index finger to
referring the
presence of a point
above letter
ذ
No match with
alphabetical letter
م
match with
alphabetical letter
ر
match with alphabetical
letter
ن
Similar form with
the middle put up
the index finger to
referring the
presence of a point
above letter
ز
match with alphabetical
letter
ه
match with
alphabetical letter
س
match with alphabetical
letter
و
No similar form,
but disperse
fingers indicating
the dots above
letter""
ش
match with alphabetical
letter
ى
Approximate at
the Format with
the letter ""
with a note placed
thumb
ص
meaning. What distinguishes nouns from verbs is that nouns
refer to entities or things. Nouns are further classified into:-
a. proper nouns and common nouns ()
b. Pronouns ()
In Arabic language pronouns are classified into three sets:
Addressee pronoun ( ), Speaker pronoun ( ) and
Absent pronoun ( ). For each set pronouns are classified
according to person (first, second, third), number (singular,
dual and plural) and gender (masculine and feminine).
c. Relative noun ()
Sometimes we add some styles and special expressions to
noun types such as
d. Condition noun (style)-()
e. Question noun or mark ()
f. Demonstratives ()
Deaf usually used several methods to express or represent
the different types of noun in ArSL.
2. Tenses (Verbs)
In the Arabic language there are three main types of the
tense: the perfect, imperfect and imperative, this category is
found in all sign languages in the world, and the Arab deaf
persons use the verb at the infinitive, often indicative of
verbal sentence starting with the timeline of verb.
a. The perfect (past) tense
The past tense is an indication of an event occurring
before the time of speaker; past tense in sign language is
classified according to timeline of verb, and represented by a
gesture hand behind the body.
b. The imperfect (present) tense
The present tense is an indication of an event occurring in
the present or the future, also the present tense in sign
language is classified according to timeline of verb, and
represented by a gesture hand parallel or directly in front of
the body.
Fig. 6. Transcription of the signs "past"and "present"tenses
c. The imperative tense
The imperative tense is an Indication demand is occurring
after the time of speaker, also the imperative tense in sign
language is inferred through gestures, bodily expression, also
the form of the verb is not different from the past tense or the
present tense, but different style which appears on a person's
body and pack into the implementation of the verb.
Fig. 7. Transcription of the sign " imperative " tense
General could say, the tense in ArSL is simply and
practically used. Past, present, and future tenses are indicated
at the beginnings of conversation chunks and only shifted
when there is a need to indicate a different tense.
d. conjugation tense
Deaf person does not have the ability to conjugation, they
strive to differentiate between the verb and the noun similar to
him in an indicative symbol such as "car", "lead" or "plane",
"fly" or "prayer", "pray"
For example, in sign languages in western countries, there
are some strategies and rules used to differentiate between the
noun and the verb, a move indicative symbol or repeat twice
to signify the verb.
TABLE II. The methods representation nouns in ArSL
Researcher suggests append or add some letters spell the
fingers of a reaction to the education of the deaf possibility of
disposal of some verbs in ArSL to distinguish the word or
symbol indicative in terms of reading and writing.
Expression
methods
ArSL
Arabic
Language
Type of
nouns
Fingerspelling
The indicative title
Where it is
encoded this
person, as
distinguishing or
special mark
دمحا
proper nouns
Descriptive Signs
Or
Non-Descriptive
Signs
God
common nouns
(vegetables, fruits,
animals,..etc)
"Mosque"
Descriptive Signs
Addressee pronoun
()
or
Speaker pronoun
()
or
absent pronoun
()
"Masculine"
Pronouns
(It is frequently used in educational activities in the classroom)
" feminine "
Replaced by
Speaker pronoun
()
+
I did wudoo
Descriptive Signs
Relative noun
"Masculine"
Relative noun
(It is frequently used in
educational activities in the
classroom)
Relative noun
+
Symbol indicative
of feminized
" feminine "
Sees, "Manal Aldraosha" The possibility of deaf education
conjugation by adding spelled fingers to the symbol or word
indicative of the verb
For example ,The possibility of adding spell fingers to the
letter """ya" at the beginning of the word or symbol
indicative verb indicative of the present tense of the masculine
time[11].
TABLE III. differentiate between noun and verb in ArSL
3. Preposition
ArSL word correspondence (signs) is limited to two basic
classes,
a. Simple forms of nouns/adjectives.
b. Simple forms of verbs without tense.
In addition to , ArSL does not have, unlike classical Arabic
(CA) the lot associated with the particles (prepositions and
several adverbs or even intensifiers).However, the
relationships and principles represented simply by
prepositions plus intensifiers, just for example, can be
portrayed by additional means. This particular could be
achieved by the particular position plus direction associated
with one sign in relation to one more regarding prepositions.
Most prepositions does not appear in the ArSL,
prepositions are used in ArSL indirectly, they are used
unobtrusively and within the context of the speech, as in the
example.
TABLE IV. Analysis of sentence in ArSL
In ArSL when analyzing the sentence indicative note does
not appear preposition, in the first instance and the
disappearance of the definite article, in the second example.
Generally, prepositions is not from within the linguistic
structure of the sentence in ArSL.
This is known as the deep structure of the language where
deaf depends on the inductive approach in the analysis and
interpretation of linguistic signs and understand the general
meaning of the sentence.
C. Pluralization in sign language
In Arabic name is divided in terms of significance to the
number to a single, dual, and plural, whether masculine or
feminine.
ArSL differs from Arabic language and other spoken
languages, This makes it difficult to compare sign languages
with their spoken counterparts. Arabic in this regard is not an
exception. As a matter of fact, many concepts used to describe
spoken languages are inadequate for the description of sign
languages. There is no singular, dual, or plural agreement in
ArSL signed sentences, as shown in table V. In Arabic
language, many nouns are countable, but it does not in ArSL.
For exampl "are expressed in sign
language by two words; " "and then sign of the number
" ,"in order to denote the dual.
TABLE V. Table showing Pluralization in sign language
In plural of sign language sometimes depends on the
principle of repetition (Reduplication) of the movement of the
symbol or word indicative as describes in the following figure
Where the focus is on repeating indicative symbol, three times
with the change of its position in the spatial space in front of
the body to signify people group.
Fig. 8. Example for plural Transcription of the sign "Responsible"
D. Derivation in sign language
One of the important features in any language is the ability
derivation. Arabic language is distinct from many other
languages for this feature, similarly, ArSL has the same
feature and so all the way back to the symbol or word
indicative, which is considered the root as in the spoken
language.
Derivation in sign language is different from what is in
Arabic language, it depends on the basic shape of the symbol
and can be added in the form of movement or the beginning or
the end to give another meaning
Fig. 9. Derivation example in ArSL
Note in the previous example, that terminology indicative
performed the same way for the symbol indicative of the word
"family" by changing the first character.
We suggest some rules that may be useful in the derivation
process for new words or symbols indicative in ArSL.
verb
noun
symbol
move indicative symbol
or repeat twice to signify
the verb " " (fly)
move indicative symbol or
repeat one time to signify
the noun " " (Plane)
move indicative symbol
or repeat twice to signify
the verb " " (pray)
move indicative symbol or
repeat one time to signify
the noun " " ( prayer )
Arabic Language
Arabic Sign Language
Examples
ArSL Syntax
Arabic
Language
Syntax
Word or symbol indicative
to Singular " "
( Mosque ) + number one
or vise versa
Singular " "
( Mosque)
Word or symbol indicative
to dual " "
( Mosques ) + number
two or vise versa
Dual " "
( Mosques)
Word or symbol indicative
to plural " "
( Mosque ) + number
three or vise versa
Plural " "
( Mosques)
Change the shape of the dominating hand, implementing
the word indicative to express a new word.
Change the shape of the dominating hand, implementing
the word indicative to express synonymous linguist.
Reverse the radix motor, that implementing of the word
indicative to express the anti-linguist.
In the dominating hand switch the shape of indicative
character by other indicative character shape in the word
indicative to express synonymous linguist
E. Analysis of the sentence indicative in ArSL
If we analyzed sentences and texts to writings of the deaf
and their messages through the linguistic culture of the Arabic
language experts, we will note a number of problems, the
researcher attempted to Illustrates when design of automatic
translation system for an ArSLto Arabic Text (ATASAT),
these problems are summarized as follows:
Lack of differentiate between the feminine and
masculine.
Exaggeration in the use of pronouns.
Lack of, use of binding tools and conjunctions between
sentences.
Do not use prepositions in sentences.
Reduction of sentences.
Overuse indicative symbols in writing.
These problems are due to, lack of knowledge of Arabic
language experts with ArSL, deaf when they write, they think
sign language and writing in the order of sentence in ArSL,
compared with the Arabic language, the ArSL is reduction
language concerned with most words and focus of the
sentences in order to reach the significance and meaning.
The researcher believes that the ArSL with deep structure
of the language, through the selection of its users for main
words of the sentence on the grounds utilitarian field
(communication and cognition) and not the formal field.
F. ArSL morphology
We refer in this section Arabic sentence structure and
ArSL sentences structure. The Arabic sentence structure is
divided into two kinds of sentences. Nominal sentences begin
with a noun or a pronoun, while verbal sentences begin with a
verb, overall, Nominal sentences it has a subject () and a
predicate (), The subject of the nominal sentence is a noun
or a pronoun, while the predicate can be a noun, adjective,
preposition and noun, or verb, while the verbal sentences
begin with a verb, and they have at least a verb () and a
subject (). And also object (). However it is
preferable to begin the ArSL sentence with subject,
The structure of sentence in ArSL contains a subject, a
verb and an object (SVO), Some believe that the sentence
structure base in ArSL as a fixed base, but the truth is through
research in the world of the deaf on the Arab scale, and use of
the language appeared many of the structures of the sentences
in ArSL.
Sentence structure in ArSL is simultaneous with a parallel
temporal and spatial configuration while language is linear;
one word followed by another [12].
Usually, Sentence structure in ArSL is begin with subject,
such as: "" ,is translated into ""
or is translated into "", In the sentence structure,
for the first translation, the translator, perform word indicative
serially front of the body, While in the sentence structure, for
the second translation the translator, perform sign of the word
"" in short to return to the sign word "" without
adherence sequence signs.
IV. CONCLUSIONS
In this paper, we put a new idea with an innovative vision
and propose a new initiative towards system design of
Automatic Translation system for an Arabic Sign Language
(ArSL)to Arabic Text (ATASAT). The research work is
aimed to serve deaf Arabic people for easy communication.
ATASAT is based on a knowledge base to solve a number of
Arabic language problems such as synonyms, derivational,
pluralization, masculine and feminization. ATASAT system
also shows how to explore finger spelling in solving some of
the ArSL challenges. We also explain some linguistic rules,
which serve the ArSL which are considered the nucleus for the
establishment of automatic translation systems. Depending on
the culture and knowledge of linguistic, serve deaf people to
clarify meaning and content of the non-deaf people. We
provide a strong motivation to deepen research into the issues,
problems and challenges of this sign language compared with
the Arabic language. The ATASAT system allows Arabian
deaf people to learn Arabic Language and to share ideas and
participate to discussions using an annotation convention
familiar to Arabic texts.
REFERENCES
[1] Abdelmoty M. Ahmed, Reda Abo Alez , Muhammad Taha , Gamal
Tharwat . " Automatic Translation of Arabic Sign To Arabic Text
(ATASAT) System ", Volume 6, Number 5, Computer Science &
Information Technology.AIRCC Publishing Corporation , Third
International Conference on Signal Processing ,CSIP 2016 .
[2] Kabil Jaballah, Mohamed Jemni, "Toward Automatic Sign Language
Recognition from Web3D Based Scenes", ICCHP, Lecture Notes in
Computer Science Springer Berlin / Heidelberg, pp.205-212 vol.6190,
Vienna Austria, July 2010.
[3] Hendriks, H. B. "Sign language varieties in Jordan and the Middle East.
Jordanian Sign Language: Aspects of grammar from a cross-linguistic
perspective",Netherlands Graduate School of Linguistics, 2008.
[4] Samreen, S., Albanali, M.: "Arabic and Quatarian Sign Language
Rules", Quatar ,2009.
[5] Kinda Al-Fityani, "Arab Sign Languages: A Lexical Comparison"
Department of Communication, University of California, San Diego,
CRL Technical Report, vol. 19 no.1, 2007.
[6] C. Dubuisso and L. Lacerte, "La glose dans la transcription et l'analyse
des langues signée," Spécificités de la recherche linguistique sur les
langues signées, Montréal: Les cahiers scientifiques de l'Acfas, 1996
[7] Samreen,S "Nature of Deaf and Dumb in Arabic Regions" Quatar ,
200.
[8] Samreen,S " Professional guide for translation and interpreter in sign
language " , Jordan,Aman ,2013.
[9] Almohimeed, A., Wald, M., Damper, R. "An Arabic Sign Language
corpus for instructional language in school", 4th Workshop on the
Representation and Processing of Sign Languages: Corpora and Sign
Language Technologies, pp. 8182. European Language Resources
Association 17-22 May 2010.
[10] NabilmenasyM., "symbols indicativ Fonts",2005.
[11] Manal Nabih Al-Darawsheh, "Methods of Deaf Teaching", Ministry of
Social Affairs, Sultanate of Oman, 2001.
[12] Samreen, S., Albanali, M "Rules of the Arabic language Sign unified ",
2010.