The Impact of Translating Resource-Rich Datasets to Low-Resource Languages Through Multi-Lingual Text Processing

Urdu is still considered a low-resource language despite being ranked as world’s 10th most spoken language with nearly 230 million speakers. The scarcity of benchmark datasets in low-resource languages has led researchers to utilize more ingenious techniques to curb the issue. One such option widely adopted is to use language translation services to replicate existing datasets from resource-rich languages such as English to low-resource languages, such as Urdu. For most natural language processing tasks, including polarity assessment, words translated via Google translator from one language to another often change the meaning. It results in a polarity shift causing the system’s performance degradation, particularly for sentiment classification and emotion detection tasks. This study evaluates the effect of translation on the sentiment classification task from a resource-rich language to a low-resource language. It identifies and enlists words causing polarity shift into five distinct categories. It further finds the correlation between the language with similar roots. Our study shows 2-3 percentage points performance degradation in sentiment classification due to polarity shift as a result of translation from resource-rich languages to low-resource languages.

NATURVETENSKAP Data- och informationsvetenskap Systemvetenskap, informationssystem och informatik hsv//swe
NATURAL SCIENCES Computer and Information Sciences Information Systems hsv//eng
Multilingual text processing
sentiment classification
polarity assessment
low resource language
language translation
BiLSTM
Conv1D
English
Urdu
German
Hindi.
Informatik
Information Systems

Imran, Ali ShariqNorwegian University of Science and Technology (NTNU), Norway (author)
Daudpota, Sher MuhammadSukkur IBA University, Pakistan (author)
Kastrati, Zenun,1984-Linnéuniversitetet,Institutionen för informatik (IK)(Swepub:lnu)zekaaa (author)
Soomro, AbdullahSukkur IBA University, Pakistan (author)
Batra, RakhiSukkur IBA University, Pakistan (author)
Wani, Mudasir AhmadNorwegian University of Science and Technology, Norway (author)
Sukkur IBA University, PakistanNorwegian University of Science and Technology (NTNU), Norway (creator_code:org_t)

By the author/editor: Ghafoor, Abdul; Imran, Ali Shari ...; Daudpota, Sher M ...; Kastrati, Zenun, ...; Soomro, Abdullah; Batra, Rakhi; show more...; Wani, Mudasir Ah ...; show less...

About the subject

NATURAL SCIENCES: NATURAL SCIENCES; and Computer and Inf ...; and Information Syst ...

Kungliga biblioteket hanterar dina personuppgifter i enlighet med EU:s dataskyddsförordning (2018), GDPR. Läs mer om hur det funkar här.
Så här hanterar KB dina uppgifter vid användning av denna tjänst.