Article Per Year (5 Year)
p-Index From 2021 - 2026
0.23
P-Index
This Author published in this journals
All Journal The Indonesian Journal of Computer Science
Waysi Naaman, Diyar
Unknown Affiliation
Computer Science & IT Electrical & Electronics Engineering Engineering

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

 1 
Architectural Evolution of Transformer Models in NLP: A Comparative Survey of Recent Developments Waysi Naaman, Diyar; Berivan Tahir Ahmed; Ibrahim Mahmood Ibrahim
The Indonesian Journal of Computer Science Vol. 14 No. 5 (2025): The Indonesian Journal of Computer Science
Publisher : AI Society & STMIK Indonesia

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.33022/ijcs.v14i5.4984

Abstract

This literature review examines the impact and advancements of XLM-RoBERTa in the field of multilingual natural language processing. As language technologies increasingly transcend linguistic boundaries, XLM-RoBERTa has emerged as a pivotal cross-lingual model that extends the capabilities of its predecessors. Through comprehensive pre-training on multilingual corpora spanning 100 languages, this model demonstrates remarkable zero-shot cross-lingual transfer capabilities while maintaining competitive performance on monolingual benchmarks. This review synthesizes research findings on XLM-RoBERTa's architecture, pre-training methodology, and performance across diverse NLP tasks including named entity recognition, question answering, and text classification. By examining comparative analyses with other multilingual models, we identify key strengths, limitations, and potential directions for future research. The findings underscore XLM-RoBERTa's significance in advancing language-agnostic representations and bridging the performance gap between high-resource and low-resource languages, with substantial implications for global accessibility of language technologies.
Co-Authors Berivan Tahir Ahmed Ibrahim Mahmood Ibrahim