Article Per Year (5 Year)
p-Index From 2021 - 2026
1.116
P-Index
This Author published in this journals
All Journal TELKOMNIKA (Telecommunication Computing Electronics and Control) JUITA : Jurnal Informatika JOIN (Jurnal Online Informatika) International Journal of Artificial Intelligence Research International Journal of New Media Technology GUIDENA: Jurnal Ilmu Pendidikan, Psikologi, Bimbingan dan Konseling Jurnal Linguistik Komputasional Journal of Applied Data Sciences
Dessi Puji Lestari
Sekolah Teknik Elektro Dan Informatika Institut Teknologi Bandung
Computer Science & IT Control & Systems Engineering Decision Sciences, Operations Research & Management Education Electrical & Electronics Engineering Engineering Social Sciences

Published : 11 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 11 Documents
Search

 1   2 
Comparing Pre-Norm and Post-Norm Transformers in Preserving Gender Information for Indonesian–English Translation through Attention-Based Signal Reinforcement Andik Wijanarko; Rinaldi Munir; Masayu Leylia Khodra; Dessi Puji Lestari
Journal of Applied Data Sciences Vol 7, No 2: May 2026
Publisher : Bright Publisher

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47738/jads.v7i2.1257

Abstract

Gender realization in Indonesian–English machine translation remains challenging due to the absence of grammatical gender in Indonesian, which often leads to unstable or ambiguous gender representations in English outputs. While Transformer-based models have demonstrated strong general translation performance, their ability to preserve gender information across encoding layers remains inconsistent and poorly understood, particularly with respect to architectural normalization strategies.This study presents a comparative analysis of Pre-Norm and Post-Norm Transformer architectures in preserving gender information, and examines the role of attention-based signal reinforcement in mitigating representational degradation. The reinforcement mechanism is introduced prior to standard encoder processing to strengthen gender-relevant token interactions without modifying the overall model structure.Four controlled configurations—Post-Norm, Pre-Norm, Post-Norm with attention-based reinforcement, and Pre-Norm with attention-based reinforcement—are trained under identical random seeds on both unbalanced and balanced datasets. Evaluation is performed on gender-ambiguous test sentences without explicit gender annotations to assess generalization. Gender preservation is assessed at the output level using gender-specific accuracy and BLEU score, and at the representation level using cosine similarity between gender cue embeddings and English gendered pronouns.The results show that Post-Norm Transformers fail to maintain stable gender representations, yielding near-random gender accuracy (~50%) and negligible BLEU scores. Pre-Norm architectures improve training stability but achieve limited gender accuracy (around 30%). Incorporating attention-based signal reinforcement substantially enhances gender preservation, with accuracy rising to over 50% and reaching up to 56% under balanced training conditions, accompanied by a consistent increase in cosine similarity values (exceeding 0.35) between gender cues and corresponding pronouns. These findings indicate that normalization strategy and attention-based reinforcement jointly determine the stability of gender representations in Transformer-based machine translation.
Co-Authors Aaz Muhammad Hafidz Azis Alivia Dewi Parahita Andik Wijanarko Aulia Rahmawati Ayu Purwarianti Devin Hoesen Dhiaulhaq, Moch Azhar Dwi Hendratmo Widyantoro Ginanjar, Rizki Rivai Ginar Santika Niwanputri Hendradjaja, Bayu Iping Supriana Suwardi Isjhar Kautsar Masayu Leylia Khodra Ranny Ranny Rinaldi Munir Rizky Faramita Tati Latifah Erawati Rajab Yanti Rubiyanti