Indonesian–Arabic Academic Translation Quality: A Comparative Content Analysis of ChatGPT and Google Translate - Alfitri, Alfitri; Zakaria, Gamal Abdul Nasir; Misran, Misran; Nurdianto, Talqis (UIN Sunan Gunung Djati Bandung, 2025)
This study compares the quality of Indonesian–Arabic academic translations produced by Google Translate and ChatGPT (GPT-4), a topic rarely examined despite widespread MT use in Arabic Language Education programs. Using a qualitative descriptive design with content analysis, the data comprised ten purposively selected undergraduate thesis titles from Indonesian university repositories, including UIN Sunan Kalijaga, UIN Suska Riau, UIN Imam Bonjol Padang, and Universitas Muhammadiyah Makassar. Titles were selected for their academic rigor and need for conceptual precision and formal Arabic register. Translations were analyzed at the phrase level using a Hybrid MQM–Nababan–Baker rubric encompassing seven dimensions: accuracy, acceptability, readability, lexical equivalence, grammatical equivalence, cohesion and coherence, and academic fluency, each rated on a three-point scale. Validity was ensured through alignment with Arabic translation theory, equivalence frameworks, and MQM standards, while iterative consistency checks supported reliability. Results reveal clear differences. ChatGPT achieved an average score of 20.0 out of 21 points (93%), which is classified as Very Good and indicates strong suitability for academic publication. Google Translate scored 13.3 out of 21 points (63%), classified as Good, but requiring post-editing. ChatGPT excelled in contextual meaning, syntactic restructuring, accurate idhafah, case governance, consistent terminology, and scholarly style, whereas Google Translate showed literal transfer. This study enriches AI-assisted translation discourse by grounding evaluation in Arabic translation theory and pedagogy, emphasizing generative AI’s pedagogical potential as a complementary tool while reaffirming the indispensable role of human expertise in maintaining linguistic accuracy, rigor, and academic standards.