Evaluating Machine Translation of Cultural Terms: Readability Comparison Between Google and Yandex - Mentari, Diana (Buletin Al-Turas, 2025)
Purpose This study aimed to analyze the readability of Google Translate (GT) and Yandex Translate (YT) translation results on dialogue texts containing cultural terms from the book Antologi Cerita Anak Indonesia (ACAI). This study evaluated the effectiveness of the Neural Machine Translation (NMT) approach in GT and the Hybrid Machine Translation (HMT) approach in YT in conveying text meanings clearly and comprehensibly to readers.Method This research employed a cloze test involving 28 participants aged 18-24 years, along with a questionnaire to assess user preferences regarding GT and YT translation results. Text readability was analyzed using the Flesch-Kincaid Grade Level and Gunning Fog Index to measure the linguistic complexity of the translations.Results/Findings The study results show that GT's readability reaches 81.1%, while YT's readability is 74.5%, both categorized as the independent level according to Rankin & Culhane's (1969) criteria. Additionally, 80% of the 20 questionnaire respondents stated that GT's translations were clearer than those of YT. Analysis using the Flesch-Kincaid Grade Level and Gunning Fog Index shows that the readability level of GT and YT translations is classified as advanced suitable for readers with a minimum education level equivalent to a bachelor's degree.Conclusion This study showed that GT has a higher readability level than YT, which might be because of its use of NMT, producing more natural sentence structures. Meanwhile, YT, which also relied on SMT, translates based on statistical patterns, making its translations more rigid. Although both systems could produce comprehensible translations, they still struggled with accurately translating cultural terms without additional context. Therefore, human involvement remained essential to improving accuracy and contextual appropriateness in machine translation.