Evaluation of Machine Translation Systems: A Literature Review on ChatGPT and Google Translate - Ohod Faisal Ahmed; Ida Kusuma Dewi; Mohammad Yunus Anis (Institut Agama Islam Negeri Palopo, 2025)
Abstract: This is a literature review discussing 15 selected papers about ChatGPT and Google Translate study results based on keyword analysis and publication year. We applied descriptive data analysis technique to analyze the data. We selected Studies on translation performance of natural language processing tools were chosen due to their increasing prominence and diverse applications, ranging from literary to technical translations. The data for this literature review was retrieved from Scopus and Google Scholar. The search was limited to the last five years to ensure the inclusion of recent advancements, particularly those reflecting improvements in ChatGPT’s GPT-4 engine and updates in Google Translate’s neural machine translation capabilities. The results showed that ChatGPT excels in fluency and contextual understanding, particularly in literary and poetic translations, outperforming Google Translate in maintaining stylistic elements and complex language structures. Both systems demonstrated strengths in specialized translations, with ChatGPT showing notable proficiency in medical literature and technical texts. However, challenges remained in low-resource languages and specialized domains, requiring further training and development. Despite technological advancements, human translators are essential for achieving culturally nuanced translations. This study has some implications for future implementing for enhancement contextual understanding, improving accuracy for low-resource languages, and addressing specific error patterns through ongoing research and collaborative efforts between human translators and machine translation tools. These recommendations aim to optimize the performance of ChatGPT and Google Translate, thereby ensuring more accurate and contextually appropriate translations across various fields.