IKRA-ITH Informatika : Jurnal Komputer dan Informatika
Vol. 10 No. 2 (2026): IKRAITH-INFORMATIKA Vol 10 No 2 Juli 2026

EVALUASI KUANTITATIF RELIABILITAS DAN KECEPATAN INFERENSI PADA TIGA ARSITEKTUR LARGE LANGUAGE MODEL (ChatGPT, DeepSeek, Gemini) MENGGUNAKAN METRIK NLP BERBASIS PYTHON

Abubakar Basri (universitas Djuanda)



Article Info

Publish Date
12 Sep 2026

Abstract

The massive adoption of Large Language Models (LLMs) across sectors demands an in-depthevaluation of the systems' intrinsic capabilities, particularly in responding to ambiguousinstructions. This study presents a structured comparative analysis of three leading LLMarchitectures ChatGPT, DeepSeek, and Gemini focusing on factual reliability and computationalefficiency (inference latency) as a direct consequence of prompt-ambiguity manipulation. Themethodology adopts a Black-Box Testing approach combined with quantitative analysis,systematically applied to 100 prompts classified into eight complexity categories. Factual reliabilitywas measured using a TF-IDF-based Cosine Similarity metric against ground truth, while latencywas recorded through end-to-end stopwatch measurement and processed with Python scripts onGoogle Colab. Results show that Gemini achieved the highest average accuracy (66.91%), followedby DeepSeek (64.89%) and ChatGPT (62.10%), while DeepSeek led in average inference speed(2.84 seconds) owing to its Mixture-of-Experts architecture. Pearson correlation tests for all threemodels produced negative coefficients (ChatGPT −0.200; Gemini −0.194; DeepSeek −0.030),disproving the traditional speed–accuracy trade-off hypothesis. These findings are expected to serveas an empirical compass for users selecting models with precision and as evaluative literature fordevelopers building more robust and efficient AI systems.

Copyrights © 2026