Assessing Conversational AI Usability: A System Usability Scale Evaluation of Dola AI

Arimbi Kurniasari (Gunadarma University)
Navizha AS Velisya (Gunadarma University)

Jurnal Ilmiah Teknik (JUIT)
Vol. 5 No. 2 (2026): Mei: Jurnal Ilmiah Teknik

The rapid development of conversational artificial intelligence has increased the need to evaluate not only system functionality but also the quality of user interaction. Dola AI is an AI-based conversational assistant designed to support users through natural-language interaction; however, widespread use does not necessarily indicate satisfactory usability. Objective: This study aims to evaluate the perceived usability of Dola AI using the System Usability Scale (SUS) and to determine its overall level of user acceptance. Method: A quantitative descriptive approach was employed involving 50 Dola AI users. Data were collected through an online questionnaire containing the ten standard SUS items measured using a five-point Likert scale. The instrument was evaluated through validity and reliability testing, followed by standardized SUS score calculation and interpretation using adjective-rating and acceptability classifications. Findings: All questionnaire items met the validity criterion, while the instrument obtained a Cronbach’s alpha of 0.6827, indicating adequate internal consistency based on the criterion adopted in the study. The overall SUS score was 69.6, placing Dola AI in the “OK” adjective category and the “Marginal” acceptability range. These results indicate that Dola AI provides an adequate level of perceived usability but has not yet reached a clearly high level of user acceptance.  Implications: The findings provide practical evidence for improving Dola AI, particularly in terms of interaction consistency, perceived complexity, learnability, and user confidence. The study also demonstrates the applicability of SUS as a concise standardized instrument for evaluating conversational AI interfaces. Originality: This study provides a standardized empirical usability assessment of Dola AI, for which quantitative usability evidence remains limited. The findings establish an initial usability benchmark that can support future comparative studies and further evaluation of conversational AI applications using broader samples and complementary usability measures.