A Comparative Performance Evaluation of Copilot, Gemini, and DeepSeek in Understanding Knot Semantic Logic - Khaerati, Khaerati; Ja'faruddin, Ja'faruddin; Fadilah, Nur; Aslindawati, Nur; Fadiyah, Wulan Nuf; Ihsan, Muh. (Jurusan Matematika FMIPA UNM, 2025)
This study examines the ability of three artificial intelligence (AI) models Copilot 3.7 Sonnet, Gemini 1.5, and DeepSeek-R1 to interpret Knot Semantic Logic (KSL), a topological framework emphasizing semantic symmetry in textual structures. A qualitative descriptive design with comparative analysis was employed. Structured prompts tested the models across three stages: basic concept mastery, analysis of symmetrical sentences, and generalization to new inputs. Performance was assessed using five indicators Conceptual Accuracy, Structural Accuracy, Generalization, Consistency, and Narrative Clarity combined into a KSL-AI Index. The results show distinct performance profiles. Copilot produced accessible explanations but lacked structural precision. Gemini demonstrated stability in recognizing semantic symmetry, supported by large-scale multimodal and multilingual training, although its technical style limited accessibility. DeepSeek showed strength in detecting simple patterns and basic logic but was less consistent and struggled with complex generalization tasks. The study validates KSL as an innovative evaluation tool, extending AI assessment beyond narrative fluency to structural semantic reasoning. It concludes that Copilot is best suited for pedagogical use, Gemini for consistent analytical tasks, and DeepSeek for exploratory analysis. Future work should integrate quantitative measures, multimodal testing, and broader model comparisons.