Muhammad Luthfi Abdillah
Universitas 'Aisyiyah Yogyakarta

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

SAST Implementation for Evaluating LLM-Generated Code Quality using Prompt Engineering Muhammad Luthfi Abdillah; Tikaridha Hardiani
Sistemasi: Jurnal Sistem Informasi Vol 15, No 5 (2026): Sistemasi: Jurnal Sistem Informasi
Publisher : Program Studi Sistem Informasi Fakultas Teknik dan Ilmu Komputer

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.32520/stmsi.v15i5.6395

Abstract

The use of Large Language Models (LLMs) for generating programming code has become increasingly widespread; however, the quality of the generated output heavily depends on the instructions or prompts provided. This study aims to evaluate the influence of prompt engineering techniques on the quality of non-functional code generated by LLMs. The research employed a quantitative experimental approach involving five Python game development tasks using four prompt variations: zero-shot, few-shot, chain-of-thought, and role-based prompting. A total of 200 code snippets were analyzed using Static Application Security Testing (SAST) with the DeepSource tool to detect issues across seven categories: secrets, bug risk, anti-pattern, security, performance, style, and documentation. The results indicate that few-shot prompting produced the lowest total number of issues overall (1,328 out of 6,932 issues), demonstrating particular advantages in the anti-pattern and performance categories. However, this technique also recorded a higher number of critical issues (3 issues) compared to zero-shot and role-based prompting (1 issue each), indicating a trade-off between the overall volume of issues and the severity of certain issues. Role-based prompting generated the highest number of issues (2,516 issues), particularly in the style and documentation categories. This study recommends few-shot prompting as a foundational approach for AI-assisted software development and highlights the importance of integrating SAST into CI/CD pipelines to ensure code security and quality.