SAST Implementation for Evaluating LLM-Generated Code Quality using Prompt Engineering - Muhammad Luthfi Abdillah; Tikaridha Hardiani (SISTEMASI, 2026)
The use of Large Language Models (LLMs) for generating programming code has become increasingly widespread; however, the quality of the generated output heavily depends on the instructions or prompts provided. This study aims to evaluate the influence of prompt engineering techniques on the quality of non-functional code generated by LLMs. The research employed a quantitative experimental approach involving five Python game development tasks using four prompt variations: zero-shot, few-shot, chain-of-thought, and role-based prompting. A total of 200 code snippets were analyzed using Static Application Security Testing (SAST) with the DeepSource tool to detect issues across seven categories: secrets, bug risk, anti-pattern, security, performance, style, and documentation. The results indicate that few-shot prompting produced the lowest total number of issues overall (1,328 out of 6,932 issues), demonstrating particular advantages in the anti-pattern and performance categories. However, this technique also recorded a higher number of critical issues (3 issues) compared to zero-shot and role-based prompting (1 issue each), indicating a trade-off between the overall volume of issues and the severity of certain issues. Role-based prompting generated the highest number of issues (2,516 issues), particularly in the style and documentation categories. This study recommends few-shot prompting as a foundational approach for AI-assisted software development and highlights the importance of integrating SAST into CI/CD pipelines to ensure code security and quality.