This study aims to present a systematic literature review (SLR) of the performance, security, and Quality of Service (QoS) of the Message Queuing Telemetry Transport (MQTT) protocol in Internet of Things (IoT) environments based on articles published between 2020 and 2026. The SLR follows the PRISMA 2020 guidelines and the Kitchenham and Charters guidelines for software engineering studies. A total of 30 core articles, comprising 7 reputable international journals and 23 nationally accredited SINTA journals, were purposively selected based on thematic relevance and methodological quality. The search was conducted using IEEE Xplore, ScienceDirect, SpringerLink, MDPI, Garuda, and SINTA with the keywords “MQTT,” “IoT,” “performance,” “security,” and “QoS.” The selection process followed the PRISMA flow, consisting of identification (n=487), screening (n=86), eligibility assessment (n=43), and included studies (n=30). The data were extracted and thematically synthesized to address four research questions (RQ1–RQ4), with Scopus-indexed international studies included as contextual comparators. The findings indicate that MQTT outperforms HTTP in terms of bandwidth efficiency and low latency, particularly on resource-constrained devices, with reported latency ranging from 45 to 210 ms depending on the QoS level and testing scenario. The dominant performance parameters include latency, throughput, and packet loss. Security approaches encompass TLS/RSA encryption, domestic cryptography, and Random Forest-based attack detection, achieving an accuracy of 97.2%, although TLS encryption may reduce throughput by up to 30%. QoS 1 represents a balanced option for non-critical applications, whereas QoS 2 is appropriate for vital data. The main research gap is the absence of integrated performance, security, and QoS testing within a single experimental framework. Therefore, future research should focus on lightweight security, context-aware adaptive QoS, and real-world-scale validation through integrative studies that examine the interaction among these three aspects.