Analisis Performa Steganografi Hybrid DCT–DWT dengan Enkripsi AES-GCM dan Kompresi Zlib

Mokhamad Yusron Rafi (Universitas Pamulang, Tangerang Selatan)
Agung Budi Susanto (Universitas Pamulang, Tangerang Selatan)
Makhsun Makhsun (Universitas Pamulang, Tangerang Selatan)

TIN: TERAPAN INFORMATIKA NUSANTARA
Vol 7 No 2 (2026): July 2026

The increasing exchange of digital data requires information security methods that not only protect message content but also conceal its existence. This study analyzes the performance of digital image steganography based on Hybrid Discrete Cosine Transform (DCT) and Discrete Wavelet Transform (DWT) combined with Advanced Encryption Standard-Galois/Counter Mode (AES-GCM) encryption and zlib compression in RGB color images. The research employed a quantitative experimental method by comparing three embedding scenarios: DCT, DWT, and Hybrid DCT-DWT. The test data consisted of RGB images with resolutions of 256×256, 512×512, and 1024×1024 pixels, while the payload consisted of DOCX and PDF documents with different sizes. The payload was compressed using zlib, encrypted using AES-GCM, and embedded into image transform coefficients. The evaluation was conducted using Peak Signal-to-Noise Ratio (PSNR), Mean Square Error (MSE), extraction success, payload capacity, histogram analysis, and robustness against image manipulation. The results show that all methods successfully embedded and extracted payloads under normal scenarios. For small payloads, the PSNR values were in the high-quality category, such as 48.66 dB for 256×256 images and 60.73 dB for 1024×1024 images. Increasing payload size reduced PSNR and increased MSE. Hybrid DCT-DWT produced visual quality relatively comparable to DCT and DWT, while offering greater flexibility by combining two transform domains.