Application of Hybrid CNN-LSTM Architecture with Optuna Optimization for Weather Image Captioning - Sulaeman Salasa; Shintami Chusnul Hidayati; Muhamad Hilmil Muchtar Aditya Pradana (Sean Institute, 2026)
Automating the description of weather phenomena through visual imagery is a crucial step in supporting efficient meteorological monitoring systems. This study aims to compare the performance of two Deep Learning architectures, ResNet101-LSTM and VGG16-LSTM, in generating automatic image captions for various weather conditions. The research methodology involves extracting visual features using Residual Learning and VGG-Net, which are subsequently processed by Long Short-Term Memory (LSTM) units for text generation. Hyperparameter optimization was conducted using the Optuna framework to ensure both models operate at their peak configurations. The results indicate that ResNet101-LSTM provides superior linguistic accuracy, achieving a BLEU-1 score of 0.7553, a BLEU-4 score of 0.4593, and a METEOR score of 0.7264. Qualitatively, this model is capable of identifying environmental details with higher precision compared to VGG16-LSTM. However, loss curve analysis reveals that VGG16-LSTM demonstrates better convergence stability (good fit), whereas ResNet101-LSTM shows signs of slight overfitting. This study concludes that while ResNet101-LSTM is superior in accuracy according to standard NLP evaluation metrics, additional regularization techniques are required to maintain its performance stability on validation data.