This study aims to implement and compare the performance of ResNet50 and Swin Transformer models in classifying palm oil leaf diseases. The decline in palm oil productivity is often caused by disease infections such as Curvularia (leaf spot) and Leaf Rust, necessitating a fast and precise automated identification system. This experimental computational research used a primary dataset of 600 digital images proportionally divided into training, validation, and testing sets. The preprocessing stage included resolution adjustment (resizing), data augmentation to prevent overfitting, and normalization. Model performance evaluation was conducted quantitatively through Confusion Matrix calculations and validated qualitatively through heatmap visualization using the Gradient-weighted Class Activation Mapping (Grad-CAM) method. The test results proved that the ResNet50 architecture outperformed the Swin Transformer with an accuracy of 98.00%, precision of 98.01%, recall of 98.00%, and F1-score of 98.00%, compared to the Swin Transformer's accuracy of 96.00%. Grad-CAM analysis also confirmed that ResNet50 is sharper in specifically localizing local infection areas. Overall, it is concluded that the ResNet50 model is more optimal, stable, and recommended for the palm oil leaf disease classification system in this dataset domain.