Classifying date fruit varieties is a challenging task due to their high visual similarity in terms of texture and color. This study aims to address this issue by developing an automated classification model that combines handcrafted Gray Level Co-occurrence Matrix (GLCM) texture features and average RGB color channels with Convolutional Neural Network (CNN) classifiers. The dataset comprises 1,658 images from nine varieties of date fruits, divided into 70% training and 30% testing subsets. The proposed workflow includes image preprocessing (resizing, normalization, grayscale conversion), extraction of GLCM features (contrast, energy, homogeneity, correlation), computation of average RGB channels, feature fusion, and CNN training using VGG16 and VGG19 architectures with Adam and Adadelta optimizers. The model performance is evaluated using accuracy, precision, recall, F1-score, and confusion matrix. Experimental results demonstrate that VGG19 with the Adam optimizer achieved the highest validation accuracy of 91%, slightly outperforming VGG16 (90%) but remaining below the 96% accuracy reported in prior studies using MobileNetV2. The integration of handcrafted features enhanced sensitivity to subtle color and texture variations, although it introduced potential feature redundancy. In conclusion, the hybrid GLCM–RGB–CNN with VGG19 and Adam achieved 91% accuracy, proving the benefit of combining handcrafted and deep features while highlighting opportunities for further enhancement through data augmentation and architectural optimization.