Evaluating the fruit’s maturity level is crucial to acquiring high-quality fruit. The skin color of some fruits may be used as one of the numerous indicators to determine whether they have achieved their peak degree of ripeness. Similar to other fruits, the skin color of an Ambarella fruit indicates its maturity. However, determining the ripeness of the Ambarella fruit was assessed manually, which is time-consuming, inefficient, taxing, requires a large number of employees, and has the potential to result in discrepancies. This study aims to classify the ripeness of the Ambarella fruit using the deep learning approach, specifically using the Convolutional Neural Network (CNN). The new family of EfficientNetV2 is trained to classify the Ambarella fruit ripeness. The pre-trained models are utilized in this work, and the training was done via transfer learning through fine-tuning. EfficientNetV2B0 achieves the highest accuracy of 100% despite having a smaller size than the other EfficientNetV2 models used in this work.