The name of the mango is Mangnifera IndicaL. It originated in India and spread to Indonesia. There are various types of mango variations with different shapes and colors according to the type. To distinguish each mango is seen by its shape and color. However, if in the harvest process mango farmers have to choose manually it takes a long time and potentially mistaken in determining the type. So it needs technology that can make it easier to differentiate the type of mango based on its shape. The study aims to create models with the best accuracy on the process of classifying 5 types of mango based on its shape. The data used in the research this time there are 5 types of mango that will be classified, namely Mangga Apel, Arumanis mango, Mangga Gedong Gincu, Golek mango and Mangga Manalagi. Used 375 images of mango as data sets. The data set before entering the previous training process is undergoing a pre-processing phase that includes the augmentation and resize process. The number of images increased to 2250. The data set is divided into three parts: 70% training data, 20% validation data, and 10% test data. Next is the process of segmentation, the segmentation used in this research is otsu segmentation. The classification process uses the Convolutional Neural Network (CNN) architecture with 3 layers of convolution 16,32 and 64, also using the Adam optimizer. 4 experimental scenarios were performed to find the best accuracy value by distinguishing between learning rate and batch size. From the confusion matrix test results, the best accuracy values were obtained from the input hyperparameter size100x100, epoch 100, learning rate 0,001 and batch size 15 with accurate values of 99.56%, precision 100%, recall 100%, and f1-score 100%.