Deep learning-based shrimp image processing has become a rapidly growing research field in recent years. This technology aims to increase efficiency and accuracy in various applications related to the fishing and aquaculture industry, such as monitoring shrimp health, disease detection, species classification, and assessing the quality and quantity of harvested crops. Based on observations to date, fish sellers and buyers in the market have difficulty distinguishing vaname shrimp cultivated in tarpaulin ponds and earthen ponds. This research aims to apply deep learning techniques to determine the classification of Litopenaeus vannamei shrimp cultivation results in earthen ponds and tarpaulin ponds. To facilitate this research, the author uses a classification method by applying two Convolutional Neural Network (CNN) architectures, namely Visual Geometry Group-16 (VGG-16) and Residual Network-50 (ResNet-50). The dataset used in this research is 2,080 images per class of vannamei shrimp from two types of shrimp ponds. The results of this research are learning rates of 0.001 and 0.0001 on the Stochastic Gradient Descent (SGD) and Adaptive Moment Estimation (ADAM) optimizer to evaluate their effectiveness in model training. The VGG-16 and ResNet-50 models were trained with a learning rate parameter of 0.0001, taking advantage of the flexibility and reasonable control provided by the SGD optimizer. Lower learning rate values were chosen to prevent overfitting and increase training stability. Model evaluation showed promising results, with both architectures achieving 100% accuracy in classifying vannamei shrimp from ground and tarpaulin ponds. The conclusion of this research is to highlight the superiority of using SGD with a learning rate of 0.0001 versus 0.001 on both architectures, then the significant impact of optimizer selection and learning rate on the effectiveness of model training in image classification tasks