This study proposes to detect the fertility of paddy soil based on texture, the power of Hydrogen (pH), and the amount of production. Fertile paddy soil provides essential nutrients and supports optimal plant growth. Therefore, monitoring and analyzing soil fertility is crucial in agricultural land management, which significantly increases rice yields. Paddy soil is categorized into three parts: very fertile soil, fertile soil, and reasonably fertile soil. This research proposes a new approach to detecting soil fertility levels based on factors that influence soil fertility using the Convolutional Neural Network (CNN) algorithm. There are 558 paddy soil datasets of 178 very fertile datasets, 135 fertile datasets, and 245 quite fertile datasets. In this research, we conducted trials using the CNN, Resnet, Enet, and VGG19 models. According to the test results, the CNN model using the Adam optimizer and a learning rate of 0.001 achieves the highest training accuracy of 0.9687 and validation accuracy of 0.8333. This suggests that this model can accurately identify the fertility of paddy soil, making it easier to calculate the fertility of paddy soil through its use. Future research can expand this study by integrating additional soil parameters, such as nitrogen, phosphorus, potassium levels, and organic matter content, to improve classification accuracy further. Additionally, employing multimodal data sources, such as remote sensing and hyperspectral imaging, could enhance the model's robustness in various environmental conditions. Further optimization of deep learning architectures and Artificial Intelligence (AI) techniques can also provide better interpretability and usability for agricultural stakeholders.