Pepper trees grow vines up to a height of 4 meters supported by other trees or using a pole called an arbor. The current problem is that during the pepper harvesting process, farmers generally pick by hand. In 2020, D3 students made a pepper picker design tool using a remote to control its movement, this tool cannot move automatically, from this problem a convolutional neural network (CNN) based pepper fruit detection system was created. This system uses the Convolutional Neural Network (CNN) method. CNN implementation uses Tensorflow tools with Python programming language. The number of datasets is 100 images of pepper and 100 images of non pepper. Based on the test results, the prediction precision level is 95%, accuracy is 89%, and recall is 90%, so it can be concluded that the detection system using a webcam can predict pepper fruit well.