Coffee productivity is an important factor in supporting the sustainability of smallholder plantations, particularly in the Sukabumi region. However, conventional estimation methods that rely on manual observation tend to be subjective and inefficient. Although previous studies have applied computer vision and machine learning for yield prediction, most approaches depend on large-scale datasets and expensive sensing technologies, and often do not integrate multiple visual plant features comprehensively. This indicates a research gap in the use of simple RGB-based imaging for productivity estimation. This study aims to analyze visual features of coffee plants and develop a coffe productivity estimation model using the Random Forest algorithm. The dataset consists of 10 coffee plant images collected directly from field observations, with extracted features including fruit count, fruit maturity percentage, canopy area, leaf color, and leaf texture. Model evaluation is performed using Leave-One-Out Cross Validation (LOOCV) method to optimize data utilization on a limited dataset. The results show that the model achieves a Mean Absolute Error (MAE) of 0.06, a Root Mean Square Error (RMSE) of 0.07, and a coefficient of determination (R²) of 0.91. These results indicate good predictive performance within the available dataset. Feature importance analysis reveals that fruit count and fruit maturity percentage are the most influential factors in determining coffee productivity. This study contributes to the development of a low-cost image-based estimation approach that is practical and potentially applicable for smart agriculture in smallholder coffee plantations, although the findings remain preliminary due to the limited dataset size.