Fundus images can be used to identify symptoms of various eye diseases. However, a limitation of using fundus images for classification is the generalization of information from the entire image, which can reduce recognition accuracy. To address this, this study proposes a combination of RGB channels from fundus images and segmented images for the identification process. Segmentation is performed using U-Net, which produces a segmentation of the blood vessels from the retinal image. The combined image is then used as input for the identification process with different ResNet models, including ResNet18, ResNet34, ResNet50, ResNet101, and ResNet152. Three model tuning scenarios are explored to obtain an optimal hybrid segmentation and classification model: learning using ResNet without U-Net (nounet), learning using a ResNet model with a frozen U-Net model (frozenunet), and learning using both the U-Net and ResNet models (hotunet). Testing is carried out to recognize normal, cataract, and glaucoma classes. The results show that the highest accuracy of 0.82 is achieved with the hybrid U-Net and ResNet152 model using frozenunet learning. This indicates that the addition of segmented images can improve identification results, with the best performance for glaucoma having a precision of 0.90, recall of 0.86, and F1-score of 0.88.