The face is the component that is most easily recognized and is often the center of attention of other people in the human body. There are often difficulties in distinguishing and analyzing large numbers of facial images manually due to the large number of similarities between males and females, which slows down the process of gender identification. This research was made to fix this problem by using the CNN method. The dataset used is 2280 images consisting of train, valid and test. The research process includes data pre-processing, model initialization, model training, hyperparameter validation and adjustment, and model performance evaluation. The test results show an increase in accuracy and a decrease in loss as training iterations increase. In this study, results were obtained with an accuracy rate of 92%, which shows the effectiveness of using a Convolutional Neural Network (CNN) with the ResNet-50 architecture in processing and classifying male and female facial images.