The face or face is part of the head, in humans it covers the area from forehead to chin, including hair, forehead, eyebrows, eyes, nose, cheeks, mouth, lips, teeth, skin and chin. The face is used for facial expressions, appearance and identity. Face detection is a stage for personal identification, monitoring systems, criminal law, human-computer interaction. The ever-evolving technological era demands the development of the world of technology to find new, more accurate and fast technological knowledge as well as many problems in the field of technological security and criminal law that require identification of facial classifications in solving a problem. The facial recognition system requires a feature from an image to be recognized then this feature will be matched with another image feature. the process requires a feature extraction method or features and K-Nearest classification. The purpose of this research is to get the best level of accuracy in the facial image classification process in determining a person's gender. The method in this study was carried out in two phases, namely the training phase and the testing phase. In the training phase, the steps taken aim to obtain a model based on a subset of images called training images. The initial step of research is to prepare image data sets to be analyzed. The image dataset used is 20 facial images and then takes 10 images. Based on the results of research conducted on facial images based on color and shape using the K-Nearest Neighbor method, it can be concluded that this method is included in an excellent algorithm for application to facial image classification based on color and shape with an accuracy value of 96%, so that the determination gender based on facial objects using data extracted from color and shape and using the K-Nearest Neighbor classification method according to the actual image data.