Segmentation is one of the critical stages in digital image processing and computer vision. However, conventional clustering-based segmentation methods, such as K-means and Fuzzy C-means (FCM), are still unable to accurately segment images whose pixels form hyperellipsoid clusters in the feature space. In addition, previous clustering methods based on Mahalanobis distance measurement require a long computational time and still have the potential to fall into local optima. Therefore, in this paper, we propose a new method for segmenting images whose pixels form hyperellipsoid clusters in the feature space, utilizing hyperellipsoid clusters merging through hierarchical analysis of hyperellipsoid clusters. The proposed method comprises eight main steps: histogram extraction, peak and valley identification, elimination of low peaks and valleys, peak combination for centroid initialization, initialization of cluster pixel members, elimination of ineffective clusters, hyperellipsoid cluster merging, and finalization of cluster members. This paper presents a novel approach to segmenting color images by employing an initial centroid discovery process and cluster analysis that considers cluster covariance for cluster merging. Based on the tests conducted using various image characteristics, the proposed method can provide 97.42% accuracy, 98.02% precision, 97.15% recall, 2.58 misclassification error, 97.54 F1-score, 95.29% intersection over union, 97.52% dice coefficient, and 15.37 seconds of computation time. The test results are superior to those of conventional methods, such as K-means and FCM. Based on these results, it can be concluded that the proposed method can effectively segment images with high accuracy. The proposed method can serve as an alternative approach to image segmentation.