This paper focuses on developing an exercise tracking system capable of recognizing simple exercises, such as push-ups, pull-ups, and sit-ups, with high accuracy, leveraging human pose estimation techniques to enhance prediction performance. Exercise tracking can help users to perform workouts correctly and improve overall physical and mental health. The system utilizes the HSiPu2 dataset for training and evaluation, employing MediaPipe as the human pose estimation input and a Multi-Layer Perceptron (MLP) model for exercise recognition. Initially, a baseline MLP with three layers was implemented, followed by an improved expand-shrink MLP architecture designed to enhance model performance. The results demonstrate that the expand-shrink MLP model has achieved a 16% higher accuracy than the baseline, showcasing its effectiveness in accurately recognizing simple exercises based on pose estimation data. This advancement highlights the potential of the model to support a broader range of exercise types, offering a robust solution for monitoring workouts. The system provides meaningful feedback to users by ensuring accurate exercise recognition and promoting safe and effective physical activity. Future research can explore integrating this system with real-time feedback mechanisms, enabling users to receive immediate corrections during workouts. Expanding the dataset to include diverse exercise routines, including complex and dynamic movements, could enhance the system’s applicability. These developments would pave the way for more comprehensive and practical exercise-tracking solutions, supporting individuals to maintain a healthy lifestyle and improving the accessibility of fitness technologies.