The rapid urbanization witnessed in cities like Bandung, Indonesia, has emerged as a pressing issue, precipitating severe traffic congestion that poses challenges to economic growth and diminishes overall quality of life. This study endeavors to confront these multifaceted challenges through the development of a sophisticated real-time traffic surveillance and control system. The proposed system utilizes the current CCTV infrastructure in the city and incorporates advanced technologies like YOLOv8 for accurate vehicle detection and ByteTrack for dynamic real-time vehicle tracking. This system utilizes a comprehensive strategy, including multi-object tracking techniques to improve the precision of congestion detection. The system was thoroughly assessed in several places in Bandung, and it showed remarkable performance metrics. Specifically, YOLOv8 achieved an impressive 80% accuracy rate in vehicle detection, showcasing its efficacy in discerning vehicles within complex urban environments. Simultaneously, ByteTrack exhibited an average error rate of 17% in vehicle counting, further Strengthening the system's capabilities in accurately quantifying vehicular traffic. Furthermore, the combination of YOLOv8 and ByteTrack in a multi-object tracking paradigm yielded an 80% accuracy rate in congestion detection, emphasizing the system's robustness in real-time traffic management scenarios. These findings underscore the immense potential of the integrated YOLOv8 and ByteTrack system in traffic management strategies and alleviating congestion in smart cities like Bandung. This research has produced precise outcomes in identifying and quantifying the traffic congestion in various scenarios.