Optimizing transportation surveillance requires accurate vehicle license plate color detection and classification; however, existing systems face significant challenges in achieving real-time accuracy and robustness, particularly in crowded traffic scenarios with varying lighting and plate conditions. In Indonesia, vehicle license plates are color-coded based on their usage, including white and black for private vehicles, yellow for public vehicles, red for government vehicles, and green for free-trade areas. Each plate color plays a crucial role in transportation management, enabling proper vehicle identification and regulation. Existing surveillance systems struggle with real-time detection accuracy, especially in distinguishing plate colors in crowded traffic. Traditional methods may not efficiently classify plate colors due to limitations in feature extraction and processing. To address this, this study implements the YOLOv7 model to improve vehicle license plate color detection (black, white, yellow, and red) while distinguishing non-plate vehicles in diverse scenarios. The model's effectiveness is evaluated using precision, recall, and F1-score to ensure robustness for surveillance applications. Results show an average precision of 95.27%, recall of 94.60%, and F1-score of 94.93%, demonstrating strong detection capabilities. Optimizing the Non-Plate category further improves system accuracy, efficiency, and scalability, enhancing transportation monitoring reliability.