The ability to classify track conditions has become a critical issue in the railway industry, as delayed detection or unaddressed adverse track conditions can profoundly impact railway safety. Current track maintenance primarily relies on manual inspections and specialized monitoring vehicles, which are constrained by their inspection frequency. Deploying models that correlate vehicle dynamic responses with track conditions in in-service trains could significantly enhance fault detection. However, existing studies utilizing machine learning approaches are notably limited in capturing complex time-series information from vehicle dynamic responses, especially when the data are derived from real measurements rather than simulations. To address these challenges, we propose the application of GoogleNet and Gramian Angular Summation Field (GASF) transformation for classifying track conditions using vehicle dynamic responses. For comparison, we will demonstrate the limitations of traditional machine learning approaches, specifically Logistic Regression and XGBoost, where only the standard deviation and peak value are extracted as features. Subsequently, we propose our approach using the GoogleNet architecture, combined with GASF to transform the time-series data into image representations. Our proposed model achieves high accuracy, in classifying vertical and lateral track conditions, significantly outperforming the machine learning model. The results of this study demonstrate that our proposed method can learn complex nonlinear features, and make accurate classifications. Additionally, the study highlights the inability of the machine learning model, to classify track conditions accurately, and provides evidence that standard deviation and peak value are insufficient as features for complex systems like vehicle dynamic responses