<![CDATA[Hate speech and offensive language have become increasingly complex problems on social media, requiring classification approaches that can effectively capture linguistic context. While transformer-based models with end-to-end fine-tuning have become the dominant approach, the use of transformers as fixed feature extractors combined with classical machine learning algorithms remains relatively underexplored, particularly in benchmark settings such as HASOC 2021. This study aims to investigate the effectiveness of a feature-based transformer approach by combining embeddings from BERT and RoBERTa with Support Vector Machine (SVM) classifiers using multiple kernel configurations, including Linear, RBF, Polynomial, and LinearSVC. Experiments were conducted on Sub-task A and Sub-task B by comparing traditional feature-based methods (TF-IDF) with transformer-based embeddings. The experimental results show that RoBERTa embeddings consistently outperform other feature extraction methods. On the test dataset, the combination of RoBERTa and SVM achieves competitive performance compared to other systems in HASOC 2021. In Sub-task B, the optimal model achieves a Macro F1-score of 0.61, outperforming several BERT-based and classical baseline systems.These findings demonstrate that using transformer embeddings as fixed feature representations combined with optimized SVM classifiers can serve as an effective alternative to fine-tuning approaches, particularly in achieving more stable performance under class imbalance conditions. This study contributes by highlighting the potential of feature-based transformer methods as a flexible and competitive strategy for hate speech and offensive language detection.]]>
