Claim Missing Document
Check
Articles

Found 3 Documents
Search
Journal : bulletin of computer science research

Implementasi Langchain dan Large Language Models Dalam Automatic Question Generation Untuk Computer Assisted Test Novri Rahman; Nazruddin Safaat Harahap; Muhammad Affandes; Pizaini Pizaini
Bulletin of Computer Science Research Vol. 5 No. 4 (2025): June 2025
Publisher : Forum Kerjasama Pendidikan Tinggi (FKPT)

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47065/bulletincsr.v5i4.558

Abstract

The advancement of Artificial Intelligence (AI), particularly Large Language Models (LLM), presents new opportunities in transforming educational assessment systems. This study aims to implement the LangChain framework integrated with LLM for an Automatic Question Generation (AQG) system within a Computer Assisted Test (CAT) platform, using eleventh-grade Biology subject matter as a case study. The methodology includes data collection from PDF-based instructional materials, text embedding using Facebook AI Similarity Search (FAISS) as the knowledge base, and automatic question generation through the GPT-4o model. The system is developed using a microservices architecture comprising frontend and backend services built with the Next.js, FastAPI, and Express.js frameworks. System evaluation was conducted using the User Acceptance Test (UAT) and the DeepEval framework. The evaluation results show a teacher satisfaction rate of 92.7% and a positive response from students at 67.5%. Meanwhile, the DeepEval assessment reported average scores of 3,69% for hallucination, 97,44% for contextual precision, 83,30% for contextual relevancy, 70,63% for answer relevancy, and 92,47% for prompt alignment. These findings indicate that the integration of LangChain and LLM is effective in generating contextually accurate and relevant questions, although improvements are still needed in answer relevancy. This study is expected to provide an efficient solution for digital-based educational assessment and contribute to future developments in educational AI.
Optimasi Klasifikasi Hate Speech dan Offensive Language melalui Frozen RoBERTa Feature Extraction dan Random Forest Marsha Cahyani Dwisyakilla; Surya Agustian; Novriyanto Novriyanto; Muhammad Affandes
Bulletin of Computer Science Research Vol. 6 No. 4 (2026): June 2026
Publisher : Forum Kerjasama Pendidikan Tinggi (FKPT)

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47065/bulletincsr.v6i4.1157

Abstract

Hate speech and offensive content detection on social media remains a significant challenge in Natural Language Processing (NLP) due to the characteristics of Twitter data, which are typically short, informal, and contain various elements such as mentions, URLs, hashtags, and emotional expressions that complicate the classification process. End-to-end Transformer fine-tuning approaches generally require substantial computational resources; therefore, this study explores a more computationally efficient approach by utilizing RoBERTa as a frozen feature extractor combined with Random Forest as the classifier. This approach enables the exploitation of contextual representations generated by Transformer models without requiring full model retraining.The study employs the HASOC 2021 English Track dataset, which consists of two classification tasks: Task A for binary classification (HOF and NOT) and Task B for multi-class classification (HATE, OFFN, PRFN, and NONE). The classification pipeline is optimized through the incorporation of handcrafted features, oversampling, Random Forest hyperparameter tuning, and threshold tuning in specific scenarios. Model performance is evaluated using accuracy, precision, recall, and F1-macro, with F1-macro serving as the primary metric due to class imbalance. The best-performing model achieved an F1-macro score of 0.80 on Task A and 0.64 on Task B. These results indicate that the combination of frozen RoBERTa representations and Random Forest provides strong performance for binary hate speech and offensive content classification. However, the performance on Task B highlights the difficulty of distinguishing linguistically similar categories, such as HATE, OFFN, and PRFN, suggesting that fine-grained multi-class classification remains a challenging task. Overall, the findings indicate that RoBERTa-based frozen feature extraction constitutes a computationally efficient alternative for hate speech detection on English Twitter data, although further improvements are required to enhance performance in multi-class classification settings.
Klasifikasi Hate Speech dan Offensive Language Menggunakan BERT dan Support Vector Machine Muhammad Tirta Syakban; Surya Agustian; Muhammad Fikry; Muhammad Affandes
Bulletin of Computer Science Research Vol. 6 No. 3 (2026): April 2026
Publisher : Forum Kerjasama Pendidikan Tinggi (FKPT)

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47065/bulletincsr.v6i3.1061

Abstract

Hate speech and offensive language have become increasingly complex problems on social media, requiring classification approaches that can effectively capture linguistic context. While transformer-based models with end-to-end fine-tuning have become the dominant approach, the use of transformers as fixed feature extractors combined with classical machine learning algorithms remains relatively underexplored, particularly in benchmark settings such as HASOC 2021. This study aims to investigate the effectiveness of a feature-based transformer approach by combining embeddings from BERT and RoBERTa with Support Vector Machine (SVM) classifiers using multiple kernel configurations, including Linear, RBF, Polynomial, and LinearSVC. Experiments were conducted on Sub-task A and Sub-task B by comparing traditional feature-based methods (TF-IDF) with transformer-based embeddings. The experimental results show that RoBERTa embeddings consistently outperform other feature extraction methods. On the test dataset, the combination of RoBERTa and SVM achieves competitive performance compared to other systems in HASOC 2021. In Sub-task B, the optimal model achieves a Macro F1-score of 0.61, outperforming several BERT-based and classical baseline systems.These findings demonstrate that using transformer embeddings as fixed feature representations combined with optimized SVM classifiers can serve as an effective alternative to fine-tuning approaches, particularly in achieving more stable performance under class imbalance conditions. This study contributes by highlighting the potential of feature-based transformer methods as a flexible and competitive strategy for hate speech and offensive language detection.