International Journal of Advances in Applied Sciences
Vol 15, No 2: June 2026

Semantic clustering of scientific abstracts with transformer embeddings and traditional text representations

Musthofa Galih Pradana (UPN Veteran Jakarta)
Pujo Hari Saputro (Sam Ratulangi University)
Ardhyansyah Mualo (Politeknik Negeri Fakfak)
Arbiati Faizah (Institut Teknologi dan Bisnis PGRI Dewantara)
Wahyuni Fithratul Zalmi (Sam Ratulangi University)



Article Info

Publish Date
01 Jun 2026

Abstract

The large and diverse quantity of scientific documents in the world, and specifically in Indonesia, makes the process of processing scientific document data an interesting study. One that represents the entire scientific document is through abstracts. The approach that can be done for the process of processing and grouping documents is to apply clustering. In this case, text-based clustering is currently heavily influenced by the feature representation of the text data used. Some popular representations of features are term frequency-inverse document frequency (TF-IDF) and count vectorizer, but they still have significant weaknesses in the context of understanding the meaning of natural language. To cover the drawbacks, it can use transformers or embedding types. In this study, several test scenarios will be carried out to obtain information and an overview of how to compare the effectiveness of the traditional TF-IDF model and the bidirectional encoder representations from transformers (BERT) and sentence bidirectional encoder representations from transformers (SBERT) embedding models in Indonesian-language scientific abstract clustering with several clustering models, such as k-means and agglomerative. The results of the study showed that the most effective clustering obtained was by using an embedding model of a combination of BERT and k-means, which was the most consistent with the most optimal number of clusters being 2 clusters.

Copyrights © 2026






Journal Info

Abbrev

IJAAS

Publisher

Subject

Earth & Planetary Sciences Environmental Science Materials Science & Nanotechnology Mathematics Physics

Description

International Journal of Advances in Applied Sciences (IJAAS) is a peer-reviewed and open access journal dedicated to publish significant research findings in the field of applied and theoretical sciences. The journal is designed to serve researchers, developers, professionals, graduate students and ...