ILKOM Jurnal Ilmiah
Vol 15, No 1 (2023)

Abstractive Text Summarization using Pre-Trained Language Model "Text-to-Text Transfer Transformer (T5)"

Qurrota A’yuna Itsnaini (Universitas Amikom Yogyakarta)
Mardhiya Hayaty (Universitas Amikom Yogyakarta)
Andriyan Dwi Putra (Universitas Amikom Yogyakarta)
Nidal A.M Jabari (Palestine technical university Kadoorie)



Article Info

Publish Date
07 Apr 2023

Abstract

Automatic Text Summarization (ATS) is one of the utilizations of technological sophistication in terms of text processing assisting humans in producing a summary or key points of a document in large quantities. We use Indonesian language as objects because there are few resources in NLP research using Indonesian language. This paper utilized PLTMs (Pre-Trained Language Models) from the transformer architecture, namely T5 (Text-to-Text Transfer Transformer) which has been completed previously with a larger dataset. Evaluation in this study was measured through comparison of the ROUGE (Recall-Oriented Understudy for Gisting Evaluation) calculation results between the reference summary and the model summary. The experiments with the pre-trained t5-base model with fine tuning parameters of 220M for the Indonesian news dataset yielded relatively high ROUGE values, namely ROUGE-1 = 0.68, ROUGE-2 = 0.61, and ROUGE-L = 0.65. The evaluation value worked well, but the resulting model has not achieved satisfactory results because in terms of abstraction, the model did not work optimally. We also found several errors in the reference summary in the dataset used.

Copyrights © 2023






Journal Info

Abbrev

ILKOM

Publisher

Subject

Computer Science & IT

Description

ILKOM Jurnal Ilmiah is an Indonesian scientific journal published by the Department of Information Technology, Faculty of Computer Science, Universitas Muslim Indonesia. ILKOM Jurnal Ilmiah covers all aspects of the latest outstanding research and developments in the field of Computer science, ...