Article Per Year (5 Year)
p-Index From 2021 - 2026
5.19
P-Index
This Author published in this journals
All Journal International Journal of Electrical and Computer Engineering IAES International Journal of Artificial Intelligence (IJ-AI) Seminar Nasional Aplikasi Teknologi Informasi (SNATI) Jurnal Ilmu Komputer dan Informasi Jurnal Teknik ITS IPTEK Journal of Science IPTEK The Journal for Technology and Science Techno.Com: Jurnal Teknologi Informasi TEKNOLOGI: Jurnal Ilmiah Sistem Informasi Jurnal Buana Informatika TELKOMNIKA (Telecommunication Computing Electronics and Control) JUTI: Jurnal Ilmiah Teknologi Informasi CommIT (Communication & Information Technology) Jurnal Ilmiah Kursor Scan : Jurnal Teknologi Informasi dan Komunikasi Jurnal EECCIS Jurnal Teknologi Informasi dan Ilmu Komputer International Journal of Advances in Intelligent Informatics Scientific Journal of Informatics Journal of Information Systems Engineering and Business Intelligence Jurnal IPTEK Register: Jurnal Ilmiah Teknologi Sistem Informasi Jurnal Inspiration JOIN (Jurnal Online Informatika) Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) INTEGER: Journal of Information Technology Syntax Literate: Jurnal Ilmiah Indonesia Seminar Nasional Teknologi Informasi Komunikasi dan Industri JITTER (Jurnal Ilmiah Teknologi Informasi Terapan) Jurnal Nasional Pendidikan Teknik Informatika (JANAPATI) JURTEKSI Indonesian Journal of Electrical Engineering and Computer Science Jurnal Teknik Informatika (JUTIF) International Journal of Robotics and Control Systems Sewagati
Nanik Suciati
Departemen Informatika Institut Teknologi Sepuluh Nopember
Arts Humanities Civil Engineering, Building, Construction & Architecture Computer Science & IT Control & Systems Engineering Decision Sciences, Operations Research & Management Dentistry Education Electrical & Electronics Engineering Engineering Environmental Science Industrial & Manufacturing Engineering Languange, Linguistic, Communication & Media Law, Crime, Criminology & Criminal Justice Library & Information Science Mathematics Mechanical Engineering Public Health Social Sciences Other

Published : 91 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search
Journal : International Journal of Advances in Intelligent Informatics

 1 
RadEval: A novel semantic evaluation framework for radiology report Tsaniya, Hilya; Fatichah, Chastine; Suciati, Nanik
International Journal of Advances in Intelligent Informatics Vol 11, No 4 (2025): November 2025
Publisher : Universitas Ahmad Dahlan

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.26555/ijain.v11i4.2151

Abstract

The evaluation of automatically generated radiology reports remains a critical challenge, as conventional metrics fail to capture the semantic, clinical, and contextual correctness required for automatic medical analysis. This study proposes RadEval, a semantic-aware evaluation framework, to assess the quality of generated radiology reports. This method integrates domain-specific knowledge and contextual embeddings to evaluate the quality of generated radiology reports using a four-level scoring system. Given a reference report and a predicted report from a radiology image, RadEval performs scoring evaluation by first extracting relevant medical entities using a fine-tuned biomedical NER model. These entities are normalized through ontology mapping using RadLex concept identifiers to resolve lexical variation. Then, semantically related entities were clustered using BioBERT's contextual embeddings to capture deeper semantic similarity. In addition, predicted abnormality tags are incorporated to weight clinically significant terms during score aggregation. The final semantic score reflects a weighted combination of exact match, ontology match, and contextual similarity, modulated by tag importance. Experiments were conducted on the MIMIC-CXR dataset, which contains over 200,000 report pairs. Comparative evaluations show that RadEval outperforms traditional metrics, achieving an F1-score of 0.69, compared to 0.56 for BERTScore. Using this method, a more precise clinical interpretation of the predicted report was captured from the reference report. These findings suggest that RadEval method provides a more accurate and clinically aligned framework for evaluating the medical report generation model.
Co-Authors Adhira Riyanti Amanda Adni Navastara, Dini Agus Eko Minarno Agus Priyono Agus Zainal Arifin Agus Zainal Arifin Ahmad Saikhu Ahmad Syauqi Ahmad Syauqi Akwila Feliciano Akwila Feliciano Akwila Feliciano Pradiptatmaka Alam Ar Raad Stone Aldinata Rizky Revanda Altriska Izzati Khairunnisa Hermawan Amelia Devi Putri Ariyanto Amirullah Andi Bramantya Andika Rahman Teja Anny Yuniarti Antonius Kevin Wiguna Ardian Yusuf Wicaksono Ari Wijayanti Aris Fanani Arrie Kurniawardhani Arsy Bilahi Tama Ary Mazharuddin Shiddiqi Arya Yudhi Wijaya Atika Faradina Randa Atikah, Luthfi Avin Maulana Awangditama, Bangun Rizki Ayu Kardina Sukmawati Ayu Septya Maulani Baso, Budiman Bryan Nandriawan Bui, Ngoc Dung Chastine Fatichah Chastine Fatichah Chilyatun Nisa' Damayanti, Putri Daniel Sugianto Darlis Herumurti Davin Masasih Diana Purwitasari Dimas Rahman Oetomo Dini Adni Navastara Dini Adni Navastara, Dini Adni Dion Devara Aryasatya Eko Prasetyo Eva Yulia Puspaningrum Evelyn Sierra Fairuuz Azmi Firas Faishal Azka Jellyanto Faizin, Muhammad 'Arif Fajar Astuti Hermawati Fandy Kuncoro Adianto Fandy Kuncoro Adianto Febri Liantoni, Febri Fiqey Indriati Eka Sari Fitri Bimantoro Ginardi, R.V. Hari Glenaya Gou Koutaki Gurat Adillion, Ilham Hafidz, Abdan Handayani Tjandrasa Handayani Tjandrasa Hani Ramadhan Haq, Arinal Hidayat, Ahmad Nur Hidayati, Shintami Chusnul Hilya Tsaniya Imagine Clara Arabella Imam Kuswardayan Imam Mustafa Kamal Irawan Rahardja, Agustinus Aldi Isye Arieshanti Isye Arieshanti Januar Adi Putra Januar Adi Putra Kautsar, Faiz Keiichi Uchimura Kevin Christian Hadinata Kevin Christian Hadinata M. Bahrul Subkhi Maulidan Bagus A.R Maulidiya, Erika Mawaddah, Saniyatul MIFTAHOL ARIFIN, MIFTAHOL Mochammad Zharif Asyam Marzuqi Muchamad Kurniawan Muchamad Kurniawan Muchamad Kurniawan, Muchamad Muhamad Nasir Muhammad 'Arif Faizin Muhammad Alif Satriadhi Muhammad Farih Muhammad Fikri Sunandar Mutmainnah Muchtar Nafa Zulfa Ni Luh Made ITS Novrindah Alvi Hasanah R Dimas Adityo R. Dimas Adityo Rachman, Rudy Rahma Fida Fadhilah Rangga Kusuma Dinata Rangga Kusuma Dinata Rayssa Ravelia Rizal A Saputra Rizal A Saputra, Rizal A Rohman Dijaya Romario Wijaya Safhira Maharani Safhira Maharani Salim Bin Usman Salim Bin Usman Salsabiil Hasanah Sarimuddin, Sarimuddin Septiana, Nuning Sherly Rosa Anggraeni Sherly Rosa Anggraeni Shintami Chusnul Hidayati Shofiya Syidada Sjahrunnisa, Anita Suastika Yulia Riska Sugianela, Yuna Surya Fadli Alamsyah Syavira Tiara Zulkarnain Tanzilal Mustaqim Tiara Anggita Tiara Anggita Tsaniya, Hilya Wahyu Saputra, Vriza Wan Sabrina Mayzura Wibowo, Della Aulia Wicaksono, Farhan Wijayanti Nurul Khotimah Yulia Niza Yulia Niza Yuna Sugianela Yuna Sugianela Yuslena Sari, Yuslena Yuwanda Purnamasari Pasrun Zakiya Azizah Cahyaningtyas Zakiya Azizah Cahyaningtyas