Claim Missing Document
Check
Articles

Evaluasi Usability Sistem Informasi Manajemen Kepegawaian Kalimantan Selatan Berdasarkan ISO 9241-11 Maulana, Syarif; Saputro, Setyo Wahyu; Abadi, Friska; Turianto Nugrahadi, Dodon; Reza Faisal, Mohammad
Jurnal Teknologi Informasi dan Ilmu Komputer Vol 13 No 3: Juni 2026
Publisher : Fakultas Ilmu Komputer, Universitas Brawijaya

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.25126/jtiik.2026133

Abstract

Penelitian ini menyajikan evaluasi usability secara komprehensif terhadap Sistem Informasi Manajemen Kepegawaian (SIMPEG) Provinsi Kalimantan Selatan berdasarkan kerangka ISO 9241-11. Evaluasi dilakukan dengan mengintegrasikan Performance Measurement, System Usability Scale (SUS), dan Retrospective Think-Aloud (RTA) untuk menilai efektivitas, efisiensi, dan kepuasan pengguna. Sebanyak 25 partisipan menyelesaikan tujuh skenario tugas yang merepresentasikan fungsi utama sistem. Hasil pengujian menunjukkan tingkat Task Completion Rate sebesar 96% dan Overall Relative Efficiency sebesar 91%, yang mengindikasikan efektivitas dan efisiensi sistem yang tinggi. Skor rata-rata SUS sebesar 79,7 menempatkan sistem pada kategori “baik” dengan tingkat akseptabilitas “dapat diterima”. Analisis inferensial menggunakan uji korelasi Spearman menun-jukkan tidak terdapat hubungan signifikan antara efisiensi objektif dan kepuasan subjektif (r = 0,129; p = 0,538), yang mengindikasikan bahwa dimensi usability dapat bersifat independen dalam konteks sistem pemerintahan. Uji Friedman menunjukkan adanya perbedaan signifikan waktu penyelesaian antar skenario tugas (χ²(6) = 141,071; p < 0,001), yang mengidentifikasi adanya bottleneck usability pada modul tertentu. Temuan kualitatif dari RTA memperkuat hasil kuantitatif dengan mengungkap kendala pada aspek kemudahan belajar, konsistensi navigasi, dan integrasi fitur. Penelitian ini memberikan kontribusi melalui integrasi analisis deskriptif dan inferensial dalam evaluasi usability sistem pemerintahan sebagai baseline pengembangan sistem selanjutnya.   Abstract   This study presents a comprehensive usability evaluation of the Civil Service Management Information System (SIMPEG) of South Kalimantan Province, based on the ISO 9241-11 framework. The evaluation integrates Performance Measurement, the System Usability Scale (SUS), and the Retrospective Think-Aloud (RTA) protocol to assess the system's effectiveness, efficiency, and user satisfaction. A total of 25 participants completed seven task scenarios representing the system’s core functions. The testing results showed a Task Completion Rate of 96% and an Overall Relative Efficiency of 91%, indicating high levels of effectiveness and efficiency. The average SUS score of 79.7 places the system in the “Good” category with an “Acceptable” level of usability. Inferential analysis using Spearman’s correlation test revealed no significant relationship between objective efficiency and subjective satisfaction (r = 0.129; p = 0.538), suggesting that usability dimensions may be independent in the context of government systems. The Friedman test indicated a significant difference in task completion times across scenarios (χ²(6) = 141.071; p < 0.001), identifying specific modules as usability bottlenecks. Qualitative findings from the RTA supported the quantitative results by uncovering challenges related to learnability, navigation consistency, and feature integration. This study contributes by integrating descriptive and inferential analysis in the usability evaluation of government systems, serving as a baseline for future system development.
Effect of SMOTE Variants on Software Defect Prediction Classification Based on Boosting Algorithm Rahmina Ulfah Aflaha; Rudy Herteno; Mohammad Reza Faisal; Friska Abadi; Setyo Wahyu Saputro
Jurnal Ilmiah Teknik Elektro Komputer dan Informatika Vol. 10 No. 2 (2024): June
Publisher : Universitas Ahmad Dahlan

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.26555/jiteki.v10i2.28521

Abstract

Detecting software defects early on is critical for avoiding significant financial losses. However, building accurate software defect prediction models can be challenging due to class imbalance, where the data for defective modules is much less than for standard modules. This research addresses this issue using the imbalanced dataset NASA MDP. To address this issue, researchers have proposed new methods that combine data level balancing approaches with 14 variations of the SMOTE algorithm to increase the amount of defective module data. An algorithm-level approach with three boosting algorithms, Catboost, LightGBM, and Gradient Boosting, is applied to classify modules as defective or non-defective. These methods aim to improve the accuracy of software defect prediction. The results show that this new method can produce a more accurate classification than previous studies. The DSMOTE and Gradient Boosting pair with 0.9161 has the highest average accuracy (0.9161). The DSMOTE and Catboost model achieved the highest average AUC value (0.9637). The ADASYN kernel and Catboost showed the best ability to perform the average G-mean value (0.9154). The research contribution to software defect prediction involves developing new techniques and evaluating their effectiveness in addressing class imbalance.
Optimized multi correlation-based feature selection in software defect prediction Muhammad Nabil Muyassar Rahman; Radityo Adi Nugroho; Mohammad Reza Faisal; Friska Abadi; Rudy Herteno
TELKOMNIKA (Telecommunication Computing Electronics and Control) Vol 22, No 3: June 2024
Publisher : Universitas Ahmad Dahlan

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.12928/telkomnika.v22i3.25793

Abstract

In software defect prediction, noisy attributes and high-dimensional data remain to be a critical challenge. This paper introduces a novel approach known as multi correlation-based feature selection (MCFS), which seeks to address these challenges. MCFS integrates two feature selection techniques, namely correlation-based feature selection (CFS) and correlation matrixbased feature selection (CMFS), intending to reduce data dimensionality and eliminate noisy attributes. To accomplish this, CFS and CMFS are applied independently to filter the datasets, and a weighted average of their outcomes is computed to determine the optimal feature selection. This approach not only reduces data dimensionality but also mitigates the impact of noisy attributes. To further enhance predictive performance, this paper leverages the particle swarm optimization (PSO) algorithm as a feature selection mechanism, specifically targeting improvements in the area under the curve (AUC). The evaluation of the proposed method is conducted on 12 benchmark datasets sourced from the NASA metrics data program (MDP) corpus, renowned for their noisy attributes, high dimensionality, and imbalanced class records. The research findings demonstrate that MCFS outperforms CFS and CMFS, yielding an average AUC value of 0.891, thereby emphasizing it is efficacy in advancing classification performance in the context of software defect prediction using k-nearest neighbors (KNN) classification.
The impact of software metrics in NASA metric data program dataset modules for software defect prediction Adinda Ayu Puspita Ramadhani; Radityo Adi Nugroho; Mohammad Reza Faisal; Friska Abadi; Rudy Herteno
TELKOMNIKA (Telecommunication Computing Electronics and Control) Vol 22, No 4: August 2024
Publisher : Universitas Ahmad Dahlan

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.12928/telkomnika.v22i4.25787

Abstract

This paper discusses software metrics and their impact on software defect prediction values in the NASA metric data program (MDP) dataset. The NASA MDP dataset consists of four categories of software metrics: halstead, McCabe, LoC, and misc. However, there is no study showing which metrics participate in increasing the area under the curve (AUC) value of the NASA MDP dataset. This study utilizes 12 modules from the NASA MDP dataset, where these 12 modules are being tested into 14 relationships of software metrics derived from the four existing metric categories. Subsequently, classification is performed using the k-nearest neighbor (kNN) method. The research concludes that software metrics have a significant impact on the AUC value, with the LoC+McCabe+misc metrics relationship influencing the improvement of the AUC value. However, the metrics relationship that has the most impact on achieving less optimal AUC values is McCabe. Halstead metric also plays a role in decreasing the performance of other metrics.
Functional Evaluation of the Logia Dashboard Using Boundary Value Testing and Cause-Effect Graph Techniques Muhammad Rizky Aulia Ramadhan; Friska Abadi; Dodon Turianto Nugrahadi; Setyo Wahyu Saputro; Rudy Herteno
Telematika Vol 18, No 2: August (2025)
Publisher : Universitas Amikom Purwokerto

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.35671/telematika.v18i2.3121

Abstract

The Logia Dashboard is a web-based information system used to manage rehabilitation plant data on post-mining land. As an alpha-stage system, Logia requires thorough functional and performance evaluation to ensure that all input validations, logical processes, and system responses operate correctly before wider implementation. This study aims to evaluate the functional reliability and performance of the Logia Dashboard by applying a combined approach of Boundary Value Testing (BVT) and Cause-Effect Graph (CEG) techniques, supported by performance testing using Google Lighthouse. The research design adopts a black-box testing approach. BVT is applied to validate input boundaries on critical features, including login, data editing, QR code generation, and account creation. Meanwhile, CEG is used to model logical relationships between input conditions and system outputs to generate systematic test cases. A total of 39 optimized functional test cases were executed in a controlled local environment. Performance testing was conducted using Lighthouse by measuring key metrics such as First Contentful Paint (FCP), Largest Contentful Paint (LCP), Total Blocking Time (TBT), and Cumulative Layout Shift (CLS). The functional testing results show that 37 out of 39 test cases passed, yielding a success rate of 94.87%. Two failed cases were identified in the login feature, indicating weaknesses in input validation feedback. Performance testing produced an average Lighthouse score of 97, demonstrating that the system has excellent load speed and interface stability, although minor layout instability was detected on certain pages. These results indicate that the combined application of BVT and CEG is effective for detecting boundary-related and logical input errors in alpha-stage web systems. The findings also provide concrete recommendations for improving login validation and interface stability, supporting further development of the Logia Dashboard toward a more reliable and robust system for post-mining land management.
Comparative Analysis of Distance Metrics in KNN and SMOTE Algorithms for Software Defect Prediction Khusnul Rahmi Maulidha; Mohammad Reza Faisal; Setyo Wahyu Saputro; Friska Abadi; Dodon Turianto Nugrahadi; Puput Dani Prasetyo Adi; Hariyady Hariyady
Telematika Vol 18, No 1: February (2025)
Publisher : Universitas Amikom Purwokerto

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.35671/telematika.v18i1.3008

Abstract

As the complexity and scale of projects increase, new challenges arise related to handling software defects. One solution uses machine learning-based software defect prediction techniques, such as the K-Nearest Neighbors (KNN) algorithm. However, KNN’s performance can be hindered by the majority vote mechanism and the distance/similarity metric choice, especially when applied to imbalanced datasets. This research compares the effectiveness of Euclidean, Hamming, Cosine, and Canberra distance metrics on KNN performance, both before and after the application of SMOTE (Synthetic Minority Over-sampling Technique). Results show significant improvements in the AUC and F-1 measure values across various datasets after the SMOTE application. Following the SMOTE application, Euclidean distance produced an AUC of 0.7752 and an F1 of 0.7311 for the EQ dataset. With Canberra distance and SMOTE, the JDT dataset produced an AUC of 0.7707 and an F-1 of 0.6342. The LC dataset improved to 0.6752 and 0.3733 in tandem with the ML dataset, which climbed to 0.6845 and 0.4261 with Canberra distance. Lastly, after using SMOTE, the PDE dataset improved to 0.6580 and 0.3957 with Canberra distance. The findings confirm that SMOTE, combined with suitable distance metrics, significantly boosts KNN’s prediction accuracy, with a P-value of 0.0001.
Automatic Analysis of Natural Disaster Messages on Social Media Using IndoBERT and Multilingual BERT Yasmin Dwi Safitri; Mohammad Reza Faisal; Dwi Kartini; Triando Hamonangan Saragih; Friska Abadi; Adam Mukharil Bachtiar
Telematika Vol 18, No 2: August (2025)
Publisher : Universitas Amikom Purwokerto

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.35671/telematika.v18i2.3140

Abstract

Information about natural disasters disseminated through social media can serve as an important data source for mitigation processes and early warning systems. Social media platforms, such as X (formerly known as Twitter), have become primary channels for conveying real-time information, especially during disaster emergencies. With the large amount of unstructured disaster-related text that must be processed, the main challenge is accurately filtering and classifying messages into three categories: eyewitness, non-eyewitness, and don’t know. This research aims to compare the performance of four BERT-based natural language processing models, namely IndoBERT, IndoBERT with Masked Language Modeling (MLM), Multilingual BERT, and Multilingual BERT with MLM, in classifying Indonesian-language disaster messages. The dataset used in this study was obtained from previous research and publicly available data on GitHub, consisting of annotated messages related to floods, earthquakes, and forest fires. The method applied is a deep learning approach using the hold-out technique with an 80:20 ratio for training and testing data, and the same ratio applied to split the training data into training and validation subsets, with stratification to maintain balanced class proportions. In addition, variations in batch size were explored to evaluate their effect on model performance stability. The results show that the IndoBERT model achieved the highest performance on the flood and earthquake datasets, with accuracies of 80.67% and 81.50%, respectively. Meanwhile, IndoBERT with MLM pre-training recorded the highest accuracy on the forest fire dataset, 88.33%. Overall, IndoBERT demonstrated the most consistent and superior performance across datasets compared to the other models. These findings indicate that IndoBERT has strong capabilities in understanding Indonesian disaster-related text, and the results can be used as a foundation for developing automatic classification systems to support real-time disaster monitoring and early warning applications
Classification of Eyewitness Social Media Messages for Natural Disaster Monitoring using BERT Variants Hanafi, Muhammad Bashir; Faisal, Mohammad Reza; Abadi, Friska; Budiman, Irwan; Saputro, Setyo Wahyu; Mbeledogu, Njideka Nkemdilim
Jurnal Teknik Informatika (Jutif) Vol. 7 No. 3 (2026): JUTIF Volume 7, Number 3, June 2026
Publisher : Informatika, Universitas Jenderal Soedirman

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.52436/1.jutif.2026.7.3.5317

Abstract

The rapid growth of disaster-related social media data demands effective monitoring. However, its real-time source presents challenges due to large volumes of unstructured and noisy data. This study aims to improve effective monitoring with BERT variants to classify eyewitness reports on Twitter/X. Earlier studies have applied machine-learning and deep-learning models to automate the monitoring of eyewitness messages on social media, but these models still have shortcomings. Traditional machine-learning models rely on handcrafted and frequency-based features, limiting their ability to capture contextual semantics. Deep-learning models offer improved performance but still face challenges in modeling long-range dependencies and handling high-volume social media streams. This issue is pronounced in social media streams. This study employs transformer-based models using several BERT variants (BERT, RoBERTa, DistilBERT, ELECTRA, and ALBERT). Each model is pre-trained with the Masked Language Modeling (MLM) objective, and batch-size optimization is applied to boost performance. Experimental results indicate that a batch size of 16 consistently yields the best performance, with the standard BERT model achieving the highest macro-F1 score of 0.762. By disaster type, macro-F1 scores reach 0.744 for hurricane, 0.793 for flood, 0.756 for earthquake, and 0.750 for wildfire. BERT (16) outperforms the other BERT variants and twelve baseline models from prior research. Unlike previous approaches, this study leverages pre-trained Masked Language Models to optimize classification on disaster-related datasets. The findings contribute to the development of transformer-based architectures for text classification in real-time disaster informatics, leading to more accurate situational awareness and reduced delays in emergency decision-making.
Comparative Performance Evaluation of Linear, Bagging, and Boosting Models Using BorutaSHAP for Software Defect Prediction on NASA MDP Datasets Kartika, Najla Putri; Herteno, Rudy; Budiman, Irwan; Nugrahadi, Dodon Turianto; Abadi, Friska; Ahmad, Umar Ali; Faisal, Mohammad Reza
Jurnal Teknik Informatika (Jutif) Vol. 6 No. 6 (2025): JUTIF Volume 6, Number 6, Desember 2025
Publisher : Informatika, Universitas Jenderal Soedirman

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.52436/1.jutif.2025.6.6.5393

Abstract

Software defect prediction aims to identify potentially defective modules early on in order to improve software reliability and reduce maintenance costs. However, challenges such as high feature dimensions, irrelevant metrics, and class imbalance often reduce the performance of prediction models. This research aims to compare the performance of three classification model groups—linear, bagging, and boosting—combined with the BorutaSHAP feature selection method to improve prediction stability and interpretability. A total of twelve datasets from the NASA Metrics Data Program (MDP) were used as test references. The research stages included data preprocessing, class balancing using the Synthetic Minority Oversampling Technique (SMOTE), feature selection with BorutaSHAP, and model training using five algorithms, namely Logistic Regression, Linear SVC, Random Forest, Extra Trees, and XGBoost. The evaluation was conducted with Stratified 5-Fold Cross-Validation using the F1-score and Area Under the Curve (AUC) metrics. The experimental results showed that tree-based ensemble models provided the most consistent performance, with Extra Trees recording the highest average AUC of 0.794 ± 0.05, followed by Random Forest (0.783 ± 0.06). The XGBoost model provided the best results on the PC4 dataset (AUC = 0.937 ± 0.008), demonstrating its ability to handle complex data patterns. These findings prove that BorutaSHAP is effective in filtering relevant features, improving classification reliability, and strengthening transparency and interpretability in the Explainable Artificial Intelligence (XAI) framework for software quality improvement.
Analysis of Static and Contextual Word Embeddings in Capsule Network for Sentiment Analysis of The Free Nutritious Meal Program on Twitter Raditya, Virgi Atha; Saragih, Triando Hamonangan; Faisal, Mohammad Reza; Abadi, Friska; Muliadi, Muliadi
Jurnal Teknik Informatika (Jutif) Vol. 7 No. 1 (2026): JUTIF Volume 7, Number 1, February 2026
Publisher : Informatika, Universitas Jenderal Soedirman

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.52436/1.jutif.2026.7.1.5424

Abstract

Public discourse surrounding Indonesia’s Makan Bergizi Gratis (MBG) program reflects diverse opinions that have not yet been systematically examined using computational methods. This study addresses that gap by evaluating the effectiveness of static and contextual word embeddings within a Capsule Network (CapsNet) framework for sentiment analysis of MBG-related tweets on Twitter. A total of 7,133 Indonesian-language tweets were collected through web crawling, preprocessed, and manually labeled into positive, neutral, and negative categories. Four embedding techniques—Word2Vec, FastText, ELMo, and IndoBERT—were tested under two preprocessing settings, raw and stemming. The experimental results show that Word2Vec on raw text achieved the highest accuracy of 96.17%, while FastText obtained the best performance on stemmed data with 94.10%. These findings indicate that morphological normalization benefits static and subword-based embeddings, whereas contextual models maintain stable performance without extensive fine-tuning. Overall, this study demonstrates the potential of combining CapsNet with appropriate embedding strategies for Indonesian-language sentiment analysis and provides evidence that natural language processing can support data-driven evaluation of public programs such as MBG.
Co-Authors A.A. Ketut Agung Cahyawan W AA Sudharmawan, AA Abdullayev, Vugar Achmad Zainudin Nur Adam Mukharil Bachtiar Adi Mu'Ammar, Rifqi Adinda Ayu Puspita Ramadhani Ahmad Juhdi Amalia, Raisa Andi Farmadi Andi Farmadi Andi Farmandi Arif, Nuuruddin Hamid Athavale, Vijay Anant Bagaskara Ridho Vandio budiman, irwan Deni Kurnia Dodon Turianto Nugrahadi Dwi Kartini Dwi Kartini, Dwi Emma Andini Fatma Indriani Fauzan Luthfi, Achmad Febrian, Muhamad Michael Halimah Hanafi, Muhammad Bashir Hariyady Hariyady Herteno, Rudy Indriani, Fatma Irwan Budiman Irwan Budiman Irwan Budiman Itqan Mazdadi, Muhammad Kartika, Najla Putri Khusnul Rahmi Maulidha M Kevin Warendra Mafazy, Muhammad Meftah Martalisa, Asri Mbeledogu, Njideka Nkemdilim Mera Kartika Delimayanti Muhamad Fawwaz Akbar Muhammad Alkaff Muhammad Alkaff Muhammad Alvin Alfando Muhammad Azmi Adhani Muhammad Denny Ersyadi Rahman Muhammad Fikri Muhammad Haekal Muhammad Itqan Mazdadi Muhammad Khairin Nahwan Muhammad Mirza Hafiz Yudianto Muhammad Nabil Muyassar Rahman Muhammad Nazar Gunawan Muhammad Noor Muhammad Reza Faisal, Muhammad Reza Muhammad Rizky Aulia Ramadhan Muhammad Sholih Afif Muliadi Muliadi Muliadi Aziz Muliadi Muliadi Muliadi Muliadi Nabella, Putri Nor Indrani Nugrahadi, Dodon Nurlatifah Amini Nursyifa Azizah Prastya, Septyan Eka Pratama, Muhammad Yoga Adha Puput Dani Prasetyo Adi Putri Nabella Raditya, Virgi Atha Radityo Adi Nugroho Rahman Hadi Rahman Rahmat Budianoor Rahmat Ramadhani Rahmina Ulfah Aflaha Reina Alya Rahma Reza Faisal, Mohammad Rezeki, Abdillah Rinaldi Riza Susanto Banner Rizal, Muhammad Nur Rizky Ananda, Muhammad Rizky, Muhammad Hevny Rudy Herteno Rudy Herteno Rudy Herteno SALLY LUTFIANI Saragih, Triando Hamonangan Sarah Monika Nooralifa Sa’diah, Halimatus Septyan Eka Prastya Setyo Wahyu Saputro Siti Fathmah Siti Napi'ah Syarif Maulana, Syarif Tri Mulyani Ulya, Azizatul Umar Ali Ahmad Vina Maulida, Vina Wahyu Dwi Styadi Yasmin Dwi Safitri Yunida, Rahmi