Claim Missing Document
Check
Articles

Found 4 Documents
Search

A Statistical Analysis of System Usability Scale (SUS) Evaluations in Online Learning Platform Suria, Ozzi
Journal of Information System and Informatics Vol 6 No 2 (2024): June
Publisher : Asosiasi Doktor Sistem Informasi Indonesia

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.51519/journalisi.v6i2.750

Abstract

This study evaluates the usability of a Moodle-based online learning system at a university using the System Usability Scale (SUS). Introduced in 2019, the platform has been instrumental in facilitating access to educational resources, enhancing the distribution of course materials, and supporting academic activities through interactive features. The SUS was employed to gather subjective feedback from 120 student respondents, providing insights into the system's efficiency, effectiveness, and user satisfaction. Statistical analysis was performed using methods including the Shapiro-Wilk test to address the skewed distribution of the original dataset with a mean SUS score of 71.52 and a high standard deviation of 16.18, indicating varied user experiences. Analysis of sample means was also conducted to achieve a more stable estimate of usability, resulting in a mean of 71.51 with a significantly reduced standard deviation of 2.44, suggesting a more consistent assessment across users. The final SUS score placed the system in the 'C+' category according to the Sauro and Lewis grading scale, indicating that while usability is acceptable, substantial improvements are necessary.
Combining SVM and Naive Bayes Models using a Soft Voting Approach for Sentiment Analysis of Tong Tji Tea House Saputra, Fendi Pradana; Suria, Ozzi
Sistemasi: Jurnal Sistem Informasi Vol 14, No 5 (2025): Sistemasi: Jurnal Sistem Informasi
Publisher : Program Studi Sistem Informasi Fakultas Teknik dan Ilmu Komputer

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.32520/stmsi.v14i5.5481

Abstract

In today’s digital technology and social media era, people are increasingly influenced to actively share reviews of restaurant services, expressing a wide range of customer opinions and perceptions. This study aims to analyze sentiment in Indonesian-language review texts using three machine learning models: Support Vector Machine (SVM), Naive Bayes (NB), and a combination of both through an Ensemble Soft Voting Classifier approach. The research focuses on user reviews of the Tong Tji Tea House, collected from the Google Maps platform, with sentiment data distributed as follows: positive (2,676 entries), neutral (670 entries), and negative (251 entries). The class imbalance poses a significant challenge in developing an optimal classification model. To address this, parameter optimization was carried out using the Grid Search method. The SVM model with a linear kernel and C=10 parameter achieved an accuracy of 0.9289 and a positive F1-score of 0.9289. The NB model recorded an accuracy of 0.8340 with an F1-score of 0.9102. Meanwhile, the Ensemble model with a soft voting approach and a 4:1 weight ratio (SVM:NB) demonstrated the best performance, achieving an accuracy of 0.9344 and a positive F1-score of 0.9750. These results indicate that the Ensemble method effectively enhances model accuracy and robustness in handling imbalanced data.
Measuring Resampling Methods on Imbalanced Educational Dataset’s Classification Performance Pratama, Irfan; Prasetyaningrum, Putri Taqwa; Chandra, Albert Yakobus; Suria, Ozzi
Register: Jurnal Ilmiah Teknologi Sistem Informasi Vol 10 No 1 (2024): January
Publisher : Information Systems - Universitas Pesantren Tinggi Darul Ulum

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.26594/register.v10i1.3397

Abstract

Imbalanced data refers to a condition that there is a different size of samples between one class with another class(es). It made the term “majority” class that represents the class with more instances number on the dataset and “minority” classes that represent the class with fewer instances number on the dataset. Under the target of educational data mining which demands accurate measurement of the student’s performance analysis, data mining requires an appropriate dataset to produce good accuracy. This study aims to measure the resampling method’s performance through the classification process on the student’s performance dataset, which is also a multi-class dataset. Thus, this study also measures how the method performs on a multi-class classification problem. Utilizing four public educational datasets, which consist of the result of an educational process, this study aims to get a better picture of which resampling methods are suitable for that kind of dataset. This research uses more than twenty resampling methods from the SMOTE variants library. as a comparison; this study implements nine classification methods to measure the performance of the resampled data with the non-resampled data. According to the results, SMOTE-ENN is generally the better resampling method since it produces a 0,97 F1 score under the Stacking classification method and the highest among others. However, the resampling method performs relatively low on the dataset with wider label variations. The future work of this study is to dig deeper into why the resampling method cannot handle the enormous class variation since the F1 score on the student dataset is lower than the other dataset.
Assessing Financial Information System Usability Using System Usability Scale (SUS) and Usability Metric for User Experience-Lite (UMUX-Lite) Suria, Ozzi
Jurnal Teknologi Sistem Informasi dan Aplikasi Vol. 7 No. 2 (2024): Jurnal Teknologi Sistem Informasi dan Aplikasi
Publisher : Program Studi Teknik Informatika Universitas Pamulang

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.32493/jtsi.v7i2.38723

Abstract

The rapid advancement in technology necessitates robust financial information systems (FIS) in educational institutions to manage the increasing complexity and volume of financial transactions. This study evaluates the usability of developed web-based Financial Information System for schools and the Foundation, designed to automate and streamline the management of student tuition fee records. The evaluation employs two prominent usability metrics: the System Usability Scale (SUS) and the Usability Metric for User Experience-Lite (UMUX-Lite). The SUS and UMUX-Lite are tools designed to quantitatively assess a system's usability and user satisfaction. The SUS provides a quick, reliable measure of a product's usability, while UMUX-Lite offers a concise evaluation of user satisfaction with less respondent time without compromising insight quality. Our research methodology included distributing questionnaires to 41 schools, employing SUS and UMUX-Lite questions to gather comprehensive feedback on the FIS's usability. Analysis of the data revealed a SUS score of 77.5, placing the system's usability in the "B+" category, indicative of good user satisfaction. The UMUX-Lite score was slightly higher at 81.69, reflecting excellent usability and aligning with user needs and ease of use. These results affirm the effectiveness of the developed FIS, highlighting its potential to significantly improve financial management processes within schools. The close correlation between SUS and UMUX-Lite scores underscores the system's robustness and intuitive design, suggesting that minor enhancements could further elevate its usability and overall user experience. This study contributes to the field by demonstrating the practical application of SUS and UMUX-Lite in evaluating the usability of financial information systems in educational settings, offering insights for future system enhancements and user-centered design practices.