Claim Missing Document
Check
Articles

Found 3 Documents
Search

Predicting the Sentiment of Review Aspects in the Peer Review Text using Machine Learning Basuki, Setio; Sari, Zamah; Tsuchiya, Masatoshi; Indrabayu, Rizky
Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control Vol. 9, No. 4, November 2024
Publisher : Universitas Muhammadiyah Malang

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.22219/kinetik.v9i4.2042

Abstract

This paper develops a Machine Learning (ML) model to classify the sentiment of review aspects in the peer review text. Reviewers use the review aspect as paper quality indicators such as motivation, originality, clarity, soundness, substance, replicability, meaningful comparison, and summary during the review process. The proposed model addresses the critique of the existing peer review process, including a high volume of submitted papers, limited reviewers, and reviewer bias. This paper uses citation functions, representing the author's motivation to cite previous research, as the main predictor. Specifically, the predictor comprises citing sentence features representing the scheme of citation functions, regular sentence features representing the scheme of citation functions for non-citation sentences, and reference-based representing the source of citation. This paper utilizes the paper dataset from the International Conference on Learning Representations (ICLR) 2017-2020, which includes sentiment values (positive or negative) for all review aspects. Our experiment on combining XGBoost, oversampling, and hyper-parameter optimization revealed that not all review aspects can be effectively estimated by the ML model. The highest results were achieved when predicting Replicability sentiment with 97.74% accuracy. It also demonstrated accuracies of 94.03% for Motivation and 93.93% for Meaningful Comparison. However, the model exhibited lower effectiveness on Originality and Substance (85.21% and 79.94%) and performed less effectively on Clarity and Soundness with accuracies of 61.22% and 61.11%, respectively. The combination predictor was the best for the 5 review aspects, while the other 2 aspects were effectively estimated by regular sentence and reference-based predictors.
Automatic Categorization of Mental Health Frame in Indonesian X (Twitter) Text using Classification and Topic Detection Techniques Basuki, Setio; Indrabayu, Rizky; Effendy, Nico Ardia
Khazanah Informatika : Jurnal Ilmu Komputer dan Informatika Vol. 10 No. 2 (2024): Oktober 2024
Publisher : Universitas Muhammadiyah Surakarta

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.23917/khif.v10i2.3328

Abstract

This paper aims to develop a machine learning model to detect mental health frames in Indonesian-language tweets on the X (Twitter) platform. This research is motivated by the gap in automatically detecting mental health frames, despite the importance of mental health issues in Indonesia. This paper addresses the problem by applying classification and topic detection methods across various mental health frames through multiple stages. First, this paper examines various mental health frames, resulting in 7 main labels: Awareness, Classification, Feelings and Problematization, Accessibility and Funding, Stigma, Service, Youth, and an additional label named Others. Second, it focuses on constructing a dataset of Indonesian tweets, totaling 29,068 data, by filtering tweets using the keywords "mental health" and "kesehatan mental". Third, this paper conducts data preprocessing and manual labeling of a random selection of 3,828 tweets, chosen due to the impracticality of labeling all data. Finally, the fourth stage involves conducting classification experiments using classical text features, non-contextual and contextual word embeddings, and performing topic detection experiments with three different algorithms. The experiments show that the BERT-based method achieved the highest accuracy, with 81% in the 'Others' vs. 'non-Others' classification, 80% in the seven main label classifications, and 92% in the seven main labels classification when using GPT-4-powered data augmentation. Topic detection experiments indicate that the Latent Dirichlet Allocation (LDA) and Latent Semantic Indexing (LSI) algorithms are more effective than the Hierarchical Dirichlet Process (HDP) in generating relevant keywords representing the characteristics of each main label.
Assistance in Preparing Engineering Prompts for Muhammadiyah School Teachers to Optimize the Use of ChatGPT in the World of Education: Pendampingan Penyusunan Prompt Engineering Bagi Guru Sekolah Muhammadiyah Untuk Mengoptimalkan Pemanfaatan ChatGPT Di Dunia Pendidikan Basuki, Setio; Faiqurrahman, Mahar; Marthasari, Gita Indah; Indrabayu, Rizky; Zachra, Fatimatus; Effendy, Nico Ardia
Dinamisia : Jurnal Pengabdian Kepada Masyarakat Vol. 8 No. 3 (2024): Dinamisia: Jurnal Pengabdian Kepada Masyarakat
Publisher : Universitas Lancang Kuning

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.31849/dinamisia.v8i3.19522

Abstract

ChatGPT, a widely used Large Language Model (LLM), enhances productivity in various sectors, including education. However, its extensive usage often lacks proficiency in writing effective prompts, resulting in less optimal, biased, and hallucinated outputs. This community service initiative by Universitas Muhammadiyah Malang (UMM) aims to educate teachers on prompt engineering, enabling them to (i) write effective prompts to utilize ChatGPT's potential, (ii) educate about potential biases and hallucinations of ChatGPT, and (iii) integrate ChatGPT into educational practices with integrity. Partnering with three Muhammadiyah Schools, the program trains 7-8 teachers from each institution. The training covers five key areas: (a) prompt engineering introduction, (b) building optimal prompts, (c) leveraging ChatGPT in teaching and learning, (d) prompt engineering for educational material creation, and (e) ethics of LLM usage in professional and academic settings. The effectiveness of this program is evaluated through pre-test and post-test questionnaires. Results indicate a significant improvement in prompt engineering proficiency, rising from 35.3% (pre-test) to 87.5% (post-test), and in the utilization of ChatGPT for learning support, increasing from 23.5% (pre-test) to 81.3% (post-test).