Article Per Year (5 Year)
p-Index From 2021 - 2026
4.299
P-Index
This Author published in this journals
All Journal Seminar Nasional Aplikasi Teknologi Informasi (SNATI) Journal of Information Systems Engineering and Business Intelligence Register: Jurnal Ilmiah Teknologi Sistem Informasi Indonesia Symposium on Computing Indonesian Journal on Computing (Indo-JC) IJoICT (International Journal on Information and Communication Technology) JOIN (Jurnal Online Informatika) Sinkron : Jurnal dan Penelitian Teknik Informatika JURNAL MEDIA INFORMATIKA BUDIDARMA Seminar Nasional Teknologi Informasi Komunikasi dan Industri Jurnal Nasional Pendidikan Teknik Informatika (JANAPATI) Journal of Computer Science and Informatics Engineering (J-Cosine) JIPI (Jurnal Ilmiah Penelitian dan Pembelajaran Informatika) Building of Informatics, Technology and Science Journal of Computer System and Informatics (JoSYC) Journal of Applied Data Sciences Charity : Jurnal Pengabdian Masyarakat Indonesian Community Journal Jurnal Pengabdian Masyarakat Akademisi eProceedings of Engineering Community Service Seminar and Community Engagement (COSECANT) LOGIC: Jurnal Penelitian Informatika
Ade Romadhony
Universitas Telkom
Agriculture, Biological Sciences & Forestry Arts Humanities Computer Science & IT Control & Systems Engineering Decision Sciences, Operations Research & Management Economics, Econometrics & Finance Education Electrical & Electronics Engineering Engineering Environmental Science Industrial & Manufacturing Engineering Law, Crime, Criminology & Criminal Justice Mathematics Public Health Social Sciences Other

Published : 45 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search
Journal : Journal of Applied Data Sciences

 1 
Assessing Large Language Models for Zero-Shot Dynamic Question Generation and Automated Leadership Competency Assessment Gheartha, I Gusti Bagus Yogiswara; Adiwijaya, Adiwijaya; Romadhony, Ade; Ardiansyah, Yusfi
Journal of Applied Data Sciences Vol 7, No 1: January 2026
Publisher : Bright Publisher

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.47738/jads.v7i1.970

Abstract

Automated interview systems powered by artificial intelligence often rely on fine-tuned models and annotated datasets, limiting their adaptability to new leadership competency frameworks. Large language models have shown potential for generating questions and assessing answers, yet their zero-shot performance, operating without task-specific retraining remains underexplored in leadership assessment. This study examines the zero-shot capability of two models, Qwen 32B and GPT-4o-mini, within a multi-turn self-interview framework. Both models dynamically generated questions, interpreted responses, and assigned scores across ten leadership competencies. Professionals representing the role of Digital Marketing and Account Manager participated, each completing two AI-led interview sessions. Model outputs were evaluated by certified experts using a structured rubric across three dimensions: quality of behavioral insights, relevance of follow-up questions, and fit of assigned scores. Results indicate that Qwen 32B generated richer insights than GPT-4o-mini (mean = 2.86 vs. 2.62; p less than 0.01) and provided more differentiated assessments across competencies. GPT-4o-mini produced more consistent follow-up questions but lacked depth in interpretation, often yielding generic outputs. Both models struggled with accurate scoring of candidate responses, reflected in low answer score ratings (Qwen mean = 2.35; GPT mean = 2.21). These findings suggest a trade-off between insight richness and scoring stability, with both models demonstrating limited ability to fully capture nuanced leadership behaviors. This study offers one of the first empirical benchmarks of zero-shot model performance in leadership interviews. It underscores both the promise and current limitations of deploying such systems for scalable assessment. Future research should explore competency-specific prompt strategies, fairness evaluation across demographic groups, and domain-adapted fine-tuning to improve accuracy, reliability, and ethical alignment in high-stakes recruitment contexts.
Co-Authors A, Subaveerapandiyan Aditia Rafif Khoerulloh Adiwijaya Affan Fattahila, Ananda Agung Toto Wibowo Al Aufar, Arya Prima Al Faraby, Said Alfian Akbar Gozali Ali Ridho Fauzi Rahman Ananda Wulandari Anditya Arifianto Anisa Herdiani Anisah Firli Ardiansyah, Yusfi Arya Prima Al Aufar Bambang Pudjoatmodjo Bambang Pudjotatmodjo Barawi, Mohamad Hardyman Bedy Purnama Bhudi Jati Prio Utomo Bimmo Satryo Wicaksono Brady Rikumahu Dadan Rahadian Dade Nurjanah Dana Kusumo Dana S Kusumo Dana S Kusumo Dodi Wisaksono Sudiharto Donni Richasdy Ema Rachmawati Ema Rachmawati Fat'hah Noor Prawira Fat’hah Noor Prawira Fat’hah Noor Prawira Fazainsyah Azka Wicaksono Fazmah Arif Yulianto Frima, Mariana Gheartha, I Gusti Bagus Yogiswara H Hasmawati Hamdy Nur Saidy Haryo Adi Nugroho Haryo Adi Nugroho Haryo Nugroho Hasmawat, Hasmawat Hasmawati Hasmawati Hasmawati Hasmawati Hasmawati Herman, Fizio Ramadhan Imelda Atastina Januarahman, Faishal Kemas Rahmat S.W Kemas Rahmat Saleh Wiharja Lintani Afina Hajar Raudhoti Luh Putri Ayu Ningsih Mahmud Dwi Sulistiyo Moch Arif Bijaksana Muhammad Arzaki Muhammad Aziz Pratama Muhammad Farrel Muhammad Iqbal Muhammad Iqbal Muhammad Taufik Wahdiat Muhammad Zaky Aonillah Nadine Azhalia Purbani Ningsih, Shabrina Retno Nugraha, Azhar Baihaqi Nur, Farhan Ahmadi Javier othman, mohd kamal Pramana, Rifki Adi Prawita, Fat’hah Noor Putu Harry Gunawan Ramanti Dharayani Rhesa Hermawan Ridea Valentini Peristiwari Siwabessy Rimba Whidiana Ciptasari Riska Junia Wulandari Rita Rismala Said Faraby Selly Meliana Setiawan, Muhammad Rizki Ramadhan Siti Saadah Tresna Ariesta, Bayu Untari Novia Wisesty Wijaya, Kurniadi Ahmad