Akam Aziz
Department of Computer Science, University of Raparin, Kurdistan Region, Iraq.

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

TrustPhish-AI: Safety-Gated Source-Aware Evaluation of Three-Class Phishing Detection Under Email-to-SMS Shift Akam Aziz
The Indonesian Journal of Computer Science Vol. 15 No. 4 (2026): The Indonesian Journal of Computer Science
Publisher : AI Society & STMIK Indonesia

Show Abstract | Download Original | Original Source | Check in Google Scholar

Abstract

Source shift can make phishing detectors appear stronger while concealing unsafe class-specific errors. This study evaluated a three-class workflow (Legitimate, Suspicious, and Phishing) using source-aware governance, controlled augmentation, family-disjoint diagnostics, and pre-specified safety gates. The frozen training set contained 3,732 records from four source families. Evaluation used a locked 584-record validation set, a 587-record internal test, a 788-record SMS compound test, and a 200-record SpamAssassin family-disjoint diagnostic. Classical word/character TF-IDF logistic regression with cell-balanced augmentation achieved internal macro-F1 of 0.9633 and compound macro-F1 of 0.5461. DistilRoBERTa augmentation improved compound macro-F1 by 0.1524 versus its reference (95% CI 0.1143-0.1914), but reduced phishing recall by 0.2374 and failed the severe-error gate. A hierarchical transformer repair achieved internal macro-F1 of 0.9668 and compound macro-F1 of 0.5119; its compound gain was 0.1774 (95% CI 0.1410-0.2152), while phishing recall remained 0.2323 below the reference. Governance review included 492 dual-human consensus records, eight independently adjudicated disagreements, and protected-overlap screening. Candidate replacement sources were excluded under provenance, licensing, channel, privacy, taxonomy, or overlap criteria, so no records were added after the training freeze. The results show that source-aware augmentation can improve targeted and aggregate metrics, but aggregate gains are insufficient when phishing-recall and severe-error safety criteria fail. The study supports a governance and evaluation contribution, not global source resistance or autonomous deployment.