This research presents a performance comparison between two approaches for identifying the base form of affixed Javanese words: the Ahmad Yusoff Sembok (AYS) rule-based stemming algorithm and the Jaro-Winkler (JW) string similarity approach. Javanese was selected as the focus because of its complex morphological structure, encompassing prefixes, suffixes, infixes, and confixes, along with significant speech-level and dialectal variation, which together pose challenges for natural language processing. The dataset comprises 720 manually annotated word lemma pairs. Evaluation was carried out using precision, recall, F1-score, accuracy, and Cohen’s Kappa, complemented by error analysis on over-stemming and under-stemming cases. Results indicate that JW achieves higher overall performance (83.19% accuracy, 83% F1-score) compared to AYS (73.19% accuracy, 73% F1-score), with AYS producing more over-stemming errors (88 cases) and JW showing more under-stemming errors (47 cases). These outcomes suggest that similarity-based approaches are more effective in addressing Javanese morphological complexity, while also contributing a benchmark dataset of manually annotated Javanese word lemma pairs, a comparative evaluation framework between rule-based and similarity-based approaches, and practical insights for the development of stemming tools in regional languages that currently lack NLP resources.