Journal of Technology Informatics and Engineering
Vol. 4 No. 2 (2025): AUGUST | JTIE : Journal of Technology Informatics and Engineering

Offline Routing for Health Language Systems with Clarification and Escalation

Zijian Cao (Electrical and Computer Engineering, University of Wisconsin-Milwaukee, WI, USA)
Justin Gao (Computer Science, Georgetown University, DC, USA)



Article Info

Publish Date
25 Aug 2025

Abstract

Health-language systems require a control decision before response generation: answer with the available information, request clarification, or escalate for professional review. This study compared eight offline routing policies on HealthBench using researcher-defined actions mapped from physician-agreed categories. The implementations used TF-IDF logistic regression, bootstrap ensembles, stacked classification, and deterministic decision rules. A cohort of 2,601 dialogues provided disjoint development partitions, a 323-dialogue in-distribution test, and a locked 453-dialogue Hard test. Six policies used dialogue-derived information; two also accessed case-specific physician rubrics. Under this privileged-information condition, the rubric-augmented router achieved the highest accuracy, 69.97% in-distribution and 65.34% on Hard. The privileged cascade had the lowest severity-weighted routing loss, 0.1564 and 0.2851. Removing its rubric stage increased Hard loss to 0.3324 and unsafe direct-answer omission from 36.72% to 43.28%. Nevertheless, the privileged cascade directly escalated only 9/16 urgent cases; 6/16 were routed to clarification and 1/16 to an answer. Rubric access improves offline discrimination, but the strongest results do not establish an equivalent deployable verifier. The framework supports controlled prototyping of pre-generation routing, with substantial residual errors and no evaluation of generated medical responses or clinical outcomes.

Copyrights © 2025






Journal Info

Abbrev

jtie

Publisher

Subject

Computer Science & IT

Description

Power Engineering Telecommunication Engineering Computer Engineering Control and Computer Systems Electronics Information technology Informatics Data and Software engineering Biomedical ...