<![CDATA[This paper presents a compact reproducible benchmark for evaluating numerical-reasoning guardrails in a quant research assistant. The revised experiment uses a fixed 2026 source snapshot derived from the SEC 2026 Q1 Financial Statement Data Sets and FRED CSV series for VIXCLS, DGS10, DGS3MO and T10Y3M. The benchmark contains 460 tasks: 300 SEC financial-ratio tasks over 50 issuer-period records, 120 FRED VIX and Treasury-rate change tasks, and 40 macro-regime classification tasks. Each answer is evaluated by five programmatic guardrails: numeric consistency, unit correctness, time-window correctness, formula correctness and citation/source consistency. Four controlled response profiles are tested: Naive-RAG, Calculator-Only, Prompted-Checklist and Guarded-Quant. These profiles are deterministic failure-mode controls rather than performance claims about any particular deployed LLM. The empirical results show that arithmetic alone is not sufficient for financial safety: Calculator-Only reaches 79.78% numeric accuracy but only 0.43% all-guardrails pass rate because source, unit, formula and window fields often fail. Guarded-Quant achieves an 88.48% all-guardrails pass rate, 97.17% numeric accuracy, 100.00% unit pass rate, 96.30% window pass rate, 98.26% formula pass rate and 96.30% citation pass rate. The findings support a modest claim: a compact benchmark can make numerical audit failures visible, but it should not be read as evidence of broad quant-assistant reliability without broader data, live model outputs and operational stress tests.]]>
Copyrights © 2026
