<![CDATA[Time-series foundation models (TSFMs) are increasingly promoted as reusable forecasting systems that can generalize across domains with zero-shot or few-shot adaptation. That claim is scientifically consequential, but current evaluation practice remains under-specified where it matters most target-domain separation, contamination control, adaptation budget definition, shift severity characterization, and aggregation across heterogeneous deployment conditions. This paper reconstructs TSFM benchmarking as a methodological problem rather than a leaderboard problem. We formalize zero-shot and few-shot forecasting under domain shift as conditional risk estimation over governed target distributions; develop a forecasting-specific taxonomy of shift covering temporal regime, entity, resolution, schema, horizon, observation-quality, intervention, and label-formation change; and propose a six-layer benchmark architecture spanning model governance, dataset governance, deterministic shift generation, evaluation tracks, metric tensors, and reporting bundles. The contribution is primarily conceptual, but to avoid a purely rhetorical framework, we also provide an executable pilot instantiation on a public electricity-transformer forecasting setting. Because large-scale TSFM execution was not conducted in this package, the pilot uses lightweight surrogate forecasters to validate the benchmark machinery itself rather than to claim new TSFM state of the art. Even this limited pilot shows that in-domain and cross-domain rankings can diverge sharply, that adaptation gains must be interpreted jointly with cost, and that robustness to observation degradation and calibration cannot be inferred from average point error alone. The paper therefore advances a benchmark doctrine: credible TSFM claims require leakage-aware governance, severity-conditioned analysis, explicit adaptation accounting, and multi-objective reporting that aligns evidence with generalization claims.]]>
Copyrights © 2026
