Article Per Year (5 Year)
p-Index From 2021 - 2026
0.23
P-Index
This Author published in this journals
All Journal Journal of Technology Informatics and Engineering
Siyu Chen
Information Management, University of Illinois Urbana-Champaign, IL, USA
Computer Science & IT

Published : 1 Documents Claim Missing Document
Claim Missing Document
Check
Articles

Found 1 Documents
Search

 1 
Narrative-Aware Scientific Claim Verification Agent with Evidence Ranking for ClimateCheck Wenhao Su; Siyu Chen; Ethan Qian
Journal of Technology Informatics and Engineering Vol. 5 No. 1 (2026): APRIL | JTIE : Journal of Technology Informatics and Engineering
Publisher : University of Science and Computer Technology

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.51903/jtie.v5i1.549

Abstract

Climate misinformation often combines a factual proposition with a recognizable narrative, such as denying observed warming, rejecting human causation, minimizing impacts, attacking mitigation, or casting doubt on climate science. This paper presents a lightweight narrative-aware scientific claim verification agent for the official ClimateCheck setting. The revised evaluation uses the official annotated ClimateCheck training data, the official publications corpus of 394,269 abstracts, and a claim-level validation split of the annotated data. The public ClimateCheck test file is treated as a blind claim list because its public fields do not contain verification or narrative labels. The system combines hashed BM25, TF-IDF retrieval, latent semantic analysis, narrative-family probabilities, and a logistic-regression verifier. Full-corpus retrieval shows that BM25 remains the strongest first-stage retriever, with Recall@10 = 0.466, while the narrative-aware hybrid obtains Recall@10 = 0.444. In the judged candidate reranking setting, the narrative-aware ranker obtains the highest Candidate Recall@1 = 0.789 and MAP = 0.848, compared with 0.759 and 0.843 for TF-IDF. End-to-end verification remains difficult: the BM25 top-1 pipeline reaches Macro-F1 = 0.408, while the narrative-aware pipeline reaches Macro-F1 = 0.355. Claim-level narrative evaluation no longer produces a perfect score; single-label top-family Macro-F1 is 0.422, and fine-grained multi-label CARDS-code Macro-F1 is 0.098. These results show that narrative information is useful for reranking already plausible evidence candidates, but it does not replace strong lexical retrieval and does not by itself solve claim verification.
Co-Authors Ethan Qian Wenhao Su