cover
Article Per Year (5 Year)
All
Home Page
OAI Link
Editorial Team
Contact
Reviewer
Google Scholar
Contact Name
Johan Reimon Batmetan
Contact Email
garuda@apji.org
Phone
+6285885852706
Journal Mail Official
danang@stekom.ac.id
Editorial Address
Jl. Majapahit No.304, Pedurungan Kidul, Kec. Pedurungan, Semarang, Provinsi Jawa Tengah, 52361
Location
Kota semarang,
Jawa tengah
INDONESIA
Journal of Technology Informatics and Engineering
Published by Universitas Sains Dan Teknologi Komputer
ISSN : 29619068     EISSN : 29618215     DOI : 10.51903
Core Subject : Science,
Computer Science & IT
Power Engineering Telecommunication Engineering Computer Engineering Control and Computer Systems Electronics Information technology Informatics Data and Software engineering Biomedical Engineering
Arjuna Subject : Ilmu Komputer (semua kategori) - Sistem Informasi
Articles 219 Documents
 18   19   20   21   22 
Evidence-Calibrated RAG for Unanswerable Question Answering: Retrieval Coverage, Abstention Calibration, and Hallucination-Proxy Analysis on SQuAD 2.0 Ziliang Samuel Zhong; Jing Chen; Eric Zhong; Xinzhuo Sun
Journal of Technology Informatics and Engineering Vol. 4 No. 2 (2025): AUGUST | JTIE : Journal of Technology Informatics and Engineering
Publisher : University of Science and Computer Technology

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.51903/jtie.v4i2.536

Abstract

This paper presents a controlled and reproducible empirical study of evidence-calibrated retrieval-augmented question answering (RAG) for answerable and unanswerable reading-comprehension tasks using the SQuAD 2.0 benchmark. The study focuses on whether a system should abstain when retrieved evidence is insufficient rather than always producing an answer. Six lightweight architectures were evaluated on the full validation set of 11,873 questions, including closed-book, BM25, dense, hybrid, reranked, and a proposed evidence-calibrated hybrid RAG model. The proposed approach combines hybrid top-25 retrieval, lexical reranking, deterministic extractive answering, and evidence sufficiency calibration trained on 43,482 examples. On the validation set, it achieved 31.65% exact match, 34.74% F1, 53.01% answerability accuracy, 53.71% refusal F1, and a 37.49% hallucination-proxy rate. Although overall QA performance remains modest, calibrated evidence sufficiency substantially reduced unsupported answers compared with a forced-answer hybrid reranker, lowering the hallucination-proxy rate from 77.80% while improving F1. However, evidence calibration itself remained weak (AUROC 0.5475, ECE 0.1144). The findings demonstrate that retrieval coverage alone is insufficient to prevent hallucinations and highlight the need for stronger evidence calibration in trustworthy RAG systems.
LLM-Inspired Offline Reranking for Financial Search: Query Rewriting, Hybrid Retrieval, and Listwise Relevance Ranking on FiQA Siquan Meng; Jing Chen; Isa Zheng
Journal of Technology Informatics and Engineering Vol. 5 No. 1 (2026): APRIL | JTIE : Journal of Technology Informatics and Engineering
Publisher : University of Science and Computer Technology

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.51903/jtie.v5i1.537

Abstract

Financial search has high practical value because investors and retail users often ask natural-language questions whose wording differs from relevant financial passages. This paper evaluates a multi-stage retrieval pipeline on FiQA, a financial question-answering retrieval collection in BEIR. The systems include BM25, Dense LSA, BM25-LSA hybrid retrieval, reciprocal-rank fusion, a compact linear reranker, fixed pointwise and listwise relevance rubrics inspired by LLM reranking, query rewriting, and the proposed query rewriting plus hybrid retrieval plus listwise reranking pipeline. The evaluation used the full 57,638-document FiQA corpus, 6,648 available queries, and the 648-query BEIR FiQA test qrels with 1,706 binary relevance judgments. BM25 was the best-performing system, with nDCG@10 = 0.2285, MAP = 0.1863, MRR = 0.2994, and Recall@100 = 0.5207. The proposed full pipeline underperformed BM25. The listwise rubric ranked second on nDCG@10 (0.2228) and improved over the pointwise rubric, suggesting that candidate-list normalization can be useful in this setting. The rubric rerankers are fixed local scoring rules, so these results should be read as an evaluation of LLM-inspired ranking logic rather than as a benchmark of an actual prompt-based LLM reranker. Dense LSA retrieval alone was weak (nDCG@10 = 0.0287), which shows the limitation of a conservative non-neural dense baseline in financial semantic matching. Query rewriting reduced average effectiveness. The findings recommend strong lexical baselines, conservative rewrite gating, and careful evaluation before adopting prompt-based or model-based LLM rerankers in financial search.
Cost-Aware LLM-Style Routing for AIOps Log Analysis: Log Parsing, Anomaly Detection, Fault Diagnosis, and Incident Summarization on LogEval Task Files Chenyu Li; Ge Liu; Zoe Zhao
Journal of Technology Informatics and Engineering Vol. 5 No. 2 (2026): AUGUST | JTIE : Journal of Technology Informatics and Engineering
Publisher : University of Science and Computer Technology

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.51903/jtie.v5i2.538

Abstract

This study investigates a local and cost-aware routing framework for AIOps log analysis using the LogEval benchmark. The evaluation covers four tasks: log parsing, anomaly detection, fault diagnosis, and incident summarization. Instead of relying on external large language model APIs, the experiment implements deterministic local policies that simulate zero-shot and few-shot LLM-style inference under controlled token-cost and latency assumptions. Six approaches were compared: regex normalization, TF-IDF with machine learning, a local character-based classifier, zero-shot policy, few-shot retrieval policy, and a routing cascade. At a risk threshold of 0.20, the router directed only 12.9% of queries to the few-shot retrieval policy while achieving parsing accuracy of 0.991, anomaly F1-score of 1.000, diagnosis accuracy of 1.000, ROUGE-L of 0.743, and BLEU-1 of 0.814. The routing strategy reduced simulated token cost by 80.1% compared with always using few-shot retrieval. Additional unseen-template evaluation revealed limited generalization for closed-label classifiers and retrieval methods when encountering unseen patterns. The findings indicate that routing can effectively reduce AIOps inference costs, while further validation with real LLMs and stronger generalization testing are required before production deployment.
Trajectory Reliability Prediction for Generalist AI Agents: Tool-Use Failure Analysis and Success Forecasting on ZClawBench Ziliang Samuel Zhong; Chenyu Li; Hengning Rao
Journal of Technology Informatics and Engineering Vol. 5 No. 1 (2026): APRIL | JTIE : Journal of Technology Informatics and Engineering
Publisher : University of Science and Computer Technology

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.51903/jtie.v5i1.539

Abstract

Generalist AI agents increasingly perform complex tasks through planning, tool execution, action revision, and artifact generation rather than isolated response generation. This study empirically investigates trajectory reliability prediction on ZClawBench, a public OpenClaw-style agent benchmark containing 696 model-task trajectories across 116 tasks, six model families, and six scenario categories. The study evaluates whether operational trajectory signals—including tool-call volume, replanning behavior, tool errors, invalid-action ratio, recovery patterns, looping indicators, trajectory length, and task category—can predict task success before manual evaluation. Since the dataset provides trajectories and identifiers but lacks explicit per-instance success labels, the binary target is reconstructed from official model-by-category score distributions and treated as a modeling assumption. Five-fold cross-validation with task-level splitting was applied to prevent task leakage. Logistic Regression, Random Forest, XGBoost, sequence TF-IDF classification, and a rubric-based trajectory judge were compared. Logistic Regression achieved the strongest calibrated performance, obtaining ROC-AUC of 0.970, F1-score of 0.913, Brier score of 0.066, and expected calibration error of 0.018. Feature analysis indicated that task difficulty, no-progress behavior, response size, tool errors, and invalid actions contributed most to reliability prediction. The findings suggest that lightweight trajectory diagnostics can support agent monitoring, failure triage, and routing decisions, while further validation with direct case-level evaluation labels is required for deployment claims.
News-Based Uncertainty and Macro-Market Fusion for VIX Direction Forecasting: Evidence from 2015-2024 FRED Panel Hailin Zhou; Kai Zhang
Journal of Technology Informatics and Engineering Vol. 4 No. 2 (2025): AUGUST | JTIE : Journal of Technology Informatics and Engineering
Publisher : University of Science and Computer Technology

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.51903/jtie.v4i2.540

Abstract

This paper evaluates news-based uncertainty and macro-market fusion for one-trading-day VIX direction forecasting using a 2015-2024 daily FRED panel. The dependent variable equals one when the next trading-day VIX close exceeds the current close. The final panel contains 2,512 processed observations from 2015-02-04 to 2024-12-30, with 1,997 training observations from 2015-2022, 257 validation observations in 2023, and 258 holdout observations in 2024. The feature set combines VIX market-state variables, the daily newspaper-based Economic Policy Uncertainty index, the effective federal funds rate, 10-year and 2-year Treasury yields, lagged CPI inflation, lagged unemployment, and interaction terms. Expanding cross-validation on the 2015-2022 training sample gives the highest average ROC AUC to Fusion Random Forest (0.5620). The 2023 validation window selects a weighted fusion ensemble with ROC AUC 0.6005; its weights are fixed before the 2024 test. In the 2024 holdout window, VIX-only Logistic achieves the highest one-day ROC AUC (0.5869) and F1 (0.5448), while the weighted fusion ensemble reaches ROC AUC 0.5638 and F1 0.4786. Event-window diagnostics show that EPU shocks following calm VIX states have a next-day VIX-up rate of 0.5672, compared with 0.4445 on other trading days. The findings support a cautious interpretation: news-based uncertainty contains conditional information, but one-day practical forecasting reliability remains modest and VIX state variables remain the strongest 2024 holdout benchmark.
Numerical-Reasoning Guardrails for a Quant Research Assistant: A Compact Reproducible Benchmark Using SEC and FRED Data Zeyi Li; Kai Zhang; Annie Wong
Journal of Technology Informatics and Engineering Vol. 5 No. 2 (2026): AUGUST | JTIE : Journal of Technology Informatics and Engineering
Publisher : University of Science and Computer Technology

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.51903/jtie.v5i2.541

Abstract

This paper presents a compact reproducible benchmark for evaluating numerical-reasoning guardrails in a quant research assistant. The revised experiment uses a fixed 2026 source snapshot derived from the SEC 2026 Q1 Financial Statement Data Sets and FRED CSV series for VIXCLS, DGS10, DGS3MO and T10Y3M. The benchmark contains 460 tasks: 300 SEC financial-ratio tasks over 50 issuer-period records, 120 FRED VIX and Treasury-rate change tasks, and 40 macro-regime classification tasks. Each answer is evaluated by five programmatic guardrails: numeric consistency, unit correctness, time-window correctness, formula correctness and citation/source consistency. Four controlled response profiles are tested: Naive-RAG, Calculator-Only, Prompted-Checklist and Guarded-Quant. These profiles are deterministic failure-mode controls rather than performance claims about any particular deployed LLM. The empirical results show that arithmetic alone is not sufficient for financial safety: Calculator-Only reaches 79.78% numeric accuracy but only 0.43% all-guardrails pass rate because source, unit, formula and window fields often fail. Guarded-Quant achieves an 88.48% all-guardrails pass rate, 97.17% numeric accuracy, 100.00% unit pass rate, 96.30% window pass rate, 98.26% formula pass rate and 96.30% citation pass rate. The findings support a modest claim: a compact benchmark can make numerical audit failures visible, but it should not be read as evidence of broad quant-assistant reliability without broader data, live model outputs and operational stress tests.
Accounting-Aware Evidence Retrieval for Institutional Due Diligence of Tokenized Trade Receivable RWA Yuanzheng Chen; Sihan Zhou; Emma Lin
Journal of Technology Informatics and Engineering Vol. 4 No. 3 (2025): DECEMBER | JTIE : Journal of Technology Informatics and Engineering
Publisher : University of Science and Computer Technology

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.51903/jtie.v4i3.542

Abstract

Institutional investors evaluating tokenized real-world asset (RWA) transactions need retrieval systems that can answer short, ambiguous, and legally loaded due-diligence questions with traceable evidence. Trade receivable pools are especially difficult because the same question may require accounting policy, financial metrics, footnote disclosure, legal covenants, insurance language, servicer reporting, or waterfall mechanics. This study implements and evaluates an accounting-aware evidence-retrieval pipeline for tokenized trade receivable RWA due diligence. The main experiment uses the official FinDER benchmark with 5,703 query-evidence-answer triples, 6,121 annotated evidence references, and 5,830 deduplicated evidence passages derived from financial disclosures. The pipeline compares vanilla sparse retrieval, accounting-aware query rewriting, feature reranking, section-aware evidence selection, and calibrated abstention. On the official FinDER evaluation, query rewriting increased Recall@10 from 28.25% to 28.62%, reranking increased Recall@10 to 33.86% and answer-support accuracy to 24.57%, and section-aware evidence selection achieved 34.44% Recall@10, 24.04% nDCG@10, 8.32% EvidencePrecision@3, and 25.23% answer-support accuracy. The accounting-relevant subset, defined as Accounting, Financials, and Footnotes, achieved 37.10% Recall@10 and 26.54% answer-support accuracy. A supplementary stress check using a public receivables purchase agreement and SEC 2026-04 financial statement notes showed that the same retrieval logic can surface schedule, lock-box, GAAP, receivable, and note-disclosure evidence, while also highlighting the need for table extraction and field-level numerical validation. The findings support a narrower deployment claim: accounting-aware RAG can improve evidence discovery and analyst review, but it is not yet suitable for autonomous investment or accounting decision-making
Budgeted Multi-Hop Retrieval Agent for Compositional Question Answering: A Retrieval-Policy Evaluation on the Official MultiHop-RAG Benchmark Wenhao Su; Siyu Chen; Chloe Zhao
Journal of Technology Informatics and Engineering Vol. 4 No. 3 (2025): DECEMBER | JTIE : Journal of Technology Informatics and Engineering
Publisher : University of Science and Computer Technology

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.51903/jtie.v4i3.543

Abstract

Multi-hop question answering requires a retrieval system to assemble several complementary evidence documents before an answer module can reason reliably. Single-shot retrieval is efficient, but it often misses later-hop evidence when a question combines source, time, comparison, and entity constraints. This paper evaluates a budgeted multi-hop retrieval agent for compositional question answering on the official MultiHop-RAG benchmark. The benchmark contains 2,556 queries and 609 news-article corpus documents, with answerable evidence distributed across two to four documents. Four retrieval policies are compared under the same sparse lexical scorer: fixed top-k retrieval, iterative retrieval, query decomposition, and the proposed budgeted retrieval agent. The revised evaluation frames the task as retrieval-policy evaluation rather than as a full free-form generative QA system: retrieval-conditioned EM/F1 are reported together with evidence recall, MRR, retrieval rounds, selected documents, and context-token cost. On the official data, the budgeted agent achieves the strongest overall retrieval-conditioned EM/F1 at 62.75% and the highest final evidence recall at 74.67%, using 3.011 average retrieval calls and 509.7 average context tokens. Query decomposition improves over fixed top-k and iterative retrieval but is less stable across question types. Fixed top-k is cheapest but incomplete on longer chains. The four-hop results remain difficult for every policy, showing that a fixed 620-token controller should be extended with hop-aware or dynamic budget allocation. The findings support a moderated contribution claim: explicit budget control is useful for auditable multi-hop retrieval, but it should be evaluated as a cost-accuracy trade-off rather than as a universally dominant RAG architecture.
Accounting-Aware Evidence-Constrained Agents for Disclosure, Settlement, and Secondary-Market Risk Monitoring in Tokenized Sihan Zhou; Yuanzheng Chen; Kenny Lee
Journal of Technology Informatics and Engineering Vol. 5 No. 2 (2026): AUGUST | JTIE : Journal of Technology Informatics and Engineering
Publisher : University of Science and Computer Technology

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.51903/jtie.v5i2.544

Abstract

Tokenized real-world asset (RWA) infrastructure exposes platform operators, investors, and reporting teams to a combined settlement, disclosure, liquidity, and accounting-quality monitoring problem. A tokenized claim can continue to trade while the underlying issuer releases new financial statements, securities fail to deliver in the reference market, or protocol-level liquidity changes in RWA venues. This paper develops an accounting-aware, evidence-constrained agent workflow for risk alerting and source-grounded report generation. The revised experiment replaces the earlier rule-generated monitoring sandbox with external datasets: SEC fails-to-deliver observations, SEC EDGAR XBRL company facts, SEC submissions metadata, Financial PhraseBank sentiment labels, and DefiLlama RWA protocol TVL. The issuer-day panel contains 2,648 surveillance tasks for eight large U.S. issuers from 2024-12-01 through 2026-03-31. Observed settlement stress is defined from external SEC FTD balances rather than from the agent's own rule. Accounting risk is computed from XBRL-derived liquidity, leverage, accrual, and cash-flow indicators. A stronger market-plus-accounting logistic baseline is added alongside single-source baselines and the proposed fusion agent. The machine-learning baseline achieves the strongest F1 score for settlement-stress detection (0.909), while the proposed fusion agent achieves the highest report faithfulness and tool-use correctness (1.000 each) and high recall (0.849). The results support a governance-oriented interpretation: an evidence-constrained agent is most useful not as an opaque high-accuracy classifier, but as an auditable layer that connects settlement evidence, filing metadata, accounting fundamentals, independent sentiment calibration, and RWA protocol liquidity into a reproducible monitoring record.
Profit-Aware Spot GPU Admission Control with Cost-Sensitive Loss and Evidence-Grounded Policy Memos for AI Workload Supply-Demand Matching Siming Zhao; Yuxuan Ren; Xiaohan Chang
Journal of Technology Informatics and Engineering Vol. 5 No. 2 (2026): AUGUST | JTIE : Journal of Technology Informatics and Engineering
Publisher : University of Science and Computer Technology

Show Abstract | Download Original | Original Source | Check in Google Scholar | DOI: 10.51903/jtie.v5i2.545

Abstract

AI clusters increasingly operate with heterogeneous GPU resources where production workloads and opportunistic spot jobs compete for limited accelerator capacity. This study presents a trace-driven admission-control framework using the Alibaba cluster-trace-v2026-spot-gpu dataset, consisting of 466,867 job records and 4,278 GPU-node records. The experiment evaluates GPU demand forecasting, profit-aware spot admission control, and evidence-grounded operational policy generation using chronological training, validation, and test splits. Hourly spot GPU demand forecasting was evaluated across six GPU models, where Ridge regression achieved the best test performance with an RMSE of 38.50 requested GPUs per hour, improving over both last-hour and seasonal naive baselines. The admission-control evaluation compared FIFO, greedy packing, classifier-based acceptance, utility ranking, and the proposed cost-sensitive policy. The proposed approach achieved a test profit of 67,278.96, improving 1.97% over the accuracy-oriented classifier while increasing spot success rate and reducing costly false acceptances by 13.17%. Sensitivity analysis showed that the optimal policy depends on the protection cost assigned to high-priority workloads. A deterministic evidence-grounded explanation layer generated 500 policy memos and passed numeric, policy, and evidence consistency checks. The findings suggest that profit-aware admission control can serve as a practical scheduling guardrail before detailed GPU placement and resource allocation decisions.

Page 21 of 22 | Total Record : 219

 18   19   20   21   22 

Filter by Year

2022 2026


Reset
Filter By Issues
All Issue Vol. 5 No. 2 (2026): AUGUST | JTIE : Journal of Technology Informatics and Engineering Vol. 5 No. 1 (2026): APRIL | JTIE : Journal of Technology Informatics and Engineering Vol. 4 No. 3 (2025): DECEMBER | JTIE : Journal of Technology Informatics and Engineering Vol. 4 No. 2 (2025): AUGUST | JTIE : Journal of Technology Informatics and Engineering Vol. 4 No. 1 (2025): APRIL | JTIE : Journal of Technology Informatics and Engineering Vol. 3 No. 3 (2024): December (Special Issue: Big Data Analytics) | JTIE: Journal of Technology Info Vol. 3 No. 2 (2024): Agustus : Journal of Technology Informatics and Engineering Vol 3 No 2 (2024): Agustus : Journal of Technology Informatics and Engineering Vol. 3 No. 1 (2024): April : Journal of Technology Informatics and Engineering Vol 3 No 1 (2024): April : Journal of Technology Informatics and Engineering Vol 2 No 3 (2023): December : Journal of Technology Informatics and Engineering Vol. 2 No. 3 (2023): December : Journal of Technology Informatics and Engineering Vol 2 No 2 (2023): August : Journal of Technology Informatics and Engineering Vol. 2 No. 2 (2023): August : Journal of Technology Informatics and Engineering Vol 2 No 1 (2023): April : Journal of Technology Informatics and Engineering Vol. 2 No. 1 (2023): April : Journal of Technology Informatics and Engineering Vol 1 No 3 (2022): Desember: Journal of Technology Informatics and Engineering Vol 1 No 3 (2022): December: Journal of Technology Informatics and Engineering Vol. 1 No. 3 (2022): December: Journal of Technology Informatics and Engineering Vol 1 No 2 (2022): August: Journal of Technology Informatics and Engineering Vol 1 No 2 (2022): Agustus: Journal of Technology Informatics and Engineering Vol. 1 No. 2 (2022): August: Journal of Technology Informatics and Engineering Vol. 1 No. 1 (2022): April: Journal of Technology Informatics and Engineering Vol 1 No 1 (2022): April: Journal of Technology Informatics and Engineering More Issue