arXiv:2609.34007v2 Announce Type: replace
Abstract: Electronic health records (EHRs) encode clinical histories as (time, modality, code) tuples, whereas pretrained language models expect text tokens. Serializing tuples as text inflates sequence length and repeatedly encodes the same structure. We introduce EHRAdapt, an adapter that maps tuples directly into a frozen language model's embedding space. Modality and time gaps enter through a learned embedding and attention biases, and event codes receive dedicated vectors. Learning them is challenging because clinical vocabularies are long-tailed, leaving rare events with few observations. EHRAdapt therefore represents each event vector as the sum of a semantic prior and an evidence residual. The prior is a frozen, ontology-trained biomedical embedding of the event's clinical description, mapped into the input space by a shared learned projection, so it supplies clinical meaning even when observations are scarce. The residual is a learned low-rank, event-specific correction that refines it as evidence accumulates. We perform continued pretraining on records from about 4 million patients with three frozen LLM backbones, OLMo2 1B, Llama3.2 1B, and OLMo2 7B, training only the adapter (0.1--0.6% of all parameters). The full adapter outperforms all ablations in held-out next-event prediction on every backbone. Removing the semantic pathway hurts rare events most, over ten times more than the most frequent ones, whereas removing the residual hurts overall prediction but improves it for the rarest events. On reportable infectious-disease and syndromic classification, EHRAdapt outperforms text-based LLM and count-based baselines and the same LLMs adapted to the same records through next-event prediction, and both pathways improve rare-disease discrimination. The two pathways therefore play complementary roles, visible only when results are broken down by event frequency.
Andre R Goncalves, Vincent Liu, Priyadip Ray
arXiv:2609.29381v2 Announce Type: replace
Abstract: Complex lung cancer decisions can involve several defensible pathways whose eligibility, sequencing and safety depend on unresolved information. Effective support must make explicit how patient conditions govern pathway eligibility, deferral and redirection. MedGPT Clinical Explorer (MCE) organizes alternatives, decision-changing unknowns, safety constraints and fallback into a conditional strategy for clinician review. To evaluate this representation in physician-authored strategies, multidisciplinary experts established case-specific references for 40 cases within a purposive 100-case corpus, and 250 physicians from 98 institutions produced 2,250 strategies under unaided, retrieval-reference and MCE-assisted conditions.
MCE-assisted strategies expressed more applicable clinical requirements, measured by the Admissible Pathway Attainment Score (APAS; 0-100), than unaided strategies (adjusted difference, 12.87; 95% CI, 11.18-14.55) and retrieval-reference strategies (5.22; 3.52-6.93). With the same knowledge base available in the retrieval-reference and MCE-assisted conditions, the additional content centered on candidate pathways, decision-critical information and safety constraints. Physicians' whole-strategy acceptability judgments correlated with APAS (Spearman's rho = 0.671), while a complementary relationship audit assessed whether candidates, conditions and subsequent actions were coherently connected.
Together, these findings identify two complementary dimensions of open-ended decision support: coverage of clinically relevant content and coherent links among pathways, conditions and subsequent actions. MCE provides a shared decision object that makes consequential omissions and pathway contingencies visible before action; prospective studies should evaluate its effects on clinical workflow and patient outcomes.
Daoyun Wang, Zhicheng Huang, Huaiyuan Sun, Jiaqi Xu, Xiaowei Xu, Zhibo Zheng, Zhongxing Bing, Yuxiao Lin, Yicheng Liang, Chao Gao, Bowen Xue, Kai Zhang, Song Xu, Wanpu Yan, Hui Xia, Lin Li, Xiang Yan, Mu Hu, Qianli Ma, Zhiqiang Xue, Xiaofang Liu, Zhihai…
arXiv:2609.17631v2 Announce Type: replace
Abstract: AI-assisted claims can appear authoritative when evidence, analysis, human authorization, presentation, and correction history describe different states. We develop Publication Authority: an exact-state, non-transferable, single-use publication capability within PAC-2026 (Publication-Accountability Calculus), an AIJIM Protocol candidate. The evaluated specification version (SF-4) defines six non-compensating obligations: evidence, runs and artifacts, measurement disclosure, authorization, surface correspondence, and lifecycle continuity. Each yields a target-bound witness, localized counterexample, or localized unverifiability. Only a fresh, complete all-pass record derives a permit consumed by one atomic publication transition. Ten models explored 110,764 distinct safe reachable states; 76 unsafe configurations yielded the expected violation or observer countermodel. A surface passing its correspondence check cannot authorize publication unless admitted by the accepted record. SF-4 separates evidence horizon from verification time and rejects authentic but causally invalid authorization. An in-house, instance-blind study (G7-B) matched 183 scored expectations after shared integration. Three author-led native comparisons identify gaps in a constructed in-toto/OPA profile: measurement-state admission, proof-expiry evaluation, and required-run identity. Repairs restore parity on the tested cases. An exploratory extension constructs exact claim-edit regions under existing e-closure semantics: a minimum retained core qualifies every permitted selection. For the full hypothesis universe (U=H), a quadratic frontier computes all strongest-core sizes. Enumeration and native controls distinguish reusable statistical qualification from the fresh publication authority each candidate requires. Results yield transferable protection rules; independent conformance and field evaluation remain open.
Torsten Olivi Tiltack, Yifei Dong, Kun Yu, Xu Wang, Wei Liu, Jianlong Zhou, Ren Ping Liu, Fang Chen
arXiv:2609.09226v2 Announce Type: replace
Abstract: Understanding the brain's mechanisms is crucial in neuroscience, particularly for simulating natural intelligence and advancing artificial general intelligence (AGI). Liu et al. (2024) and Deco et al. (2025) propose that quantum entanglement may underlie the synchronization of millions of neurons, introducing the Liu-Chen-Ao (LCA) hypothesis of entangled nerve fibers. However, directly testing this idea remains challenging. Here we employ a nonlocal many-body wave equation, introducing a generalized probability-wave framework to model collective behaviors among adaptive, interacting agents, particles, or neurons. This approach enables testable eigenmodes, offering an indirect means of examining the LCA hypothesis by observing patterns in collective trading behaviors. These patterns, rooted in behavioral psychology, are thought to mirror the internal brain's decision-making and underlying mechanisms. Our analysis of Chinese intraday stock market data shows that adaptive entangled game modes explain 82%-94% of decision patterns, sharply contrasting neoclassical finance predictions assuming independent rational agents. Additionally, 2-12% of actions adapt to news, events, and environmental changes, marked by rapid equilibrium shifts, while truly independent behaviors make up less than 5%. These results empirically support the LCA hypothesis and underscore the need to simulate the brain's mechanisms and generate adaptive, entangled modules for AGI. They also expose the shortcomings of conventional artificial neural networks (ANNs) that depend on vast, opaque parameters. By combining ANN-based AI with probability-wave-based entangled brain simulations, machine learning enables the creation of human-like processing units (HPUs). Such HPUs promise more efficient and robust AGI, particularly for embodied intelligence and robotics.
Haochen Li, Xinshuai Guo, Jingdong Ouyang, Wei Zhang, Leilei Shi
arXiv:2609.07299v2 Announce Type: replace
Abstract: Learned world models typically assume that observations arrive synchronously, an abstraction inherited from simulators that return a complete state vector at each environment step. Physical sensing instead operates at heterogeneous rates, leaving most observation channels stale at any given instant. Interpolating stale channels introduces measurements that were never observed, while downsampling to the slowest sensor discards valid measurements. A natural alternative is to zero-order-hold the most recent reading and provide the known sampling schedule to the model through two features, staleness and time-to-refresh. We test this prediction using transformer world models across three regimes of increasing causal coupling: open-loop rollouts in continuous-control locomotion, closed-loop model-predictive planning in which each learned model serves as the planner dynamics, and a linear latched-actuator system in which refresh events apply a zero-order-held command to the plant. Our findings show that the effectiveness of time-to-refresh depends on the causal role of the sampling schedule, specifically when refresh events affect the system rather than merely report its state. These results establish when sampling schedules provide useful information for predictive world models operating under asynchronous physical observations.
Akash Anand, Abhay Anand, Yash Vishe
arXiv:2609.03966v3 Announce Type: replace
Abstract: Tool-use evaluations typically measure calls through a serving stack, which can report zero calls even when model outputs contain call-shaped payloads. We study interface-induced trajectory censoring: a mismatch between model emissions and the interface that converts them into actions. On selected BFCL v4 subsets, changing only the serving adapter while holding model weights and other evaluation settings fixed moves scores under the official scorer from 0.00 to 0.96 on simple_python and from 0.00 to 0.19 on multi_turn_base. A template-parser factorial experiment shows strong complementarity: neither component replacement alone restores parsing from the baseline, whereas their joint replacement recovers 196/200 first-request parses. Across five Qwen2.5-Coder checkpoints, classifier-positive emissions increase with model size while the mismatched interface reports no calls; a separate matched-interface Qwen3 ladder exhibits few silent emissions through 8B. Experiments on 115 tau-bench retail tasks extend the mechanism to a stateful, non-code environment. In a preregistered, single-seed, 150-step 7B LoRA experiment, changing only the parser registered in verl increases tool executions from 10 to 16,844. However, neither a shared programmatic-feedback evaluation nor a post-hoc common native function-calling evaluation detects a held-out success gain from repair; rescue counts do not increase. Interface repair thus restores tool-mediated experience without a detected performance gain in this setting. These findings show that tool-use measurements depend on the model-interface configuration and motivate layer-wise observability and conformance checks before attributing evaluation or training outcomes to the model alone.
arXiv:2608.26950v2 Announce Type: replace
Abstract: Large Language Models (LLMs) are evolving from performing end-to-end mathematical reasoning to integrating agentic intelligence. However, most existing math benchmarks evaluate only final answers. This outcome-oriented evaluation provides limited diagnostic value for identifying process-level failures or rigorous logic, failing to guide the transformation of LLMs into robust agents. To bridge this gap, we present a process-level benchmark designed to evaluate the inherent agentic mathematical reasoning abilities of LLMs. Our framework aligns problem-solving agentic behaviors with a structured taxonomy of reusable mathematical atomic capabilities. We design a comprehensive suite of planning, action, and feedback tasks across both textual and multimodal contexts, supported by an automated pipeline that synthesizes high-quality trajectories and produces fine-grained annotations via controlled LLM rewriting. Experiments reveal that models with similar end-to-end accuracy can exhibit markedly different agentic capability profiles. This demonstrates that process-level evaluation is crucial for interpreting the true potential of LLMs and guiding the development of next-generation mathematical agents.
Jiayi Kuang, Yinghui Li, Yunze Song, Keyu Chen, Zhifeng Shen, Yangning Li, Yidong Wang, Di Yin, Ruizhi Qiao, Xing Sun, Kai Jin, Ying Shen, Liang Lin, Philip S. Yu
arXiv:2608.22887v2 Announce Type: replace
Abstract: Large language models are increasingly used to judge people, and their answers can depend on inputs correlated with a protected attribute such as race, even when that attribute is never shown. Interpreting this reliance requires knowing how much predictive information those inputs add, but audits often report only whether a decision changes. We introduce an evidence-referenced audit that compares proxy reliance with two references, the known outcome rule and a Bayesian linear regression fitted to the same labeled examples, while measuring ranking accuracy separately. We apply it to four models in a simulated patient-ranking task with known field weights. We find four results. First, at zero predictive value, the models still relied on fields that carried no information, while the same-example learner did not. Second, as proxies became predictive, reliance grew far less than the reference. Third, display format mattered. Ordered labels drew more reliance than arbitrary labels, a direction that held in all four models. Fourth, ranking accuracy did not consistently track these changes in reliance. Based on these findings, we argue that decision changes and accuracy cannot show whether proxy reliance is warranted by the evidence. Audits should report reliance against an explicit predictive reference, alongside accuracy. Ordered fields whose values move with group membership, such as scores or income bands, are a useful starting point. Together, these results make it possible to measure how far reliance departs from the evidence, rather than infer it from observed choices alone.
arXiv:2608.14590v3 Announce Type: replace
Abstract: Large Language Model (LLM) agents take irreversible real-world actions, yet no system gives formally grounded, task-level safety guarantees for their plans. In a PRISMA 2020 systematic review, we searched six databases (20-29 May 2026) for work on LLM agent specification, verification, and enforcement published in 2022-2026. We narratively synthesized 38 studies, assessed risk of bias per study, and graded evidence certainty with GRADE. Three authors independently screened a repeated search to validate the single-reviewer screening (agreement: Fleiss' $\kappa = 0.81$). First, specification is the bottleneck: on one benchmark, only 24.8-34.2% of LLM translations from natural language into Planning Domain Definition Language (PDDL) are semantically correct. Second, runtime monitoring is the most mature enforcement strategy; it reduced unsafe actions by 65.37% in one household-agent study, but its guarantees cover only single steps or short horizons. Third, in one $\tau$-Bench study of transactional tool-use agents, blocking 94% of unsafe actions still left fewer than 5% of tasks completed both safely and correctly in most settings, because agents found other unsafe routes (the verifier tax). Fourth, no approach simultaneously achieves soundness, completeness, scalability, semantic grounding, and utility preservation. For seven of the nine principal claims, GRADE gives the lowest certainty rating (Very Low), meaning further studies are likely to produce substantially different figures; each rests on one or two studies in narrow domains. Two of the open problems we rank highest are translating requirements into formal specifications that keep their meaning and enforcing safety over whole plans rather than single actions. We contribute a three-level taxonomy and an agenda of ten open problems.
Pierre Dantas, Lucas Cordeiro, Ehsan Nowroozi, Norbert Tihanyi
arXiv:2608.12002v2 Announce Type: replace
Abstract: While autonomous agents show promise for network operations, existing telecom troubleshooting evaluations fail to capture the partial observability and heterogeneity of real-world telecom environments. We introduce CTBench, a public benchmark evaluating agent troubleshooting capabilities in root cause analysis (RCA) and path restoration. Constructed by domain experts, CTBench features rich metadata and golden evidence trajectories, utilizing metrics that assess both final accuracy and diagnostic reasoning. Our evaluation of state-of-the-art agents reveals that while they excel at path endpoint identification, they severely underperform in RCA. Crucially, even when agents produce correct answers, they frequently fail to gather the requisite diagnostic evidence demanded in operational practice. Finally, we demonstrate that increased computational expenditure does not guarantee improved diagnostic competence, highlighting critical gaps in current agent capabilities.
Xingyu Yan, Tingting Dai, Antonio De Domenico, Mohamed Sana, Nicola Piovesan, Changchang Li, Bowen Liu, Kun Jiang, Mengjie Zhang, Dingcheng Shan, Jing-Cheng Pang, Chenwei Wu, Sijie Wu, Lianying Chao, Haoran Cai, Jiantao Ye, Xubin Li, Simon Mark Lucas,…
arXiv:2607.22602v2 Announce Type: replace
Abstract: Test-time scaling improves large language model reasoning, but spends compute uniformly along a reasoning trace, even in stretches that are already stable. Motivated by the observation that reasoning failures show an early onset of uncertainty, we introduce DeepLook, a training-free decoding framework that monitors segment-level confidence, branches only at uncertainty bottlenecks, and ranks candidate continuations by Average Lookahead Confidence (ALC)---their mean confidence over fixed-horizon rollouts---before completing and voting over a pruned subset. On four competition mathematics benchmarks across four open models, DeepLook matches or exceeds the accuracy of DeepConf-low in 12 of 16 settings while generating 87.3% fewer tokens, showing that a model's own confidence is informative enough to decide where extra decoding compute is worth spending. On knowledge-intensive GPQA-Diamond the accuracy advantage disappears even though the token saving does not: coherent hallucinations also receive high ALC, so internal confidence localizes instability, not falsehood. Code is available at https://github.com/txyang1/deeplook.
Tingxin Yang, Zefeng Wang, Mengyue Wang, Jinhe Bi, Xingcheng Zhou, Volker Tresp, Yunpu Ma
arXiv:2607.20491v4 Announce Type: replace
Abstract: When a financial agent repeats its decision, how much does that tell us about the actions behind it? DFAH-Bench examines this question through replay experiments, a controlled test of omitted tool calls, and banking tasks with authorization checks. Across 570 eligible prospective replay episodes, decisions agreed at rates of 94-95%, while tool paths, including inputs and results, agreed at 45-52%; one condition fell one group short of the planned minimum. In a separate 48-execution diagnostic, deleting the same calls from repeated records left agreement perfect. In banking probes, repeated authorization decisions sometimes conflicted with constructed rule labels. The $\tau$-Knowledge banking study scheduled 1,080 episodes, with 1,033 known outcomes. Structural checks alone completed more primary tasks than either model-based gate-and-recovery bundle. Input limits and recovery behavior materially shape these results. The typed-choice bundle had lower mean episode cost than the generative bundle on complete task pairs, but fewer successes across the primary schedule under every assignment of unknown outcomes. Missing outcomes prevented the planned confirmatory tests, so the banking comparisons are descriptive. Together, the findings show that agreement alone cannot establish reliability: it can conceal changes in execution, systematic omissions, and unsupported decisions. Evaluation therefore needs to connect what the agent did, the evidence and rules governing its action, and whether it completed the task.
arXiv:2606.20208v2 Announce Type: replace
Abstract: Machine learning models are predominantly evaluated through predictive performance metrics such as ranking quality, prediction error, or classification accuracy. While these metrics effectively quantify how closely predictions match the ground truth, they do not assess whether model outputs respect predefined logical or domain-specific constraints. In high-stakes applications, including healthcare, finance, and autonomous systems, logical consistency can be as critical as predictive accuracy, yet no standard metric captures this dimension. We introduce the Rule Violation Score (RVS), a complementary evaluation metric that quantifies the extent to which a predictive model respects a given set of logical rules, independently of predictive accuracy. RVS treats hard rules (strict constraints) and soft rules (statistical regularities) differently, can be evaluated on any dataset and on any predictive model expressed over a relational vocabulary, and can be computed using SQL queries that are automatically generated for Horn rules. Beyond evaluating models, RVS can also evaluate the logical consistency of training datasets and help identify poorly defined rules. We evaluate RVS on three benchmarks covering knowledge graph link prediction and relational regression, including rule-based, embedding-based, and neuro-symbolic predictive models. Our results demonstrate that two models achieving comparable predictive accuracy can exhibit substantially different levels of logical compliance, revealing differences in model behavior that standard metrics fail to capture.
Guillaume Olivier Delplanque (TYREX), Pierre Genev\`es (TYREX), Nabil Laya\"ida (TYREX)
arXiv:2606.16567v3 Announce Type: replace
Abstract: Neural ordinary differential equations (neural ODE) gained attention in safety critical settings such as continuous-time controllers for cyber-physical systems and classifiers integrated into automated decision pipelines, raising the question whether their behavior can be formally verified. Existing tools dedicated to neural ODE provide only a single reachability call without iterative input-set refinement, limiting the precision of their verdicts to whatever one reachability call can deliver. We present TNODEV, the first formal verifier for neural ODE that integrates a falsification checker, a fast interval-based reachability backend based on continuous-time mixed monotonicity, a verification and refinement loop with three input-set splitting heuristics, and a parallel scheduler in a single end-to-end pipeline. TNODEV supports safe-set inclusion verification on pure neural ODE, neural ODE in closed loop with a neural network controller and general neural ODE (GNODE), with the safe set specified either as an interval or as the half-space intersection induced by a target classification label. We evaluate TNODEV on a range of benchmarks across safe-set inclusion and classification-robustness properties, including a direct reachability comparison against NNV 2.0 and CORA and a verification comparison against NNV 2.0 on MNIST general neural ODE classifiers.
Abdelrahman Sayed Sayed, Pierre-Jean Meyer, Mohamed Ghazel
arXiv:2606.10279v2 Announce Type: replace
Abstract: Supervised fine-tuning with synthetic rationales is widely assumed to improve clinical prediction by teaching models not only what to predict but why. We test this assumption on five-year Alzheimer's disease and related dementias (ADRD) prediction from longitudinal health histories. In this setting, we generate training rationales using the correct labels, then fine-tune models to output a rationale followed by a label. Across a large-scale controlled experiment of 504 configurations, we find that Rationale First SFT consistently and substantially hurts predictive performance relative to Label Only fine-tuning. The degradation persists across model families and data scales, and is not resolved by using a reasoning-oriented base model. A fixed-protocol cardiovascular disease (CVD) validation reproduces similar findings. Crucially, the failure is not explained by poor rationale quality: human expert annotation confirms that the generated rationales are medically accurate and faithfully grounded in patient-specific evidence, and few-shot experiments show that the same rationales improve performance when used as inference-time demonstrations rather than training targets. A simple label-ordering intervention clarifies the mechanism: placing the prediction before the same rationale restores predictive performance. This suggests that conditioning predictions on generated rationales contributes to the degradation. We hope our work paves the way toward a more precise understanding of when and how rationale-based supervision helps and when it does not, guiding the responsible development of language models for high-stakes clinical prediction.
Buxin Su, Bingxuan Li, Cheng Qian, Yiwei Wang, Jin Jin, Bingxin Zhao
arXiv:2606.06523v3 Announce Type: replace
Abstract: Equipping Large Language Models (LLMs) to execute reliable multi-step workflows has become a central challenge in artificial intelligence. Despite recent advances in LLMs' agentic capabilities, most agent systems still lack formal methods for specifying, verifying, and debugging their workflow and execution trajectories. This challenge mirrors a long-standing problem in mathematics, where the ambiguity of natural languages (NLs) motivates the development of formal languages (FLs). Inspired by this paradigm, we propose **Lean4Agent**, to the best of our knowledge, the first framework that uses Lean4, a dependent-type FL to model and verify agent behavior. **Lean4Agent** launches **FormalAgentLib**, an extensible Lean4 library for formally modeling and verifying agent workflows' semantic consistency under explicit assumptions, and enabling localization of execution-time failures revealed by trajectories. Building on **FormalAgentLib**, we further develop **LeanEvolve**, which applies results in **FormalAgentLib** to revise workflows to enhance its capability. Extensive experiments on a hard problem subset of SWE-Bench-Verified and a subset of ELAIP-Bench across 5 leading LLMs indicate that the verification-passing workflows outperform the failing ones by an average of **11.94%**, and **LeanEvolve** further improves SWE performance by **7.47%** on average. Furthermore, **Lean4Agent** establishes a foundation for a new field of using expressive dependent-type FL to formally model and verify agent behavior.
Ruida Wang, Jerry Huang, Pengcheng Wang, Xuanqing Liu, Luyang Kong, Tong Zhang
arXiv:2606.05433v3 Announce Type: replace
Abstract: Proposals for Frontier AI rules often use cumulative training compute as the main criterion for identifying high-impact models. Recent governance analyses call zero-knowledge proofs promising but currently impractical at frontier scale [26, 4]. We argue this barrier reflects the current paradigm, not a fundamental limit. Our frontier dense-pre-training design combines a pre-committed specification, inter-node network observations, and live Merkle commitments to intermediate computation. Our reference zero-knowledge virtual machine (zkVM) provides a succinct proof layer with native BF16/FP32 precompiles. We qualitatively compare zkVMs, custom circuits, and direct interactive audits by transferability, soundness, and cost. Public randomness beacons are a separate audit-selection choice.
At challenged positions, proofs check the committed computation against declared BF16/FP32 semantics bit-for-bit. A private training specification targets model-architecture confidentiality. Genesis proofs cover initialisation; then, in-training step proofs verify selected steps across the run. Ex-ante attestations enforce policy-relevant claims as running invariants on every proved step. Together they create a governance-auditable training record. Detection-probability formulas quantify sampling guarantees from the sampling budget and the share of computation altered. We give numerical examples and explain what sampling does not cover.
We note, however, that weight smuggling could reuse or merge weights trained elsewhere without charging that work to the current run. We then argue that regimes with global training budgets are more robust than regimes with per-run FLOP caps. This advantage depends on counting all contributing training. We catalogue remaining research and engineering problems for external contributors.
Pierre Peign\'e, Ky Nguyen, Paul Wang
arXiv:2605.09511v2 Announce Type: replace
Abstract: Many decisions in complex terrain require fast wind estimates at specific locations and heights rather than dense forecasts on fixed grids. We present WindINR, a latent-state implicit neural representation framework for continuous high-resolution local wind queries and correction using sparse observations. A latent-conditioned decoder maps terrain descriptors, a low-resolution background field, and continuous query coordinates to local wind estimates. WindINR separates reusable representation learning from sample-specific latent correction. During training, a privileged encoder infers reference latent states from high-resolution supervision, while a deployable predictor estimates initial states from inference-time inputs. Their discrepancies define a dataset-adaptive Gaussian prior over latent corrections. At inference time, network weights remain fixed, and only the latent state is updated through a regularized objective incorporating sparse observations and their uncertainty. We train WindINR on the large-scale FuXi-CFD dataset and evaluate it using controlled observing system simulation experiments, simulated helicopter trajectories, and real UAV measurements over Troms{\o}. Across these settings, sparse-observation assimilation consistently improves local wind predictions, with WindINR outperforming several baselines, including WindSeer. Updating only a low-dimensional latent state yields a 14-fold speedup over full-network fine-tuning while preserving continuous queries at arbitrary spatial coordinates. WindINR thus provides an efficient interface between kilometer-scale background products, sparse local observations, and local wind queries in complex terrain.
Yi Xiao, Qilong Jia, Hang Fan, Pascal Fua, Robert Jenssen, Xiaosong Ma, Wei Xue
arXiv:2605.07002v2 Announce Type: replace
Abstract: A major bottleneck in characterizing the failure modes of generative AI systems is the cost and time of annotation and evaluation. Consequently, adaptive testing paradigms have gained popularity, where one opportunistically decides which cases and how many to annotate based on past results. While this framework is highly practical, its extreme flexibility makes it difficult to draw statistically rigorous conclusions, as it violates classical assumptions: the number of observations is typically limited (often 10 to 50 cases) and decisions regarding sampling and stopping are made in the midst of data collection rather than based a pre-specified rule. To characterize what statistical inferences can be drawn from highly adaptive audits, we introduce a hypothesis testing framework from two "dueling" perspectives: (i) the model's null that asserts there is no failure mode with performance below a target threshold versus (ii) the auditor's null that asserts they have a sampling strategy that will uncover a failure mode. Leveraging Safe Anytime-Valid Inference (SAVI), we formalize the auditor as conducting "testing by betting," which translates into simultaneous e-processes for testing the dueling null hypotheses. Furthermore, if the auditor is sufficiently powerful, we prove that these two hypotheses are asymptotically inverses of each other, in that passage of such an audit certifies the AI system as being globally robust. Empirically, we demonstrate that our proposed testing procedures maintain anytime-valid Type-I error control, outperform pre-specified testing methods, and can reach statistically rigorous conclusions sometimes with as few as 20 observations.
Siyu Zhou, Patrick Vossler, Venkatesh Sivaraman, Yifan Mai, Jean Feng
arXiv:2604.04074v5 Announce Type: replace
Abstract: AI is accelerating scientific research, but providing timely and trustworthy feedback remains challenging. Such feedback requires evaluating specific claims against verified evidence, helping researchers and autonomous research systems determine which findings can guide subsequent work. Thorough verification is time-consuming, and reviewers under time pressure may overlook or misjudge critical issues. Existing AI reviewers have yet to fill this gap because they often assess claims without adequately verifying the supporting evidence. We introduce FactReview, a framework that verifies claims in papers to provide more trustworthy feedback. It checks claims against the literature and examines derivations, while also executing the paper's code to test empirical claims. Across 300 papers, FactReview ranks best among all evaluated systems on five of seven metrics, achieving 6.8% false support and 79.2% classification accuracy. In a study of 36 papers, reviewers using FactReview detect 9.4 issues per paper compared with 5.6 independently, while spending 74.1% less time. FactReview thus supports more reliable peer review, with the potential to accelerate research iteration. The code is public at https://github.com/DEFENSE-SEU/FactReview.
Ling Yue, Chaoqian Ouyang, Ruijun Huang, Hang Xu, Lu Li, Libin Zheng, Yuchen Liu, Wei Liu, Shaowu Pan, Shimin Di, Min-Ling Zhang
2 minutos 54 segundos ago
cs.AI updates on the arXiv.org e-print archive.
Suscribirse a Fuente de noticias arXiv AI