A False Average: Pooled CoT-Monitor Accuracy Conceals a Reasoning-Dependent Fragility

6 días 1 hora ago
arXiv:2608.00583v4 Announce Type: replace-cross Abstract: Chain-of-thought (CoT) monitors are reported by their aggregate accuracy on a pool of reward hacks. We show that this number is a false average. On Terminal Wrench, about 77% of hacks are given away by the actions alone, and the monitor's pooled accuracy is dominated by them; on the remaining 23%, which an action-only monitor misses, the same monitor is fragile. We expose the fragility with a controlled attack: we rewrite only the agent's reasoning to read as good-faith engineering, leaving every command and output byte-identical. One gradient-free rewrite drops a held-out monitor's catch rate on that subset from about 95% to between 4 and 11%, while the pooled rate falls only about 25 points, near what the subset's share of the pool times its collapse predicts. The rewrite stays truthful about what happened and lies only about intent, so the trace-only monitors we test recover little, even when warned about the attack; of our defenses, only information from outside the trace helps much. The attack transfers across monitor and agent families, and the dataset's own hack labels show the same gradient with no monitor in the definition. Live agents told to hack on 27 tasks, with no monitor feedback, produce hollow hacks, but a monitor calibrated on each model's own honest solves catches nearly all of them. Applying the same rewrite to these live hacks keeps all 111 action-visible ones caught and lowers catch on the 16 whose actions score below 0.5 only from 16 to 13: the effect lands where the false average predicts, but the collapse stays a controlled result. A linear probe on a surrogate monitor's activations out-reads its verdict on the obfuscated hacks (AUC up to 0.98), but a behavioral control shows this edge does not, by itself, mean the monitor knows more than it says. Monitor accuracy should be reported on the reasoning-dependent subset, where the defense is load-bearing.
Shikhar Shiromani, Leo Richter, Noah Siegel

What the Abstention Option Measures: Refusal Loss, Sampling Churn, and Option Capture in Distilled Small Language Models

6 días 1 hora ago
arXiv:2607.28639v3 Announce Type: replace-cross Abstract: Small instruction-tuned language models reach phones, cars, and offline assistants through knowledge distillation, and whether a distilled model is fair enough to ship is often decided by a multiple-choice bias score on a benchmark that offers an abstention option. Three mechanisms move behavior around the abstention option: refusal loss, where the student stops declining on items its pre-distillation reference declined; sampling churn, the same change observed between two runs of the reference itself; and option capture, where the student takes an abstention option whenever one is offered. Injecting each mechanism into a reference's own records shows that aggregate scores see only outright loss, per-item scores see churn and remapping, and only conditioning on committed answers stops moved refusal mass from reading as fairness. Resampling a reference across inference seeds already turns 10% to 27% of its refusals into stereotype answers, so a per-item refusal metric reported without that floor overstates damage. Option capture is what corpus repair teaches: students trained with refusal exemplars take an offered abstention option on up to 95% of science questions where abstention is never correct, and the habit carries to three further benchmarks. Free-form evaluation shows that the refusals multiple choice loses are not where free-form stereotyping occurs. We distill these findings into a per-condition report anchored to the reference, with measured floors and ceilings, and apply it to four student families against a fine-tuning control: distillation keeps fewer of the reference's refusals than fine-tuning on the same prompts where the reference's refusals are reliable, by a few points, while the common metrics report improvements whenever refusal mass moves. A bias score quoted for a distilled model should say which of these it reports.
Plawan Kumar Rath

Towards Practical Algorithm Selection for Unsupervised Domain Adaptation in Medical Imaging

6 días 1 hora ago
arXiv:2607.28125v3 Announce Type: replace-cross Abstract: Numerous unsupervised domain adaptation (UDA) algorithms exist, but for clinical practice, selecting the best-suited one along with proper hyperparameters often remains unclear, as the unlabeled deployment (target) domain prevents direct evaluation. We propose a label-free criterion that jointly selects the algorithm and hyperparameters for UDA. Given a pool of candidate models from multiple algorithms trained with different hyperparameters, our approach scores each candidate against an agreement reference, and selects the one with the highest score. The agreement reference is constructed in two levels without using target labels. First, we leverage multiple label-free selection signals, using each to nominate a model within every algorithm. Second, the nominated models are aggregated across algorithms to form a reference prediction for each unlabeled target sample. The candidate whose predictions agree most with this reference is then selected for deployment. Experimental results on four brain MRI and four chest X-ray datasets across seven clinically relevant transfer scenarios show that our method achieves better selection performance than other methods and remains effective across different algorithm pools. Our approach takes a step towards practical, label-free algorithm selection for clinical deployment of UDA.
Yiheng Xiong, Luisa Gall\'ee, Daniel Santak Wolf, Heiko Hillenhagen, Michael G\"otz

SpectONet: A Physics-Guided Spectral Deep Operator Network for Euler-Bernoulli Beam Dynamics

6 días 1 hora ago
arXiv:2607.25790v2 Announce Type: replace-cross Abstract: This paper proposes a novel physics-guided spectral deep operator network, termed SpectONet, for solving Euler-Bernoulli beam (EBB) vibration problems. The proposed framework integrates the operator-learning capability of DeepONet with physics-informed constraints and Chebyshev-Gauss-Lobatto (CGL) sensor placement. Unlike conventional DeepONet frameworks, which commonly employ uniformly distributed sensors, SpectONet uses nonuniform spectral sensor locations with a higher concentration of points near the domain boundaries. This sampling strategy improves the finite-dimensional representation of boundary-sensitive structural responses while requiring only a limited number of branch-network inputs. The governing beam equation, together with the associated initial and boundary conditions, is incorporated into the training objective to promote physically consistent and generalizable predictions. Numerical experiments on three EBB vibration problems demonstrate the effectiveness of the proposed framework. Comparisons with strong baseline models such as Vanilla DeepONet, PI-DeepONet, PINN, and CNN-UNet show that SpectONet consistently achieves lower prediction errors across all considered evaluation metrics. In particular, SpectONet achieves an average reduction of approximately 79% in the mean relative $\mathcal{L}_2$ error compared with the best-performing baseline across the three benchmark problems. These results demonstrate the effectiveness of SpectONet as an accurate physics-guided operator-learning framework for structural vibration analysis.
Shivani Saini, Ramesh Kumar Vats, Arup Kumar Sahoo

GTIN: A Unified Scalable Framework for Joint Event and Time Prediction in Temporal Graphs

6 días 1 hora ago
arXiv:2607.23556v3 Announce Type: replace-cross Abstract: Temporal graphs are increasingly used to model dynamic systems across diverse domains, including social, financial, and traffic networks. Predicting both what the next event will be and when it will occur in these systems is crucial for understanding and anticipating complex behaviors, but has not been studied much yet, especially in large-scale graphs. To address this gap, we propose a scalable unified mathematical framework that captures varying degrees of complexity across temporal graphs with events occurring on the edges. Our framework is flexible and expressive enough to accommodate a wide range of network structures and temporal dynamics. Building upon this analysis, we introduce our novel scalable approach for jointly predicting the next event and its occurrence time. Empirical evaluations across multiple datasets demonstrate that our method consistently outperforms existing techniques, particularly in scenarios involving irregular event patterns and complex temporal dependencies. These findings highlight the potential of our framework as a robust foundation for future research in temporal event prediction.
Mohammad Ostadmohammadi, Sepehr Kazemi, Hamid R. Rabiee

CANDOR: Chance-Calibrated Neighborhood Discordance in Frozen Encoders for Medical Imaging

6 días 1 hora ago
arXiv:2607.18451v4 Announce Type: replace-cross Abstract: In medical imaging, one frozen foundation encoder is often reused for many tasks. The encoder is usually chosen by the area under the receiver operating characteristic curve (AUROC) of a head trained on its features. The AUROC does not measure whether the encoder separates positive from negative images in its feature space. Chance-calibrated neighborhood discordance (CANDOR) measures this separation without training a head. Among images acquired the same way, a positive image is discordant when its k nearest negatives are more similar to it on average than its k nearest positives are. The discordance rate D is the share of discordant positives. When the neighbors come from positive and negative candidate sets of equal size, D has the chance level 1/2 for features without information about the label. With unequal candidate sets, all 11 chest encoders have rates above 1/2 (0.732 to 0.927) on the 12 chest findings of MIMIC-CXR and CheXpert. These rates increase as the finding becomes rarer. With equal sets, the same encoders have rates below 1/2 (0.277 to 0.463). On the chest test split, linear heads miss 0.301 of the discordant positives and 0.165 of the other positives. Across 22 encoders, 605,443 images, and 8 tasks in 7 domains, the best encoder of every task has a rate below 1/2. Because computing D takes one forward pass per image, a hospital can compare candidate encoders before it trains any head.
Soroosh Tayebi Arasteh, Sven Nebelung, Daniel Truhn

Loop the Loopies!

6 días 1 hora ago
arXiv:2607.16051v3 Announce Type: replace-cross Abstract: We present Loopie, the most powerful open-source looped Transformer to date. The Loopie series consists of two Mixture-of-Experts (MoE) models: a 20B-parameter model with 2B active parameters and a 6B parameter model with 0.6B active parameters. Looped Transformers have long faced a challenge: given an N-fold pre-training compute, increasing the parameter count by a factor of N usually outperforms looping N times. Loopie addresses this challenge. Extensive ablation studies, including comparisons with a vanilla 30B-A3B model, show that Loopie substantially outperforms vanilla Transformer baselines trained with the same compute budget. With a novel post-training method, Loopie develops strong reasoning abilities and achieves frontier-level reasoning performance.
Zitian Gao, Yilong Chen, Yihao Xiao, Xinyu Yang, Ran Tao, Joey Zhou, Bryan Dai

Compaction as Epistemic Failure: How Agentic LLM Tools Fabricate Confirmed Results from Killed Processes

6 días 1 hora ago
arXiv:2607.13071v2 Announce Type: replace-cross Abstract: Agentic LLM coding tools compress long session histories into compaction summaries that subsequent sessions inherit as ground truth. This paper documents a failure mode in Claude Code where partial standard output from timed-out commands (exit code 143) is recorded in compaction summaries as confirmed results, propagating false positives across sessions and model versions without re-verification. The underlying mechanism is a conflation of observation and persistence, where information that appeared in the terminal is treated as equivalent to information written to durable storage. This finding extends the analysis of LLM self-evaluation failures reported in prior work on non-determinism in LLM-as-judge grading by showing that agentic tools exhibit analogous reliability deficits when reporting on their own operational outcomes. The failure has direct implications for any workflow that relies on agentic session continuity for data processing, scientific computation, or multi-step automation.
Hiroki Tamba

Safeguard-Conditioned Uplift: Measuring Utility-Risk Frontiers for Dual-Use Biology Assistants

6 días 1 hora ago
arXiv:2607.13039v3 Announce Type: replace-cross Abstract: Refusal rates conflate upstream provider behavior with downstream safeguards and do not reveal what information remains in a delivered answer. Safeguard-conditioned uplift is an evidence-gated framework for evaluating benchmark utility and risk-related proxies along deployed access paths for dual-use biology assistants. It attributes interventions to their source, selects thresholds under a legitimate-access budget, and supports content claims only when answer verifiers pass frozen qualification tests. In a prospectively frozen route and generation study, two of six configurations pass the LAB-Bench-calibrated action-level rule, but both inherit the same Claude Opus provider effect rather than independent downstream gains. Positive selectivity persists on 104 previously unused matched pairs, but benign intervention exceeds the $20\%$ limit. No joint-scoring verifier meets the frozen criteria; a factorial follow-up shows that criterion isolation and ordinal outputs improve end-to-end edit-direction accuracy on controlled template edits. The evidence supports action-level selectivity and verifier interface effects, but not calibrated access, verified content removal, or biological-risk reduction.
Dipesh Tharu Mahato

SLAB: Time-Lag-Aware Deep Reinforcement Learning for Flexible Job-Shop Scheduling in PPVC Module Factories

6 días 1 hora ago
arXiv:2607.11725v2 Announce Type: replace-cross Abstract: Prefabricated prefinished volumetric construction (PPVC) builds room-sized modules in factories that operate as flexible job shops. A module waits 12 to 48 h after casting, waterproofing, and painting, while its station is free for other modules. Existing learned dispatchers release the next operation before this wait ends, so their plans need repair. This paper presents SLAB (Scheduling under time-Lags with an Admissible Bound), a decision process for dispatchers trained by deep reinforcement learning. Its transition releases a module only after its wait, and its reward is the decrease of a proven lower bound that includes the waits. Training on 10-module orders with halved waits also teaches the policy the station contention of larger orders. On three existing networks, repaired plans are 42% to 133% longer than the same network's schedules inside SLAB. On orders generated from Singapore's PPVC guidebook, one trained policy gives schedules 2.2% to 8.1% shorter than the shortest-processing-time rule at 10 to 80 modules. At 50 and 80 modules, it outperforms a time-lag memetic algorithm by 6.7% and 7.4% and constraint-programming search started from that rule's schedule and given 600 s by 6.6% and 7.6%. The memetic algorithm gives shorter schedules on orders of 10 modules, and the same search given 300 s on orders of 10 and 20 modules. A module factory can therefore train one dispatcher on small orders and run it without a solver on production batches of 50 to 80 modules.
Ziheng Zhang, Dhivya Dharshini Kannan, Wei Zhang