Dooly: Configuration-Agnostic, Redundancy-Aware Profiling for LLM Inference Simulation

1 día 22 horas ago
arXiv:2605.07985v3 Announce Type: replace-cross Abstract: Selecting the optimal LLM inference configuration requires evaluation across hardware, serving engines, attention backends, and model architectures, since no single choice performs best across all workloads. Profile-based simulators are the standard tool, yet they hardcode their operation set to a specific configuration and re-profile every operation from scratch, making exploration prohibitively expensive. This cost stems from a missing structural understanding: every input dimension of each operation is fixed by the model configuration or determined by the incoming request. Many model-configuration values (e.g., head size, layer count) recur across models, so the same operation runs in many configurations; a single sweep over the request-dependent dimensions can serve them all. We present Dooly, which exploits this structure to achieve configuration-agnostic, redundancy-aware profiling. Dooly performs a single inference pass, labels each input dimension with its origin via taint propagation, and selectively profiles only operations absent from its latency database; stateful operations such as attention are isolated by reusing the serving engine's own initialization code, eliminating manual instrumentation. From this database, it builds latency regression models that serve as a drop-in backend for existing simulators. Across two GPU platforms, three attention backends, and diverse model architectures, Dooly achieves simulation accuracy within 5% MAPE for TTFT and 8% for TPOT while reducing profiling GPU-hours across 12 models by 56.4% compared to the existing profiling approach. We have open-sourced Dooly at https://dooly-project.github.io.
Joon Ha Kim, Geon-Woo Kim, Anoop Rachakonda, Daehyeok Kim

ProMORNA: An End-to-End Generative Framework for Full-Length mRNA Design

1 día 22 horas ago
arXiv:2605.01513v2 Announce Type: replace-cross Abstract: Designing therapeutic messenger RNA (mRNA) requires generating full-length transcripts that optimally balance intracellular stability and translation efficiency. While generative models have advanced mRNA design, existing approaches rely on region-wise generation, expensive inference-time search, or target-specific model updates. To overcome these limitations, we introduce ProMORNA, a zero-shot, end-to-end generative framework for full-length mRNA design. ProMORNA aligns the design process with modern Large Language Model training pipelines: it utilizes a BART-style architecture pre-trained on over 6 million natural protein-mRNA pairs, followed by a two-stage post-training process comprising cold-start supervised fine-tuning and multi-objective reinforcement learning. By standardizing and aggregating relative advantages at the individual metric level with domain-specific stabilization, our post-training stage directly aligns the generative policy with conflicting biological constraints while avoiding direct scalarization of raw objective scores. In silico evaluations demonstrate that ProMORNA pushes the Pareto frontier of predicted stability and translation efficiency beyond standard baselines and GEMORNA, a leading state-of-the-art method. In three independent in vitro Fluc2P experiments, Candidate 2 achieved a 12-72h reporter-activity AUC 3.33 times that of the conventional Benchmark (Holm-adjusted $P=0.01285$), while its estimated 1.29-fold advantage over the reported best GEMORNA candidate remained exploratory. These results support ProMORNA as a promising framework for generating functionally active full-length mRNA candidates.
Zixi Shao, Tao Wang, Ye Shen, Tianyi Huang, Yibei Xiao

SUDP: Secret-Use Delegation Protocol for Agentic Systems

1 día 22 horas ago
arXiv:2604.24920v4 Announce Type: replace-cross Abstract: Agentic systems now act with user secrets for APIs, messaging platforms, and cloud services. Today's agent runtimes typically implement authorization by exposure: enabling action means placing a reusable secret, or a reusable artifact derived from it, inside the runtime, so a transient prompt-injection or tool-side compromise becomes durable account compromise. Existing defenses cover adjacent pieces such as secret storage, scoped delegation, sender-constrained tokens, and runtime monitoring, but none states the combined requirement: an untrusted autonomous requester should be able to cause a user-authorized secret-backed operation without gaining reusable authority over it. We formalize this as the Agent Secret Use (ASU) problem and identify seven security properties any solution must satisfy, spanning authorization integrity and secret confidentiality. We propose the Secret-Use Delegation Protocol (SUDP), in which a requester proposes a canonical operation, the user authorizes it with a fresh authenticator-backed grant, and a custodian redeems the grant to perform the bounded use; reusable authority never crosses the requester boundary. We specialize SUDP for LLM-driven agents, where it applies whenever a tool call would exercise user-enrolled authority-bearing material. Under standard cryptographic assumptions, SUDP satisfies six of the seven properties; the seventh, runtime-memory confidentiality, is closed by composition with a hardware-rooted runtime. A reference implementation, available at https://github.com/xhyumiracle/sudp, adds sub-millisecond protocol overhead per mediated call.
Xiaohang Yu, Hejia Geng, Xinmeng Zeng, William Knottenbelt

On the Complementarity of Quantum and Classical Features: Adaptive Hybrid Quantum-Classical Feature Fusion for Breast Cancer Classification

1 día 22 horas ago
arXiv:2604.22903v2 Announce Type: replace-cross Abstract: The integration of quantum machine learning with classical deep learning offers promising avenues for medical image analysis by mapping data into high-dimensional Hilbert spaces. However, unifying these paradigms remains challenging due to optimization asymmetries. We propose a hybrid quantum-classical architecture for breast cancer diagnosis based on a dual-branch feature-extraction pipeline. Our framework unifies complementary representations from classical models and trainable or deterministic (non-trainable) quantum circuits. Three fusion strategies are introduced: Static Hybrid Fusion (SHF) for offline extraction, Dynamic Hybrid Fusion (DHF) for end-to-end co-adaptation, and Temperature-Scaled Hybrid Fusion (TSHF). Inspired by multimodal learning, TSHF uses a learnable scalar to balance branch gradient dynamics and address optimization bottlenecks. We evaluate BreastMNIST, INbreast, and BUS-UCLM, spanning ultrasound and mammography, using ResNet-18 and a shallow CNN, with five independent runs per configuration. Fusion improves over the strongest classical baseline on all three benchmarks. On BreastMNIST, TSHF with ResNet-18 and a non-trainable circuit achieves 88.33% accuracy, 92.09% F1-score, and 92.09% AUC-ROC, versus 84.74% accuracy and 87.08% AUC-ROC for ResNet-18 alone. On INbreast, SHF with the same pairing performs best, at 70.42% accuracy, 72.96% F1-score, and 80.28% AUC-ROC. On BUS-UCLM, SHF with a trainable circuit achieves 78.46% accuracy and 84.66% F1-score, over seven accuracy points above the best standalone branch. Adaptive fusion is not universally beneficial: static fusion performs best on the two smaller datasets, suggesting an additional estimation burden from co-adaptation and learned calibration under limited data. Clinical applicability requires validation beyond the small, low-resolution datasets and simulated quantum circuits considered here.
Yasmin Rodrigues Sobrinho, Jo\~ao Renato Ribeiro Manesco, Jo\~ao Paulo Papa

Fine-Tuning Small Reasoning Models for Quantum Field Theory

1 día 22 horas ago
arXiv:2604.18936v2 Announce Type: replace-cross Abstract: Despite the growing application of large language models (LLMs) to theoretical physics, there has been little academic exploration of how domain-specific physics reasoning ability develops during training. To investigate this, we perform the first academic comparison of reinforcement learning and supervised fine-tuning for LLMs on verifiable problem-solving tasks in theoretical physics. Because the open-source verifiable training data required to develop such capabilities is scarce, we built a robust data generation pipeline that can both create synthetic problems and make existing human-authored problems suitable for model training. Selecting quantum field theory (QFT) as our primary domain, we generated over 2,500 synthetic problems alongside a curated collection of human-adapted problems sourced from arXiv and standard pedagogical resources. We conduct both reinforcement learning (RL) and supervised fine-tuning (SFT) experiments, benchmarking performance gains as well as generalization to other physics domains. Although both methods improve over the base model on every dataset, SFT gains are concentrated on in-distribution synthetic tasks, while RL generalizes better, outperforming SFT on the human-adapted datasets and TPBench. We perform an extensive analysis of model chain-of-thought (CoT) reasoning before and after fine-tuning to understand how reasoning errors evolve during RL and SFT. Finally, we publicly release our data pipeline, verifiable QFT training data, and ~200M tokens of QFT reasoning traces.
Nathaniel S. Woodward, Zhiqi Gao, Yurii Kvasiuk, Kendrick M. Smith, Frederic Sala, Moritz M\"unchmeyer

On the Expressive Power and Limitations of Multi-Layer SSMs

1 día 22 horas ago
arXiv:2604.14501v3 Announce Type: replace-cross Abstract: We study how depth, finite precision, state dimension, and chain-of-thought (CoT) affect the expressive power of multi-layer state-space models (SSMs). For the explicit-table $K$-function-composition problem, a canonical benchmark for sequential information propagation, we prove that any $L$-layer SSM solving $(L+3)$-function composition must satisfy $d^2p=\Omega(N/L^3)$, where $d$ is the state dimension and $p$ is the per-scalar precision. Conversely, $K$-function composition is solved exactly by a $(K+1)$-layer generalized SSM with $d=1$ and $p=\Theta(\log N)$. This gives a worst-case depth hierarchy for this formal problem family. We then distinguish post-input reasoning, in which all thought tokens are generated after the input, from input-interleaved reasoning, in which thought tokens may be inserted while the input stream is being read. Post-input reasoning does not circumvent our communication-based lower-bound pipeline, whereas input-interleaved reasoning admits bidirectional simulations with general deterministic one-pass streaming algorithms at the granularity of persistent memory. Finally, width and precision are not interchangeable under exact step-preserving simulation in the base affine-state model, but become interchangeable through the streaming-memory characterization once input-interleaved reasoning is allowed.
Nikola Zubi\'c, Qian Li, Yuyi Wang, Davide Scaramuzza

It Takes a Forum: Resources & Stakeholders Sustaining the Online Synthetic Non-Consensual Intimate Image Creation Ecosystem

1 día 22 horas ago
arXiv:2604.12190v2 Announce Type: replace-cross Abstract: Many malicious actors responsible for disseminating synthetic non-consensual intimate imagery (SNCII) operate within internet forums to exchange resources, strategies, and generated content across multiple platforms. Technically-sophisticated actors gravitate toward certain communities (e.g., 4chan), while lower-sophistication end-users are more active on others (e.g., Reddit). To characterize key stakeholders in the broader ecosystem, we perform an integrated analysis of multiple communities, analyzing 282,154 4chan comments and 78,308 Reddit submissions spanning 165 days between June and November 2025 to characterize involved actors, actions, and resources. We uncover a litany of resources both shared and discussed by myriad actors on both forums, including 156 low-ranking adaptation (LoRA) files used to finetune models to target 141 female celebrities (on 4chan), and links to 54 commercial nudification applications (on Reddit). Our results illustrate: (a) that users with differing levels of technical sophistication employ and share a wide range of primary resources facilitating SNCII content creation as well as numerous secondary resources facilitating dissemination; and (b) that knowledge transfer between experts and newcomers facilitates propagation of these illicit resources. Based on our empirical analysis, we identify gaps in existing SNCII regulatory infrastructure and synthesize several critical intervention points for bolstering deterrence.
Bernardo B. P. Medeiros (University of Florida), Malvika Jadhav (University of Florida), Allison Lu (University of Florida), Tadayoshi Kohno (Georgetown University), Vincent Bindschaedler (University of Florida), Kevin R. B. Butler (University of Florida)

Generalization and Scaling Laws for Mixture-of-Experts Transformers

1 día 22 horas ago
arXiv:2604.09175v2 Announce Type: replace-cross Abstract: We develop a theory of generalization and scaling for Mixture-of-Experts (MoE) Transformers that cleanly separates \emph{active} per-input capacity from routing combinatorics. By conditioning on fixed routing patterns and union-bounding across them, we derive a sup-norm covering-number bound whose metric entropy scales with the active parameter budget and incurs a MoE-specific routing overhead. Combined with a standard ERM analysis for squared loss, this yields a generalization bound under a $d$-dimensional manifold data model and $C^\beta$ targets, showing that approximation and estimation trade off as in dense networks once active parameters are accounted for appropriately. We further prove a constructive approximation theorem for MoE architectures, showing that, under the approximation construction, error can decrease either by scaling active capacity or by increasing the number of experts, depending on the dominant bottleneck. From these results we derive neural scaling laws for model size, data size, and compute-optimal tradeoffs. Overall, our results provide a transparent statistical reference point for reasoning about MoE scaling, clarifying which behaviors are certified by worst-case theory and which must arise from data-dependent routing structure or optimization dynamics.
Mansour Zoubeirou a Mayaki

The Stepwise Informativeness Assumption: When Do Entropy Dynamics Track Reasoning Correctness in LLMs?

1 día 22 horas ago
arXiv:2604.06192v2 Announce Type: replace-cross Abstract: Recent work uses entropy-based signals at multiple representation levels to study reasoning in large language models, but the field remains largely empirical. From a theoretical perspective, reductions in model uncertainty do not necessarily reflect progress toward correctness. In this work, we characterize when such alignment can be expected through the Stepwise Informativeness Assumption (SIA), which requires entropy-reducing reasoning-prefix extensions increase the probability assigned to valid answers in expectation. We show that next-token prediction can transfer answer-relevant information from training data to the model, but does not, by itself, guarantee alignment with correctness. Instead, we propose that this alignment depends on how the training regime couples reasoning traces to correct answers. Furthermore, we hypothesize observable signatures of this phenomenon and characterize its expected failure regimes. We empirically investigate how entropy dynamics relate to correctness across four reasoning benchmarks (GSM8K, ARC, SVAMP, MATH) and diverse open-weight language models spanning base, instruction-tuned, and RL-trained regimes. Our results show that entropy-correctness alignment strengthens with post-training but can weaken when task difficulty exceeds model capabilities.
Mar Gonz\`alez I Catal\`a, Haitz S\'aez de Oc\'ariz Borde, George D. Monta\~nez, Pietro Li\`o

MCLR: Improving Conditional Modeling via Inter-Class Likelihood-Ratio Maximization and Unifying Classifier-Free Guidance with Alignment Objectives

1 día 22 horas ago
arXiv:2603.22364v4 Announce Type: replace-cross Abstract: Diffusion models achieve strong generative performance but often rely on classifier-free guidance (CFG), an inference-time intervention that modifies the sampling trajectory. Since standard denoising score matching (DSM) should recover the target distribution when the score is learned accurately, the practical necessity of CFG suggests deficiencies in learned conditional scores. We identify weak inter-class separation as one such limitation: samples generated from the same noise under different labels can retain similar class-agnostic structure. To address this deficiency, we introduce MCLR, a principled fine-tuning objective that explicitly maximizes inter-class likelihood-ratios, thereby strengthening class-specific structures under standard sampling. Across diverse models and datasets, MCLR substantially improves guidance-free conditional generation and narrows the gap to CFG. Theoretically, we prove that the CFG-guided score is the unique minimizer of a sample-adaptively weighted, ELBO-approximated MCLR objective, interpreting CFG as an implicit inference-time contrastive alignment procedure. We further compare MCLR with related contrastive objectives, including DPO, CCA, and CFM, clarifying their connections and key differences in an otherwise fragmented literature.
Xiang Li, Yixuan Jia, Xiao Li, Jeffrey A. Fessler, Rongrong Wang, Qing Qu