arxiv1m agobullish

A general optimization solver based on OP-to-MaxSAT reduction

arXiv:2604.21961v1 Announce Type: cross Abstract: Optimization problems are fundamental in diverse fields, such as engineering, economics, and scientific computing. However, current algorithms are mostly designed for specific problem types and exhibit limited generality in solving multiple types of

#optimization #algorithm #research Read on arxiv →

arxiv4d ago

LayerTracer: A Joint Task-Particle and Vulnerable-Layer Analysis framework for Arbitrary Large Language Model Architectures

arXiv:2604.20556v1 Announce Type: cross Abstract: Currently, Large Language Models (LLMs) feature a diversified architectural landscape, including traditional Transformer, GateDeltaNet, and Mamba. However, the evolutionary laws of hierarchical representations, task knowledge formation positions, and

TRGAMA3 models #large-language-models #architecture #interpretability Read on arxiv →

arxivApr 18bullish

Calibrate-Then-Delegate: Safety Monitoring with Risk and Budget Guarantees via Model Cascades

arXiv:2604.14251v1 Announce Type: new Abstract: Monitoring LLM safety at scale requires balancing cost and accuracy: a cheap latent-space probe can screen every input, but hard cases should be escalated to a more expensive expert. Existing cascades delegate based on probe uncertainty, but uncertaint

CA1 model #safety #machine-learning #optimization Read on arxiv →

arxivApr 18

Safe Reinforcement Learning using Action Projection: Safeguard the Policy or the Environment?

arXiv:2509.12833v2 Announce Type: replace Abstract: Projection-based safety filters, which modify unsafe actions by mapping them to the closest safe alternative, are widely used to enforce safety constraints in reinforcement learning (RL). Two integration strategies are commonly considered: Safe env

#reinforcement-learning #safety #optimization Read on arxiv →

arxivApr 17bullish

Preconditioned Test-Time Adaptation for Out-of-Distribution Debiasing in Narrative Generation

arXiv:2603.13683v2 Announce Type: replace Abstract: Although debiased large language models (LLMs) excel at handling known or low-bias prompts, they often fail on unfamiliar and high-bias prompts. We demonstrate via out-of-distribution (OOD) detection that these high-bias prompts cause a distributio

#debiasing #optimization #language-models Read on arxiv →

arxivApr 16bullish

Neural Two-Stage Stochastic Optimization for Solving Unit Commitment Problem

arXiv:2507.09503v4 Announce Type: replace-cross Abstract: This paper proposes a neural stochastic optimization method for efficiently solving the two-stage stochastic unit commitment (2S-SUC) problem under high-dimensional uncertainty scenarios. The proposed method approximates the second-stage reco

NE1 model #optimization #machine-learning #scalability Read on arxiv →

arxivApr 15

On the Convergence Analysis of Muon

arXiv:2505.23737v2 Announce Type: replace-cross Abstract: The majority of parameters in neural networks are naturally represented as matrices. However, most commonly used optimizers treat these matrix parameters as flattened vectors during optimization, potentially overlooking their inherent structu

MUGR2 models #optimization #neural-networks #machine-learning Read on arxiv →

arxivApr 14bullish

Record-Remix-Replay: Hierarchical GPU Kernel Optimization using Evolutionary Search

arXiv:2604.11109v1 Announce Type: cross Abstract: As high-performance computing and AI workloads become increasingly dependent on GPUs, maintaining high performance across rapidly evolving hardware generations has become a major challenge. Developers often spend months tuning scientific applications

RELL2 models #optimization #gpu #performance Read on arxiv →

arxivApr 13bullish

CSAttention: Centroid-Scoring Attention for Accelerating LLM Inference

arXiv:2604.08584v1 Announce Type: cross Abstract: Long-context LLMs increasingly rely on extended, reusable prefill prompts for agents and domain Q&A, pushing attention and KV-cache to become the dominant decode-time bottlenecks. While sparse attention reduces computation and transfer costs, it ofte

#sparse-attention #long-context #machine-learning Read on arxiv →

arxivApr 10bullish

LoRA-DA: Data-Aware Initialization for Low-Rank Adaptation via Asymptotic Analysis

arXiv:2510.24561v2 Announce Type: replace-cross Abstract: LoRA has become a widely adopted method for PEFT, and its initialization methods have attracted increasing attention. However, existing methods have notable limitations: many methods do not incorporate target-domain data, while gradient-based

LOLO2 models #machine-learning #optimization #initialization Read on arxiv →

arxivApr 10bullish

We Still Don't Understand High-Dimensional Bayesian Optimization

arXiv:2512.00170v2 Announce Type: replace Abstract: Existing high-dimensional Bayesian optimization (BO) methods aim to overcome the curse of dimensionality by carefully encoding structural assumptions, from locality to sparsity to smoothness, into the optimization procedure. Surprisingly, we demons

BAGA2 models #optimization #machine-learning #bayesian-methods Read on arxiv →

arxivApr 10bullish

FVD: Inference-Time Alignment of Diffusion Models via Fleming-Viot Resampling

arXiv:2604.06779v1 Announce Type: new Abstract: We introduce Fleming-Viot Diffusion (FVD), an inference-time alignment method that resolves the diversity collapse commonly observed in Sequential Monte Carlo (SMC) based diffusion samplers. Existing SMC-based diffusion samplers often rely on multinomi

#diffusion #monte-carlo #inference Read on arxiv →

arxivApr 10bullish

SAGE: Sign-Adaptive Gradient for Memory-Efficient LLM Optimization

arXiv:2604.07663v1 Announce Type: new Abstract: The AdamW optimizer, while standard for LLM pretraining, is a critical memory bottleneck, consuming optimizer states equivalent to twice the model's size. Although light-state optimizers like SinkGD attempt to address this issue, we identify the embedd

ME1 model #optimization #memory-efficiency #large-language-models Read on arxiv →

arxivApr 9bullish

STQuant: Spatio-Temporal Adaptive Framework for Optimizer Quantization in Large Multimodal Model Training

arXiv:2604.06836v1 Announce Type: new Abstract: Quantization is an effective way to reduce the memory cost of large-scale model training. However, most existing methods adopt fixed-precision policies, which ignore the fact that optimizer-state distributions vary significantly across layers and train

GPVI2 models #optimization #quantization #memory-reduction Read on arxiv →

arxivApr 8bullish

StateX: Enhancing RNN Recall via Post-training State Expansion

arXiv:2509.22630v2 Announce Type: replace-cross Abstract: Recurrent neural networks (RNNs), such as linear attention and state-space models, have gained popularity due to their constant per-token complexity when processing long contexts. However, these recurrent models struggle with tasks that requi

#rnn #state-space #post-training Read on arxiv →

arxivApr 8bullish

HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model Inference

arXiv:2604.05887v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) have advanced unified reasoning over text, images, and videos, but their inference is hindered by the rapid growth of key-value (KV) caches. Each visual input expands into thousands of tokens, causing caches to

QW1 model #multimodal #compression #optimization Read on arxiv →

arxivApr 7bullish

Fine-tuning is Not Enough: A Parallel Framework for Collaborative Imitation and Reinforcement Learning in End-to-end Autonomous Driving

arXiv:2603.13842v2 Announce Type: replace-cross Abstract: End-to-end autonomous driving is typically built upon imitation learning (IL), yet its performance is constrained by the quality of human demonstrations. To overcome this limitation, recent methods incorporate reinforcement learning (RL) thro

PATRDI3 models #autonomous driving #imitation learning #reinforcement learning Read on arxiv →

arxivApr 7bullish

TSPO: Breaking the Double Homogenization Dilemma in Multi-turn Search Policy Optimization

arXiv:2601.22776v2 Announce Type: replace Abstract: Multi-turn tool-integrated reasoning enables Large Language Models (LLMs) to solve complex tasks through iterative information retrieval. However, current reinforcement learning (RL) frameworks for search-augmented reasoning predominantly rely on s

QWQW2 models #reinforcement learning #large language models #reasoning Read on arxiv →

arxivApr 6bullish

Multi-Turn Reinforcement Learning for Tool-Calling Agents with Iterative Reward Calibration

arXiv:2604.02869v1 Announce Type: new Abstract: Training tool-calling agents with reinforcement learning on multi-turn tasks remains challenging due to sparse outcome rewards and difficult credit assignment across conversation turns. We present the first application of MT-GRPO (Multi-Turn Group Rela

QWQWGP5 models · +2 #reinforcement-learning #conversational-ai #benchmark Read on arxiv →

arxivApr 6

Low-Dimensional and Transversely Curved Optimization Dynamics in Grokking

arXiv:2602.16746v3 Announce Type: replace Abstract: Grokking -- the delayed transition from memorization to generalization in small algorithmic tasks -- remains poorly understood. We present a geometric analysis of optimization dynamics in transformers trained on modular arithmetic. PCA of attention

TR1 model #machine-learning #optimization #generalization Read on arxiv →

arxivApr 6

Communication-Efficient Distributed Learning with Differential Privacy

arXiv:2604.02558v1 Announce Type: new Abstract: We address nonconvex learning problems over undirected networks. In particular, we focus on the challenge of designing an algorithm that is both communication-efficient and that guarantees the privacy of the agents' data. The first goal is achieved thr

#machine-learning #optimization #privacy Read on arxiv →

arxivApr 4

How to measure the optimality of word or gesture order with respect to the principle of swap distance minimization

arXiv:2604.01938v1 Announce Type: new Abstract: The structure of all the permutations of a sequence can be represented as a permutohedron, a graph where vertices are permutations and two vertices are linked if a swap of adjacent elements in the permutation of one of the vertices produces the permuta

#language #optimization #research Read on arxiv →

arxivApr 2bullish

SkillRouter: Skill Routing for LLM Agents at Scale

arXiv:2603.22455v4 Announce Type: replace Abstract: Reusable skills let LLM agents package task-specific procedures, tool affordances, and execution guidance into modular building blocks. As skill ecosystems grow to tens of thousands of entries, exposing every skill at inference time becomes infeasi

SK1 model #machine-learning #benchmark #routing Read on arxiv →

arxivApr 2bullish

Learning to Shuffle: Block Reshuffling and Reversal Schemes for Stochastic Optimization

arXiv:2604.00260v1 Announce Type: new Abstract: Shuffling strategies for stochastic gradient descent (SGD), including incremental gradient, shuffle-once, and random reshuffling, are supported by rigorous convergence analyses for arbitrary within-epoch permutations. In particular, random reshuffling

LA1 model #optimization #machine-learning #research Read on arxiv →

arxivApr 2bullish

Beyond Softmax and Entropy: Convergence Rates of Policy Gradients with f-SoftArgmax Parameterization & Coupled Regularization

arXiv:2601.12604v2 Announce Type: replace Abstract: Policy gradient methods are known to be highly sensitive to the choice of policy parameterization. In particular, the widely used softmax parameterization can induce ill-conditioned optimization landscapes and lead to exponentially slow convergence

#machine-learning #optimization #policy-gradient Read on arxiv →

arxivApr 2

Reconsidering Dependency Networks from an Information Geometry Perspective

arXiv:2604.01117v1 Announce Type: new Abstract: Dependency networks (Heckerman et al., 2000) provide a flexible framework for modeling complex systems with many variables by combining independently learned local conditional distributions through pseudo-Gibbs sampling. Despite their computational adv

#machine-learning #research #optimization Read on arxiv →