·
DataBubble
  • Home
  • Models
  • News
  • Compare
  • Boards
  • Pricing
  • About
  • Newsletter
  • Methodology
  • Contact
Latest
Bringing Value Models Back: Generative Critics for Value Modeling in LLM Reinforcement Learning9h◆Subagents vs Agent Skills: Executing Reusable Knowledge for Long-Horizon Agentic Tasks9h◆Distribution-Consistent Inference for Dynamic Sparse Mixture-of-Experts9h◆In RAG We Trust? Measuring Robustness of Retrieval-Augmented Generation Under Document Poisoning9h◆FrontierChallenge: Evaluating Scientific Workflow Completion9h◆IBIB: A Protocol for Measuring Enterprise AI Systems by Serving Route, Not Model Identifier9h◆AgenticGen: Reward-Guided Agentic Video Generation for Advertising9h◆Strangers to Themselves: What Language Models Say About Themselves Is Generic9h◆Omni Interaction Agent Technical Report9h◆CoGReV: A Confidence-Gated Post-Hoc Non-Monotonic Belief Revision Framework for Phishing Website Classification9h◆Bit-Flip Attacks on Vision-Language-Action Models: Action-Decoding Architecture Shapes the Vulnerability9h◆Can Foundation Models Moderate Online Content? Evaluating Instruction- vs. Example-Driven Policy Operationalization9h◆RiLM: Parameter-Efficient Language Modeling via Geodesic Decoding9h◆Tracing Computation Density in LLMs9h◆Cultural Binding Heads in Language Models9h◆Direct Diversity Optimization for Diverse Successful Trajectories in Preference Post-Training9h◆Left-Branching Transformers Excel at Right-Branching Languages: Data Shapes Word Order Preferences in Language Models9h◆Palmyra x6 Technical Report: An Agentic, Tool-Use Model Post-Trained via Anchored Supervised Fine-Tuning9h◆'Ghaib in Translation' aka Unseen Harm: Measuring Cross-Script Safety Inconsistency with 'Missed-in-Urdu' Scores in LLM Hate Speech Detection9h◆DexterSQL: Deep Schema Exploration and Rule-based Correction for Text-to-SQL Generation9h◆Bringing Value Models Back: Generative Critics for Value Modeling in LLM Reinforcement Learning9h◆Subagents vs Agent Skills: Executing Reusable Knowledge for Long-Horizon Agentic Tasks9h◆Distribution-Consistent Inference for Dynamic Sparse Mixture-of-Experts9h◆In RAG We Trust? Measuring Robustness of Retrieval-Augmented Generation Under Document Poisoning9h◆FrontierChallenge: Evaluating Scientific Workflow Completion9h◆IBIB: A Protocol for Measuring Enterprise AI Systems by Serving Route, Not Model Identifier9h◆AgenticGen: Reward-Guided Agentic Video Generation for Advertising9h◆Strangers to Themselves: What Language Models Say About Themselves Is Generic9h◆Omni Interaction Agent Technical Report9h◆CoGReV: A Confidence-Gated Post-Hoc Non-Monotonic Belief Revision Framework for Phishing Website Classification9h◆Bit-Flip Attacks on Vision-Language-Action Models: Action-Decoding Architecture Shapes the Vulnerability9h◆Can Foundation Models Moderate Online Content? Evaluating Instruction- vs. Example-Driven Policy Operationalization9h◆RiLM: Parameter-Efficient Language Modeling via Geodesic Decoding9h◆Tracing Computation Density in LLMs9h◆Cultural Binding Heads in Language Models9h◆Direct Diversity Optimization for Diverse Successful Trajectories in Preference Post-Training9h◆Left-Branching Transformers Excel at Right-Branching Languages: Data Shapes Word Order Preferences in Language Models9h◆Palmyra x6 Technical Report: An Agentic, Tool-Use Model Post-Trained via Anchored Supervised Fine-Tuning9h◆'Ghaib in Translation' aka Unseen Harm: Measuring Cross-Script Safety Inconsistency with 'Missed-in-Urdu' Scores in LLM Hate Speech Detection9h◆DexterSQL: Deep Schema Exploration and Rule-based Correction for Text-to-SQL Generation9h◆
Tag

#pretraining

9 articles tagged #pretraining

arxivJul 29bullish

Seesaw: Accelerating Training by Balancing Learning Rate and Batch Size Scheduling

arXiv:2510.14717v2 Announce Type: replace Abstract: Increasing the batch size during training -- a ''batch ramp'' -- is a promising strategy to accelerate large language model pretraining. While for SGD, doubling the batch size can be equivalent to halving the learning rate, the optimal strategy for

SE1 model#optimization#pretraining#accelerationRead on arxiv →
HomeModelsNews
arxivJul 18bullish

Reachability-Aware Pretraining for Efficient Target-Oriented Path Exploration in Temporal Knowledge Graph Reasoning

arXiv:2607.14886v1 Announce Type: new Abstract: Temporal Knowledge Graph (TKG) reasoning under the extrapolation setting focuses on forecasting future time-stamped events (facts) from historical data in a temporal knowledge graph. Existing approaches, reinforcement learning (RL)-based multi-hop reas

RA1 model#temporal-knowledge-graph#reinforcement-learning#pretrainingRead on arxiv →
arxivJul 18bullish

Function-Aware Fill-in-the-Middle as Mid-Training for Coding Agent Foundation Models

arXiv:2607.12463v2 Announce Type: replace Abstract: Coding agents must integrate external tool returns into ongoing reasoning - a capability that standard left-to-right pretraining on code exposes only in its forward direction. We observe that the action-observation-continuation loop of a coding age

QWQW2 models#pretraining#self-supervised#mid-trainingRead on arxiv →
arxivJun 29

Copy First, Translate Later: Interpreting Translation Dynamics in Multilingual Pretraining

arXiv:2604.17633v2 Announce Type: replace Abstract: Large language models exhibit impressive cross-lingual capabilities. However, prior work analyzes this phenomenon through isolated factors and at sparse points during training, limiting our understanding of how cross-lingual generalization emerges-

MU1 model#multilingual#pretraining#language-modelsRead on arxiv →
arxivJun 19

Characterizing Narrative Content in Web-scale LLM Pretraining Data

arXiv:2606.19468v1 Announce Type: new Abstract: The narrative composition of web-scale LLM pretraining corpora remains largely unexplored even though narrative is a fundamental mode of human communication. We present the first fine-grained study of narrative features in Dolma, a 3-trillion-token ope

NAFADO3 models#narrative-analysis#pretraining#llmRead on arxiv →
arxivJun 12

Small Experiments, Cheaper Decisions: A Case Study in Staged Promotion for Micro-Pretraining

arXiv:2606.11387v1 Announce Type: cross Abstract: Short pretraining runs can reduce experimental cost, but they can also over-promote configurations that only look strong at tiny budgets. We study an auditable staged-promotion protocol for a fixed micro-pretraining runner on two heterogeneous host b

#pretraining#optimization#machine-learningRead on arxiv →
arxivMay 22bullish

Billion-Scale Graph Foundation Models

arXiv:2602.04768v2 Announce Type: replace Abstract: Graph-structured data underpins many critical applications. While foundation models have transformed language and vision via large-scale pretraining and lightweight adaptation, extending this paradigm to general, real-world graphs is challenging. I

GR1 model#graph-learning#foundation-models#pretrainingRead on arxiv →
arxivApr 16bullish

Concrete Jungle: Towards Concreteness Paved Contrastive Negative Mining for Compositional Understanding

arXiv:2604.13313v1 Announce Type: new Abstract: Vision-Language Models demonstrate remarkable capabilities but often struggle with compositional reasoning, exhibiting vulnerabilities regarding word order and attribute binding. This limitation arises from a scarcity of informative samples needed to d

COSL2 models#machine-learning#vision-language#compositional-reasoningRead on arxiv →
arxivApr 6bullish

Contrastive Language-Colored Pointmap Pretraining for Unified 3D Scene Understanding

arXiv:2604.02546v1 Announce Type: cross Abstract: Pretraining 3D encoders by aligning with Contrastive Language Image Pretraining (CLIP) has emerged as a promising direction to learn generalizable representations for 3D scene understanding. In this paper, we propose UniScene3D, a transformer-based e

OPUN2 models#computer-vision#3d-scene-understanding#transformerRead on arxiv →