This is a remedial run for missed papers from 08/18/2026 to 08/18/2026.

Results generated on 09/13/2026.

Personalized Daily ArXiv Papers 2026-08-19

_{Token counts are not reported for this run. 16 of 18 model calls succeeded, 5,381s of model wall clock.}

Topic Coverage:

Table of contents by topic:

Large-Scale Training Systems and Efficiency (5)

1. Belayer: Efficient Fault Tolerance for LLM Agentic RL Training Authors: Jiecheng Zhou, Qinghao Hu, Peng Sun, Xingcheng Zhang, Weiming Zhang

2. Row-Stochastic Matrices Can Provably Outperform Doubly Stochastic Matrices in Decentralized Learning Authors: Bing Liu, Boao Kong, Limin Lu, Kun Yuan, Chengcheng Zhao

3. Federated Compositional Muon Optimizer for Matrix-Wise Models Authors: Wang Yan, Feihu Huang

4. Efficient Resource Optimization for Split Federated Learning Authors: Wei Wei, Xianhao Chen

5. Accelerating Visual On-Policy Distillation with Batched Speculative Jacobi Rollouts Authors: Bingqi Shan, Zhehao Yu, Kenhong Lin, Baoquan Zhang

Architecture and Training Dynamics (14)

1. Dynamic Compression in Recurrent Networks Authors: Jyothish Pari, Ryan Bahlous-Boldi, Pulkit Agrawal

2. Think Shallow, Solve Deep: Controlling Recurrent Dynamics for Reliable Test-Time Depth Authors: Ivan Viakhirev, Kirill Borodin, Amirah Almutairi, Serguei Barannikov, Maxim Abramov, Grach Mkrtchian

3. Allocating Recurrent Compute in Looped Language Models Authors: Ruhai Lin, Yiyang Guo, Rui-Jie Zhu, Hao Ye, Jason K. Eshraghian

4. Center-Manifold Reduction of Learning at Bifurcations: Interference and Rich Learning in Recurrent Neural Networks Authors: James Hazelden, Eric Shea-Brown

5. Understanding Curriculum Learning in Large Language Models via Cross-Difficulty Optimization Dynamics Authors: Zhikai Ding, Ziyi Ye

6. SingularClip: Preventing Spectral Collapse to Maintain Plasticity in Continual and Reinforcement Learning Authors: Tyler Kastner, Nimrod De La Vega, Amir-massoud Farahmand

7. Demystifying Training-Time Augmentation for Data-Constrained Language Model Pretraining Authors: Michael K. Chen, Xikun Zhang, Fan Bai, Zhengding Hu, Zhen Wang

8. When to Review: Spaced Repetition for Continual Pre-Training of Language Models Authors: Alankar Atreya, Devesh Batra, Yoages Kumar Mantri, Geremy Bantug, Greig A Cowan, Raad Khraishi

9. HyPE-GT: where Graph Transformers meet Hyperbolic Positional Encodings Authors: Kushal Bose, Swagatam Das

10. Low-dimensional topology of deep neural networks Authors: Junyu Ren, Lek-Heng Lim

11. SurgicalMamba: Dual-Path SSD with State Regramming for Online Surgical Phase Recognition Authors: Sukju Oh, Sukkyu Sun

12. Spikformer V2: Join the High Accuracy Club on ImageNet with an SNN Ticket Authors: Zhaokun Zhou, Yijie Lu, Kaiwei Che, Wei Fang, Keyu Tian, Qihao Peng, Yuesheng Zhu, Shuicheng Yan, Yonghong Tian, Li Yuan

13. Where a New Concept Must Enter: Entry Point Gates Cross-Task Usability in Unified Multimodal Models Authors: Zongyang Qiu, Yihan Wu, Kaixuan Fan, Bo Li, Hui Xiong

14. LoopVLA: Learning Sufficiency in Recurrent Refinement for Vision-Language-Action Models Authors: Boyang Shen, Kaixiang Yang, Hao Wang, Qiuyu Yu, Qiang Xie, Qiang Li, Zhiwei Wang

Efficiency, Compression, and Large-Scale Training (6)

1. TileMix: Tile-Centric Mixed-Precision Attention for LLM Inference Acceleration Authors: Hanzhi Zhang, Qiao Zhang, Qinglei Cao, Heng Fan, Yan Huang, Kewei Sha, Yunhe Feng

2. MoNe: Modular Neural Memory for Efficient Long Context Inference Authors: Wonguk Cho, Kyubyung Chae, Tribhuvanesh Orekondy, Sunghyun Park, Hyoungwoo Park, Jeongho Kim, Arash Behboodi, Kyuwoong Hwang, Sungrack Yun

3. Optimize Your Sampling: Tuned Diffusion Sampling with Bayesian Optimization Authors: Travis Zhang, Christian Belardi, Justin Lovelace, Jin Peng Zhou, Saebyeol Shin, Carla P. Gomes, Kilian Q. Weinberger

4. WAM-Diff2: Hierarchical AR-to-Diffusion Distillation for Highly Efficient Autonomous Driving VLA Authors: Zhihao Zhu, Hanlin Shang, Mingwang Xu, Feipeng Cai, Zhuolin He, Yaoyi Li, Jianhua Han, Hang Xu, Siyu Zhu

5. CORAM: Coherent Orthogonal Rotation for Model Merging Authors: Xinyi Sui, Ziran Liu, Nam Ling, Wei Wang, Wei Jiang

6. The Integer Alibi: Localizing Cross-Kernel Divergence in INT8-Quantized LLM Inference Authors: Teng-Ruei Chen

Large-Scale Training Systems and Efficiency (5)

1. Belayer: Efficient Fault Tolerance for LLM Agentic RL Training

ArXiv ID: 2608.14635

Primary Topic: Large-Scale Training Systems and Efficiency

Authors: Jiecheng Zhou, Qinghao Hu, Peng Sun, Xingcheng Zhang, Weiming Zhang

Abstract: Large language model (LLM) agents are increasingly trained with reinforcement learning in long-horizon, sandboxed environments. Unlike conventional RL, agentic RL couples GPU-intensive rollout engines with stateful environment containers whose actions may produce visible side effects, such as file edits, command execution, and dependency installation. A single trajectory can span many rounds of gen- eration and environment interaction, so a component failure can discard completed work or expose the model to an environment state that is inconsistent with its context. However, existing systems lack efficient and correct recovery mechanisms for this distributed execution model. This paper presents Belayer, an efficient fault-tolerant system for LLM agentic RL training. Belayer handles failures in both rollout engines and environment execution while targeting low failure-free overhead. For scoped worker-local rollout failures, Belayer equips each pre-initialized shadow worker with a selective GPU-state reuse protocol that retains independently owned weights and raw KV-arena allocations after owner and GPU health checks, reinitializes worker-local state, and rebuilds request-specific KV contents from logged token prefixes. For environment failures, Belayer introduces full checkpoint and full restore to jointly capture and restore container file-system and runtime state, and coordinates the recovered environment with the LLM context to preserve prefix consistency. An adaptive policy opportunistically overlaps full-state checkpointing with natural LLM inference bubbles when the predicted interval is long enough. Empirical results show low measured overhead during failure-free training, a worker-recovery-time reduction of up to 42 times faster compared with a full engine cold start, and 1.5 to 3.5 times faster recovery from environment failures.

Comment: Selective GPU-state reuse and coordinated environment checkpoints reduce recovery costs while preserving training-state consistency.

Topic Match: Distributed recovery protocols and checkpoint scheduling are substantive training-systems contributions, despite their agentic-RL setting.

Relevance: 8 Novelty: 7

2. Row-Stochastic Matrices Can Provably Outperform Doubly Stochastic Matrices in Decentralized Learning

ArXiv ID: 2511.19513

Primary Topic: Large-Scale Training Systems and Efficiency

Authors: Bing Liu, Boao Kong, Limin Lu, Kun Yuan, Chengcheng Zhao

Abstract: Decentralized learning often involves a weighted global loss with heterogeneous node weights $λ$. We revisit two natural strategies for incorporating these weights: (i) embedding them into the local losses to retain a uniform weight (and thus a doubly stochastic matrix), and (ii) keeping the original losses while employing a $λ$-induced row-stochastic matrix. Although prior work shows that both strategies target the same $λ$-weighted global loss, it remains unclear whether the Euclidean-space guarantees are tight and what fundamentally differentiates their behaviors. To clarify this, we develop a weighted Hilbert-space framework $L^2(λ;\mathbb{R}^d)$ and obtain convergence rates that are strictly tighter than those from standard Euclidean analysis. In this geometry, the row-stochastic matrix becomes \emph{self-adjoint} whereas the doubly stochastic one does not, creating additional \emph{penalty terms} that amplify consensus error, thereby slowing convergence. Consequently, the difference in convergence arises not only from spectral gaps but also from these penalty terms. We then derive sufficient conditions under which the row-stochastic design converges faster even with a smaller spectral gap. Finally, by using a Rayleigh-quotient and Loewner-order eigenvalue comparison, we further obtain topology conditions that guarantee this advantage and yield practical topology-design guidelines.

Comment: Weighted-Hilbert-space analysis yields conditions where row-stochastic decentralized optimization converges faster.

Topic Match: Its contribution is a distributed learning algorithm and convergence analysis with practical communication-topology implications.

Relevance: 7 Novelty: 8

3. Federated Compositional Muon Optimizer for Matrix-Wise Models

ArXiv ID: 2608.12710

Primary Topic: Large-Scale Training Systems and Efficiency

Authors: Wang Yan, Feihu Huang

Abstract: Muon, a more recently developed optimizer, is useful for matrix-wise models in AI areas. Although many works have studied Muon and its variants, these methods are still not particularly well-suited for hierarchical structured problems. To fill this gap, we propose an effective federated compositional Muon (FedCoMuon) optimizer to solve distributed matrix-wise compositional optimization problems. Specifically, our FedCoMuon optimizer builds on compositional gradient tracking and orthogonalized momentum. Moreover, we propose a variance reduced variant of FedCoMuon (FedCoMuon-VR) based on a momentum-based variance reduced technique. In theory, we analyze the convergence properties of our algorithms under the non-i.i.d. and non-convex settings. In particular, we prove that our FedCoMuon-VR obtains a lower sample complexity of $O(ε^{-3})$ for finding an $ε$-stationary solution than the existing FedMuon algorithms. Extensive numerical experiments on robust federated learning and task-distributed risk-sensitive meta learning show that our proposed methods are competitive with existing compositional baselines and achieve the best reported accuracy in several settings.

Comment: Combines compositional gradient tracking, orthogonalized momentum, and variance reduction in a federated matrix optimizer with convergence guarantees.

Topic Match: A distributed optimizer is the core contribution, although evaluation on federated compositional problems leaves large-scale pretraining relevance indirect.

Relevance: 7 Novelty: 7

4. Efficient Resource Optimization for Split Federated Learning

ArXiv ID: 2608.17849

Primary Topic: Large-Scale Training Systems and Efficiency

Authors: Wei Wei, Xianhao Chen

Abstract: Split federated learning (SFL) has emerged as a powerful paradigm for model training at the edge. However, SFL inherently involves discrete decision variables for model splitting and resource allocation, resulting in a challenging mixed-integer problem. Consequently, prior optimization schemes for SFL are either \textit{heuristic} or \textit{computationally inefficient}, which cannot handle large-scale user populations. To address this limitation, this work establishes an efficient optimization framework for SFL under resource-constrained networks. Our framework jointly optimizes model splitting and resource allocation to minimize training cost, which is defined as the weighted sum of latency and energy costs. We first study the model splitting problem and develop a polynomial-time algorithm that achieves the global optimum. Then, we extend the approach to the joint model splitting and resource allocation problem. In this case, we formulate it as a two-dimensional master problem and develop an efficient approximation method with a $(1+ε)$-approximation guarantee. Extensive experiments show that the proposed approach provides efficient solutions to strike the optimal energy--latency tradeoff.

Comment: Provides polynomial-time optimal model splitting and a (1+epsilon)-approximation for joint splitting and resource allocation.

Topic Match: New partitioning and allocation algorithms directly optimize distributed training cost, although the setting is edge federated learning rather than large-model pretraining.

Relevance: 7 Novelty: 7

5. Accelerating Visual On-Policy Distillation with Batched Speculative Jacobi Rollouts

ArXiv ID: 2608.18183

Primary Topic: Large-Scale Training Systems and Efficiency

Also Matches: Efficiency, Compression, and Large-Scale Training

Authors: Bingqi Shan, Zhehao Yu, Kenhong Lin, Baoquan Zhang

Abstract: Visual on-policy distillation (OPD) improves the training of compact visual autoregressive models by learning from trajectories generated by the current student. However, these online rollouts are still produced token by token with autoregressive decoding, which adds substantial cost to every on-policy training step. Speculative Jacobi Decoding (SJD) provides an alternative because it can process multiple tokens in parallel without an auxiliary draft model, but the original method is designed for single-sequence inference. We introduce HB-SJD, a batched SJD rollout backend for visual OPD. HB-SJD allows each image to advance independently according to its own decoding progress, while images at different sequence positions are still verified in batched model forwards. As images finish, HB-SJD switches between Full and Compact execution to reduce the cost of later rollout rounds. HB-SJD only replaces the student rollout backend and leaves the teacher, distillation objective, and optimization procedure unchanged. Experiments with LlamaGen show that HB-SJD substantially reduces rollout and end-to-end training time while preserving the generation quality of the distilled student.

Comment: Batches independently advancing speculative Jacobi rollouts and compacts execution as images finish, reducing on-policy training time.

Topic Match: The core contribution is a training-rollout execution schedule; parallel decoding and adaptive compaction also directly address computational efficiency.

Relevance: 7 Novelty: 6

Architecture and Training Dynamics (14)

1. Dynamic Compression in Recurrent Networks

ArXiv ID: 2608.17896

Primary Topic: Architecture and Training Dynamics

Also Matches: Efficiency, Compression, and Large-Scale Training

Authors: Jyothish Pari, Ryan Bahlous-Boldi, Pulkit Agrawal

Abstract: Recurrent models process long contexts efficiently by compressing their history into a fixed-size state, but modern architectures typically do so in a single causal pass over the sequence. Each input must therefore be compressed before the model knows how it will later be used, forcing a limited state to compromise across possible future demands. We introduce dynamic compression, which allows a recurrent model to selectively revisit past tokens and revise its fixed-size state through additional recurrent updates. The model need not preserve every part of the history at uniformly high fidelity in its recurrent state, because lower-fidelity information can be revisited from the retained raw sequence when it becomes relevant. We study this in a controlled setting where the model first learns multiple functions in-context and, later in the same sequence, encounters a series of few-shot tasks that each require it to identify and reuse one of those functions. A single-pass model must preserve every function at sufficient fidelity for any future task, whereas selective re-scanning allows the model to revisit and refine only the function currently needed. We find that dynamic compression substantially reduces the recurrent state required for accurate reuse and scales more favorably as the number of stored functions grows. These results demonstrate a computation--memory tradeoff in which recurrent models can spend more computation revisiting their history to make more effective use of a fixed-size state.

Comment: Selective history rescanning lets recurrent models revise fixed-size states through additional recurrent updates.

Topic Match: The core contribution changes recurrent sequence computation and establishes a state-capacity versus compute tradeoff; evidence comes from controlled tasks with retained raw history.

Relevance: 9 Novelty: 7

2. Think Shallow, Solve Deep: Controlling Recurrent Dynamics for Reliable Test-Time Depth

ArXiv ID: 2608.18222

Primary Topic: Architecture and Training Dynamics

Authors: Ivan Viakhirev, Kirill Borodin, Amirah Almutairi, Serguei Barannikov, Maxim Abramov, Grach Mkrtchian

Abstract: Recurrent-depth reasoners aim to solve harder problems by iterating their update longer at test time, but additional iterations can improve, preserve, or degrade an answer. We show that a measurable property of the trained operator, its finite-time dynamical regime (estimated as settling, marginal, or drifting), indicates which of these occurs. We give a sufficient condition for depth-safety: once an operator's per-step displacement is small relative to the decoder margin, the decoded answer cannot change under further iterations. Empirically, on algorithmic tasks trained from $800$ unaugmented examples per difficulty tier, settling operators do not degrade with added depth, and on some tasks convert it into higher accuracy on harder unseen instances (Sudoku, $0.19$ to $0.34$ past the training horizon). A single terminal fixed-point objective moves the regime and the depth behavior together: removing it induces drift and removes the gains, and adding it to a generic recurrence yields depth-safe extrapolation on carry propagation. We give four operational criteria for useful test-time depth, use them to catalogue failure modes, and, as a consistency check, apply the same measurements to Huginn-3.5B, which falls in the non-settling family.

Comment: A terminal fixed-point objective controls recurrent dynamics to improve reliability when increasing test-time depth.

Topic Match: Directly analyzes recurrent computation through dynamical regimes, a sufficient depth-safety condition, and training interventions that change extrapolation behavior.

Relevance: 9 Novelty: 7

3. Allocating Recurrent Compute in Looped Language Models

ArXiv ID: 2608.18230

Primary Topic: Architecture and Training Dynamics

Also Matches: Efficiency, Compression, and Large-Scale Training

Authors: Ruhai Lin, Yiyang Guo, Rui-Jie Zhu, Hao Ye, Jason K. Eshraghian

Abstract: Looped language models improve reasoning and knowledge manipulation by applying shared computation repeatedly. Existing systems usually repeat an entire layer stack, although a mixer and a dense feed-forward network (FFN) perform different operations and have different costs. We ask a narrower question: what should loop? We view recurrence as repeated composition of a state update and argue that an application is valuable when it exposes a new cross-position influence direction that remains observable at the task readout. Iterative Transport Rank (ITR) describes the cumulative influence trajectory; marginal ITR describes the nonredundant influence contributed by successive applications. This view motivates MixerLoop, which repeats each Gated DeltaNet mixer while applying its dense FFN once. We compare MixerLoop with no recurrence and full-block recurrence at 15M and 110M parameters under the same data, initialization, and architecture. A finite context-off intervention tests whether later mixer applications produce distinct, non-negligible, and beneficial changes at the final language-model readout. MixerLoop surpasses FullLoop on aggregate CORE at 15M and retains 41.5% of its CORE improvement at 110M while reducing recurrent-backbone projection FLOPs by 45.9%. These results show that the benefits of recurrent depth can be retained without repeatedly executing the dense FFN.

Comment: Loops Gated DeltaNet mixers while executing each dense FFN once, reducing projection FLOPs in the recurrent backbone by 45.9%.

Topic Match: Selective recurrence directly changes architectural compute allocation, supported by influence analysis and controlled measurements of capability versus computation.

Relevance: 9 Novelty: 7

4. Center-Manifold Reduction of Learning at Bifurcations: Interference and Rich Learning in Recurrent Neural Networks

ArXiv ID: 2605.12763

Primary Topic: Architecture and Training Dynamics

Authors: James Hazelden, Eric Shea-Brown

Abstract: Rich learning in recurrent neural networks often proceeds through sudden transitions in latent dynamics, but there is little theory predicting how gradient descent behaves during these events. We study the local learning geometry near codimension-one bifurcations through the global empirical Neural Tangent Kernel (GeNTK). Under local center-manifold conditions, and when bifurcation-related sensitivity dominates bounded residual terms, we show that the global parameter-to-state Jacobian (D_θh) is approximated by a low-rank normal-form operator. The induced GeNTK and Fisher information matrix therefore become strongly amplified and anisotropic, concentrating toward a rank-one channel for the four scalar codimension-one bifurcations and a rank-two real channel for a Neimark--Sacker bifurcation. Controlled high-dimensional RNN experiments validate this operator reduction. In learned RNNs, the same low-rank concentration coincides with abrupt loss changes and subtask interference, while a local projection predicts the sign of these effects near isolated events. Finally, in an input-driven 15-task LeakyRNN, GeNTK amplification aligns with continuation-detected changes in the MemoryPro dynamics. These results suggest a tractable operator-level description of learning near dynamical transitions, together with scalable diagnostics for amplified low-dimensional learning geometry.

Comment: Center-manifold reduction explains low-rank gradient geometry and abrupt loss changes near RNN bifurcations.

Topic Match: The work directly analyzes optimization and interference dynamics induced by architectural state transitions.

Relevance: 8 Novelty: 8

5. Understanding Curriculum Learning in Large Language Models via Cross-Difficulty Optimization Dynamics

ArXiv ID: 2608.17268

Primary Topic: Architecture and Training Dynamics

Authors: Zhikai Ding, Ziyi Ye

Abstract: Curriculum learning has been widely adopted in the post-training of large language models by organizing training data from easy to hard. However, its effectiveness varies substantially across reasoning tasks, suggesting that no single curriculum is universally optimal and raising a fundamental question: what determines when curriculum learning works? In this paper, we answer this question by analyzing the optimization dynamics induced by different curriculum schedules. We show that the transfer relationship between different difficulty levels characterizes the optimization dynamics induced by curriculum learning, which in turn explains the effectiveness of different curriculum schedules, and formalize this relationship as Relative Transfer, a principled measure of cross-difficulty knowledge transfer. Based on this measurement, we derive Transfer-aware Dynamic Curriculum Sampling (TDCS), which dynamically adjusts the sampling distribution according to the estimated transfer relationship throughout training. Extensive experiments on multiple reasoning benchmarks demonstrate that TDCS consistently outperforms representative scheduling strategies across different tasks, model scales, and training paradigms. More importantly, our work provides a unified optimization-based explanation of curriculum learning through cross-difficulty transfer.

Comment: Uses cross-difficulty Relative Transfer to explain curriculum optimization dynamics and derive adaptive sampling.

Topic Match: The core contribution explains how data scheduling changes optimization dynamics, qualifying despite its reasoning post-training setting.

Relevance: 8 Novelty: 7

6. SingularClip: Preventing Spectral Collapse to Maintain Plasticity in Continual and Reinforcement Learning

ArXiv ID: 2608.18319

Primary Topic: Architecture and Training Dynamics

Authors: Tyler Kastner, Nimrod De La Vega, Amir-massoud Farahmand

Abstract: Neural networks trained on nonstationary tasks frequently lose the ability to fit new targets, a phenomenon referred to as loss of plasticity. We identify a novel source of plasticity loss due to the growing anisotropy of weight matrices' singular values during training, and analyze this phenomenon both empirically and theoretically. To mitigate this issue, we introduce SingularClip, a procedure that periodically clips the singular values of all weight matrices. We show that SingularClip performs strongly against baselines across a range of tasks in both continual supervised learning and deep reinforcement learning.

Comment: Links weight-spectrum anisotropy to plasticity loss and periodically clips singular values to preserve trainability.

Topic Match: A mechanistic analysis and intervention for lost trainability form the core contribution; continual-learning and RL experiments test that training-dynamics mechanism.

Relevance: 8 Novelty: 7

7. Demystifying Training-Time Augmentation for Data-Constrained Language Model Pretraining

ArXiv ID: 2606.16246

Primary Topic: Architecture and Training Dynamics

Also Matches: Efficiency, Compression, and Large-Scale Training

Authors: Michael K. Chen, Xikun Zhang, Fan Bai, Zhengding Hu, Zhen Wang

Abstract: As AI labs approach a data ceiling where compute capacity outpaces the rate of new high-quality text generation, language model pretraining is shifting toward a data-constrained, compute-abundant regime that demands productive multi-epoch training on fixed corpora. Standard autoregressive (AR) pretraining overfits severely in this setting, reaching its optimum early and then continuously deteriorating. We investigate training-time data augmentation as a regularizer to mitigate this overfitting and enable productive training for hundreds of epochs on the same data. We introduce three orthogonal categories of augmentation for AR pretraining: token-level noise (masking, random replacement), sequence permutations (right-to-left prediction, Fill-in-the-Middle), and target offset prediction ($x_{t+i}$ for $i > 1$). Through systematic ablations, we find that individual augmentations delay overfitting and lower validation loss relative to the baseline, with random token replacement achieving the best minimum loss among individual methods. Combining augmentation categories further lowers the minimum validation loss. Our experiments demonstrate that data augmentations mitigate AR pretraining's data inefficiency and offer a promising solution to the data-constrained regime~\footnote{All code and data are available at https://github.com/ michaelchen-lab/ data-augmentations-for-pretraining.

Comment: Training-time augmentation delays multi-epoch pretraining overfitting on fixed corpora.

Topic Match: The central result concerns autoregressive pretraining dynamics, with secondary gains in data and compute efficiency.

Relevance: 8 Novelty: 7

8. When to Review: Spaced Repetition for Continual Pre-Training of Language Models

ArXiv ID: 2608.17530

Primary Topic: Architecture and Training Dynamics

Authors: Alankar Atreya, Devesh Batra, Yoages Kumar Mantri, Geremy Bantug, Greig A Cowan, Raad Khraishi

Abstract: Continual pre-training of large language models must acquire new information without erasing old knowledge. Existing replay methods often choose a global old/new mixture and sample uniformly, ignoring that examples differ in how quickly they are forgotten. We formulate continual pre-training as adaptive review scheduling: the training loop should decide not only how much history to replay, but which examples should return at each step. We introduce Spaced Repetition Training (SRT), a continual learning framework inspired by cognitive science, which schedules sample-rehearsal using the SuperMemo-2 (SM-2) algorithm. SRT maintains per-example review state, maps per-example perplexity to a recall-quality signal, and schedules historical examples for retention and new examples for consolidation while leaving the model, objective, and optimizer unchanged. On temporally separated Wikipedia and code corpora, SRT improves the stability-plasticity trade-off, recovering 5 to 37 percentage points of old-knowledge accuracy lost by naive continual pre-training across model scales while preserving or improving new-knowledge acquisition. At larger scale, SRT preserves broad benchmark performance that naive continual pre-training and uniform replay substantially degrade. Experiments with vision and tabular data further suggest that the scheduling principle extends beyond language when paired with an appropriate recall signal.

Comment: Schedules per-example replay using perplexity-based recall signals and SM-2 review state during continual pretraining.

Topic Match: Adaptive replay directly changes the retention and acquisition dynamics of language-model pretraining; novelty comes from adapting an established scheduling principle.

Relevance: 8 Novelty: 6

9. HyPE-GT: where Graph Transformers meet Hyperbolic Positional Encodings

ArXiv ID: 2312.06576

Primary Topic: Architecture and Training Dynamics

Authors: Kushal Bose, Swagatam Das

Abstract: Graph Transformers (GTs) facilitate the comprehension of complex relationships on graph-structured data by leveraging self-attention of the possible pairs of nodes. The structural information or inductive bias of the input graph is provided as positional encodings to the GT. The positional encodings are mostly Euclidean and are not able to capture the complex hierarchical relationships of the corresponding nodes. To address the limitation, we introduce a novel and efficient framework, HyPE, that generates learnable positional encodings in the non-Euclidean hyperbolic space that capture the intricate hierarchical relationships of the underlying graphs. Unlike existing methods, HyPE can generate a set of hyperbolic positional encodings, empowering us to explore diverse options for the optimal selection of PEs for specific downstream tasks. Additionally, we repurpose the generated hyperbolic positional encodings to mitigate the impact of oversmoothing in deep Graph Neural Networks (GNNs). Furthermore, we provide extensive theoretical underpinnings to offer insights into the working mechanism of the HyPE framework. Comprehensive experiments on four molecular benchmarks, including the four large-scale Open Graph Benchmark (OGB) datasets, substantiate the effectiveness of hyperbolic positional encodings in enhancing the performance of Graph Transformers. We also consider Coauthor and Copurchase networks to establish the efficacy of HyPE in controlling oversmoothing in deep GNNs.

Comment: Learnable hyperbolic positional encodings capture graph hierarchy and mitigate oversmoothing in deep graph networks.

Topic Match: The contribution introduces and theoretically analyzes a positional-encoding mechanism, with applicability and evidence concentrated on graph architectures.

Relevance: 7 Novelty: 7

10. Low-dimensional topology of deep neural networks

ArXiv ID: 2606.31856

Primary Topic: Architecture and Training Dynamics

Authors: Junyu Ren, Lek-Heng Lim

Abstract: We study layered models, including feedforward networks, ResNets, and transformers, by limiting each layer to a width of $d = 3$, i.e., $\mathbb{R}^3$ as representation space. This allows us to track how a neural network changes low-dimensional topological invariants through its layers. Just about any topological structure may be simplified or even trivialized by simply increasing dimension; e.g., any knot is equivalent to an unknot in $\mathbb{R}^4$. By restricting to $\mathbb{R}^3$, we not only isolate the effects of activation and depth from that of width, we work in a space that lends itself to easy visualization. We focus on linking number here, deferring other invariants like link groups, Milnor's $\barμ$-invariants, knot types, ambient cobordisms, to a sequel. We provide full proofs and empirical experiments to justify the following insights: When measured by their power to effect changes in linking numbers, the layer-skipping feature in ResNets is as powerful as the attention mechanism in transformers; both ResNets and transformers are strictly more powerful than feedforward neural networks with monotonic activations, which are in turn more powerful than invertible and flow-based models; but replacing monotonic activation with a nonmonotonic one elevates a feedforward network into the same expressivity class as ResNets and transformers. These results suggest that low-dimensional topology can be a useful tool to guide designs of AI architectures. We also generalize our results from $d = 3$ to arbitrary $d > 3$.

Comment: Uses linking-number changes to isolate the expressive effects of residual skips, attention, and nonmonotonic activations.

Topic Match: Directly analyzes architectural mechanisms through topological expressivity, although implications for large-model training remain indirect.

Relevance: 7 Novelty: 7

11. SurgicalMamba: Dual-Path SSD with State Regramming for Online Surgical Phase Recognition

ArXiv ID: 2605.14889

Primary Topic: Architecture and Training Dynamics

Authors: Sukju Oh, Sukkyu Sun

Abstract: Online surgical phase recognition must commit to a prediction at every frame of a procedure that runs for hours, from past frames alone and at a per-frame cost that does not grow with elapsed length. Structured state-space duality (SSD) meets that constraint, but only by having the scan see a per-head scalar transition, which fixes both where the state puts a frame and how fast it decays. The same views recur through an operation, so repeated content is written over itself and can afterwards be told apart only by age. How fast to decay is left to the step, and when the past stops being useful has to be inferred from a loss that never marks the moment. Procedures run long and change little visually from frame to frame, leaving the step with little to select on. Phases also vary widely in length, so no fixed rate serves as a fallback. We address the two with two mechanisms. State regramming rotates the carried state at each chunk boundary, by an amount the chunk's content decides, so where a frame is written also depends on what has passed since: two occurrences of the same view are held apart when different phases intervene, which no decay rate can achieve once both have aged. Intensity-modulated stepping increases the decay at the annotated phase transitions, so the state empties quickly where a phase ends and slowly in between and the decay itself can be set for the longest phase. Both leave SSD's N-semiseparable structure and O(d) per-frame cost intact. Across seven public benchmarks SurgicalMamba reaches state-of-the-art online accuracy and phase-level Jaccard (94.6%/82.7% on Cholec80, 89.5%/68.9% on AutoLaparo) at 312.88 fps on a single GPU. Adding the rotation alone to a plain Mamba2 improves multi-query associative recall (MQAR) wherever the recurrent state is the binding constraint, indicating that the mechanism is not specific to surgical video.

Comment: Adds content-dependent state rotations to SSD recurrence while preserving its semiseparable structure and linear per-frame cost.

Topic Match: State regramming changes the recurrent computational mechanism, with associative-recall experiments supporting applicability beyond surgical recognition.

Relevance: 7 Novelty: 7

12. Spikformer V2: Join the High Accuracy Club on ImageNet with an SNN Ticket

ArXiv ID: 2401.02020

Primary Topic: Architecture and Training Dynamics

Also Matches: Efficiency, Compression, and Large-Scale Training

Authors: Zhaokun Zhou, Yijie Lu, Kaiwei Che, Wei Fang, Keyu Tian, Qihao Peng, Yuesheng Zhu, Shuicheng Yan, Yonghong Tian, Li Yuan

Abstract: Spiking Neural Networks (SNNs), known for their biologically plausible architecture, face the challenge of limited performance. The self-attention mechanism, which is the cornerstone of the high-performance Transformer and also a biologically inspired structure, is absent in existing SNNs. To this end, we explore the potential of leveraging both self-attention capability and biological properties of SNNs, and propose a novel Spiking Self-Attention (SSA) and Spiking Transformer (Spikformer). The SSA mechanism eliminates the need for softmax and captures the sparse visual feature employing spike-based Query, Key, and Value. This sparse computation without multiplication makes SSA efficient and energy-saving. Further, we develop a Spiking Convolutional Stem (SCS) with supplementary convolutional layers to enhance the architecture of Spikformer. The Spikformer enhanced with the SCS is referred to as Spikformer V2. To train larger and deeper Spikformer V2, we introduce a pioneering exploration of Self-Supervised Learning (SSL) within the SNN. Specifically, we pre-train Spikformer V2 with masking and reconstruction style inspired by the mainstream self-supervised Transformer, and then finetune the Spikformer V2 on the image classification on ImageNet. Extensive experiments show that Spikformer V2 outperforms other previous surrogate training and ANN2SNN methods. An 8-layer Spikformer V2 achieves an accuracy of 80.38% using 4 time steps, and after SSL, a 172M 16-layer Spikformer V2 reaches an accuracy of 81.10% with just 1 time step. To the best of our knowledge, this is the first time that the SNN achieves 80+% accuracy on ImageNet. The code will be available at Spikformer V2.

Comment: Multiply-free spiking self-attention introduces sparse, energy-efficient Transformer computation.

Topic Match: The primary contribution is a new attention architecture, with sparse spike-based computation providing a secondary efficiency match.

Relevance: 7 Novelty: 7

13. Where a New Concept Must Enter: Entry Point Gates Cross-Task Usability in Unified Multimodal Models

ArXiv ID: 2608.17564

Primary Topic: Architecture and Training Dynamics

Authors: Zongyang Qiu, Yihan Wu, Kaixuan Fan, Bo Li, Hui Xiong

Abstract: Unified multimodal models (UMMs) are motivated by the hope that understanding and generation reinforce each other but controlled ablations repeatedly find that adding a generation objective leaves understanding flat. Joint-training studies cannot settle the disagreement: with overlapping supervision, a gain cannot be attributed to the architecture rather than the data. To further investigate the relationship between the two directions in UMMs, we separate them by construction. A novel visual entity, a rendered 3D asset paired with a pseudo-word screened for absence from the frozen model's behavior, is bound through exactly one task direction, and the untrained direction is then measured. We find that the channel is real in both directions, but the directions differ in kind: generation training installs a name the model can only match among candidates; understanding training installs one it can also produce. What governs cross-task usability is where the binding enters the shared computation. An alignment probe predicts export across 36 configurations (Spearman $ρ= +0.68$). That objective's alignment term, maximized in closed form over activations with every weight frozen, makes a concept drawable when injected at layer 7 of 28 and is indistinguishable from the base model from layer 14 on, while the weight-based version of the same edit peaks at layers 10-14. In an observational series of four models, this window appears only where the understanding pathway is a semantic vision encoder, suggesting that unified weights are not enough: the two directions must share a semantic format at the entry point. Exploiting the rule, a mid-stack alignment objective acquires the concept for a $0.1\%$ relative loss of the model's general text-to-image ability, against $41\%$ for the standard generative route. Our code is at https://github.com/Zane-ZYQiu/entry-point-umm.

Comment: Identifies a layer-dependent semantic-alignment window governing transfer between multimodal understanding and generation.

Topic Match: Shared-pathway interventions provide an architectural connection, but the central contribution concerns concept binding and representation transfer, which this feed generally excludes.

Relevance: 6 Novelty: 8

14. LoopVLA: Learning Sufficiency in Recurrent Refinement for Vision-Language-Action Models

ArXiv ID: 2605.09948

Primary Topic: Architecture and Training Dynamics

Also Matches: Efficiency, Compression, and Large-Scale Training

Authors: Boyang Shen, Kaixiang Yang, Hao Wang, Qiuyu Yu, Qiang Xie, Qiang Li, Zhiwei Wang

Abstract: Current Vision-Language-Action (VLA) models typically treat the deepest representation of a vision-language backbone as universally optimal for action prediction. However, robotic manipulation is composed of many frequent closed-loop spatial adjustments, for which excessive abstraction may waste computation and weaken low-level geometric cues essential for precise control. Existing early-exit strategies attempt to reduce computation by stopping at predefined layers or applying heuristic rules such as action consistency, but they do not directly answer when a representation is actually sufficient for action. In this paper, we present LoopVLA, a recurrent VLA architecture that jointly learns representation refinement, action prediction, and sufficiency estimation. LoopVLA iteratively applies a shared Transformer block to refine multimodal tokens, and at each iteration produces both a candidate action and a sufficiency score that estimates whether further refinement is necessary. By sharing parameters across iterations, LoopVLA decouples refinement from absolute layer indices and grounds sufficiency estimation in the evolving representation itself. Since sufficiency has no direct supervision, we introduce a self-supervised distribution alignment objective, where intermediate confidence scores are trained to match the relative action quality across refinement steps, thereby linking sufficiency learning to policy optimization signals. Experiments on LIBERO, LIBERO-Plus, and VLA-Arena show that LoopVLA pushes the efficiency-performance frontier of VLA policies, reducing parameters by 45% and improving inference throughput by up to 1.7 times while matching or outperforming strong baselines in task success.

Comment: Shared-block recurrence with learned sufficiency scores allocates refinement depth according to action quality.

Topic Match: Learned adaptive computation is the central architectural mechanism, with parameter and inference savings; its formulation remains specific to robotic action prediction.

Relevance: 7 Novelty: 6

Efficiency, Compression, and Large-Scale Training (6)

1. TileMix: Tile-Centric Mixed-Precision Attention for LLM Inference Acceleration

ArXiv ID: 2608.17336

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Hanzhi Zhang, Qiao Zhang, Qinglei Cao, Heng Fan, Yan Huang, Kewei Sha, Yunhe Feng

Abstract: Long-context prefill in large language models (LLMs) incurs substantial computation and memory traffic because dense self-attention computes quadratic query-key scores. Existing methods either use a uniform low-precision path or select token interactions, leaving spatial precision routing over hardware-aligned score tiles outside fused dense attention. We introduce TileMix, a tile-centric precision-routing kernel that makes numerical precision an executable spatial decision over score-tile groups within fused dense attention. TileMix partitions the attention matrix into hardware-aligned score tiles, packs routing decisions into compact bitmasks, and dispatches each tile group through FP16 or INT8 score computation while both paths update a shared online-softmax state. Scalable precision grouping lets each routing bit govern multiple adjacent key tiles, preserving hardware-aligned compute tiles and compact metadata at long contexts. By routing all legal tile groups, TileMix preserves dense token connectivity, requires no training, and supports grouped-query attention, variable-length batches, and INT8 key/value caches. Across LongEval, LV-Eval, and A100 prefill benchmarks on LLaMA, Qwen, and Vicuna, TileMix recovers long-context quality lost under uniform INT8 and improves prefill throughput over FP16, yielding a controllable accuracy-efficiency frontier across model families. The implementation is available at https://github.com/HanzhiZhang-Ulrica/TileMix.

Comment: Routes attention score tiles through FP16 or INT8 computation inside a fused kernel with shared online softmax.

Topic Match: A new spatial mixed-precision kernel directly improves long-context prefill efficiency while preserving dense attention connectivity.

Relevance: 9 Novelty: 7

2. MoNe: Modular Neural Memory for Efficient Long Context Inference

ArXiv ID: 2608.17616

Primary Topic: Efficiency, Compression, and Large-Scale Training

Also Matches: Architecture and Training Dynamics

Authors: Wonguk Cho, Kyubyung Chae, Tribhuvanesh Orekondy, Sunghyun Park, Hyoungwoo Park, Jeongho Kim, Arash Behboodi, Kyuwoong Hwang, Sungrack Yun

Abstract: We present MoNe, a lightweight modular neural memory that attaches to any frozen pretrained Transformer to enable long-context inference without retraining. MoNe reads context in fixed-size segments via test-time learning of fast-weight neural memory networks with layer-localized gradient updates; at inference, the memory generates keys and values from the query tokens alone, with no context tokens re-read. This two-phase design decouples inference cost from context length, achieving $O(N)$ preprocessing and $O(1)$ query cost with peak GPU memory that does not grow with $N$. At 128K tokens, MoNe reduces both compute and peak GPU memory by approximately 80% compared to ICL with only 6.4% parameter overhead. MoNe generalizes to context lengths far beyond the backbone's native window, achieving strong performance on needle-in-a-haystack and word extraction benchmarks from RULER, where ICL degrades sharply.

Comment: Fast-weight key/value generation reduces long-context query compute and GPU memory without backbone retraining.

Topic Match: The modular mechanism decouples query cost from context length and reports approximately 80% compute and peak-memory reductions at 128K, providing a concrete efficiency contribution.

Relevance: 8 Novelty: 7

3. Optimize Your Sampling: Tuned Diffusion Sampling with Bayesian Optimization

ArXiv ID: 2608.18040

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Travis Zhang, Christian Belardi, Justin Lovelace, Jin Peng Zhou, Saebyeol Shin, Carla P. Gomes, Kilian Q. Weinberger

Abstract: Sampling from a diffusion model typically requires many forward passes through a large neural network, making generation computationally expensive. While much work has focused on efficient solvers and samplers, comparatively little attention has been paid to selecting the sampling timesteps themselves. A recent line of work optimizes theoretically derived surrogates for sample quality rather than the quality metric itself. We propose Optimizing Your Sampling (OYS), which instead treats timestep selection as a black-box optimization problem, optimizing the target metric directly with Bayesian optimization. OYS outperforms both the default schedules and those of Align Your Steps on text-to-image generation, and improves over the default schedules on inpainting and other image tasks, in both quantitative and human evaluations. OYS requires no additional training, is applicable even to distilled models, and improves both simple and sophisticated samplers such as Euler and DPM-Solver++. A 5-step OYS schedule retains 89%-94% of the quality of a 50-step schedule while reducing inference cost by 10x.

Comment: Bayesian-optimized timestep schedules reduce diffusion inference cost by roughly 10x.

Topic Match: Its primary contribution materially reduces model-running cost without retraining.

Relevance: 8 Novelty: 6

4. WAM-Diff2: Hierarchical AR-to-Diffusion Distillation for Highly Efficient Autonomous Driving VLA

ArXiv ID: 2608.01035

Primary Topic: Efficiency, Compression, and Large-Scale Training

Also Matches: Architecture and Training Dynamics

Authors: Zhihao Zhu, Hanlin Shang, Mingwang Xu, Feipeng Cai, Zhuolin He, Yaoyi Li, Jianhua Han, Hang Xu, Siyu Zhu

Abstract: Vision-Language-Action (VLA) models have emerged as a prominent paradigm for end-to-end autonomous driving; however, their efficient deployment is severely constrained by high computational latency and exposure bias arising from sequential autoregressive decoding. Conversely, while specialized diffusion policies enable low-latency, parallel execution, training them from scratch typically yields narrow, single-task architectures that lack holistic visual-linguistic reasoning. Successfully transforming pre-trained autoregressive generalists into parallel diffusion models could combine multi-task cognitive intelligence with execution efficiency, yet this transition presents a formidable architectural challenge due to mismatched attention patterns (causal versus bidirectional) and divergent optimization objectives. To bridge this divide, we introduce WAM-Diff2, a multi-task discrete diffusion VLA framework powered by a three-stage hierarchical distillation strategy. By structuring the architectural shift through progressive block-wise adaptation, block-wise distillation, and model-wise cross-scale distillation, WAM-Diff2 preserves the underlying semantic foundations of the base model while accelerating inference. Extensive evaluations across driving understanding, perception, and planning benchmarks demonstrate that WAM-Diff2 effectively mitigates exposure bias and achieves performance parity with autoregressive baselines. Crucially, the autoregressive-to-diffusion transition yields a 2.8x decoding speedup, which scales to an ultimate 15.1x acceleration when combined with system-level optimizations including FlashInfer and CUDA Graphs.

Comment: Three-stage hierarchical distillation converts autoregressive VLAs into parallel diffusion decoders, delivering a reported 2.8x decoding speedup.

Topic Match: The core conversion method changes model computation and directly reduces decoding cost, qualifying despite driving-specific validation.

Relevance: 7 Novelty: 6

5. CORAM: Coherent Orthogonal Rotation for Model Merging

ArXiv ID: 2608.17366

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Xinyi Sui, Ziran Liu, Nam Ling, Wei Wang, Wei Jiang

Abstract: Merging finetuned models combines specialized capabilities without joint training or access to the original data. Most methods operate by linear arithmetic in Euclidean weight space, which cannot carry the geometry of the update. Orthogonal Model Merging (OrthoMerge) uses a single orthogonal transform for each weight matrix, but such a transform cannot change singular values. We propose CORAM, which partitions each target matrix into row slices, represents every expert slice by its singular value decomposition in the corresponding base-model SVD frame, and merges the task-specific factors on their corresponding manifolds. Because manifold averaging contracts the merged update, CORAM applies an amplification coefficient $λ=κ\hat{c}$. The scale c_hat is estimated from the expert and merged update norms and is approximately $\sqrt{N}$ for $N$ experts with comparable update magnitudes. The restoration strength kappa is selected from the dispersion of expert updates without evaluating candidate merged models. This rule remains within 0.72 points of the best swept value on all evaluated suites. CORAM also includes spread slicing to distribute highly updated rows across slices and a residual pathway for non-target layers. Across four suites covering three model families, 3B to 9B scales, and language and vision-language experts, CORAM improves over OrthoMerge by 0.25 to 1.35 points and matches or exceeds the strongest weight-space baselines.

Comment: SVD-factor manifold averaging merges specialist checkpoints without joint retraining.

Topic Match: The efficiency connection comes from avoiding joint retraining; the central advance improves checkpoint-merging geometry and quality, with no demonstrated change to training or execution cost.

Relevance: 6 Novelty: 6

6. The Integer Alibi: Localizing Cross-Kernel Divergence in INT8-Quantized LLM Inference

ArXiv ID: 2608.13756

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Teng-Ruei Chen

Abstract: Two GPU kernels implementing the same scaled INT8 GEMM interface are usually treated as interchangeable. We test that assumption: holding the checkpoint, prompts, hardware, inference engine, decoding, and quantization configuration fixed, we swap only the INT8 linear kernel (CUTLASS versus Triton) inside vLLM. At 1.7B each arm reproduces itself bit-for-bit across cold restarts, yet the arms agree on no sequence in any end-to-end comparison we ran (0/8, 0/16, and 0/64). What makes this more than a benchmark discrepancy is an integer alibi: for shared INT8 operands under a verified no-overflow bound, the INT32 dot product is exact and order-independent, so the accumulator cannot be the source of any difference. Feeding both kernels identical operands from every linear layer of Qwen3-1.7B and 8B (196 and 252 layers), we find bit-identical outputs under power-of-two scales, confirming a pinned prediction list 196/196 and 252/252 (pre-registered at 1.7B, pinned but not blind at 8B), and observed differences of at most one bfloat16 spacing under the checkpoints' real scales. This localizes the divergence to scale application and output rounding after the exact accumulator. Applied as a probe checkpoint, the same intervention restores end-to-end bitwise agreement (8/8 and 16/16 sequences). Cross-implementation FP8 GEMM shows a different signature: both the prevalence and the magnitude of differences grow with reduction depth, while the INT8 fraction stays at parts per million and within one spacing over a 64x range of K. Teacher-forced replay ties layers to tokens: flips concentrate at small logit margins, which predict flip risk with ROC-AUC 0.94 on 16,384 positions. We will release the pre-registration, per-layer predictions, manifests with kernel-selection evidence, and a conformance procedure that turns these controls into a concrete check for kernel interchangeability.

Comment: Localizes INT8-kernel output divergence to scale application and rounding after exact integer accumulation.

Topic Match: Quantized-kernel numerics directly touches efficiency methods, but the contribution establishes reproducibility and conformance rather than reducing model computation or memory costs.

Relevance: 6 Novelty: 6

Paper Selection Prompt

System Prompt

You are a helpful paper reading assistant whose job is to read daily posts from ArXiv and identify a few papers that your friend will enjoy reading. Your job is to carefully read the paper titles and abstracts below and find the ones that match the criteria below.

User Prompt

Instructions

Respond in JSONL. Output exactly one JSON object per paper, one per line:

{"ARXIVID":"...","COMMENT":"...","RELEVANCE":0,"NOVELTY":0,"PRIMARY_TOPIC_ID":"...","MATCHED_TOPIC_IDS":[],"TOPIC_MATCH_COMMENT":"...","HOTSPOT_PAPER_TAGS":[],"HOTSPOT_PAPER_COMMENT":"..."}

Rules: - ARXIVID: the arXiv ID. - COMMENT: identify the single strongest matching criterion. Be brief and specific. Do not rely on generic phrases like "language modeling" or "advancement". Do not mention non-matching criteria. - RELEVANCE: integer from 1 to 10. - NOVELTY: integer from 1 to 10. - PRIMARY_TOPIC_ID: exactly one stable topic ID from the allowed topic registry. - MATCHED_TOPIC_IDS: zero or more stable topic IDs from the same allowed set. Include PRIMARY_TOPIC_ID when there are multiple matches. - TOPIC_MATCH_COMMENT: briefly explain why the primary topic is the best fit. - HOTSPOT_PAPER_TAGS: zero or more tags from this exact set only: daily_hot, new_frontier. - HOTSPOT_PAPER_COMMENT: briefly explain why the paper belongs in the daily hotspot paper feed when HOTSPOT_PAPER_TAGS is non-empty; otherwise use an empty string. - Use HOTSPOT_PAPER_TAGS sparingly. Most papers should return []. - daily_hot means the paper feels broadly important to the day and belongs in the daily hotspot paper section even if it is not part of the personalized foundational reading list. - new_frontier means the paper appears to open a genuinely new direction, paradigm, or field, even if the work is still early. - Do not output markdown, code fences, or any extra text.

Scoring Criteria

Relevance and Novelty are independent axes. Score both from 1 to 10.

Relevance Scoring

• 9-10: directly centered on the target foundational topics; highest when the core contribution is clearly within them.
• 7-8: substantially related, but partly peripheral or focused on a narrower aspect.
• 5-6: touches the target topics, but the main contribution is elsewhere.
• 3-4: largely outside the target topics, often application-focused or domain-specific.
• 1-2: unrelated.

Important: Broad frontier relevance, major launch status, or daily buzz is not enough for a high Relevance score here. Those cases belong in the hotspot digest unless the paper strongly matches the specialized paper topics.

Novelty Scoring

• 9-10: new paradigm, theory, or major methodological breakthrough.
• 7-8: substantial methodological advance or strong new insight.
• 5-6: meaningful but incremental extension or refinement.
• 3-4: minor, narrow, or mostly engineering or domain-specific improvement.
• 1-2: little originality; mainly standard application of existing methods.

Topic Registry

Use exactly one PRIMARY_TOPIC_ID chosen from the stable topic IDs below. - moe_training: MoE Training - Mixture-of-Experts training: routing, load balancing, expert granularity and shared experts, training stability, MoE parallelism and communication, and MoE kernels. - training_systems: Large-Scale Training Systems and Efficiency - Large-scale training systems and efficiency: distributed training algorithms, optimizers, and communication. - architecture_training: Architecture and Training Dynamics - Core architectural or computational mechanisms, dynamic computation, and training-stability dynamics. - efficiency_scaling: Efficiency, Compression, and Large-Scale Training - Compression, sparsity, memory or cache efficiency, and large-scale training systems that materially change cost or behavior.

Papers

[PAPER LIST HERE]

Relevant Topics

This is a training-side feed. The centre of gravity is Mixture-of-Experts training and the large-scale training systems it sits on; architecture and efficiency work is kept when it bears on how a large model is trained or what it costs to train.

Keep a paper when its CORE CONTRIBUTION falls in one of the four topics below. Filter it when the topic is merely the setting for a downstream application, a benchmark, or a product result.

1. MoE Training (primary) - Keep: expert routing and gating mechanisms; load balancing, with or without auxiliary losses; expert granularity, fine-grained and shared experts; MoE training stability (loss spikes, router collapse, z-loss and related fixes); expert and hybrid parallelism; all-to-all dispatch/combine and the communication schedules MoE needs; MoE kernels and grouped-GEMM style implementations; expert capacity, token dropping, and upcycling dense checkpoints into MoE. - Filter: papers that merely train on top of a MoE model without a new routing, balancing, stability, parallelism, or kernel contribution; "mixture of experts" in the classical ensemble or recommender sense.

2. Large-Scale Training Systems - Keep: distributed training algorithms (data, tensor, pipeline, sequence, and expert parallelism, FSDP and ZeRO-style sharding); optimizers and preconditioners for large-scale pretraining; communication algorithms, overlap, and collective design; throughput and MFU work that materially changes what a training run costs or whether it converges; scaling-law work that informs how such runs are configured. - Filter: routine infrastructure tuning with no new algorithmic or systems idea.

3. Architecture and Training Dynamics - Keep: new or analysed architectural mechanisms (attention variants, normalization and residual design, state-space and recurrent sequence modelling, dynamic or modular computation); optimisation and training-dynamics analysis that explains why large models train the way they do. - Filter: applying an existing architecture to a new task or benchmark with no mechanistic insight.

4. Efficiency and Compression - Keep: quantization, sparsity, pruning, low-rank adaptation, KV-cache and memory-efficient designs, and other work that materially changes the cost of running or training a large model, especially when the mechanism is new rather than a tuned variant. - Filter: straightforward application of standard efficiency methods, and deployment or serving engineering with no new idea.

Leave these to the hotspot digest unless the core contribution clearly falls in one of the four topics above: - major model or product releases - agent, tooling, and RAG launches - benchmark, leaderboard, or evaluation-only papers - representation-learning theory, interpretability, and feature-formation analysis - memory mechanisms and agent memory systems - world models, exploration, and reinforcement learning - alignment and post-training (RLHF, DPO, GRPO, RFT, instruction tuning) - downstream applications in medical imaging, segmentation, 3D vision, video understanding, information retrieval, summarization, recommendation, machine translation, speech recognition, time series, knowledge graphs, and similar domains