This is a remedial run for missed papers from 07/23/2026 to 07/23/2026.

Results generated on 09/13/2026.

Personalized Daily ArXiv Papers 2026-07-24

_{Token counts are not reported for this run. 5 of 5 model calls succeeded, 2,234s of model wall clock.}

Topic Coverage:

Table of contents by topic:

Large-Scale Training Systems and Efficiency (2)

1. Spectral Transformation for Layer-wise Global Rank Discovery in Federated LoRA for Vision Transformers Authors: Hariharan Ramesh, Jyotikrishna Dass

2. Searching the Space of Feed-Forward Neural-Network Weight-Update Rules with Fixed Depth Symbolic Regression Authors: Charles Brum, Edward Finkelstein

Architecture and Training Dynamics (9)

1. Mean-to-Score Discrete Diffusion: Posterior-Mean Denoisers for Score Entropy Authors: Jingyuan Li, Xiaoyi Jiang, Yixuan Jiang, Wei Liu, Yi Zhu, Zuoqiang Shi, Pipi Hu

2. Context-weighted Discrete Flow Matching Authors: Daniil Cherniavskii, Daniel Severo, Karen Ullrich

3. Limit Theorems for Stochastic Gradient Descent in High-Dimensional Single-Layer Networks Authors: Parsa Rangriz

4. Gumbel Distillation for Parallel Text Generation Authors: Chi Zhang, Xixi Hu, Bo Liu, Qiang Liu

5. LatentChem: From Textual CoT to Latent Thinking in Chemical Reasoning Authors: Xinwu Ye, Yicheng Mao, Yuxuan Liao, Jia Zhang, Yimeng Liu, Li Hao, Fang Wu, Zhiwei Li, Zehong Wang, Zhiyuan Liu, Zhenfei Yin, Li Yuan, Philip Torr, Huan Sun, xiangxiang Zeng, Mengdi Wang, Le Cong, Shenghua Gao, Xiangru Tang

6. Theoretical Aspects of Lie Groupoid and Lie Algebroid Equivariant Convolutional Neural Networks Authors: Michael Astwood

7. AG-REPA: Causal Layer Selection for Representation Alignment in Audio Flow Matching Authors: Pengfei Zhang, Tianxin Xie, Minghao Yang, Li Liu

8. CHERRY: Compressed Hierarchical Experts with Recurrent Representational Yield Authors: Dohyeon Kwon, Youngjin Park

9. From Noise to Diversity: Random Embedding Injection in LLM Reasoning Authors: Heejun Kim, Seungpil Lee, Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Taesup Kim, Sundong Kim

Efficiency, Compression, and Large-Scale Training (15)

1. KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers Authors: Yann Bouquet, Alireza Khodamoradi, Kristof Denolf, Mathieu Salzmann

2. LinearARD: Linear-Memory Attention Distillation for RoPE Restoration Authors: Ning Yang, Hengyu Zhong, Wentao Wang, Baoliang Tian, Haijun Zhang, Jun Wang

3. Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context Authors: Alagappan Valliappan

4. Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs Authors: Yidu Wu, Xiang Wang, Kejie Zhao, Zhangchi Wang, Qinghai Guo, Xiaoying Tang

5. RED-PIM: Reducing Data Movement for Transformers using Processing-in-Memory Authors: Zahra Yousefijamarani, Alaa Alameldeen

6. PreMoE: Proactive Inference for Efficient Mixture-of-Experts Authors: Zehua Pei, Ying Zhang, Hui-Ling Zhen, Tao Yuan, Xianzhi Yu, Zhenhua Dong, Sinno Jialin Pan, Mingxuan Yuan, Bei Yu

7. DriftXpress: Faster Drifting Models via Projected RKHS Fields Authors: Ali Falahati, Elliot Creager, Gautam Kamath, Shubhankar Mohapatra

8. EvoSpec: Evolving Speculative Decoding via Real-Time Vocabulary and Parameter Adaptation Authors: Shuyu Zhang, Lingfeng Pan, Qicheng Wang, Yaqi Shi, Yueyang Tan, Ruyu Yan, Jiaqi Chen, Lixing Du, Lu Wang

9. Three-Pronged Spectral Control for Federated Parameter Efficient Fine Tuning Authors: Shiva Raj Pokhrel, Dipsan Bhattarai, Anwar Walid

10. How Many Bits Can an Adapter Write? Measuring the Capacity and Memorization of Parameter-Efficient Fine-Tuning Authors: Kaizhen Tan, Heqing Du, Yang Feng

11. AnchorPrune: Relevance-Anchored Contextual Expansion for Visual Token Pruning Authors: Kyuan Oh, Bumsoo Kim

12. Adversarial Prompts for Acceptance Collapse in Speculative Decoding Authors: Run Wang, Chaoyi Zhou, Xi Liu, Yi Zhu, Amir Salarpour, Pedram MohajerAnsari, Zhi-Qi Cheng, Feng Luo, Siyu Huang, Mert D. Pesé

13. Source-Prior-Driven Selective Adaptation for Efficient Diffusion Model Finetuning Authors: Yi Xiong, Yuan-Yuan Cheng, Xiao-Ming Fu

14. Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts Authors: Renuka Oladri, Mohan Vamsi Varadaraju Priya, Jerry Wu

15. Gibbs randomness-compression proposition Authors: M Süzen

Large-Scale Training Systems and Efficiency (2)

1. Spectral Transformation for Layer-wise Global Rank Discovery in Federated LoRA for Vision Transformers

ArXiv ID: 2607.21074

Primary Topic: Large-Scale Training Systems and Efficiency

Also Matches: Efficiency, Compression, and Large-Scale Training

Authors: Hariharan Ramesh, Jyotikrishna Dass

Abstract: Fine-tuning Vision Transformers (ViTs) with low-rank adapters (LoRA) promises better communication efficiency under federated setup, yet existing aggregation strategies face fundamental limitations. Independently averaging these LoRA factors is mathematically inconsistent, introducing cross-term aggregation error. In contrast, approaches that preserve heterogeneous client ranks by concatenating local adapters on the server substantially increase download cost and often require merging global LoRA updates into pretrained weights on the clients, causing reinitialization lag and unstable convergence. Other approaches further increase server-side overhead by reconstructing dense weight updates or training auxiliary models to refine aggregation error. In this work, we propose SpecTraL, spectral transformation for layer-wise global rank discovery, that resolves these challenges within a unified design. SpecTraL stacks local LoRA modules from clients and performs orthonormal Householder Transformation of the stacked adapters directly in the low-rank latent space, eliminating dense reconstruction of the global update and any auxiliary refinement on the server. By leveraging the Spiked Covariance Model from Random Matrix Theory, SpecTraL analytically separates the global consensus signal from non-IID noise, discovering optimal layer-wise global ranks without manual hyperparameter tuning. To match local ranks in subsequent rounds, we introduce a padding-aware initialization framework that lets clients incorporate residual LoRA dimensions without re-merging them into the pre-trained base model. Experiments on federated fine-tuning of ViT-B/16 and ViT-L/16 over DomainNet and NICO++ demonstrate improved accuracy-communication trade-offs, reduced server computation, and elimination of hyperparameter search for rank selection. Our code is available at https://github.com/DASS-Lab-Group/SpecTraL

Comment: Latent-space spectral aggregation discovers layer-wise global LoRA ranks without reconstructing dense updates.

Topic Match: The core method improves distributed LoRA aggregation, communication, and server computation in federated training.

Relevance: 7 Novelty: 7

2. Searching the Space of Feed-Forward Neural-Network Weight-Update Rules with Fixed Depth Symbolic Regression

ArXiv ID: 2607.21855

Primary Topic: Large-Scale Training Systems and Efficiency

Authors: Charles Brum, Edward Finkelstein

Abstract: We investigate whether symbolic regression can discover explicit neural network weight-update rules that outperform standard hand-designed optimizers on small symbolic regression benchmarks. Candidate update rules are represented as fixed-depth symbolic expressions over operands derived from common optimizers, including gradient, momentum, adaptive-gradient, and moment-estimate quantities. Across 30 benchmark/neural network combinations, the symbolic regression procedure found an update rule outperforming the best hyperparameter-tuned established optimizer in 25 cases, with an aggregate MSE reduction of 44.47\% over the improved cases. The discovered rules do not all share a single common symbolic form, but many combine adaptive normalization, momentum-like quantities, nonlinear transformations, and rational expressions. These results suggest that symbolic regression can serve as a lightweight mechanism for discovering compact optimizer variants, while also highlighting the need for larger-scale validation.

Comment: Uses fixed-depth symbolic regression to discover explicit neural-network optimizer update rules.

Topic Match: Optimizer design is the closest systems-training topic, although validation remains small-scale.

Relevance: 6 Novelty: 6

Architecture and Training Dynamics (9)

1. Mean-to-Score Discrete Diffusion: Posterior-Mean Denoisers for Score Entropy

ArXiv ID: 2607.21372

Primary Topic: Architecture and Training Dynamics

Authors: Jingyuan Li, Xiaoyi Jiang, Yixuan Jiang, Wei Liu, Yi Zhu, Zuoqiang Shi, Pipi Hu

Abstract: Score Entropy Discrete Diffusion (SEDD) parameterizes discrete reverse processes with unconstrained positive score ratios. While positivity guarantees nonnegative reverse jump rates, it does not ensure Bayes realizability: ratios at a noisy state need not be jointly induced by any clean-token posterior under the forward kernel. The score-entropy loss has the correct population optimum but does not enforce this constraint away from it. In a trained pure-uniform SEDD checkpoint, roughly one quarter of complete score vectors violate the coordinate box, while more than half lie inside it yet remain materially incompatible with any valid posterior. Such violations can produce negative pre-normalization weights in finite-step sampling. Projecting raw scores onto the bridge polytope removes all observed negative weights and improves external generative PPL from $203.6$ to $175.1$ without changing the sampler. We introduce \emph{mean-to-score} (M2S), which predicts a clean-token posterior mean and converts it to the score through an exact kernel-dependent linear map. The construction applies to any known coordinate-wise continuous-time Markov chain (CTMC) satisfying a mild support condition. For uniform corruption, it maps the probability simplex onto the bridge polytope; for absorbing-mask corruption, the resulting objective recovers MD4 exactly. In a controlled 28.4M-parameter CIFAR-10 comparison, M2S lowers test BPD from $3.173$ to $3.129$ and FID-50k from $\CifarSEDDFID$ to $\CifarMtwoSFID$. A 170M-parameter M2S model trained on about 262B OpenWebText token slots outperforms the evaluated pure-uniform SEDD, GIDD, and Neural CTMC checkpoints at every tested sampling budget, reaching generative PPL $143.3$ at 128 steps versus $183.6$ for the strongest pure-uniform baseline.

Comment: Posterior-mean parameterization constrains discrete diffusion scores to be Bayes-realizable by construction.

Topic Match: The paper introduces a fundamental parameterization and training-objective correction for discrete diffusion models.

Relevance: 8 Novelty: 8

2. Context-weighted Discrete Flow Matching

ArXiv ID: 2607.21427

Primary Topic: Architecture and Training Dynamics

Authors: Daniil Cherniavskii, Daniel Severo, Karen Ullrich

Abstract: Discrete flow matching provides a flexible framework for generative modeling on discrete structures. However, the standard factorized training objective exposes the model to targets of varying difficulty, mixing well-conditioned, predictable tokens with ambiguous, high-entropy ones. We empirically demonstrate that the uncertainty over the value of each token is closely related to the density of available context in its neighborhood. Motivated by this observation, we propose a simple modification to the underlying continuous-time Markov chain (CTMC) that incorporates local context information. Our context-weighted sampler improves generation quality with negligible computational overhead, while our scaled cross-entropy loss function reweights the training signal from different tokens and reduces generative perplexity by up to 63% on OpenWebText. Moreover, our approach matches a strong semi-autoregressive block diffusion baseline in quality while retaining the ability to perform generation in any order. These results highlight the role of local context as an important factor in discrete generative modeling and show that simple context-aware modifications can significantly improve both sampling and training efficiency.

Comment: Modifies discrete flow matching's CTMC and training loss using local context density.

Topic Match: The core contribution changes both the generative transition mechanism and its token-level training dynamics.

Relevance: 8 Novelty: 7

3. Limit Theorems for Stochastic Gradient Descent in High-Dimensional Single-Layer Networks

ArXiv ID: 2511.02258

Primary Topic: Architecture and Training Dynamics

Authors: Parsa Rangriz

Abstract: This paper studies the high-dimensional scaling limits of online stochastic gradient descent (SGD). Building on the work of Ben Arous, Gheissari, and Jagannath on the effective dynamics of SGD, we study the critical scaling regime of the step size for single-layer networks. Below this regime the effective dynamics are governed by deterministic (ballistic) limits, whereas at the critical scale a correction term emerges that changes the phase diagram. Near the fixed points of these dynamics, we show that the diffusive (SDE) limit of the rescaled correlation is an Ornstein-Uhlenbeck process. More precisely, it is mean-reverting whenever the information exponent is at least three. At information exponent two the drift has no universal sign, and the fixed point may become repelling; we show this explicitly for phase retrieval, where the sign is determined by the step size and the noise level. These results illustrate the limitations of deterministic scaling limits in capturing stochastic fluctuations in high-dimensional learning dynamics.

Comment: Derives critical-scale stochastic limits that expose fluctuation-driven changes in high-dimensional SGD dynamics.

Topic Match: It directly analyzes optimization dynamics and failure of deterministic approximations near critical scaling.

Relevance: 8 Novelty: 7

4. Gumbel Distillation for Parallel Text Generation

ArXiv ID: 2603.22216

Primary Topic: Architecture and Training Dynamics

Also Matches: Efficiency, Compression, and Large-Scale Training

Authors: Chi Zhang, Xixi Hu, Bo Liu, Qiang Liu

Abstract: The slow, sequential nature of autoregressive (AR) language models has driven the adoption of parallel decoding methods. However, these non-AR models often sacrifice generation quality as they struggle to model the complex joint distribution of token sequences. To narrow this performance gap, we introduce Gumbel Distillation, a novel distillation technique that enables parallel decoders to learn this distribution effectively. Our method leverages the Gumbel-Max trick to create a deterministic mapping from a latent Gumbel noise space to the output tokens of a high-performing AR teacher. As a model-agnostic technique, Gumbel Distillation seamlessly integrates with diverse parallel decoding architectures, including MDLM and BD3-LM. Experiments on LM1B and OpenWebText show that Gumbel Distillation substantially improves the generation quality of parallel language models, achieving a 30.0% improvement in MAUVE score and 10.5% in generative perplexity over MDLM trained on OpenWebText dataset. Code available at https://github.com/hxixixh/gumbel-distill.

Comment: Gumbel-noise coupling lets parallel decoders distill the autoregressive teacher's joint sequence distribution.

Topic Match: The core contribution is a new training mechanism for high-quality parallel generation architectures.

Relevance: 7 Novelty: 8

5. LatentChem: From Textual CoT to Latent Thinking in Chemical Reasoning

ArXiv ID: 2602.07075

Primary Topic: Architecture and Training Dynamics

Authors: Xinwu Ye, Yicheng Mao, Yuxuan Liao, Jia Zhang, Yimeng Liu, Li Hao, Fang Wu, Zhiwei Li, Zehong Wang, Zhiyuan Liu, Zhenfei Yin, Li Yuan, Philip Torr, Huan Sun, xiangxiang Zeng, Mengdi Wang, Le Cong, Shenghua Gao, Xiangru Tang

Abstract: Current chemical large language models (LLMs) predominantly rely on explicit Chain-of-Thought (CoT) to solve complex reasoning problems. However, forcing nonverbal tacit chemical logic into discrete natural language imposes a fundamental ``modality mismatch,'' creating an artificial bottleneck for reasoning. We introduce LatentChem, a reasoning interface that decouples chemical logic from linguistic generation, enabling the model to process information via continuous thought vectors and dynamic perception. Our investigation reveals a pivotal emergent behavior: spontaneous internalization, defined here as self-selected under outcome-only optimization. When optimized for task success, the model abandons verbose textual derivations in favor of implicit latent computation, suggesting that it identifies the continuous manifold as a more native substrate for chemical logic. This paradigm shift also proves to be a superior computational strategy: LatentChem achieves a 59.88\% non-tie win rate against the strong CoT baseline on the rigorous ChemCoTBench, while delivering a broad 10.84$\times$ average reduction in reasoning step overhead (5.96$\times$ wall-clock speedup) across all evaluated benchmarks. Our results provide empirical evidence that chemical reasoning is more naturally and effectively realized as continuous latent dynamics rather than discretized linguistic trajectories.

Comment: Replaces explicit textual reasoning with dynamically selected continuous latent computation.

Topic Match: The latent-reasoning interface is a new computational mechanism, despite its chemistry-focused evaluation.

Relevance: 7 Novelty: 8

6. Theoretical Aspects of Lie Groupoid and Lie Algebroid Equivariant Convolutional Neural Networks

ArXiv ID: 2606.02758

Primary Topic: Architecture and Training Dynamics

Authors: Michael Astwood

Abstract: We introduce Lie groupoid equivariant neural networks as a specialization of recently proposed topological category-equivariant neural networks to the differentiable setting. Lie groupoid equivariant neural networks are composed from Lie groupoid lifting convolutions and Lie groupoid convolution layers, and we show how for suitable Lie groupoids they are equivalent to certain Lie algebroid-equivariant neural networks. We additionally describe groupoid invariant global pooling as a generalization of group invariant global pooling. Furthermore, we show that each of the aforementioned layers is a special case of recently introduced admissible category-equivariant layers by demonstrating that they define continuous natural transformations between continuous feature functors.

Comment: Lie groupoid lifting and convolution layers define a differentiable class of generalized equivariant networks.

Topic Match: It is a foundational architectural construction, although peripheral to large-model training systems.

Relevance: 6 Novelty: 8

7. AG-REPA: Causal Layer Selection for Representation Alignment in Audio Flow Matching

ArXiv ID: 2603.01006

Primary Topic: Architecture and Training Dynamics

Authors: Pengfei Zhang, Tianxin Xie, Minghao Yang, Li Liu

Abstract: REPresentation Alignment (REPA) improves the training of generative flow models by aligning intermediate hidden states with pretrained teacher features, but its effectiveness in token-conditioned audio Flow Matching critically depends on the choice of supervised layers, which is typically made heuristically based on the depth. In this work, we introduce Attribution-Guided REPresentation Alignment (AG-REPA), a novel causal layer selection strategy for representation alignment in audio Flow Matching. Firstly, we find that layers that best store semantic/acoustic information (high teacher-space similarity) are not necessarily the layers that contribute most to the velocity field that drives generation, and we call it Store-Contribute Dissociation (SCD). To turn this insight into an actionable training guidance, we propose a forward-only gate ablation (FoG-A) that quantifies each layer's causal contribution via the induced change in the predicted velocity field, enabling sparse layer selection and adaptive weighting for alignment. Across unified speech and general-audio training (LibriSpeech + AudioSet) under different token-conditioning topologies, AG-REPA consistently outperforms REPA baselines. Overall, our results show that alignment is most effective when applied to the causally dominant layers that drive the velocity field, rather than to layers that are representationally rich but functionally passive.

Comment: Causal gate ablation identifies which intermediate layers should receive representation-alignment supervision.

Topic Match: The central contribution is a mechanistic layer-selection rule that changes how flow models are trained.

Relevance: 6 Novelty: 7

8. CHERRY: Compressed Hierarchical Experts with Recurrent Representational Yield

ArXiv ID: 2606.31796

Primary Topic: Architecture and Training Dynamics

Also Matches: Efficiency, Compression, and Large-Scale Training

Authors: Dohyeon Kwon, Youngjin Park

Abstract: Frontier language capability is usually bought with frontier compute; CHERRY shows a different trade. It is a sovereign Korean model family built on one principle: supervise the tokens that decide the answer, and let shared weights carry the rest. Under matched compute this exposes a sharp, reproducible dissociation---selected-token supervision preserves held-out discrimination yet collapses free generation, and a full-sequence anchor recovers only part of the gap. The same signal drives a heal-after-merge recurrent-representational-yield loop that collapses 48 layers to 6 unique blocks at near-dense parity (227M at loss 2.934 vs a 566M dense model at 2.926) and composes them by MoEE fusion (2.789)---a recurrent-compression direction independently pursued by concurrent frontier looped-MoE work, which we project (not yet measure) to frontier scale. It also installs metacognition from two-token supervision (200 held-out KO prompts/type, kappa>0.82, +/-6.9pp): self-correction 12->47% and jailbreak 23->4% at 97.6% loss-retention on 1.2B, with a pre-registered 1B->13.7B ablation localising the operand-binding limit to capacity (1B lookup vs 13.7B H-PRESERVE); and it specializes a 1.8B model to reported human-expert level on CyberMetric (75.0% vs a 30-expert average 72.24%). The released 1.8B tokenizer is 9.2% more Korean-efficient than Gemma-4; the from-scratch 12B adds a sovereign Korean tokenizer (vocab 131,037). On the government-operated K-AI Korean-LLM leaderboard the released 1.8B leads the HLE(Ko) column (0.123 vs 0.077; overall 51/78). At the frontier we fine-tune and serve an adapted 122B model on one 120GB accelerator (83GB measured peak). Provenance differs by member and we state it exactly; the architecture and recipe are ours throughout, and every claim is bound to a released curated measurement.

Comment: Recurrent block sharing and post-merge healing compress a 48-layer model into six unique blocks.

Topic Match: Its strongest foundational contribution is the recurrent layer-sharing and compression recipe.

Relevance: 6 Novelty: 7

9. From Noise to Diversity: Random Embedding Injection in LLM Reasoning

ArXiv ID: 2605.11936

Primary Topic: Architecture and Training Dynamics

Authors: Heejun Kim, Seungpil Lee, Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Taesup Kim, Sundong Kim

Abstract: Recent soft prompt research has tried to improve reasoning by inserting trained vectors into LLM inputs, yet whether the gain comes from the learned content or from the act of injection itself has not been carefully separated. We study Random Soft Prompts (RSPs), which drop the training step entirely and append a freshly drawn sequence of random embedding vectors to the input. Each RSP vector is sampled from an isotropic Gaussian fitted to the entrywise mean and variance of the pretrained embedding table; the sequence carries no learned content, and yet reaches accuracy comparable to optimized soft prompts on math reasoning benchmarks in several settings. The mechanism unfolds in two stages: because attention has to absorb a never-seen-before random position, the distribution over the first few generated tokens flattens and reasoning trajectories branch, and as generation continues this influence dilutes naturally so the response commits to a single completion. We show that during inference RSPs lift early-stage token diversity and, combined with temperature sampling, widen Pass@N, the probability that at least one out of N attempts is correct. Beyond inference, we carry the same effect into DAPO training and demonstrate practical gains. Our contributions are: (i) RSP isolates the simplest form of soft prompt -- training-free, freshly resampled -- providing a unified lens for the structural effect of injection that variants otherwise differing in training and form all share; (ii) a theoretical and empirical validation of the underlying mechanism; and (iii) an extension from inference to training.

Comment: Shows that random soft-prompt vectors reshape early-token uncertainty and diversify reasoning trajectories.

Topic Match: Its strongest fit is the mechanistic analysis of how injected embeddings alter attention and generation dynamics.

Relevance: 6 Novelty: 7

Efficiency, Compression, and Large-Scale Training (15)

1. KroQuant: Kronecker-Structured Block Transforms for Efficient Post-Training Quantization of Diffusion Transformers

ArXiv ID: 2607.21446

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Yann Bouquet, Alireza Khodamoradi, Kristof Denolf, Mathieu Salzmann

Abstract: Post-training quantization (PTQ) of diffusion transformers (DiTs) to W4A4 severely degrades output quality, because activations entering each linear layer contain outliers that 4-bit formats cannot represent. The standard fix applies an invertible linear transform to the activations and its inverse to the weights before quantizing both. Normalization layers between blocks force this transform to run online at every denoising step, making its inference computation cost the binding design constraint. Existing options trade quantization quality for inference cost: per-channel scaling (SmoothQuant) is computationally cheap but impacts the magnitude of the channels, which can harm quantization accuracy; fixed Hadamard transforms yield better quantization accuracy but require large block sizes that incur a high online cost; learned full-$d$ invertible transforms calibrate best but entail a prohibitive dense $d \times d$ matrix multiplication (GEMM) per layer per step. We propose KroQuant, a PTQ method that applies a learned Kronecker-structured invertible transform to each 32-element block of the activation, storing less than half the parameters of per-channel scaling. The block-local structure runs as small tensor-core GEMMs, and on an MI350 GPU the KroQuant quantizer kernel is up to $14\%$ faster than the SmoothQuant kernel. Offline LoRaQ weight calibration then absorbs the residual per-weight quantization error. On PixArt-$Σ$, SANA, and FLUX.1-schnell at W4A4 (MXFP4e2), KroQuant produces outputs closer to the FP reference than SVDQuant and LoRaQ on MJHQ-30K and SDCI, while preserving or improving image quality.

Comment: Learned Kronecker-structured block transforms enable accurate and efficient W4A4 diffusion-transformer quantization.

Topic Match: The paper's core contribution is a new structured transformation for low-bit post-training quantization.

Relevance: 9 Novelty: 8

2. LinearARD: Linear-Memory Attention Distillation for RoPE Restoration

ArXiv ID: 2604.00004

Primary Topic: Efficiency, Compression, and Large-Scale Training

Also Matches: Architecture and Training Dynamics

Authors: Ning Yang, Hengyu Zhong, Wentao Wang, Baoliang Tian, Haijun Zhang, Jun Wang

Abstract: The extension of context windows in Large Language Models is typically facilitated by scaling positional encodings followed by lightweight Continual Pre-Training (CPT). While effective for processing long sequences, this paradigm often disrupts original model capabilities, leading to performance degradation on standard short-text benchmarks. We propose LinearARD, a self-distillation method that restores Rotary Position Embeddings (RoPE)-scaled students through attention-structure consistency with a frozen native-RoPE teacher. Rather than matching opaque hidden states, LinearARD aligns the row-wise distributions of dense $Q/Q$, $K/K$, and $V/V$ self-relation matrices to directly supervise attention dynamics. To overcome the quadratic memory bottleneck of $n \times n$ relation maps, we introduce a linear-memory kernel. This kernel leverages per-token log-sum-exp statistics and fuses logit recomputation into the backward pass to compute exact Kullback-Leibler divergence and gradients. On LLaMA2-7B extended from 4K to 32K, LinearARD recovers 98.3\% of the short-text performance of state-of-the-art baselines while surpassing them on long-context benchmarks. Notably, our method achieves these results using only \textbf{4.25M} training tokens compared to the \textbf{256M} tokens required by LongReD and CPT. Our code is available at https://github.com/gracefulning/LinearARD.

Comment: An exact linear-memory backward kernel distills dense attention relations while restoring RoPE-scaled long-context models.

Topic Match: The strongest contribution removes quadratic training memory from attention-structure distillation.

Relevance: 8 Novelty: 8

3. Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context

ArXiv ID: 2607.21535

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Alagappan Valliappan

Abstract: Speculative decoding accelerates autoregressive generation by having a cheap draft propose tokens that a target verifies in parallel. Frontier models increasingly ship a built-in Multi-Token-Prediction (MTP/NEXTN) draft head under the assumption that the draft is negligibly cheap. At million-token context this breaks: an MTP draft head typically runs full attention over the entire KV cache at every draft step, so its read grows linearly with context and comes to dominate the draft cost -- precisely where speculation is most valuable. The effect compounds with draft length (a deep native draft can turn net-negative, slower than no speculation) and sharpens under hybrid/linear-attention targets, where cheaper verification leaves the draft's full-attention read exposed. We apply a StreamingLLM-style sliding window plus attention sink to the draft's attention only (Windowed-MTP), leaving full-attention verification intact. It is training-free, drop-in, and lossless by construction: the full-attention target still decides every accepted token, so windowing changes only which tokens are proposed, never which are accepted. It bounds the draft's KV working set to a constant, dropping ~99% of KV entries at 1M. Across three architecture families (Qwen GDN-MoE 35B/122B and a Mamba2-hybrid NoPE 120B) at 1M context on a single GPU in SGLang, windowing cuts the per-decode-step cost over the shipping native MTP draft by +28% to +44%, an input-invariant margin that widens with context. Since per-token latency is this cost divided by acceptance length, at matched acceptance end-to-end decode latency improves by the same amount, and more where windowing also lifts acceptance, while preserving the target's verified output distribution. Finally, the unread draft KV -- 7.7-11% of total KV at 1M -- is reclaimed via a compact ring buffer at no acceptance or quality cost.

Comment: Windowing only the MTP draft attention bounds draft KV-cache cost at million-token context.

Topic Match: It directly introduces a KV-memory optimization that materially reduces long-context decoding cost.

Relevance: 8 Novelty: 7

4. Adaptive Depth Sparse Framework: Similarity-Driven Resource Allocation for Pre-Trained LLMs

ArXiv ID: 2607.21291

Primary Topic: Efficiency, Compression, and Large-Scale Training

Also Matches: Architecture and Training Dynamics

Authors: Yidu Wu, Xiang Wang, Kejie Zhao, Zhangchi Wang, Qinghai Guo, Xiaoying Tang

Abstract: Large language models (LLMs) achieve strong generation and reasoning performance, but the Transformer architecture incurs high inference cost. Existing acceleration methods often rely on task-specific fine-tuning or training from scratch, increasing adaptation cost and limiting cross-task usability. We present an Adaptive Depth Sparse Framework (AdaDSF) that converts off-the-shelf pre-trained LLMs into depth-sparse models without full retraining. Our key insight is that layers contribute unequally to representation transformation, characterized by the cosine similarity between layer input and output hidden states. Based on this, AdaDSF assigns layer-wise token retention ratios from similarity statistics, uses a lightweight router to select informative tokens at each layer, and introduces a feature-preserving alignment objective to match intermediate and final representations between sparse and dense models. On GPT-NeoX and Qwen2.5 over language modeling and commonsense reasoning, AdaDSF substantially reduces inference FLOPs while preserving performance close to dense counterparts. Under comparable sparsity, AdaDSF consistently yields smaller accuracy degradation than strong baselines including MoD, D-LLM, and DLO.

Comment: Similarity-guided token routing assigns layer-specific sparsity to pretrained LLMs without full retraining.

Topic Match: The primary contribution is adaptive depth and token sparsity that reduces inference computation.

Relevance: 8 Novelty: 7

5. RED-PIM: Reducing Data Movement for Transformers using Processing-in-Memory

ArXiv ID: 2607.21731

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Zahra Yousefijamarani, Alaa Alameldeen

Abstract: Transformers are widely used across many domains, including natural language processing, computer vision, web search, and DNA sequence analysis. Given their broad applicability, improving the performance of transformer models is critical. However, the high volume of data movement between processing units and memory during attention operations significantly limits their efficiency. Processing-In-Memory (PIM) mitigates this issue by performing computations directly inside memory. While prior work has proposed PIM-based transformer implementations, they suffer from costly inter-bank communication, and struggle to scale due to the limited capacity of memory banks. As a result, attention-related data must be split across banks, diminishing the potential benefits of PIM. In this work, we propose RED-PIM, an algorithm-architecture co-design that reduces attention latency by minimizing inter-bank data movement from O(N^2) to O(N) and shrinking intermediate attention matrices from N x N to d x d. By reorganizing matrix operations, performing computations locally, and employing an optimized data transfer strategy, RED-PIM significantly reduces computation cost and interconnect traffic. Compared to baseline PIM implementation, RED-PIM achieves inference time reductions ranging from 16.05% to 99.99% (geometric mean of 66.42%), with the largest gains on longer sequences. On real-world datasets, RED-PIM improves performance by 99.60% for long documents and 13.44% for shorter ones, while maintaining or improving accuracy. These results demonstrate RED-PIM's effectiveness for scalable and efficient transformer inference.

Comment: An algorithm-architecture co-design reduces PIM attention communication from quadratic to linear in sequence length.

Topic Match: It materially lowers transformer inference cost through a new memory-local attention organization.

Relevance: 7 Novelty: 7

6. PreMoE: Proactive Inference for Efficient Mixture-of-Experts

ArXiv ID: 2505.17639

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Zehua Pei, Ying Zhang, Hui-Ling Zhen, Tao Yuan, Xianzhi Yu, Zhenhua Dong, Sinno Jialin Pan, Mingxuan Yuan, Bei Yu

Abstract: Mixture-of-Experts (MoE) models offer dynamic computation, but are typically deployed as static full-capacity models, missing opportunities for deployment-specific specialization. We introduce PreMoE, a training-free framework that proactively compiles sparse MoE variants for targeted deployment scenarios. At its core is Predicted Expert Utility (PEU), a robust metric for estimating expert importance from router logits through high-confidence threshold filtering and logit transformation, which together stabilize utility estimation under aggressive sparsity. Using PEU scores computed on a small calibration set, PreMoE produces domain-aware expert rankings that can be used to compile either domain-specific specialists or high-efficiency multi-domain generalists, without any retraining. Across MoE models ranging from 30B to 718B parameters, PreMoE achieves up to 50\% sparsity with nearly no performance loss. It further exposes a practical deployment trade-off: specialists maximize in-domain efficiency, while synthesized generalists retain broader cross-domain capability at the same sparsity budget. Code: https://github.com/JarvisPei/PreMoE

Comment: Router-logit utility estimates support training-free, domain-aware expert pruning in very large MoE models.

Topic Match: The main contribution is deployment-time MoE compression rather than router or expert training.

Relevance: 7 Novelty: 7

7. DriftXpress: Faster Drifting Models via Projected RKHS Fields

ArXiv ID: 2605.12183

Primary Topic: Efficiency, Compression, and Large-Scale Training

Also Matches: Architecture and Training Dynamics

Authors: Ali Falahati, Elliot Creager, Gautam Kamath, Shubhankar Mohapatra

Abstract: Drifting Models have emerged as a new paradigm for one-step generative modeling, achieving strong image quality without iterative inference. The premise is to replace the iterative denoising process in diffusion models with a single evaluation of a generator. However, this creates a different trade-off: drifting reduces inference cost by moving much of the computation into training. We introduce DriftXpress, an accelerated formulation of drifting models based on projected RKHS fields. DriftXpress approximates the drifting kernel in a low-rank feature space. This preserves the attraction-repulsion structure of the original drifting field while reducing the cost of field evaluation. Across image-generation benchmarks, DriftXpress achieves comparable FID to standard drifting while reducing wall-clock training cost. These results show that the training-inference trade-off of drifting models can be pushed further without giving up their one-step inference advantage.

Comment: Projected low-rank RKHS fields reduce the wall-clock training cost of one-step drifting models.

Topic Match: The primary contribution is a low-rank computational reformulation that changes the training-inference cost trade-off.

Relevance: 7 Novelty: 7

8. EvoSpec: Evolving Speculative Decoding via Real-Time Vocabulary and Parameter Adaptation

ArXiv ID: 2605.27390

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Shuyu Zhang, Lingfeng Pan, Qicheng Wang, Yaqi Shi, Yueyang Tan, Ruyu Yan, Jiaqi Chen, Lixing Du, Lu Wang

Abstract: Speculative decoding accelerates Large Language Model inference through draft-then-verify generation, yet lightweight draft models face coupled efficiency and quality limitations: large-vocabulary output projection is costly, while limited draft capacity and static parameters reduce acceptance under specialized or shifting inputs. Vocabulary pruning lowers projection cost, but static variants miss locally important long-tail tokens, while dynamic variants remain sensitive to preset selection policies and budgets. Moreover, limited draft capacity can leave the draft distribution misaligned even when the target token is covered. Online alignment improves draft quality, but full-parameter updates introduce substantial memory and latency overhead. We introduce EvoSpec, which jointly adapts the active vocabulary and lightweight draft parameters from verification feedback. EvoSpec asynchronously retrieves semantic and statistical token neighbors and performs curriculum-weighted online LoRA alignment while preserving exact target-model verification. On Qwen3-8B/EAGLE-2, EvoSpec reaches a $2.18\times$ speedup over vanilla decoding and a $1.20\times$ gain over EAGLE-2, while improving specialized-domain coverage and using $27\%$ less auxiliary GPU adaptation memory than full-parameter online adaptation.

Comment: Verification feedback jointly adapts a speculative draft's active vocabulary and lightweight LoRA parameters online.

Topic Match: The method materially improves speculative-decoding efficiency through adaptive vocabulary and parameter compression.

Relevance: 7 Novelty: 7

9. Three-Pronged Spectral Control for Federated Parameter Efficient Fine Tuning

ArXiv ID: 2607.20914

Primary Topic: Efficiency, Compression, and Large-Scale Training

Also Matches: Large-Scale Training Systems and Efficiency

Authors: Shiva Raj Pokhrel, Dipsan Bhattarai, Anwar Walid

Abstract: Federated parameter-efficient fine-tuning (PEFT) enables communication-efficient adaptation of large pretrained models on decentralized edge data, but it remains fragile under non-IID client heterogeneity. In low-rank adaptation (LoRA), different clients may learn locally useful but spectrally misaligned update subspaces, causing high-variance aggregation and poor global transfer. We propose TRISHUL, a spectral-control framework for robust federated PEFT. TRISHUL follows the FL no-raw-data-sharing setting but does not itself provide formal privacy guarantees. TRISHUL uses shared frozen multi-head low-rank bases to obtain algebraically exact aggregation of compact core updates, applies nuclear norm proximal shrinkage to suppress client-specific high-rank spectral components before upload, and allocates adaptation heads non-uniformly across layers using a concave water filling budget rule derived from pretrained layer capacity. Because shrinkage is performed only on small core matrices, TRISHUL adds negligible computation and no extra per-round communication over the underlying multi-head PEFT protocol. Across vision and language benchmarks, including CIFAR-100, SVHN, 20 Newsgroups, MRQA, and GLUE with LLaMA3.2-1B, TRISHUL improves convergence, stability, and final performance over federated LoRA baselines, with greater gains under stronger heterogeneity.

Comment: Stabilizes federated LoRA aggregation through shared low-rank bases and proximal spectral shrinkage.

Topic Match: The core mechanism controls compact adapter updates, while federated aggregation also gives it a distributed-training systems connection.

Relevance: 7 Novelty: 7

10. How Many Bits Can an Adapter Write? Measuring the Capacity and Memorization of Parameter-Efficient Fine-Tuning

ArXiv ID: 2607.21351

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Kaizhen Tan, Heqing Du, Yang Feng

Abstract: A LoRA adapter is a few megabytes that almost everyone treats as a skill rather than a record of the data behind it. We put that assumption on a scale. Extending compression-based memorization analysis to the frozen-base setting, we measure directly, in bits, how much a low-rank adapter writes into a model it never changes. The answer is both smaller than full fine-tuning and less lawful than parameter counting would predict. Adapters store a couple of bits per trainable parameter, well short of a full model's budget, but that figure turns less on how many parameters an adapter carries than on where they sit. Move the same parameter budget from attention into the MLP and it holds nearly twice as much; strip the frozen base of its structure and the capacity all but disappears. Applied to realistic fine-tunes of Qwen2.5, the same instrument shows privacy leakage rising with the bits an adapter writes rather than the parameters it nominally has, and it draws a clean line between supervised and reinforcement learning: the secrets that supervised fine-tuning copies down verbatim, an adapter trained on verifiable rewards never records. Measuring what fine-tuning writes, rather than attacking it after the fact, turns a piece of folklore into a quantity one can design against.

Comment: Measures LoRA memorization capacity directly in bits and shows that module placement matters more than parameter count.

Topic Match: It offers a mechanistic capacity analysis of low-rank adaptation, a central parameter-efficient training method.

Relevance: 7 Novelty: 7

11. AnchorPrune: Relevance-Anchored Contextual Expansion for Visual Token Pruning

ArXiv ID: 2607.07033

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Kyuan Oh, Bumsoo Kim

Abstract: Large vision-language models incur substantial inference costs because high-resolution inputs introduce thousands of visual tokens, many of which are redundant for a given query. Existing pruning methods often combine query relevance and token diversity, yet these objectives can conflict under aggressive compression: relevance-driven selection may overconcentrate the budget on correlated local evidence, while diversity-driven selection may suppress indispensable tokens or retain distinct but uninformative regions. We introduce AnchorPrune, a training-free framework that first constructs a protected relevance anchor and then expands it with complementary visual context. AnchorPrune adaptively determines the anchor size from the novelty profile of relevance-ranked tokens, preserving a compact set of query-critical evidence, and allocates the remaining budget through importance-weighted novelty to recover informative, non-redundant context relative to the anchor. This ordered design prevents contextual expansion from displacing indispensable query cues while improving overall visual coverage. AnchorPrune is lightweight, architecture-aware, and requires neither retraining nor model modification. Across image and video vision-language models and benchmarks, it consistently improves the accuracy-efficiency trade-off over training-free baselines, particularly under severe compression. On LLaVA-NeXT-7B, AnchorPrune preserves 97.6% of full-token performance using only 160 of 2,880 visual tokens. These results establish relevance-anchored contextual expansion as an effective principle for efficient multimodal inference. Code is available at https://github.com/MULTI-cau/AnchorPrune.

Comment: Prunes visual tokens by protecting relevance anchors before adding complementary contextual coverage.

Topic Match: The central contribution is a new token-pruning mechanism that sharply reduces multimodal inference cost.

Relevance: 7 Novelty: 6

12. Adversarial Prompts for Acceptance Collapse in Speculative Decoding

ArXiv ID: 2607.21804

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Run Wang, Chaoyi Zhou, Xi Liu, Yi Zhu, Amir Salarpour, Pedram MohajerAnsari, Zhi-Qi Cheng, Feng Luo, Siyu Huang, Mert D. Pesé

Abstract: Lossless acceleration schemes, such as speculative decoding, promise significant inference speedups by relying on dynamic token-level alignment between a draft and a target model. However, this guarantee of semantic equivalence masks a severe operational vulnerability: draft-target alignment can be systematically attacked. In this paper, we introduce ADSD, which, to the best of our knowledge, is the first prompt-suffix attack that collapses verifier acceptance by pushing draft probability mass toward tokens the target is unlikely to accept. ADSD uses Soft-Collapse, a verifier-aligned surrogate derived from the asymmetric speculative acceptance rule, together with a target-preservation objective that discourages obvious task corruption. ADSD successfully generates highly effective adversarial suffixes. On the GSM8K dataset, our attack increases the mean sample time by 62.3% while preserving the task quality. We further show that this vulnerability exists across different domains, speculative decoding strategies, and model architectures.

Comment: A verifier-aligned suffix attack systematically collapses speculative-decoding acceptance and increases latency.

Topic Match: It reveals a new operational failure mode of a major lossless inference-efficiency mechanism.

Relevance: 6 Novelty: 7

13. Source-Prior-Driven Selective Adaptation for Efficient Diffusion Model Finetuning

ArXiv ID: 2607.20913

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Yi Xiong, Yuan-Yuan Cheng, Xiao-Ming Fu

Abstract: Fine-tuning large diffusion models for new domains or styles involves a trade-off: improving target-specific generation often degrades the pretrained model's broad generative capability. Existing full and parameter-efficient fine-tuning methods typically handle this trade-off only implicitly. In this work, we propose a novel source-prior-driven selective adaptation method to efficiently fine-tune diffusion models, achieving a favorable trade-off. Our method relies on two key observations: (1) the loss of general generative capability is highly inconsistent across pretrained parameters, and (2) parameters that have a relatively small impact on the model's general generative capability remain structurally inconsistent across layers and parameter types. Motivated by these observations, we first learn a static mask to explicitly identify parameters better suited for downstream adaptation, and then construct structured update strategies for the selected subset. Experiments show that our method achieves a better adaptation-retention trade-off than existing strong baselines.

Comment: A source-prior mask restricts diffusion-model adaptation to parameters least important for pretrained capability retention.

Topic Match: The central method is structured sparse parameter-efficient adaptation.

Relevance: 6 Novelty: 6

14. Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts

ArXiv ID: 2607.09999

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Renuka Oladri, Mohan Vamsi Varadaraju Priya, Jerry Wu

Abstract: We show that post-training quantization can silently alter how large language models reason even when task accuracy is preserved. Using a six-category failure taxonomy validated by two independent human annotators (Cohen's $κ$ = 0.906), we classify 30,000 chain-of-thought outputs from five instruction-tuned LLMs (3B--14B parameters) across three quantization precisions (FP32, FP16, NF4) and four reasoning benchmarks. We find that while accuracy is robust across precisions (maximum 3.1 pp drop), Hollow Convergence (correct answers reached through incomplete or unverifiable reasoning) shows a significant size-dependent shift under NF4, dropping sharply for the two smallest models tested but remaining invariant for models at 12B parameters and above. This effect is also benchmark-specific: GSM8K is categorically immune while LogiQA and ARC-Challenge show the largest shifts. Furthermore, under NF4, Shortcut Collapse rises from 44% to 78% of wrong-answer failures in LLaMA 3.2-3B while Confidence Snowballing collapses from 15.8% to near zero, a qualitative shift invisible to accuracy metrics. Finally, we show Hollow Convergence cannot be reliably detected from surface-level text features (best F1 = 0.53), establishing it as a deployment-relevant failure mode that standard evaluation pipelines cannot catch.

Comment: Failure-mode analysis shows that NF4 quantization can alter reasoning trajectories while preserving answer accuracy.

Topic Match: It provides mechanistic evidence about behavioral risks introduced by a core compression technique.

Relevance: 6 Novelty: 6

15. Gibbs randomness-compression proposition

ArXiv ID: 2505.23869

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: M Süzen

Abstract: A proposition that connects randomness and compression is put forward via Gibbs entropy over set of measurement vectors associated with a lossy compression process. In building this connection, we use a performance of a learning task as a probe of compression in iterative compress-train cycles. This can be thought as iterative coarse-graining from statistical mechanics perspective using thermodynamic efficiency as a probe. We formulate this connection via comonotonic relationship within a very small decrease in compression ratio and the performance. We have showcase the validity of this proposition with a canonical vision task in deep learning with three different model compression processes as {\it a baseline model}. We use the following, simpler to more complex model compression approaches: (1) random pruning,(2) magnitude pruning, and (3) a more complex compression by using dual tomographic compression, which utilizes compressed sensing in dual fashion which is introduced as a new method. We use remaining weights of deep learning network as a measurement vector where we measure the Gibbs entropy. We show case the idea that there is an inherent computable connection between compression probed by performance and randomness from an entropy measure on the learned model.

Comment: Connects pruning-induced compression to Gibbs entropy through iterative compress-train cycles.

Topic Match: Its central analysis concerns model compression behavior and a new compression construction.

Relevance: 6 Novelty: 6

Paper Selection Prompt

System Prompt

You are a helpful paper reading assistant whose job is to read daily posts from ArXiv and identify a few papers that your friend will enjoy reading. Your job is to carefully read the paper titles and abstracts below and find the ones that match the criteria below.

User Prompt

Instructions

Respond in JSONL. Output exactly one JSON object per paper, one per line:

{"ARXIVID":"...","COMMENT":"...","RELEVANCE":0,"NOVELTY":0,"PRIMARY_TOPIC_ID":"...","MATCHED_TOPIC_IDS":[],"TOPIC_MATCH_COMMENT":"...","HOTSPOT_PAPER_TAGS":[],"HOTSPOT_PAPER_COMMENT":"..."}

Rules: - ARXIVID: the arXiv ID. - COMMENT: identify the single strongest matching criterion. Be brief and specific. Do not rely on generic phrases like "language modeling" or "advancement". Do not mention non-matching criteria. - RELEVANCE: integer from 1 to 10. - NOVELTY: integer from 1 to 10. - PRIMARY_TOPIC_ID: exactly one stable topic ID from the allowed topic registry. - MATCHED_TOPIC_IDS: zero or more stable topic IDs from the same allowed set. Include PRIMARY_TOPIC_ID when there are multiple matches. - TOPIC_MATCH_COMMENT: briefly explain why the primary topic is the best fit. - HOTSPOT_PAPER_TAGS: zero or more tags from this exact set only: daily_hot, new_frontier. - HOTSPOT_PAPER_COMMENT: briefly explain why the paper belongs in the daily hotspot paper feed when HOTSPOT_PAPER_TAGS is non-empty; otherwise use an empty string. - Use HOTSPOT_PAPER_TAGS sparingly. Most papers should return []. - daily_hot means the paper feels broadly important to the day and belongs in the daily hotspot paper section even if it is not part of the personalized foundational reading list. - new_frontier means the paper appears to open a genuinely new direction, paradigm, or field, even if the work is still early. - Do not output markdown, code fences, or any extra text.

Scoring Criteria

Relevance and Novelty are independent axes. Score both from 1 to 10.

Relevance Scoring

• 9-10: directly centered on the target foundational topics; highest when the core contribution is clearly within them.
• 7-8: substantially related, but partly peripheral or focused on a narrower aspect.
• 5-6: touches the target topics, but the main contribution is elsewhere.
• 3-4: largely outside the target topics, often application-focused or domain-specific.
• 1-2: unrelated.

Important: Broad frontier relevance, major launch status, or daily buzz is not enough for a high Relevance score here. Those cases belong in the hotspot digest unless the paper strongly matches the specialized paper topics.

Novelty Scoring

• 9-10: new paradigm, theory, or major methodological breakthrough.
• 7-8: substantial methodological advance or strong new insight.
• 5-6: meaningful but incremental extension or refinement.
• 3-4: minor, narrow, or mostly engineering or domain-specific improvement.
• 1-2: little originality; mainly standard application of existing methods.

Topic Registry

Use exactly one PRIMARY_TOPIC_ID chosen from the stable topic IDs below. - moe_training: MoE Training - Mixture-of-Experts training: routing, load balancing, expert granularity and shared experts, training stability, MoE parallelism and communication, and MoE kernels. - training_systems: Large-Scale Training Systems and Efficiency - Large-scale training systems and efficiency: distributed training algorithms, optimizers, and communication. - architecture_training: Architecture and Training Dynamics - Core architectural or computational mechanisms, dynamic computation, and training-stability dynamics. - efficiency_scaling: Efficiency, Compression, and Large-Scale Training - Compression, sparsity, memory or cache efficiency, and large-scale training systems that materially change cost or behavior.

Papers

[PAPER LIST HERE]

Relevant Topics

This is a training-side feed. The centre of gravity is Mixture-of-Experts training and the large-scale training systems it sits on; architecture and efficiency work is kept when it bears on how a large model is trained or what it costs to train.

Keep a paper when its CORE CONTRIBUTION falls in one of the four topics below. Filter it when the topic is merely the setting for a downstream application, a benchmark, or a product result.

1. MoE Training (primary) - Keep: expert routing and gating mechanisms; load balancing, with or without auxiliary losses; expert granularity, fine-grained and shared experts; MoE training stability (loss spikes, router collapse, z-loss and related fixes); expert and hybrid parallelism; all-to-all dispatch/combine and the communication schedules MoE needs; MoE kernels and grouped-GEMM style implementations; expert capacity, token dropping, and upcycling dense checkpoints into MoE. - Filter: papers that merely train on top of a MoE model without a new routing, balancing, stability, parallelism, or kernel contribution; "mixture of experts" in the classical ensemble or recommender sense.

2. Large-Scale Training Systems - Keep: distributed training algorithms (data, tensor, pipeline, sequence, and expert parallelism, FSDP and ZeRO-style sharding); optimizers and preconditioners for large-scale pretraining; communication algorithms, overlap, and collective design; throughput and MFU work that materially changes what a training run costs or whether it converges; scaling-law work that informs how such runs are configured. - Filter: routine infrastructure tuning with no new algorithmic or systems idea.

3. Architecture and Training Dynamics - Keep: new or analysed architectural mechanisms (attention variants, normalization and residual design, state-space and recurrent sequence modelling, dynamic or modular computation); optimisation and training-dynamics analysis that explains why large models train the way they do. - Filter: applying an existing architecture to a new task or benchmark with no mechanistic insight.

4. Efficiency and Compression - Keep: quantization, sparsity, pruning, low-rank adaptation, KV-cache and memory-efficient designs, and other work that materially changes the cost of running or training a large model, especially when the mechanism is new rather than a tuned variant. - Filter: straightforward application of standard efficiency methods, and deployment or serving engineering with no new idea.

Leave these to the hotspot digest unless the core contribution clearly falls in one of the four topics above: - major model or product releases - agent, tooling, and RAG launches - benchmark, leaderboard, or evaluation-only papers - representation-learning theory, interpretability, and feature-formation analysis - memory mechanisms and agent memory systems - world models, exploration, and reinforcement learning - alignment and post-training (RLHF, DPO, GRPO, RFT, instruction tuning) - downstream applications in medical imaging, segmentation, 3D vision, video understanding, information retrieval, summarization, recommendation, machine translation, speech recognition, time series, knowledge graphs, and similar domains