This is a remedial run for missed papers from 07/08/2026 to 07/08/2026.

Results generated on 09/12/2026.

Personalized Daily ArXiv Papers 2026-07-09

_{Token counts are not reported for this run. 6 of 13 model calls succeeded, 6,530s of model wall clock.}

Topic Coverage:

Table of contents by topic:

Large-Scale Training Systems and Efficiency (2)

1. Robust Federated Learning Under Real-World Client Churn Authors: Dhruv Garg, Neha Lakhani, Debopam Sanyal, Myungjin Lee, Alexey Tumanov, Ada Gavrilovska

2. Restricted Dynamic Geometric Complexity: Path-Space Reduction and Möbius--Jacobi Response Authors: Zavier Li

Architecture and Training Dynamics (13)

1. Adaptive Depth in Looped Transformers: Diagnosing Learned Halting Gates and Trajectory Readouts Authors: Andrei Cristian Popescu, Haitz Sáez de Ocáriz Borde, Pietro Liò

2. Linear Attention Architectures: Mechanisms, Trade-offs, and Cross-Layer Routing Authors: Tommaso Cerruti, Tim Rieder, George Rowlands, Lingfeng Jin, Imanol Schlag

3. Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE Authors: Haozhan Tang, Zerui Wang, Yuxian Gu, Song Han, Han Cai

4. Explaining Near-Zero Hessian Eigenvalues Through Approximate Symmetries in Neural Networks Authors: Marcel Kühn, Bernd Rosenow

5. An optimal control approach for neural network architecture adaptation with a posteriori error estimation Authors: C G Krishnanunni, Thomas Scott, Tan Bui-Thanh

6. Architecture Generalization with MetaNCA Authors: Meet Barot, Daniel Berenberg, Sina Khajehabdollahi

7. Eigenbasis-Independent Learnable Spectral Positional Encodings for Directed Graphs via Hermitian Block Krylov Subspaces Authors: Jiaqing Xie, Yuxin Wang

8. Causal Optimizer Interaction Calculus: Hidden Geometric Relaxation and Identifiable Interventions Authors: Zavier Li

9. Intrinsic Green's Learning: Supervised Learning on Manifolds via Inverse PDE Authors: Alexandre Quemy

10. FourierQK: Spectral Preprocessing of Query-Key Projections Improves Transformer Attention Authors: Athanasios Zeris

11. Optimal Learning Rate Scaling Depends on Data in Deep Scalar Linear Networks Authors: Yedi Zhang, Peter E. Latham, Leena Chennuru Vankadara, Andrew Saxe

12. Reward-Aware Population Scaling of Evolutionary Strategies in LLM Fine-Tuning Authors: Sung Cho, Gyubin Han

13. The Anatomy of Implicit Bias: Information Allocation in Neural Network Training Authors: Zhang Gongyue, Wang Zhiyong, Liu Donghan, Ren Weihong, Sheng Yixuan, Liu Honghai

Efficiency, Compression, and Large-Scale Training (7)

1. CTA-Pipelining: A Latency-Oriented Spatial Scaling Method for Multi-GPU Systems Authors: Tingkai Liu, Muralidhar Andoorveedu, Sanjoy Das, Sanjay Patel, Volodymyr Kindratenko

2. Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata Authors: Meihua Dang, Stefano Ermon

3. Fractal KV-Cache Archives: Lossless Symbolic Storage with In-Place Retrieval for Long-Context LLM Inference Authors: Vladimir Gusev

4. LoCA: Spatially-Aware Low-Rank Convolutional Adaptation of Vision Foundation Models Authors: Sojung An, Junha Lee, Sujeong You, Nam Ik Cho, Donghyun Kim

5. AnchorPrune: Relevance-Anchored Contextual Expansion for Visual Token Pruning Authors: Kyuan Oh, Bumsoo Kim

6. EdgeCompress: Coupling Multidimensional Model Compression and Dynamic Inference for EdgeAI Authors: Hao Kong, Di Liu, Shuo Huai, Xiangzhong Luo, Ravi Subramaniam, Christian Makaya, Qian Lin, Weichen Liu

7. Guidance Breaks the Fitted Operator: A Terminal-Fitted Repair for Classifier-Free Guidance Authors: Shiheng Zhang

Large-Scale Training Systems and Efficiency (2)

1. Robust Federated Learning Under Real-World Client Churn

ArXiv ID: 2607.06979

Primary Topic: Large-Scale Training Systems and Efficiency

Authors: Dhruv Garg, Neha Lakhani, Debopam Sanyal, Myungjin Lee, Alexey Tumanov, Ada Gavrilovska

Abstract: Federated Learning (FL) enables training shared models on private, on-device data, but production deployments remain constrained to slow, multi-day refresh cycles due to the complexity of coordinating massive client populations. For applications such as feed ranking, ad targeting, and personalized recommendation, model freshness: the ability to rapidly adapt to new user-local data is critical for maximizing objectives like click-through rate. This lag leaves models stale and unresponsive to volatile data distributions driven by viral trends and shifting user intent. Bridging this gap requires addressing three challenges overlooked by existing FL systems: transient client availability, dynamic data heterogeneity, and delays between model predictions and observable outcomes. We present FeLiX, an FL orchestration framework that minimizes wall-clock time-to-target accuracy on live interaction streams. FeLiX introduces three primitives: (i) streaming-aware availability tiers that leverage lightweight telemetry to identify ready clients at scale; (ii) fresh-utility selection, a dual-tier mechanism that prioritizes statistically valuable updates from devices able to meet tight refresh deadlines; and (iii) informativeness-aware, delay-robust aggregation that incorporates late, high-value updates containing ground-truth outcomes without biasing the global model toward stale distributions. Unlike prior systems that rely on unrealistic oracular knowledge of client availability, FeLiX achieves near-oracular performance in real-world settings. Across CIFAR-10, Google Speech, and realistic low-availability traces, FeLiX reduces wall-clock time-to-target accuracy by up to 2.37X while reducing communication bandwidth by 1.30X compared to state-of-the-art synchronous and asynchronous FL baselines.

Comment: Client selection and aggregation of delayed updates reduce distributed training time under client churn.

Topic Match: The core is distributed training coordination that improves time-to-accuracy and communication cost, with relevance narrowed by the federated setting.

Relevance: 7 Novelty: 6

2. Restricted Dynamic Geometric Complexity: Path-Space Reduction and Möbius--Jacobi Response

ArXiv ID: 2607.07204

Primary Topic: Large-Scale Training Systems and Efficiency

Authors: Zavier Li

Abstract: Structured preconditioners restrict optimization to a small family of positive metrics, but endpoint condition-number reachability does not measure the geometric effort required to reach a useful metric. We formulate this effort as a path-space value problem. Restricted dynamic geometric complexity is the least affine-invariant length of an admissible metric path whose endpoint reaches a Hessian-relative generalized-eigenvalue condition target. Path elimination gives an exact min-plus semigroup and Bellman principle, while fixed-horizon kinetic energy is exactly squared complexity divided by twice the horizon. The main response result is global on a Hadamard state space: geodesic convexity produces a smooth intervention-cube path branch and a uniformly coercive Jacobi form, while one Green inverse generates the value Hessian, two-sided force-to-curvature bounds, exact Möbius effects, and arbitrary prescribed finite-order responses. For the hard condition target, a bordered Jacobi--KKT theorem differentiates the moving projection endpoint and multiplier on every regular active spectral stratum; its indefinite inverse also explains why hard-target interactions need not share the unconstrained sign. The theory specializes to affine-invariant positive-definite geometry. A determinant-one two-dimensional diagonal model has an exact target interval, a closed-form forced path, and a strictly negative-definite interaction matrix. A moving diagonal Hessian gives a closed-form hard-target projection, multiplier, and pair effects of either sign, while a coordinate-sequential three-dimensional protocol yields an exact path metric strictly larger than the ambient projection distance. Thus the global Green and bordered hard-target responses are explicit laws of restricted metric-path elimination built on Bellman composition.

Comment: Quantifies the metric-path effort required for structured preconditioners to reach Hessian-conditioning targets.

Topic Match: Structured preconditioning is the closest fit, but the contribution is abstract metric-path theory with no demonstrated large-scale training method.

Relevance: 6 Novelty: 7

Architecture and Training Dynamics (13)

1. Adaptive Depth in Looped Transformers: Diagnosing Learned Halting Gates and Trajectory Readouts

ArXiv ID: 2607.20519

Primary Topic: Architecture and Training Dynamics

Also Matches: Efficiency, Compression, and Large-Scale Training

Authors: Andrei Cristian Popescu, Haitz Sáez de Ocáriz Borde, Pietro Liò

Abstract: Looped Transformers increase test-time computation by repeatedly applying a shared recurrent block. Learned halting objectives in looped Transformers typically use a single exit distribution both as the inference-time stopping rule and as the training-time weighting of per-depth losses. This entangles exit selection with trajectory formation: the gate not only chooses which recurrent state to use, but also determines how strongly each intermediate state is supervised. Consequently, poor adaptive-compute performance can arise from the readout, the induced trajectory, or their interaction. We study adaptive depth in looped Transformers through this trajectory--readout lens, across controlled synthetic tasks (modular arithmetic and binary parity) and large-scale Ouro-1.4B and 2.6B checkpoints. We find that fixed-prior depth supervision, which shapes the trajectory without an input-dependent halting policy, produces difficulty-aware trajectories whose intermediate states expose useful stopping signals, and that simple post-hoc confidence readouts often match or outperform learned linear and MLP gates. Fitting gates on frozen trajectories localizes the failure: it appears to stem mainly from the trajectory induced by joint gate training rather than from limited gate expressivity. The same pattern is present in Ouro evaluations, where pretrained ponder gates are competitive but not uniformly Pareto-optimal, and measured latency confirms that the resulting reductions in average exit depth translate into practical inference-time savings. Our systematic diagnostic evaluation reframes adaptive depth in looped Transformers as a joint problem of trajectory formation and exit readout, rather than gate learning alone, highlighting a distinction that prior learned-halting work has often left implicit.

Comment: Shows that joint halting-gate training can impair recurrent trajectories, motivating fixed-prior depth supervision for adaptive computation.

Topic Match: Directly analyzes how supervision shapes adaptive-depth Transformer computation, with billion-parameter evidence and measured inference savings.

Relevance: 9 Novelty: 7

2. Linear Attention Architectures: Mechanisms, Trade-offs, and Cross-Layer Routing

ArXiv ID: 2607.07953

Primary Topic: Architecture and Training Dynamics

Authors: Tommaso Cerruti, Tim Rieder, George Rowlands, Lingfeng Jin, Imanol Schlag

Abstract: Self-attention lets each token retrieve information from the full context, but its quadratic cost in sequence length limits training and inference at long context. This paper presents a comparative study of softmax attention and four recent recurrent linear-attention architectures: DeltaNet, Gated DeltaNet, Kimi Delta Attention, and Gated DeltaNet-2. We express these mechanisms in a common recurrent-memory notation, making explicit how they differ in expressivity, memory decay, erase and write control, training throughput, and implementation complexity. Our experiments center on 350M-parameter models trained for 15B tokens, and include optimizer and learning-rate comparisons, hybrid-versus-pure stack comparisons, sequence-length runtime measurements, larger DeltaNet runs at 1.3B and 3B parameters, and a small set of downstream evaluations. The reported speed results measure training throughput and iteration time; we do not provide an empirical inference-speed benchmark. Within the reported 350M-parameter, 15B-token sweep, Kimi Delta Attention with Muon reaches the lowest final validation loss, a pure Gated DeltaNet stack trained with AdamW has the highest normalized training throughput, hybrid stacks generally improve loss at a throughput cost, and Muon consistently lowers final validation loss relative to AdamW in the matched architecture settings we evaluate. We introduce and evaluate lightweight cross-layer routing mechanisms for DeltaNet-style memories. The most natural DeltaNet-inspired formulation, forwarding a lower layer's delta-rule write error into the next layer's value target, does not improve over matched baselines. Routing into the aligned hidden stream and forwarding the write value instead yields a modest improvement in the matched runs we report: Cross-Layer Value Routing (CLVR) lowers final validation loss for both DeltaNet and Gated DeltaNet.

Comment: Cross-layer value routing modifies delta-rule recurrent computation and modestly improves loss in matched pretraining runs.

Topic Match: Directly analyzes linear-attention mechanisms and introduces a cross-layer architectural connection, supported by controlled training comparisons.

Relevance: 9 Novelty: 6

3. Jet-Long: Efficient Long-Context Extension with Dynamic Bifocal RoPE

ArXiv ID: 2607.07740

Primary Topic: Architecture and Training Dynamics

Also Matches: Efficiency, Compression, and Large-Scale Training

Authors: Haozhan Tang, Zerui Wang, Yuxian Gu, Song Han, Han Cai

Abstract: Modern LLMs are increasingly deployed in long-context applications such as retrieval-augmented generation, repository-level coding, and agentic workflows whose accumulated reasoning and tool traces routinely push the input an order of magnitude past the pretraining window, making zero-shot context extension the dominant deployment path for open-weight checkpoints. The dominant zero-shot methods (YaRN, Self-Extend, DCA) fix a single rescaling factor up front, so an aggressive factor sacrifices short-context fidelity while a conservative one breaks down at long contexts; recent length-aware variants adapt the mapping, but with a fitted or distance-dependent schedule. We propose Jet-Long, a tuning-free zero-shot method that pairs a local RoPE-faithful window with a long-range window whose rescaling factor adapts dynamically to the current sequence length via a parameter-free analytic schedule, recovering the base model exactly at short inputs while extrapolating cleanly at long ones. An inclusion-exclusion attention merge and an on-the-fly RoPE correction rotation make the bifocal construction essentially free at inference; fused into a single CuTe kernel, long-context prefill reaches up to $1.39\times$ FA2 throughput on H100 (approaching the Hopper-only FA4), and single-batch generation incurs $\le 4\%$ overhead at every length. On Qwen3-1.7B/4B/8B up to 128K context, Jet-Long leads RULER by $+4.79$/$+2.18$/$+2.03$ pp over the strongest baseline at 1.7B/4B/8B, achieves the best overall accuracy on HELMET-RAG (a benchmark identified by HELMET as the most efficient predictor of downstream long-context performance) and attains the lowest PG-19 perplexity. Jet-Long also generalizes to hybrid attention architectures such as Jet-Nemotron for further long-context improvement without retraining, and remains hyperparameter-resilient for ease of deployment.

Comment: Length-adaptive bifocal RoPE extends context while preserving the base model's short-input positional mapping.

Topic Match: The primary contribution changes attention and positional computation; a fused kernel also improves inference efficiency.

Relevance: 8 Novelty: 7

4. Explaining Near-Zero Hessian Eigenvalues Through Approximate Symmetries in Neural Networks

ArXiv ID: 2607.07845

Primary Topic: Architecture and Training Dynamics

Authors: Marcel Kühn, Bernd Rosenow

Abstract: The Hessian of the training loss governs the local geometry of the loss landscape, yet despite existing explanations for its largest eigenvalues, the origin of the vast multitude of vanishingly small eigenvalues remains elusive. We argue that the bulk consists of the weakly lifted pseudo-Goldstone modes of the continuous symmetries of the network parametrization. In deep linear networks these symmetries are exact: they generate flat directions and hence exact zero modes, whose eigenvectors we construct explicitly. Introducing a ReLU nonlinearity as a perturbation, we show that it breaks these symmetries weakly and explicitly. Resolving the spectrum at the level of eigenvectors, we find that the high-curvature directions are orthogonal to the symmetry subspace, while the bulk lies almost entirely within it. We demonstrate the mechanism in a two-layer ReLU student--teacher model and in a network trained on CIFAR-10. A convolutional example demonstrates that the same diagnostic extends beyond fully connected layers. Together, these results link the Hessian bulk to weakly broken symmetries and clarify the origin of near-zero modes.

Comment: Weakly broken parameter symmetries explain near-zero Hessian modes in the training loss.

Topic Match: Mechanistic loss-landscape analysis directly addresses optimization geometry, with evidence from linear, ReLU, and convolutional networks.

Relevance: 8 Novelty: 7

5. An optimal control approach for neural network architecture adaptation with a posteriori error estimation

ArXiv ID: 2607.07637

Primary Topic: Architecture and Training Dynamics

Authors: C G Krishnanunni, Thomas Scott, Tan Bui-Thanh

Abstract: This work presents a novel approach for adapting neural network architecture along the depth based on a posteriori error estimation. By formulating neural network training as a continuous-time optimal control problem, we derive rigorous error estimates that quantify how approximation error distributes across network layers. This error decomposition enables a principled depth adaptation strategy: new layers are inserted at locations of maximum estimated error, allowing the network to efficiently capture complex, nonlinear variations in the underlying problem. Our framework introduces a novel network architecture that treats weights and biases as piecewise linear functions varying across layers, with the error estimator bounding the discrepancy between this discrete representation and the true continuous optimal control solution. The approach leverages dual weighted residual methodology from finite element analysis to derive computable upper bounds on the functional error. A key theoretical contribution is the derivation of explicit error bounds that decompose the total approximation error into interval-wise contributions, providing a rigorous basis for targeted architecture refinement. We demonstrate the effectiveness of our method on scientific datasets, including learning the observable-to-parameter map for the Navier-Stokes equation. Numerical results reveal that our approach consistently outperforms existing architecture adaptation methods in terms of generalization performance.

Comment: Uses computable layerwise error estimates to insert new layers where architectural refinement is most needed.

Topic Match: Principled depth adaptation and a new parameterization across layers are core architectural contributions, currently demonstrated on scientific learning problems.

Relevance: 8 Novelty: 7

6. Architecture Generalization with MetaNCA

ArXiv ID: 2607.07743

Primary Topic: Architecture and Training Dynamics

Authors: Meet Barot, Daniel Berenberg, Sina Khajehabdollahi

Abstract: Self-organization is an emergent property of life, driven by the collective behavior of individual components acting on local information. Biological neurons, through local interactions transmitted through synapses, are able to learn efficiently and can adapt their connections over an organism's lifespan. Motivated by these desirable properties of adaptability and local interaction, neural cellular automata (NCA) models have been successful at learning morphogenesis solely through local update rules, demonstrating stability over many updates and robustness to perturbations. In this work, we introduce Meta Neural Cellular Automata (MetaNCA), a framework that learns local rules which self-organize the weights of artificial neural networks. A learned rule network iteratively updates the weights of a task network using only local interactions on the computation graph. We propose a novel Weight Transformer architecture for the local rule network, which uses linear attention to aggregate signals from neighboring weights and hidden states. Once trained, the rule network generates task networks of diverse architectures without backpropagation. We show that MetaNCA generates weights for feedforward MLPs, CNNs, and ResNets on MNIST and CIFAR-100, scaling to networks of 2 million parameters. We further show that MetaNCA generalizes to architectures not seen during meta-training, and that architectural diversity in the training phase strengthens this generalization.

Comment: Meta-trained local update rules construct weights for previously unseen network architectures.

Topic Match: The core is an alternative weight-learning mechanism that generalizes across architectures, demonstrated up to 2M parameters.

Relevance: 7 Novelty: 8

7. Eigenbasis-Independent Learnable Spectral Positional Encodings for Directed Graphs via Hermitian Block Krylov Subspaces

ArXiv ID: 2607.07032

Primary Topic: Architecture and Training Dynamics

Authors: Jiaqing Xie, Yuxin Wang

Abstract: Spectral positional encodings (PEs) for \emph{directed} graphs face two obstacles: magnetic Laplacians require an $O(n^3)$ Hermitian eigendecomposition per potential, and their complex eigenvectors are defined only up to unitary gauge, which prior work handles with basis-invariant architectures. We propose learnable spectral PEs of the form $h_θ(A_q)\,R$, where $A_q$ is a normalized magnetic operator, $h_θ$ a learnable scalar spectral response, and $R$ a block of random probes. Because the PE is a \emph{matrix function} of the operator, it is gauge-invariant by construction. We compute it in a Hermitian block Krylov subspace from sparse matrix--vector products only, prove that $k = O(\log(1/\varepsilon))$ block steps suffice uniformly over heat--resolvent response families, and give a covering-number argument for why low-dimensional structured families generalize where free per-eigenvalue weights overfit. On a directed SBM whose symmetrization is uninformative by construction, direction-blind PEs stay at chance while magnetic Krylov PEs converge to the exact-eigendecomposition oracle as the depth grows. The same probes yield gauge-invariant pairwise features with $1/\sqrt{s}$ Monte-Carlo error, and the undirected $q{=}0$ case improves heterophilous benchmarks over no-PE and polynomial baselines.

Comment: Learnable matrix-function positional encodings remove eigenbasis ambiguity through block-Krylov computation.

Topic Match: The contribution is a general positional-encoding mechanism for directed graphs, with a principled alternative to eigendecomposition.

Relevance: 7 Novelty: 7

8. Causal Optimizer Interaction Calculus: Hidden Geometric Relaxation and Identifiable Interventions

ArXiv ID: 2607.07206

Primary Topic: Architecture and Training Dynamics

Authors: Zavier Li

Abstract: Optimizer experiments observe responses to algorithmic configurations without uniquely revealing hidden mechanisms. We develop a causal optimizer interaction calculus that separates pathwise realization, Mobius decomposition, and experimental identification. Under a fixed innovation coupling, every finite-horizon innovation-driven optimizer admits a behaviorally minimal pathwise realization. For any finite effect support and intervention design, an incidence operator gives the complete observational gauge, exact identifiability, sharp quotient stability, held-out predictions, and exact noiseless configuration complexity. Smooth hidden relaxation generates interactions through inverse hidden-state stiffness. Building on this structural law, we prove an observable-readout transfer theorem: arbitrary smooth update or trace readouts inherit an explicit five-term interaction through first and second hidden responses. Unlike the reduced optimal value, a general readout has no universal interaction sign. Its Boolean effects remain exact integrals of continuous interaction curvature and can therefore be identified by factorial interventions. We also derive Gaussian quotient minimax risk, exact confidence sets and tests, misspecification decomposition, certified downstream decisions, and optimal replication. A controlled real-data experiment on a 65-dimensional strongly convex logistic model validates the complete reduced-value chain. Boolean effects and independently integrated curvature agree within 4.21e-11, while nine held-out continuous intensities agree within 8.88e-13. Gaussian campaigns attain the predicted coverage and power, and 4,500 real-minibatch observations reject an order-two interaction model. Neural trace audits provide complementary evidence that the declared response classes remain informative in nonconvex training.

Comment: Relates optimizer-component interactions to hidden-state relaxation and identifiable responses in observable update traces.

Topic Match: The central contribution explains optimizer interaction dynamics; validation is primarily a small convex model with supplementary neural trace audits.

Relevance: 7 Novelty: 7

9. Intrinsic Green's Learning: Supervised Learning on Manifolds via Inverse PDE

ArXiv ID: 2607.07034

Primary Topic: Architecture and Training Dynamics

Authors: Alexandre Quemy

Abstract: We introduce Intrinsic Green's Learning (IGL), a framework that models a target function on a manifold as the solution to a linear PDE whose source term is learned from data. Rather than approximating the target directly, IGL learns a source and integrates it against a Green's kernel. An encoder discovers a low-dimensional coordinate chart on the manifold where both the source and the kernel decompose as low-rank tensors, collapsing a high-dimensional integral into independent one-dimensional integrals with cost linear in the intrinsic dimension. A two-stage algorithm separates coordinate discovery from source fitting, a near-convex linear solve, preventing the dimensional collapse of joint training. Learnable gates on each coordinate automatically discover the intrinsic dimension of the manifold. We validate IGL on synthetic manifolds and on MNIST, where it simultaneously achieves near-optimal classification and automatic recovery of the intrinsic dimension.

Comment: Replaces direct function prediction with a learned source integrated through a low-rank Green's kernel.

Topic Match: Introduces a computational architecture with staged training to prevent dimensional collapse, although large-model applicability remains unestablished.

Relevance: 7 Novelty: 7

10. FourierQK: Spectral Preprocessing of Query-Key Projections Improves Transformer Attention

ArXiv ID: 2607.07478

Primary Topic: Architecture and Training Dynamics

Authors: Athanasios Zeris

Abstract: FFT-based spectral preprocessing of learned query-key (Q/K) projections substantially improves transformer attention on character-level language modelling. On TinyShakespeare: a fixed random spectral filter achieves val=1.031 (Delta=+0.443); a single learned frequency at paragraph scale achieves val=0.608 (Delta=+0.867); and four learned frequencies spanning paragraph to word scale achieve val=0.309 (Delta=+1.166), a 79% reduction over standard dot-product attention. The single-frequency result is confirmed across three random seeds (mean val=0.236, std=0.019). The four frequencies converge to a near-geometric multi-scale ordering (49, 27, 10, 6 tokens/cycle) corresponding to paragraph, sub-paragraph, phrase, and word scales. The gain is specific to spectral preprocessing: random orthogonal and non-orthogonal projections of Q/K produce no measurable improvement, suggesting the benefit comes from global frequency-domain mixing rather than metric distortion. All results are verified by a shuffled-validation diagnostic against positional leakage. Causal filters (Gaussian, Mexican Hat, Morlet) do not improve over standard attention at character-level tokenisation: the bilateral FFT kernel is structurally non-causal, coupling every position to future tokens. This defines an architectural boundary between bilateral spectral attention (this paper) and genuinely causal spectral attention at word-scale tokenisation (companion paper MorletQK). This work is architecturally distinct from FNet (Lee-Thorp et al., 2021), which replaces attention with Fourier mixing of token embeddings. Here, spectral preprocessing applies only to Q/K projections while the full attention score structure is preserved.

Comment: FFT preprocessing introduces global spectral mixing into query-key attention computation.

Topic Match: Direct attention-mechanism work, but evidence is tiny-scale and bilateral filtering exposes future tokens, limiting autoregressive applicability.

Relevance: 7 Novelty: 6

11. Optimal Learning Rate Scaling Depends on Data in Deep Scalar Linear Networks

ArXiv ID: 2607.07884

Primary Topic: Architecture and Training Dynamics

Authors: Yedi Zhang, Peter E. Latham, Leena Chennuru Vankadara, Andrew Saxe

Abstract: In this short note we consider the gradient descent dynamics of deep scalar linear networks, $f(x) = \prod_{l=1}^L w_l x$, which enjoy exact time-course solutions for any integer depth. We show that even in this minimal model, the optimal depth-wise learning rate scaling depends on data, whereas data-agnostic scaling rules fail to transfer across depths. Under the data-dependent optimal scaling, the learning dynamics is independent of data and weakly dependent on depth, resulting in a constant linear convergence rate across all depths including infinity. We further show similar data-dependent effects in deep scalar linear networks with residual connections.

Comment: Optimal depth-wise learning-rate scaling depends on data in exactly solvable deep linear networks.

Topic Match: Direct optimization-dynamics theory about depth and learning-rate scaling, with conclusions established in scalar linear models.

Relevance: 7 Novelty: 6

12. Reward-Aware Population Scaling of Evolutionary Strategies in LLM Fine-Tuning

ArXiv ID: 2607.19408

Primary Topic: Architecture and Training Dynamics

Authors: Sung Cho, Gyubin Han

Abstract: Using Evolutionary Strategies (ES) for fine-tuning large language models is attractive because it is memory-efficient, parallel, and compatible with black-box or discrete rewards. Yet its population-size conclusions conflict sharply: fine-tuning with cross-entropy (CE) reward succeeds with $N=1$, while binary-reward training often needs $N \approx 30$. We show this gap is largely about reward design and normalization, not population size. In the capable-model regime we study, z-score advantage normalization can cause $N=2$ to fail. Disabling normalization lets binary-reward ES with $N=2$ improve on GSM8K and TREC across capable models spanning 0.5B-7B, where the normalized variant collapses or degrades. This small-$N$ risk is set by reward granularity: binary accuracy reward induces a zero-advantage probability $q$ that depends in closed form on base accuracy, batch size, and intra-pair correctness correlation; a zero-training probe on Qwen2.5-Instruct/GSM8K matches the formula with mean absolute error 0.020 across 12 configurations and finds the availability threshold $N_{\mathrm{avail}}$ to be small in this capable-model regime. The implication is not that $N=2$ is universally sufficient, but that small-population failure in capable-model binary ES can be an implementation artifact rather than an intrinsic population limit.

Comment: Explains small-population optimizer collapse through reward granularity and z-score advantage normalization.

Topic Match: The core contribution isolates a normalization-induced optimization failure in LLMs, providing a direct but narrow training-dynamics insight.

Relevance: 7 Novelty: 6

13. The Anatomy of Implicit Bias: Information Allocation in Neural Network Training

ArXiv ID: 2607.07156

Primary Topic: Architecture and Training Dynamics

Authors: Zhang Gongyue, Wang Zhiyong, Liu Donghan, Ren Weihong, Sheng Yixuan, Liu Honghai

Abstract: Implicit bias is usually explained as the preference of an optimization process for certain final solutions and their geometry. This view helps explain where a model finally stops. It gives less direct explanation of how this bias is formed during training. This paper proposes a training-time information allocation view. Under this view, optimization forms a writing pattern for error signals across parameter paths, coordinate channels, and sample regions. This paper builds a set of observable allocation diagnostics. These diagnostics include gradient demand, actual update injection, coordinate gain induced by exponential moving averages, channel-level update ratios, and sample-wise loss distributions. To separate training progress from internal allocation, this paper introduces a collapse--persistence analysis. Under matched training loss, if external loss statistics collapse but internal allocation ratios remain separated, then the factor changes the internal allocation of the training signal. Overall, this paper extends the analysis of implicit bias from final-solution geometry to training-time signal allocation. The main claim is that implicit bias is not only reflected by the final solution. It is also reflected by which parameter paths, coordinate channels, and sample regions receive the error signal first and more strongly during training. Based on this view, this paper places different training factors into a unified information-allocation diagnostic framework. The framework gives a mechanism-level explanation of training-time implicit bias. It also provides a basis for future optimization methods that control training progress and signal allocation separately.

Comment: Uses matched-loss comparisons to distinguish training progress from optimizer-induced allocation of updates across parameters and samples.

Topic Match: The core concerns how optimization distributes training signals over time, making it training-dynamics analysis with unproven large-scale implications.

Relevance: 7 Novelty: 6

Efficiency, Compression, and Large-Scale Training (7)

1. CTA-Pipelining: A Latency-Oriented Spatial Scaling Method for Multi-GPU Systems

ArXiv ID: 2607.07862

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Tingkai Liu, Muralidhar Andoorveedu, Sanjoy Das, Sanjay Patel, Volodymyr Kindratenko

Abstract: The evolution of compute infrastructure has transformed multi-GPU systems into tightly integrated shared-memory structures. However, current software still mostly treats these coherent interconnects simply as high-speed networks. Simultaneously, the demand for serving Large Language Models under latency constraints has shifted GPU workload optimization from being throughput-driven to latency-bound, necessitating latency-oriented scaling methods beyond Tensor Parallelism (TP). Thus, we introduce CTA-pipelining, an execution paradigm designed to exploit shared-memory multi-GPU systems. As a latency-oriented spatial scaling technique, CTA-pipelining leverages dependencies at the Cooperative Thread Array level, enabling concurrent execution of dependent kernels across GPUs. We demonstrate its capability using CUTLASS, cuBLAS, and NCCL libraries on 8-GPU H200 and B200 systems. Results show on 2-layer GEMM, representing the MLP operation, CTA-pipelining reduces latency by up to 31.8% compared to micro-batching, and 29.6% compared to TP. It can also be combined with TP as an orthogonal scaling dimension to further push the latency boundary.

Comment: CTA-level dependencies pipeline dependent kernels across GPUs to reduce execution latency.

Topic Match: A new multi-GPU execution mechanism targets inference efficiency, with latency evidence on MLP-like two-layer GEMMs.

Relevance: 8 Novelty: 8

2. Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automata

ArXiv ID: 2607.07026

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Meihua Dang, Stefano Ermon

Abstract: Constrained decoding is essential for serving LLMs, ensuring that generated outputs follow specific structures such as JSON schema-formatted function calls. Existing systems are designed for autoregressive models and assume left-to-right generation, masking out invalid next tokens at each step. Diffusion language models, however, break this assumption: they sample multiple positions simultaneously from a fully-factorized mean-field distribution at each denoising step. In this paper, we present an exact and tractable algorithm for sampling from the constrained mean-field posterior under any constraint expressible as a finite automaton. Viewing finite automata as graphical models, we obtain tractable representations of the constrained distribution that enable efficient inference. The approach guarantees constraint satisfaction by construction, supports both greedy and sampling-based decoding, and is compatible with parallel and block-wise decoding under arbitrary remasking schedules. Applying depth-reduction techniques from arithmetic circuit theory, we further reduce sampling depth from linear to logarithmic in the sequence length. Empirical evaluations on Dream-7B and LLaDA-8B show substantial accuracy gains across various tasks including function calling (xLAM, BFCL), planning (Sudoku, Countdown), text-to-SQL (Spider), and math reasoning (GSM-Symbolic), with little inference overhead relative to unconstrained decoding. For example, on BFCL-Live, our approach improves Dream-7B's greedy decoding accuracy from 63.9% to 71.5%, and stochastic sampling accuracy from 22.3% to 69.0%, where the unconstrained baseline collapses, with under 5% wall-clock overhead.

Comment: Finite-automaton inference enables exact constrained sampling with logarithmic parallel depth.

Topic Match: The core is an efficient decoding algorithm for diffusion language models, including a substantive reduction in sampling depth.

Relevance: 7 Novelty: 8

3. Fractal KV-Cache Archives: Lossless Symbolic Storage with In-Place Retrieval for Long-Context LLM Inference

ArXiv ID: 2607.07144

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Vladimir Gusev

Abstract: The key-value (KV) cache dominates the memory cost of long-context autoregressive inference, and a growing body of work compresses it through quantization, eviction, or offloading. We study a complementary question: once a position's KV state has been quantized to codebook indices, how should the resulting symbol stream be stored, and can the storage layer do more than store? A family of contractive iterated-map codes that serialize a symbol sequence into a sequence of low-dimensional real vectors is revisited, and it is shown that they form a natural archive format for a quantized KV cache with the following features. The method provides exactly the access pattern a growing cache requires. It is lossless, it runs in linear time, and supports O(1) random access and O(1) amortized append. A controlled study of the quantizer feeding this archive is conducted on GPT-2 with 1024-token contexts. Keeping a small exact window (4 attention sinks plus 32 recent tokens) and archiving the rest, per-head residual vector quantization reduces the archived cache by 36-54x relative to an fp16 cache at a perplexity cost of 11-15%, and we quantify a sharp key/value asymmetry - quantizing keys is roughly 4x more damaging than quantizing values, consistent with prior low-bit KV work - and use it to allocate bits in a hybrid scheme. Finally, we show the archive is simultaneously a search index: approximate substring queries execute directly on the stored vectors, and matched context is decoded from the matched vector without ever materializing the surrounding text. We further characterize the archive's operating range: truncating stored points yields a lossy regime whose distortion we localize, and probability-weighting the maps recovers arithmetic coding, exposing a trade-off between rate efficiency, random access, and memory. We release all code; every number reproduces on a laptop CPU.

Comment: An appendable archive supports constant-time random access to quantized KV-cache symbols.

Topic Match: Direct KV-cache storage work; losslessness applies after quantization, and validation uses GPT-2 with 1K-token contexts.

Relevance: 8 Novelty: 6

4. LoCA: Spatially-Aware Low-Rank Convolutional Adaptation of Vision Foundation Models

ArXiv ID: 2607.06918

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Sojung An, Junha Lee, Sujeong You, Nam Ik Cho, Donghyun Kim

Abstract: Pre-trained Vision Foundation Models (VFMs) provide strong visual representations for diverse downstream tasks. The key challenge of VFM adaptation stems from the prohibitive costs of full fine-tuning and catastrophic forgetting. To address this, Low-Rank Adaptation (LoRA) has emerged as the prevailing paradigm for Parameter-Efficient Fine-Tuning (PEFT). However, LoRA is typically designed for transformer self-attention layers parameterized by 2D matrices. Since convolutional kernels inherently couple spatial and channel information within a 4D tensor, forcing them into a monolithic 2D matrix disrupts the inherent spatial topology. In this paper, we propose Low-Rank Convolutional Adaptation (LoCA), a convolution-aware PEFT framework that addresses spatial-channel entanglement by decoupling channel and spatial adaptation. LoCA introduces a low-rank channel adaptation for dense cross-channel mixing and refines spatial bases extracted from pre-trained kernels via Singular Value Decomposition (SVD). Experimental results show that LoCA preserves pre-trained spatial priors and achieves competitive or state-of-the-art performance across fine-grained classification, domain-generalized semantic segmentation, and generative benchmarks.

Comment: Separates convolutional adaptation into low-rank channel updates and refinement of SVD-derived spatial bases.

Topic Match: The core mechanism is structured parameter-efficient adaptation that preserves convolutional spatial structure, directly matching low-rank adaptation.

Relevance: 8 Novelty: 6

5. AnchorPrune: Relevance-Anchored Contextual Expansion for Visual Token Pruning

ArXiv ID: 2607.07033

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Kyuan Oh, Bumsoo Kim

Abstract: Large vision-language models incur substantial inference costs because high-resolution inputs introduce thousands of visual tokens, many of which are redundant for a given query. Existing pruning methods often combine query relevance and token diversity, yet these objectives can conflict under aggressive compression: relevance-driven selection may overconcentrate the budget on correlated local evidence, while diversity-driven selection may suppress indispensable tokens or retain distinct but uninformative regions. We introduce AnchorPrune, a training-free framework that first constructs a protected relevance anchor and then expands it with complementary visual context. AnchorPrune adaptively determines the anchor size from the novelty profile of relevance-ranked tokens, preserving a compact set of query-critical evidence, and allocates the remaining budget through importance-weighted novelty to recover informative, non-redundant context relative to the anchor. This ordered design prevents contextual expansion from displacing indispensable query cues while improving overall visual coverage. AnchorPrune is lightweight, architecture-aware, and requires neither retraining nor model modification. Across image and video vision-language models and benchmarks, it consistently improves the accuracy-efficiency trade-off over training-free baselines, particularly under severe compression. On LLaVA-NeXT-7B, AnchorPrune preserves 97.6% of full-token performance using only 160 of 2,880 visual tokens. These results establish relevance-anchored contextual expansion as an effective principle for efficient multimodal inference. Code is available at https://github.com/MULTI-cau/AnchorPrune.

Comment: Prunes visual tokens by protecting query-relevant anchors before allocating the remaining budget to complementary context.

Topic Match: The core contribution is a token-pruning mechanism that substantially reduces large vision-language model inference computation.

Relevance: 8 Novelty: 6

6. EdgeCompress: Coupling Multidimensional Model Compression and Dynamic Inference for EdgeAI

ArXiv ID: 2607.06982

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Hao Kong, Di Liu, Shuo Huai, Xiangzhong Luo, Ravi Subramaniam, Christian Makaya, Qian Lin, Weichen Liu

Abstract: Convolutional neural networks (CNNs) have demonstrated encouraging results in image classification tasks. However, the prohibitive computational cost of CNNs hinders the deployment of CNNs onto resource-constrained embedded devices. To address this issue, we propose EdgeCompress, a comprehensive compression framework to reduce the computational overhead of CNNs. In EdgeCompress, we first introduce dynamic image cropping (DIC), where we design a lightweight foreground predictor to accurately crop the most informative foreground object of input images for inference, which avoids redundant computation on background regions. Subsequently, we present compound shrinking (CS) to collaboratively compress the three dimensions (depth, width, and resolution) of CNNs according to their contribution to accuracy and model computation. DIC and CS together constitute a multidimensional CNN compression framework, which is able to comprehensively reduce the computational redundancy in both input images and neural network architectures, thereby improving the inference efficiency of CNNs. Further, we present a dynamic inference framework to efficiently process input images with different recognition difficulties, where we cascade multiple models with different complexities from our compression framework and dynamically adopt different models for different input images, which further compresses the computational redundancy and improves the inference efficiency of CNNs, facilitating the deployment of advanced CNNs onto embedded hardware. Experiments on ImageNet-1K demonstrate that EdgeCompress reduces the computation of ResNet-50 by 48.8% while improving the top-1 accuracy by 0.8%. Meanwhile, we improve the accuracy by 4.1% with similar computation compared to HRank, the state-of-the-art compression framework. The source code and models are available at https://github.com/ntuliuteam/edge-compress

Comment: Jointly shrinks CNN depth, width, and input resolution according to their accuracy and computational contributions.

Topic Match: Model compression is the core contribution, with a narrower focus on CNN inference costs and embedded deployment.

Relevance: 7 Novelty: 6

7. Guidance Breaks the Fitted Operator: A Terminal-Fitted Repair for Classifier-Free Guidance

ArXiv ID: 2607.07665

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Shiheng Zhang

Abstract: Classifier-free guidance (CFG) is the standard way to strengthen class-conditioning in diffusion and flow-matching samplers, yet at large guidance it oversaturates and destabilizes, symptoms practitioners suppress with more steps or limited-interval schedules. We analyze CFG through an asymptotic-preserving, numerical-analysis lens. Building on a recent result that the deterministic DDIM step is the unique fitted operator for the unguided terminal layer, exact on the final small-sigma stretch of sampling, we show that guidance re-stiffens exactly the discriminative subspace to an anomalous exponent 1+w. DDIM is therefore no longer fitted there, and on coarse meshes its guided residual diverges as sigma_min goes to zero. We prove a guided clock barrier with three ordered step-size thresholds, and read one-step oversaturation as its endpoint: a solver artifact on the calibration model rather than the continuous guided law. The same analysis yields a one-coefficient, zero-extra-NFE repair: replace CFG's w(r-1) by r^(1+w)-r on the guidance direction. On the calibration model's discriminative crossover, this removes CFG's sigma_min-divergent blow-up and is first-order accurate against the exact guided flow as sigma_min goes to zero. On learned CIFAR-10 checkpoints, and as a cross-domain smoke test on Stable Diffusion 1.5 DDIM, it acts as a high-guidance stabilizer at no extra cost rather than a universal quality knob: it cuts residual amplification and saturation, gives 9/9 point-FID wins over CFG on the tested grid, and preserves classifier-proxy target accuracy in the hard-cell blocks. We report the limits alongside: it is not a universal image-quality win, and against a dense vanilla-CFG reference it is not a uniformly better integrator of that field.

Comment: An analytic guidance correction removes terminal residual blow-up without extra model evaluations.

Topic Match: Closest to inference efficiency; the main result repairs guided sampling dynamics at unchanged evaluation cost.

Relevance: 6 Novelty: 7

Paper Selection Prompt

System Prompt

You are a helpful paper reading assistant whose job is to read daily posts from ArXiv and identify a few papers that your friend will enjoy reading. Your job is to carefully read the paper titles and abstracts below and find the ones that match the criteria below.

User Prompt

Instructions

Respond in JSONL. Output exactly one JSON object per paper, one per line:

{"ARXIVID":"...","COMMENT":"...","RELEVANCE":0,"NOVELTY":0,"PRIMARY_TOPIC_ID":"...","MATCHED_TOPIC_IDS":[],"TOPIC_MATCH_COMMENT":"...","HOTSPOT_PAPER_TAGS":[],"HOTSPOT_PAPER_COMMENT":"..."}

Rules: - ARXIVID: the arXiv ID. - COMMENT: identify the single strongest matching criterion. Be brief and specific. Do not rely on generic phrases like "language modeling" or "advancement". Do not mention non-matching criteria. - RELEVANCE: integer from 1 to 10. - NOVELTY: integer from 1 to 10. - PRIMARY_TOPIC_ID: exactly one stable topic ID from the allowed topic registry. - MATCHED_TOPIC_IDS: zero or more stable topic IDs from the same allowed set. Include PRIMARY_TOPIC_ID when there are multiple matches. - TOPIC_MATCH_COMMENT: briefly explain why the primary topic is the best fit. - HOTSPOT_PAPER_TAGS: zero or more tags from this exact set only: daily_hot, new_frontier. - HOTSPOT_PAPER_COMMENT: briefly explain why the paper belongs in the daily hotspot paper feed when HOTSPOT_PAPER_TAGS is non-empty; otherwise use an empty string. - Use HOTSPOT_PAPER_TAGS sparingly. Most papers should return []. - daily_hot means the paper feels broadly important to the day and belongs in the daily hotspot paper section even if it is not part of the personalized foundational reading list. - new_frontier means the paper appears to open a genuinely new direction, paradigm, or field, even if the work is still early. - Do not output markdown, code fences, or any extra text.

Scoring Criteria

Relevance and Novelty are independent axes. Score both from 1 to 10.

Relevance Scoring

• 9-10: directly centered on the target foundational topics; highest when the core contribution is clearly within them.
• 7-8: substantially related, but partly peripheral or focused on a narrower aspect.
• 5-6: touches the target topics, but the main contribution is elsewhere.
• 3-4: largely outside the target topics, often application-focused or domain-specific.
• 1-2: unrelated.

Important: Broad frontier relevance, major launch status, or daily buzz is not enough for a high Relevance score here. Those cases belong in the hotspot digest unless the paper strongly matches the specialized paper topics.

Novelty Scoring

• 9-10: new paradigm, theory, or major methodological breakthrough.
• 7-8: substantial methodological advance or strong new insight.
• 5-6: meaningful but incremental extension or refinement.
• 3-4: minor, narrow, or mostly engineering or domain-specific improvement.
• 1-2: little originality; mainly standard application of existing methods.

Topic Registry

Use exactly one PRIMARY_TOPIC_ID chosen from the stable topic IDs below. - moe_training: MoE Training - Mixture-of-Experts training: routing, load balancing, expert granularity and shared experts, training stability, MoE parallelism and communication, and MoE kernels. - training_systems: Large-Scale Training Systems and Efficiency - Large-scale training systems and efficiency: distributed training algorithms, optimizers, and communication. - architecture_training: Architecture and Training Dynamics - Core architectural or computational mechanisms, dynamic computation, and training-stability dynamics. - efficiency_scaling: Efficiency, Compression, and Large-Scale Training - Compression, sparsity, memory or cache efficiency, and large-scale training systems that materially change cost or behavior.

Papers

[PAPER LIST HERE]

Relevant Topics

This is a training-side feed. The centre of gravity is Mixture-of-Experts training and the large-scale training systems it sits on; architecture and efficiency work is kept when it bears on how a large model is trained or what it costs to train.

Keep a paper when its CORE CONTRIBUTION falls in one of the four topics below. Filter it when the topic is merely the setting for a downstream application, a benchmark, or a product result.

1. MoE Training (primary) - Keep: expert routing and gating mechanisms; load balancing, with or without auxiliary losses; expert granularity, fine-grained and shared experts; MoE training stability (loss spikes, router collapse, z-loss and related fixes); expert and hybrid parallelism; all-to-all dispatch/combine and the communication schedules MoE needs; MoE kernels and grouped-GEMM style implementations; expert capacity, token dropping, and upcycling dense checkpoints into MoE. - Filter: papers that merely train on top of a MoE model without a new routing, balancing, stability, parallelism, or kernel contribution; "mixture of experts" in the classical ensemble or recommender sense.

2. Large-Scale Training Systems - Keep: distributed training algorithms (data, tensor, pipeline, sequence, and expert parallelism, FSDP and ZeRO-style sharding); optimizers and preconditioners for large-scale pretraining; communication algorithms, overlap, and collective design; throughput and MFU work that materially changes what a training run costs or whether it converges; scaling-law work that informs how such runs are configured. - Filter: routine infrastructure tuning with no new algorithmic or systems idea.

3. Architecture and Training Dynamics - Keep: new or analysed architectural mechanisms (attention variants, normalization and residual design, state-space and recurrent sequence modelling, dynamic or modular computation); optimisation and training-dynamics analysis that explains why large models train the way they do. - Filter: applying an existing architecture to a new task or benchmark with no mechanistic insight.

4. Efficiency and Compression - Keep: quantization, sparsity, pruning, low-rank adaptation, KV-cache and memory-efficient designs, and other work that materially changes the cost of running or training a large model, especially when the mechanism is new rather than a tuned variant. - Filter: straightforward application of standard efficiency methods, and deployment or serving engineering with no new idea.

Leave these to the hotspot digest unless the core contribution clearly falls in one of the four topics above: - major model or product releases - agent, tooling, and RAG launches - benchmark, leaderboard, or evaluation-only papers - representation-learning theory, interpretability, and feature-formation analysis - memory mechanisms and agent memory systems - world models, exploration, and reinforcement learning - alignment and post-training (RLHF, DPO, GRPO, RFT, instruction tuning) - downstream applications in medical imaging, segmentation, 3D vision, video understanding, information retrieval, summarization, recommendation, machine translation, speech recognition, time series, knowledge graphs, and similar domains