This is a remedial run for missed papers from 07/20/2026 to 07/20/2026.

Results generated on 09/12/2026.

Personalized Daily ArXiv Papers 2026-07-21

_{Token counts are not reported for this run. 5 of 8 model calls succeeded, 3,131s of model wall clock.}

Topic Coverage:

Table of contents by topic:

Large-Scale Training Systems and Efficiency (3)

1. Optimizing the Preconditioner: A Black-box Online-to-Nonconvex Conversion with Static Regret Minimization Oracles Authors: Haichen Hu, David Simchi-Levi

2. AutoEncoder-Compressed Parallel Split Learning for Pre-trained Model Fine-Tuning Authors: Bas Meuwissen, Vasileios Tsouvalas, Nirvana Meratnia

3. Federated Lightweight Fine-Tuning Authors: Radhakrishna Achanta, Will Reed

Architecture and Training Dynamics (13)

1. An Adjoint-Sensitivity Framework for Lost-in-the-Middle Phenomena in Causal Residual Transformers Authors: Cheng Huan, Hongwei Yuan

2. Phasor Attention: Mean Root Square Normalization for Phase Manifold Preservation Authors: Sungwoo Goo, Hwi-yeol Yun, Sangkeun Jung

3. After the Euclidean Highway: Hyperbolic Expert AI as the Next Innovation Authors: Kwan Soo Shin, In Seok Kang, Munho Lee

4. Signed Rectified Flow: Negativity-Controlled Generation Authors: Runlong Liao, Baiyu Su, Lizhang Chen, Qiang Liu

5. L1 Augmented Attention as an Improved Vector Similarity Metric Authors: Kurt Godden

6. Conditioned Direct Feedback Alignment via Activity and Error Geometry Authors: Houman Safaai, Varun Reddy, Bernardo L. Sabatini

7. Model of Models: When Does Emitting a Specialist Beat Attending, Adapting, or Tuning? Authors: John C. Howell

8. On the Diverse Dynamical Behaviors Arising in Deep Linear Transformers Authors: Sixu Li, Thomas Jacob Maranzatto, Jan Peszek, Trevor Teolis, Semih Akkoc, Konstantin Riedl, Sennur Ulukus, Nicolás García Trillos

9. PAC--Bayes Bounds on Quotient Parameter Spaces: Geometry-induced Implicit-Bias Priors Authors: Nicola Aladrah, Fabio Anselmi

10. Organization of computation in reservoir computing Authors: Mohab Abdalla, Damien Rontani

11. A Weisfeiler-Leman Characterization of Global-Attention Graph Transformers for Mixed-Integer Linear Programs Authors: Md Abrar Jahin, Craig A. Knoblock, Jay Pujara

12. Adaptive Mamba Neural Operators Authors: Zeyuan Song, Zheyu Jiang

13. Brain-Aligned Multi-Stream Video Transformers with Sparse Self-Selection Authors: Amir Hosein Fadaei, Mahyar Maleki, Mohammad-Reza A. Dehaqani

Efficiency, Compression, and Large-Scale Training (12)

1. CoCurve: Cross-Module Co-Pruning Curvature for Training-Free Structured LLM Pruning Authors: Zhiren Gong, Zihao Zeng, Zijie Wang, Tiantong Wang, Chau Yuen, Wei Yang Bryan Lim

2. MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference Authors: Simla Burcu Harma, Danila Mishin, Zhengyuan Su, Ayan Chakraborty, Elizaveta Kostenok, Dongho Ha, Babak Falsafi, Martin Jaggi, Yunho Oh, Amir Yazdanbakhsh

3. PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer Authors: Nikhil Ghosh, Tetiana Parshakova, Robert M. Gower

4. Manifold-Constrained Hyper-Connections for Parameter-Efficient Finetuning Authors: Valentijn Oldenburg, Floris de Kam, Bente Zuijdam, Lieve Eberson, Nicky van Zutphen, Stef de Wildt, Ivo Verhoeven

5. BRIM: Workload-Balanced Dual-Sided Bit-Serial Sparse Inference Accelerator Authors: Varun Manjunath, Ruokai Yin, Donghyun Lee, Arkapravo Ghosh, Priyadarshini Panda

6. FlowBlock: Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models Authors: Bing Tian, Haikun Liu, Xiaocheng Zhong, Zhuohui Duan, Zhaokai Luo, Huayi Jin, Zhiyong Wang, Xiaofei Liao

7. SelectInfer: Selective Neuron Loading and Computation for On-Device LLMs Authors: Huzaifa Shaaban Kabakibo, Eric Schniedermeyer, Artem Burchanow, Lin Wang

8. Recti-Q: Feature-Space Rectification for Out-of-Distribution-Robust Quantized Perception in Edge Robotics Authors: Hamidreza Yaghoubi Araghi, Parastoo Pilevar, Ming C. Lin

9. DA-MergeLoRA: Hypernetwork-Based LoRA Merging for Few-Shot Test-Time Domain Adaptation Authors: Siobhan Reid, Zhixiang Chi, Li Gu, Omid Reza Heidari, Ziqiang Wang, Yang Wang

10. SALT: Salience-Aware Lexical Trie for Long-Context Compression Authors: Oteo Mamo, Hyunjin Yi, Joydhriti Choudhury, Shangqian Gao, Weikuan Yu

11. PRISM: Sensitivity-Aware PolynoMial PRuning for EffIcient Neural Network Encryption Authors: Sahaj Majavdia, Mahdi Taheri

12. Opto-ViT-v2: Noise-Resilient On-Chip Fine-Tuning for Photonic Near-Sensor Vision Transformer Accelerators Authors: Xuming Chen, Deniz Najafi, Mehrdad Morsali, Chengwei Zhou, Zahra Ghanaatianjobzari, Mahdi Nikdast, Shaahin Angizi, Gourav Datta

Large-Scale Training Systems and Efficiency (3)

1. Optimizing the Preconditioner: A Black-box Online-to-Nonconvex Conversion with Static Regret Minimization Oracles

ArXiv ID: 2607.17607

Primary Topic: Large-Scale Training Systems and Efficiency

Authors: Haichen Hu, David Simchi-Levi

Abstract: We study whether stochastic nonconvex optimization can be reduced to ordinary static regret minimization in online convex optimization in a black-box manner. For smooth nonconvex objectives, our reduction maintains a predictable gradient tracker, while a black-box online learner selects a preconditioner that determines how this tracker is transformed into the update direction. The learner receives linear convex losses and is evaluated against a single fixed comparator over one undiscounted online game. For a $β$-smooth objective with range bounded by $M$ and an unbiased stochastic-gradient oracle with variance bounded by (σ^2), we establish $$\frac{1}{T}\sum_{t=1}^T \mathbb E!\left[|\nabla f(x_t)|_2^2\right] \lesssim \frac{σ\sqrt{Mβ}}{\sqrt T} + \frac{\sqrt{Mβ}\, \mathscr R_T(\mathcal A,I_d)}{T} + \frac{Mβ}{T}.$$ Consequently, any black-box OCO algorithm with $\mathscr R_T(\mathcal A,I_d)=O(\sqrt T)$ recovers the classical $O(\frac{1}{\sqrt{T}})$ convergence rate. We further show that the same black-box framework extends beyond the smooth setting to Lipschitz nonconvex objectives without Lipschitz continuous gradients. Importantly, this extension continues to rely only on an ordinary static-regret guarantee and requires no stronger notion of online regret. When the OCO oracle admits square-root static regret, the resulting conversion achieves the optimal $O(T^{-2/7})$ convergence rate for the corresponding Goldstein stationary point. These results resolve the open problem posed by Chen and Hazan (2024). More broadly, our framework separates optimizer design into gradient prediction and online preconditioner selection, providing a principled perspective on how adaptive optimization methods may be understood through static regret and applied in nonconvex optimization.

Comment: Reduces stochastic nonconvex optimization to static-regret learning of a gradient preconditioner with optimal convergence guarantees.

Topic Match: Its black-box preconditioner-selection framework is directly relevant to optimizer design, though it is not evaluated on large-model pre-training.

Relevance: 7 Novelty: 8

2. AutoEncoder-Compressed Parallel Split Learning for Pre-trained Model Fine-Tuning

ArXiv ID: 2607.17913

Primary Topic: Large-Scale Training Systems and Efficiency

Also Matches: Efficiency, Compression, and Large-Scale Training

Authors: Bas Meuwissen, Vasileios Tsouvalas, Nirvana Meratnia

Abstract: Distributed Fine-Tuning (DFT) of large-scale Foundation Models (FMs) on resource-constrained edge devices is limited by local compute constraints and communication overhead. Parallel Split Learning (PSL) reduces client-side computation by keeping few model layers on each client and offloading the remaining computation to the server; however, clients must exchange intermediate activations and gradients with the server at every training step. Existing SL communication-compression methods mainly rely on task-agnostic heuristics, such as sparsification and quantization. While learnable SL compressors can better adapt to intermediate representations, they require co-training with the target model. Therefore, directly inserting them into off-the-shelf FMs introduces feature-distribution misalignment and degrades DFT performance. To address this, we propose AE-PSL, a communication-efficient PSL framework that compresses intermediate activations and gradients using a lightweight AutoEncoder (AE) placed at the split layer. To ensure compatibility of AE compression with pre-trained FMs, AE-PSL introduces a novel two-stage alignment mechanism, which adapts the AE to the pre-trained model's feature manifold and client-specific feature distributions before DFT.

Comment: Compresses split-learning activations and gradients with a two-stage aligned autoencoder.

Topic Match: The core contribution reduces communication in distributed foundation-model fine-tuning.

Relevance: 8 Novelty: 6

3. Federated Lightweight Fine-Tuning

ArXiv ID: 2607.18343

Primary Topic: Large-Scale Training Systems and Efficiency

Also Matches: Efficiency, Compression, and Large-Scale Training

Authors: Radhakrishna Achanta, Will Reed

Abstract: Federated fine-tuning is bottlenecked by communication: FedAvg and pseudo-gradient schemes transmit a payload that scales with the model, and gradient compression shrinks it by only a constant factor. We take a different lever. Mapping networks generate a network's weights from a small trainable latent through a frozen affine projection; because the map is shared and affine, averaging latents is exactly averaging the generated weights. We turn this into a practical low-bandwidth federated channel with two changes: a low-rank, seed-regenerable factorisation of the projection (cutting generator memory from ~80 GB to ~10 MB), and a delta formulation $θ= θ^{\mathrm{pre}} + U V^{\top} z$ that learns an additive correction around a shared centrally-pretrained base -- federated fine-tuning, which is what makes the method work at scale. A frozen orthogonal classifier head further removes the head from the payload while improving accuracy. On CIFAR-100 with ResNet-18+GroupNorm, our method (FLITE, Federated Low-rank Iterative Training Engine) communicates 1,280 floats (~5 KB) per client per round -- an 8718x reduction -- and reaches 74.67%, within ~0.5 pp of full-weight FedAvg. The averaging identity holds to floating-point precision ($6 \times 10^{-8}$); the method sits one to two orders of magnitude below PowerSGD and top-k on the bandwidth-accuracy Pareto; it matches or exceeds full-weight FedAvg under strong non-IID skew. int4 latents reach 648 bytes per round at unchanged accuracy, whereas int4 full-weight FedAvg collapses to chance.

Comment: Introduces an exact-averaging low-rank latent channel that cuts federated fine-tuning communication by over four orders of magnitude.

Topic Match: Its main contribution is a communication-efficient distributed training protocol, supported by a low-rank parameterization.

Relevance: 7 Novelty: 7

Architecture and Training Dynamics (13)

1. An Adjoint-Sensitivity Framework for Lost-in-the-Middle Phenomena in Causal Residual Transformers

ArXiv ID: 2607.17696

Primary Topic: Architecture and Training Dynamics

Authors: Cheng Huan, Hongwei Yuan

Abstract: We develop an adjoint-sensitivity framework for positional influence in causal residual Transformers and separate unconditional analytic results from conditional boundary-shape conclusions. The principal unconditional theorem is the residual-to-depth-flow estimate for layer controls converging in $L^1$, complemented by a finite-token-to-Volterra attention estimate that explicitly controls the first cells near the causal endpoint. We define a normalized adjoint-energy influence density and derive its exact evolution along full-batch gradient flow. The adjoint admits an exact generator-term decomposition into residual transmission, nonlocal Volterra, and local channels, including all covariance cross terms. Causal masking can amplify early-position sensitivity and residual identity paths can transmit a right-localized terminal bias, but neither mechanism alone forces a U-shaped profile. We therefore state boundary advantages under independently checkable energy, correlation, and local-channel bounds; these conditions are sufficient rather than necessary. Finite-token influence balancing, positional reweighting, and task-aligned observability are presented as diagnostics or regularizers with explicit differentiation requirements, computational costs, and limitations. Controlled simulations illustrate that each intervention controls its designated surrogate, while observability balance or outer-loop reweighting need not monotonically reduce the influence-based Lost-in-the-Middle diagnostic.

Comment: Derives an adjoint-sensitivity framework that decomposes positional influence across residual, nonlocal, and local channels.

Topic Match: It directly explains positional influence and training dynamics in causal residual Transformers.

Relevance: 8 Novelty: 8

2. Phasor Attention: Mean Root Square Normalization for Phase Manifold Preservation

ArXiv ID: 2607.17822

Primary Topic: Architecture and Training Dynamics

Authors: Sungwoo Goo, Hwi-yeol Yun, Sangkeun Jung

Abstract: While Root Mean Square Normalization has become the de facto standard for accelerating modern sequence models, its reliance on the quadratic accumulation of independent scalars ($\sum x^2$) inherently triggers outlier-induced numerical instability, gradient starvation, and anisotropic phase distortion. We introduce Mean Root Square Normalization (MRSNorm). By structurally pairing channels into 2D phasors, MRSNorm mathematically inverts the traditional scaling paradigm: it computes the localized $L_2$ magnitudes (Root Square) before aggregating them via a global $L_1$ average (Mean). This operational inversion strictly constrains activations to a phasor manifold, preserving conformal invariance. By sharing a single affine weight across phasor components, MRSNorm halves the total number of learnable parameters, proving that unconstrained spatial scaling in standard norms is a harmful redundancy. We analytically demonstrate that this geometric constraint yields a built-in, trigonometric gradient clipper governed by the Pythagorean identity, unconditionally equalizing the local gradient norm to ensure Gradient Homogeneity. Empirical evaluations on a ResNet with CIFAR-100 show that despite halved parameters, MRSNorm provides critical structural stability under rigorous stress tests. Under extreme hyperparameter settings where standard normalizations suffer from gradient divergence, MRSNorm successfully prevents numerical explosion and secures stable optimization trajectories. Our findings propose a fundamental paradigm shift toward phasor-based deep representation learning. The implementation of MRSNorm is available at Appendix C.

Comment: Proposes phasor-paired normalization intended to prevent outlier-driven instability and equalize local gradient norms.

Topic Match: Normalization design and its effect on gradient stability are directly within architecture and training dynamics.

Relevance: 8 Novelty: 7

3. After the Euclidean Highway: Hyperbolic Expert AI as the Next Innovation

ArXiv ID: 2607.17513

Primary Topic: Architecture and Training Dynamics

Authors: Kwan Soo Shin, In Seok Kang, Munho Lee

Abstract: Expert domains are trees; the Euclidean transformer is not, diluting parent-child structure exponentially at depth. The hyperbolic turn left one question unasked: not how much of a network to curve, but where curvature may touch the gradient. Placement is a law, not a knob: the same geometry on a trainable adapter collapses training (seventeen training collapses, ~220 GPU-hours), yet at the loss layer alone it trains without one -- this is HySAT (Hyperbolic Structure-Aware Training), hyperbolic losses at the loss layer only. Across six expert SLMs we constructed and deployed (Llama 3.1 and EXAONE 3.5; four adapter strategies; 18.0M-sample corpus; zero NaN over ~317K optimizer steps), a matched four-arm ablation isolates the preserved manifold invariant, and three propositions and a lemma prove why loss-only placement is stable where adapter-on-manifold is not. Four models are operationally deployed (one live, consumer-facing), two open-weight, with per-step traces and a seventeen-incident failure ledger on Zenodo (CC-BY-4.0).

Comment: Identifies loss-only hyperbolic geometry as a stable placement while documenting collapse when the same geometry enters trainable adapters.

Topic Match: The central contribution is a mechanistic account of how geometric placement changes optimization stability.

Relevance: 8 Novelty: 7

4. Signed Rectified Flow: Negativity-Controlled Generation

ArXiv ID: 2607.18516

Primary Topic: Architecture and Training Dynamics

Authors: Runlong Liao, Baiyu Su, Lizhang Chen, Qiang Liu

Abstract: We introduce Signed Rectified Flow (Signed RF), a generalization of Rectified Flow that targets the signed measure $π^{sign} = (1+α)π^+ - απ^-$, where $α>0$, $π^+$ is the distribution to promote, and $π^-$ is the distribution to suppress. Although direct sampling from a signed measure is not well-defined, Signed RF induces a valid generative process that concentrates probability in regions where the signed measure is positive while provably excluding regions dominated by its negative component. It therefore provides a principled framework for incorporating negative information and exclusion constraints into generative modeling. We analyze the signed continuity equation underlying Signed RF and use a charged-particle interpretation to explain how negative mass forms exclusion barriers. This theory further motivates practical adaptive guidance algorithms. Across several applications, Signed RF improves the fidelity-diversity trade-off on ImageNet, reduces nearest-neighbor similarity in anti-memorization experiments, and reduces nudity induced by adversarial prompts in Stable Diffusion 3.5 while preserving CLIP and aesthetic scores.

Comment: Generalizes rectified flow to signed measures, enabling learned attraction toward desired data and exclusion of negative regions.

Topic Match: Its central contribution is a new generative-flow mechanism with accompanying continuity-equation analysis.

Relevance: 7 Novelty: 8

5. L1 Augmented Attention as an Improved Vector Similarity Metric

ArXiv ID: 2607.18027

Primary Topic: Architecture and Training Dynamics

Authors: Kurt Godden

Abstract: Scaled dot product attention conflates directional alignment and vector magnitude, limiting its effectiveness as a similarity metric in Transformer models. We introduce L1 augmented attention, a simple and computationally parallelizable modification that subtracts a learned, head specific L1 distance between queries and keys from the dot product score. This hybrid similarity captures complementary geometric information. Dot product rewards directional alignment, while L1 penalizes coordinate deviations. To reduce the cost of L1 computation, we project queries and keys into low dimensional subspaces whose parameters specialize to preserve informative L1 structure. Evaluated on WikiText 2 using a compact transformer, L1 augmented attention achieves up to a 14.5% reduction in perplexity over the original transformer baseline and outperforms an RBF L2 kernel. Analysis of norm variance and learned L1 weights reveals distinct geometric roles across layers and strong head level specialization. These results demonstrate that enriching attention with L1 geometry provides a principled and effective improvement to similarity computation in modern language models, with practical benefits for both accuracy and parallel efficiency.

Comment: Augments dot-product attention with learned low-dimensional L1 distance to improve similarity geometry.

Topic Match: The core contribution is a new attention scoring mechanism with analyzed head specialization.

Relevance: 8 Novelty: 6

6. Conditioned Direct Feedback Alignment via Activity and Error Geometry

ArXiv ID: 2607.18574

Primary Topic: Architecture and Training Dynamics

Authors: Houman Safaai, Varun Reddy, Bernardo L. Sabatini

Abstract: Direct feedback alignment (DFA) trains hidden layers with fixed random projections of the output error, avoiding the transposed-weight backward pass of backpropagation (BP). We study a failure mode of DFA training that is distinct from feedback quality: the local weight update is calculated by an outer product, so anisotropy can enter through either its presynaptic-activity factor or its local-error factor. Our analyses with controlled synthetic regimes isolate the first failure mode and show an approximately 40-percentage-point activity-conditioning gain when high-variance directions contain task-irrelevant nuisance. Three clean confirmations isolate a different regime: error conditioning improves raw DFA by 1.77--7.53 percentage points, and combining independently selected activity and error factors adds 0.40--0.90 points over activity conditioning. The signs hold for tanh/one-vs-rest MNIST and preregistered Fashion-MNIST, and replicate on eight fresh seeds in a ReLU/softmax MNIST model. This factorization yields a symmetric block-local family of normalized DFA (nDFA): activity nDFA right-preconditions by an inverse activity second moment, error nDFA left-preconditions by an inverse local-error second moment, and K-nDFA applies both factors with separately tuned damping. A linearized post-alignment calculation gives an exact input-side spectral identity and a Kronecker-factor motivation for the two-sided rule, whereas norm matching rules out a scalar step-size explanation. The error factor is fragile when under-damped, BatchNorm is a strong activity-side alternative, and convnet gains remain partial. We therefore frame conditioned DFA as a factor-level study of when local outer-product rules fail, not as a general replacement for BP or a solution to all-layer convolutional credit assignment.

Comment: Preconditions both activity and error factors in direct feedback alignment to correct anisotropic local updates.

Topic Match: The paper mechanistically analyzes and improves the training dynamics of a local credit-assignment rule.

Relevance: 7 Novelty: 7

7. Model of Models: When Does Emitting a Specialist Beat Attending, Adapting, or Tuning?

ArXiv ID: 2608.21386

Primary Topic: Architecture and Training Dynamics

Authors: John C. Howell

Abstract: Given a task described by a few examples, how should a model be specialized to it? Four mechanisms are available -- zero-shot, in-context attention, test-time gradient adaptation, and emitting specialist weights from a hypernetwork -- yet the operating regime of the last is rarely mapped. We run the identical four-way comparison across six tasks spanning regression, generation, language modeling, reinforcement learning, and clinical and genomic classification, holding the specialist, the context, and (where we can) the training budget fixed. The clearest wins for emission are about cost at matched quality: it ties the state-of-the-art amortized tabular model (TabPFN) on clinical few-shot classification while emitting a reusable specialist instead of re-attending the support set per query, and reaches noise-floor shape generation with a $132$-float per-instance program. On few-shot sinusoid regression it is $2$--$3$ orders of magnitude below MAML at zero test-time gradient steps -- a margin that narrows to $\sim$$30\times$ but persists once training budgets are equalized. Emission cannot match in-context attention on high-dimensional sequence modeling: under matched-budget pre-training a one-pass adapter recovers only a minority of the in-context gain ($14.0\pm0.9\%$ at $5$M, $11.2\pm0.5\%$ at $15$M), and a LoRA-rank sweep shows this shortfall is a partial capacity limit -- capture climbs from $5\%$ to $21\%$ as rank grows but plateaus far below full recovery. Mechanism ablations confirm the emitted specialist is genuinely task-conditioned, not a memorized prior; and, more speculatively, emitted specialists compose in weight space -- interpolating two of them tracks the corresponding blend of their functions. We close with a falsifiable thesis, operationalized through a per-task resolution measure, bounding when each conditioning mechanism should be preferred.

Comment: Maps when a hypernetwork-emitted specialist is preferable to attention, gradient adaptation, or tuning under controlled budgets.

Topic Match: The paper analyzes task-conditioned weight emission as an alternative architectural mechanism for specialization.

Relevance: 7 Novelty: 7

8. On the Diverse Dynamical Behaviors Arising in Deep Linear Transformers

ArXiv ID: 2607.18584

Primary Topic: Architecture and Training Dynamics

Authors: Sixu Li, Thomas Jacob Maranzatto, Jan Peszek, Trevor Teolis, Semih Akkoc, Konstantin Riedl, Sennur Ulukus, Nicolás García Trillos

Abstract: We study the inference-time behavior of deep linear encoder-only transformers through the lens of interacting particle systems. In this perspective, tokens are modeled as particles that interact dynamically through successive linear self-attention layers. We show that in embedding dimension two, for any key, query, and value matrices, the dynamics can be reformulated as a generalized Kuramoto-type model with pure second-harmonic coupling. This formulation is amenable to Watanabe--Strogatz theory which reveals the dynamics are intrinsically low-dimensional regardless of the parameter matrices. For a class of token initializations associated with the Ott--Antonsen (OA) manifold, we show that the parameter matrices induce a diverse variety of long-time behaviors in linear transformers, including clustering, oscillations, and bifurcations. The oscillations and bifurcations are characterized by uncovering a hidden Hamiltonian structure in the dynamics. By establishing a structural stability result, we further show that dynamics initialized near the OA manifold exhibit the same long-time behavior as those initialized exactly on the manifold. Motivated by our theory in dimension two, we conduct numerical experiments for analogous parameter regimes in higher-dimensional transformers. Our numerical experiments suggest that the long-time behaviors characterized in our theoretical results persist in higher dimensions.

Comment: Derives a low-dimensional dynamical account of stacked linear self-attention, including clustering, oscillations, and bifurcations.

Topic Match: Its central contribution is a mechanistic analysis of transformer computation and long-term token dynamics.

Relevance: 6 Novelty: 8

9. PAC--Bayes Bounds on Quotient Parameter Spaces: Geometry-induced Implicit-Bias Priors

ArXiv ID: 2607.18422

Primary Topic: Architecture and Training Dynamics

Authors: Nicola Aladrah, Fabio Anselmi

Abstract: Overparameterized models often have continuous parameter symmetries, so different parameters define the same predictor. We show that PAC--Bayesian analysis should be performed on the quotient predictor space: pushing a prior and posterior to the quotient preserves the empirical and population Gibbs risks while removing the nonnegative KL contribution caused solely by how the two distributions differ among parameterizations of the same predictor. Quotienting alone does not determine which prior to use. We construct a canonical choice of one parameterization for each predictor and account for the geometric volume of its equivalent parameterizations. This transforms a neutral reference prior into a data-independent prior that reflects the model's implicit bias. It approximates the ideal but inadmissible posterior-matched prior, which would minimize the KL term by depending on the training data. The resulting certificate is tighter exactly when this geometry-induced prior has smaller KL divergence from the learned quotient posterior than the neutral prior. We test this prediction in Fourier regression with a Hadamard parameterization and in Query-Key attention, using ordinary SGD without an explicit regularizer. The implicit-bias prior reduces the mean quotient-space KL by (40.69\%) and the mean PAC--Bayes certificate by (21.40\%) in the Fourier-Hadamard experiment. The smaller, prior-scale-dependent improvement in Query-Key attention confirms the predicted conditional nature of the effect.

Comment: Derives geometry-induced priors on quotient parameter spaces to explain and tighten certificates for SGD implicit bias.

Topic Match: The paper primarily analyzes how parameter symmetry shapes optimization and implicit training bias.

Relevance: 6 Novelty: 8

10. Organization of computation in reservoir computing

ArXiv ID: 2607.17858

Primary Topic: Architecture and Training Dynamics

Authors: Mohab Abdalla, Damien Rontani

Abstract: Reservoir computing exploits nonlinear dynamical systems to encode temporal inputs into high-dimensional state space representations. Although reservoir performance is often characterized through memory, nonlinearity, and their tradeoff, such aggregate measures do not reveal how task-relevant information is organized within the state space. Here, we introduce an eigen-spectral decomposition framework linking the degree-wise information processing capacity to the corresponding state space modes. As a result, we are able to quantify the degree-wise representation energy, and show that in some cases, substantial amounts of information processing capacity may reside in low-energy modes that are vulnerable to experimental noise. These results suggest that useful reservoir computation depends not only on dimensionality expansion, but also on the geometric organization of task-relevant information, with direct implications on physical reservoir computers.

Comment: Links degree-wise information-processing capacity to specific spectral modes in recurrent reservoir state spaces.

Topic Match: It provides mechanistic insight into how a recurrent computational architecture organizes task-relevant information.

Relevance: 6 Novelty: 7

11. A Weisfeiler-Leman Characterization of Global-Attention Graph Transformers for Mixed-Integer Linear Programs

ArXiv ID: 2607.17570

Primary Topic: Architecture and Training Dynamics

Authors: Md Abrar Jahin, Craig A. Knoblock, Jay Pujara

Abstract: Graph foundation models (GFMs) with global attention are increasingly used to represent mixed-integer linear programs (MILPs), aiming to capture structure beyond the locality of standard graph neural networks. We study their expressive power through graph isomorphism testing, asking which MILP instances they map to identical representations. We prove that a broad class of hierarchical graph transformers combining global linear attention, edge-weighted cross-attention, and bipartite message passing is bounded by the one-dimensional Weisfeiler-Leman (1-WL) test: under any parameter setting, 1-WL-equivalent MILP graphs receive identical graph embeddings. Our compositional proof shows that each architectural component is a symmetric multiset function and thus preserves 1-WL equivalence. We validate this characterization across ten diverse graph encoders, including Graphormer-, GraphGPS-, Set-Transformer-, and Gasse-style models. Across model capacities, graph scales, and pooling operators, every tested encoder maps 1-WL-equivalent non-isomorphic graph pairs to numerically identical embeddings. Consequently, graph invariants that vary within a 1-WL equivalence class cannot be recovered from these representations. We further show that expressiveness beyond 1-WL arises from input encoding rather than attention: random-walk positional encodings separate the constructed pairs, while additional constructions expose the limits of this remedy. These results characterize the expressive power of global-attention GFMs and provide an encoder-agnostic diagnostic for detecting 1-WL-induced representation equivalence.

Comment: Characterizes a broad family of global-attention graph Transformers through a compositional 1-WL expressivity bound.

Topic Match: The primary contribution is a mechanistic limitation of attention-based graph architectures.

Relevance: 6 Novelty: 7

12. Adaptive Mamba Neural Operators

ArXiv ID: 2607.18043

Primary Topic: Architecture and Training Dynamics

Authors: Zeyuan Song, Zheyu Jiang

Abstract: Accurately solving partial differential equations (PDEs) on arbitrary geometries and a variety of meshes is an important task in science and engineering applications. In this paper, we propose Adaptive Mamba Neural Operators (AMO), which integrates reproducing kernels for state-space models (SSMs) rather than the kernel integral formulation of SSMs. This is achieved by constructing Takenaka-Malmquist systems for the PDEs. AMO offers new representations that align well with the adaptive Fourier decomposition (AFD) theory and can approximate the solution manifold of PDEs on a wide range of geometries and meshes. In several challenging benchmark PDE problems in the fields of fluid physics, solid physics, and finance on point clouds, structured meshes, regular grids, and irregular domains, AMO consistently outperforms state-of-the-art solvers in terms of relative $L^2$ error. Overall, this work presents a new paradigm for designing explainable neural operator frameworks.

Comment: Reformulates Mamba neural operators using reproducing kernels and adaptive Takenaka-Malmquist systems.

Topic Match: The core contribution is a new state-space architectural representation.

Relevance: 6 Novelty: 7

13. Brain-Aligned Multi-Stream Video Transformers with Sparse Self-Selection

ArXiv ID: 2607.17625

Primary Topic: Architecture and Training Dynamics

Also Matches: Efficiency, Compression, and Large-Scale Training

Authors: Amir Hosein Fadaei, Mahyar Maleki, Mohammad-Reza A. Dehaqani

Abstract: Modern video transformers typically ignore principles from primate vision and are rarely evaluated against neural data, limiting their biological interpretability. We introduce a sparse winner-takes-all token selection module that replaces dense self-attention to improve efficiency and approximate competitive routing observed in biological visual circuits. We further propose a neuro-inspired split-and-fuse video transformer which uses two complementary pathways: a high-resolution, low-frame-rate "what" stream and a low-resolution, high-frame-rate "where" stream, fused before classification. On Kinetics-400 and Something-Something V2, our best variant operates on the Pareto frontier of accuracy versus inference time among models of comparable scale and pretraining, and showing improved robustness to spatial perturbations. Using representational similarity analysis between model embeddings and time-resolved EEG recordings for the same video stimuli, our model attains a peak brain-model correlation of 0.18 (about 78% of the noise ceiling) and consistently outperforms strong video transformer baselines, suggesting that pathway specialization and sparse competition are useful inductive biases for efficient, brain-aligned video understanding.

Comment: Replaces dense self-attention with sparse winner-takes-all token selection in a split-stream Transformer.

Topic Match: The sparse selection mechanism is primarily an architectural change with secondary inference-efficiency benefits.

Relevance: 6 Novelty: 6

Efficiency, Compression, and Large-Scale Training (12)

1. CoCurve: Cross-Module Co-Pruning Curvature for Training-Free Structured LLM Pruning

ArXiv ID: 2607.17568

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Zhiren Gong, Zihao Zeng, Zijie Wang, Tiantong Wang, Chau Yuen, Wei Yang Bryan Lim

Abstract: Structured pruning compresses large language models (LLMs) by removing whole computational units, such as attention heads and feed-forward (FFN) channel groups. Most training-free methods, however, rank these units independently, implicitly treating the loss from pruning a set as the sum of its individual losses. This view fails for Transformers, whose sublayers are coupled through a shared residual stream. Two individually weak units can thus be jointly indispensable, yet independent scoring is blind to such dependence and removes them together. We introduce CoCurve (Cross-Module Co-Pruning Curvature), a calibration-only, fine-tuning-free method that prunes attention and FFN units jointly. A second-order Taylor expansion of the token-level KL between the frozen model and its masked copy yields a single Fisher matrix whose diagonal is classical node saliency and whose off-diagonal entries are co-pruning curvature edges: the extra damage of removing two units together. Under a single-ablation additivity approximation this matrix reduces to a Gram product of single-unit ablation features, so the full M x M interaction is recovered from M forward passes, with no pairwise sweeps or gradients. Pruning then reduces to one budgeted quadratic program, solved in a single shot under a shared attention--FFN budget, with no labels, fine-tuning, or recovery.

Comment: Introduces joint attention–FFN structured pruning using cross-unit curvature recovered from single-unit ablations.

Topic Match: The paper directly advances training-free structured compression through interaction-aware pruning.

Relevance: 9 Novelty: 7

2. MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference

ArXiv ID: 2607.17733

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Simla Burcu Harma, Danila Mishin, Zhengyuan Su, Ayan Chakraborty, Elizaveta Kostenok, Dongho Ha, Babak Falsafi, Martin Jaggi, Yunho Oh, Amir Yazdanbakhsh

Abstract: 4-bit quantization enables efficient LLM inference, but suffers from significant accuracy degradation due to outliers. Prior work addresses this problem via data rotation or mixed-precision integer quantization, but often relies on software-managed scaling and frequent dequantization, incurring substantial overhead. Microscaling formats, such as MXINT, eliminate these inefficiencies by encoding scales in hardware, yet remain incompatible with rotation-based methods. Our analysis reveals that outliers vary in severity, from rare extremes to frequent mild deviations, and that quantization sensitivity is unevenly distributed across layers and columns. These insights motivate a fine-grained, sensitivity-guided approach. We introduce MXSens, a training-free method that assigns mixed mantissa bitwidths (4/6/8) based on column- and layer-wise sensitivity, naturally leveraging the block-wise structure of MXINT. MXSens outperforms state-of-the-art quantization methods across a range of models and tasks. Under the W4A4KV4 setting, MXSens achieves perplexities of 3.77 and 7.63 on LLaMA-2-70B and LLaMA-3-8B, respectively, substantially improving over existing baselines on WikiText-2. Our work establishes a new balance between accuracy and resource efficiency for LLM quantization.

Comment: Assigns 4-, 6-, and 8-bit microscaling precision by column- and layer-level quantization sensitivity.

Topic Match: This is directly centered on training-free mixed-precision LLM quantization.

Relevance: 9 Novelty: 7

3. PoLoRA: A Preconditioned Orthogonalized LoRA Optimizer

ArXiv ID: 2607.17620

Primary Topic: Efficiency, Compression, and Large-Scale Training

Also Matches: Large-Scale Training Systems and Efficiency

Authors: Nikhil Ghosh, Tetiana Parshakova, Robert M. Gower

Abstract: Low-rank adaptation (LoRA) makes finetuning large language models cheaper by adding to each weight matrix a trainable low-rank update parameterized as the product of two matrices. These matrices are usually trained with Adam, which treats them as a single flat vector of parameters and ignores both the matrix and product structure of LoRA. Applying a matrix-aware optimizer such as Muon to each factor does not consistently improve over Adam, and neither do the product-aware Muon variants proposed in concurrent works. To realize consistent gains, we introduce PoLoRA, a Preconditioned Orthogonalized LoRA optimizer built from three ingredients: a product-aware spectral update direction, curvature preconditioning derived from controlling the per-sample loss change, and a magnitude rule that controls the sizes of both the factor and merged updates. We evaluate PoLoRA on instruction-tuning datasets for code and math across models from 1B to 8B parameters, and find that it reaches the final held-out loss achieved by tuned Adam in 1.2-1.7 times fewer steps, while adding at most 3% per-step overhead. Compared to Adam, PoLoRA is also less sensitive to the learning rate, and its optimal learning rate is stable across ranks.

Comment: Builds a product-aware, curvature-preconditioned optimizer for low-rank adaptation factors.

Topic Match: The method primarily improves parameter-efficient adaptation while also contributing a structured optimizer.

Relevance: 8 Novelty: 7

4. Manifold-Constrained Hyper-Connections for Parameter-Efficient Finetuning

ArXiv ID: 2607.18130

Primary Topic: Efficiency, Compression, and Large-Scale Training

Also Matches: Architecture and Training Dynamics

Authors: Valentijn Oldenburg, Floris de Kam, Bente Zuijdam, Lieve Eberson, Nicky van Zutphen, Stef de Wildt, Ivo Verhoeven

Abstract: Most parameter-efficient finetuning (PEFT) methods adapt weights or activations, thus leaving one of the key Transformer components unchanged: residual connections. This paper investigates Manifold-Constrained Hyper-Connections (mHC), a generalisation of residual connections, as a novel PEFT approach, wrapping frozen OLMo-2 backbones with learned residual routing modules. We find that mHC can finetune frozen Transformers, but that its role differs fundamentally from the original pre-training setting: in finetuning, fixing the residual mixing matrix to identity often improves performance. As a standalone PEFT method, mHC does not consistently outperform LoRA. However, at matched trainable parameter budgets, mHC+LoRA combinations improve language-modelling loss and show task-dependent benchmark gains at both 1B and 7B scale. Overall, our results identify residual routing as a distinct and promising novel PEFT axis.

Comment: Uses learned residual routing as a parameter-efficient adaptation axis complementary to LoRA.

Topic Match: Parameter-efficient adaptation is the main goal, implemented through a new residual architecture.

Relevance: 8 Novelty: 7

5. BRIM: Workload-Balanced Dual-Sided Bit-Serial Sparse Inference Accelerator

ArXiv ID: 2607.19431

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Varun Manjunath, Ruokai Yin, Donghyun Lee, Arkapravo Ghosh, Priyadarshini Panda

Abstract: Bit-serial accelerators exploit bit-level sparsity to reduce DNN inference cost, but existing designs exploit sparsity on only one operand, bounding the speedup. Extending sparsity exploitation to both operands simultaneously yields compounding reductions in partial products but introduces a critical new bottleneck: workload imbalance. Because each concurrent weight - activation pair's execution cost depends on the product of two independently varying operand non-zero bit counts, pairs that must complete together finish at vastly different times, leaving faster computations idle. We show this limits PE utilization to 56 - 64% in existing dual-sided designs. We present BRIM, a hardware - software co-designed dual-sided bit-serial sparse accelerator that directly targets this bottleneck. BRIM combines two integrated mechanisms: 1) Cyclic-Balanced Pruning (CBP), a post-training weight optimization that reshapes weight representations based on profiled activation statistics to equalize expected workloads across concurrently processed pairs offline; and 2) Pairwise Slot Donation, a lightweight hardware mechanism that absorbs residual runtime imbalance with negligible area overhead. Evaluated across CNNs, ViTs, and LLMs under iso-area constraints, BRIM achieves over 90% PE utilization, up to 2.37x speedup, and up to 1.63x energy efficiency improvement over prior dual-sided designs.

Comment: Combines workload-aware pruning with runtime slot donation for balanced dual-sided bit-sparse inference.

Topic Match: Its hardware-software sparsity mechanisms materially reduce inference time and energy.

Relevance: 8 Novelty: 7

6. FlowBlock: Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models

ArXiv ID: 2607.17652

Primary Topic: Efficiency, Compression, and Large-Scale Training

Also Matches: Architecture and Training Dynamics

Authors: Bing Tian, Haikun Liu, Xiaocheng Zhong, Zhuohui Duan, Zhaokai Luo, Huayi Jin, Zhiyong Wang, Xiaofei Liao

Abstract: Block-wise diffusion large language models (dLLMs) decode sequentially at the block level, enabling effective KV-cache reuse across blocks but making inter-block decoding strictly serial. Prior work has attempted to unlock inter-block parallelism through post-training methods, but achieves only modest speedups and often degrades accuracy. We observe that self-correcting dLLMs offer a training-free alternative: token-to-token (T2T) editing can repair tokens drafted with a slightly stale upstream context, so a downstream block requires only an informative draft rather than a finalized predecessor. This turns block finality from a hard dependency into a scheduling resource. We propose \textbf{\flowblock{}}, a training-free parallel decoding framework built on two mechanisms. (i) \emph{Gated Wavefront Decoding} admits blocks into a bounded wavefront only when a readiness gate is satisfied, jointly refines active blocks via T2T editing, and commits blocks in order under a windowed block-causal mask that preserves exact frozen-prefix KV caches reuse. (ii) \emph{Heterogeneous Wavefront Packing} assigns each request an independent wavefront while packing asynchronous windows into dense, shape-stable batched forwards. Across different benchmarks, \flowblock{} improves tokens per second (TPS) over LLaDA-2.1 and LLaDA-2.0, two serial block-wise dLLMs, by up to 2.95$\times$ and 4.01$\times$, while reducing latency by up to 53.6\% and 77.1\%, respectively. It also improves average accuracy by 1.3 points. Compared with D2F, a training-based inter-block-parallel baseline, \flowblock{} achieves higher accuracy and up to 16$\times$ higher batched serving throughput.

Comment: Relaxes serial diffusion-block dependencies through gated wavefront scheduling while preserving frozen-prefix KV-cache reuse.

Topic Match: The primary result is a new decoding schedule that materially improves large-model throughput and latency.

Relevance: 7 Novelty: 8

7. SelectInfer: Selective Neuron Loading and Computation for On-Device LLMs

ArXiv ID: 2607.18081

Primary Topic: Efficiency, Compression, and Large-Scale Training

Also Matches: Architecture and Training Dynamics

Authors: Huzaifa Shaaban Kabakibo, Eric Schniedermeyer, Artem Burchanow, Lin Wang

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities across a range of Natural Language Processing (NLP) tasks, but their high computational and memory demands pose significant challenges for deployment on resource-constrained edge devices. Existing approaches to model compression and optimization often rely on coarse-grained pruning or quantization, which can compromise accuracy or require re-training and fine-tuning. In this work, we introduce SelectInfer, a neuron-level optimization framework that enables efficient LLM inference on edge devices through selective neuron loading and computation. By profiling and identifying both task-specific and general-purpose neurons using an offline LLM profiler, SelectInfer implements two key optimizations: selective loading, which reduces memory footprint by selectively loading a subset of neurons that were identified to be most important during the offline stage, and selective computation, which dynamically computes only the most relevant neurons at runtime. Evaluation across multiple datasets shows that SelectInfer achieves significant reductions in memory footprint and computation while preserving task performance, making it a practical step towards enabling LLM deployment on edge devices

Comment: Selectively loads and computes task-relevant neurons to reduce on-device LLM memory and computation.

Topic Match: Its primary goal is inference efficiency through dynamic neuron-level sparsity.

Relevance: 8 Novelty: 6

8. Recti-Q: Feature-Space Rectification for Out-of-Distribution-Robust Quantized Perception in Edge Robotics

ArXiv ID: 2607.18540

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Hamidreza Yaghoubi Araghi, Parastoo Pilevar, Ming C. Lin

Abstract: Robotic perception pipelines increasingly rely on large vision backbones deployed on SWaP-constrained edge platforms, making post-training quantization (PTQ) attractive for real-time inference. However, while PTQ often preserves clean in-distribution accuracy, we show that it can substantially degrade reliability under deployment-relevant distribution shifts (e.g., sensor noise, severe weather, and novel operating environments), creating a Quantization-Induced Robustness Gap. Across foundational vision benchmarks (ImageNet-C and PACS), 4-bit PTQ models exhibit pronounced robustness degradation despite negligible ID accuracy loss. To address this, we propose Recti-Q, a lightweight feature-space rectification framework that freezes the quantized backbone and trains a small classifier-head LoRA adapter using only source data. Recti-Q is architecture-agnostic across CNNs and Transformers, supports efficient teacher-free training, and recovers a significant portion of the lost robustness, in some cases matching or exceeding FP32 performance. At less than 1% parameter overhead (as small as 6 KB), Recti-Q preserves over 99% of PTQ memory savings, adds negligible compute, and enables low-bandwidth Over-The-Air (OTA) resilience patching for deployed robotic fleets operating in unpredictable physical environments.

Comment: Adds a lightweight feature-space adapter that recovers distribution-shift robustness lost under 4-bit post-training quantization.

Topic Match: The core method addresses a failure mode of aggressive quantization while preserving nearly all memory savings.

Relevance: 7 Novelty: 6

9. DA-MergeLoRA: Hypernetwork-Based LoRA Merging for Few-Shot Test-Time Domain Adaptation

ArXiv ID: 2607.17467

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Siobhan Reid, Zhixiang Chi, Li Gu, Omid Reza Heidari, Ziqiang Wang, Yang Wang

Abstract: Few-shot Test-Time Domain Adaptation (FSTT-DA) seeks to adapt models to novel domains using only a handful of unlabeled target samples. This setting is more realistic than typical domain adaptation setups, which assume access to target data during source training. However, prior FSTT-DA approaches fail to effectively leverage source domain-specific knowledge, relying on shallow batch normalization updates, prompt-based methods that treat the model as a black box, or ensembling strategies that do not capture cross-domain relationships. To address these limitations, we introduce a new FSTT-DA framework that integrates LoRA fine-tuning with model merging. In our approach, separate LoRA modules are fine-tuned on CLIP's vision encoder for each source domain. Since LoRA modifies only a small fraction of the model's parameters, it retains the base model's generalized knowledge while internally learning domain-specific features. To adapt the learned knowledge to a specific target domain, we propose a hypernetwork trained via meta-learning that generates per-column merging factors to combine LoRA modules. Given a small batch of target images, the hypernetwork produces merging weights that fuse source LoRA modules into a single adapted representation. Our results demonstrate state-of-the-art performance across various domain adaptation datasets. Our code is publicly available at https://github.com/nahbois4321/DA-MergeLoRA.

Comment: Uses a meta-learned hypernetwork to merge domain-specific LoRA modules with per-column coefficients.

Topic Match: The core mechanism is a new low-rank adaptation and parameter-merging method.

Relevance: 7 Novelty: 6

10. SALT: Salience-Aware Lexical Trie for Long-Context Compression

ArXiv ID: 2607.17486

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Oteo Mamo, Hyunjin Yi, Joydhriti Choudhury, Shangqian Gao, Weikuan Yu

Abstract: As large language models (LLMs) process increasingly longer prompts, computation and KV-cache memory costs have emerged as major bottlenecks in inference systems. Existing input-level prompt compression methods address this, but rank each sentence by a scalar relevance score, treating the document as an unstructured pool of words and sentences. Under tight budgets, this causes theme collapse, where the dominant theme(s) of a document consumes the budget, discarding less-frequent yet task-relevant themes. Preserving thematic coverage instead requires allocating the budget across recurring themes rather than scoring sentences in isolation. To this end, we propose SALT, a model-agnostic extractive framework that organizes per-sentence keywords into a trie ordered by sentence frequency (SF), a lightweight, reusable proxy for document thematic structure. This trie-based organization smooths memory allocation and prevents dominant themes from monopolizing the budget. Multi-anchor retrieval activates trie nodes labeled by query keywords at any depth, and the trie persists across dialogue turns, supporting multi-turn use without re-encoding the document. By preserving document themes, SALT reduces the prefill computation and memory cost of long-context prompts while remaining composable with KV-cache methods that target decoding-time latency and memory.

Comment: Compresses long-context inputs with a persistent salience-aware trie to reduce prefill computation and memory.

Topic Match: The central contribution reduces long-context inference cost through structured prompt compression.

Relevance: 7 Novelty: 6

11. PRISM: Sensitivity-Aware PolynoMial PRuning for EffIcient Neural Network Encryption

ArXiv ID: 2607.18342

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Sahaj Majavdia, Mahdi Taheri

Abstract: Structured pruning is essential for making neural network inference feasible under homomorphic encryption (HE), yet its impact on model reliability has remained unexplored. This paper presents a systematic reliability characterization of pruned CKKS-encrypted neural networks and introduces Polynomial-Sensitivity-Aware Pruning (PSAP), a structured pruning method that is inherently reliability-aware. PSAP scores filters jointly by weight magnitude, polynomial activation sensitivity, and rotation cost, which concentrates pruning in fault-tolerant regions. Across two architectures, two datasets, two numerical representations, and five bit-error rates (40 full-model and 108 per-layer experiments), PSAP-pruned models limit catastrophic (>10 pp accuracy drop) layers to at most two versus 5--14 for magnitude-pruned baselines, reducing worst-case vulnerability by up to 29 times under int32 bit-flip injection. Direct CKKS encrypted fault injection indicates a safe operating boundary near BER~ 10^{-5}, supporting int32 injection as a conservative reliability proxy. The fault-critical structural layers account for only 1.1% of parameters, enabling selective hardening at minimal overhead. These reliability gains are obtained alongside competitive efficiency: PSAP reduces Halevi--Shoup rotations by up to 45.2\% on ResNet-32, and an adaptive mixed-degree allocation scheme lowers multiplicative depth from 66 to 56 levels, enabling leveled inference without bootstrapping.

Comment: Prunes filters using joint weight, polynomial-sensitivity, and encrypted-rotation costs to retain reliability under homomorphic inference.

Topic Match: Structured pruning and mixed-degree allocation are the core mechanisms for reducing encrypted-model inference cost.

Relevance: 7 Novelty: 6

12. Opto-ViT-v2: Noise-Resilient On-Chip Fine-Tuning for Photonic Near-Sensor Vision Transformer Accelerators

ArXiv ID: 2607.19421

Primary Topic: Efficiency, Compression, and Large-Scale Training

Authors: Xuming Chen, Deniz Najafi, Mehrdad Morsali, Chengwei Zhou, Zahra Ghanaatianjobzari, Mahdi Nikdast, Shaahin Angizi, Gourav Datta

Abstract: Silicon-photonic (SiPh) accelerators have emerged as a promising platform for Vision Transformer (ViT) inference by performing matrix multiplications on microring-resonator (MRR) banks with high throughput and energy efficiency. Extending these platforms to support on-chip fine-tuning remains challenging because backpropagation requires large activation storage, frequent weight write-back to MRRs, and tolerance to device-level noise. We present Opto-ViT-v2, the first framework for parameter-efficient fine-tuning (PEFT) on a near-sensor SiPh ViT accelerator. Our tensorized low-rank decomposition separates pretrained optical weights from a small set of trainable electronic factors (as few as 8K parameters for ViT-Base), greatly reducing activation storage and weight updates while enabling practical on-chip training. We further introduce a gradient-accumulated sparse classifier that freezes low-importance weights through one-shot top-k gradient masking, reducing classifier training cost by about 40 percent. We also develop the first system-level noise model for photonic on-chip training, capturing the effects of MRR crosstalk, thermal drift, and laser amplitude noise during both forward and backward propagation. Calibrated using measurements from more than 200 fabricated MRR devices, the model shows that low-rank factor updates are more robust than full fine-tuning and conventional layer-wise low-rank adaptation under identical noise conditions. Experiments on VTAB-1K (19 tasks) and FGVC few-shot benchmarks demonstrate that Opto-ViT-v2 recovers within 0.3 to 0.8 percent of clean software accuracy under measured photonic noise while achieving more than 100 KFPS/W, enabling practical on-chip domain adaptation for photonic edge vision systems.

Comment: Combines tensorized low-rank updates with sparse classifier training for noise-resilient photonic fine-tuning.

Topic Match: Its main contribution is reducing the storage, update, and energy costs of hardware-constrained adaptation.

Relevance: 6 Novelty: 7

Paper Selection Prompt

System Prompt

You are a helpful paper reading assistant whose job is to read daily posts from ArXiv and identify a few papers that your friend will enjoy reading. Your job is to carefully read the paper titles and abstracts below and find the ones that match the criteria below.

User Prompt

Instructions

Respond in JSONL. Output exactly one JSON object per paper, one per line:

{"ARXIVID":"...","COMMENT":"...","RELEVANCE":0,"NOVELTY":0,"PRIMARY_TOPIC_ID":"...","MATCHED_TOPIC_IDS":[],"TOPIC_MATCH_COMMENT":"...","HOTSPOT_PAPER_TAGS":[],"HOTSPOT_PAPER_COMMENT":"..."}

Rules: - ARXIVID: the arXiv ID. - COMMENT: identify the single strongest matching criterion. Be brief and specific. Do not rely on generic phrases like "language modeling" or "advancement". Do not mention non-matching criteria. - RELEVANCE: integer from 1 to 10. - NOVELTY: integer from 1 to 10. - PRIMARY_TOPIC_ID: exactly one stable topic ID from the allowed topic registry. - MATCHED_TOPIC_IDS: zero or more stable topic IDs from the same allowed set. Include PRIMARY_TOPIC_ID when there are multiple matches. - TOPIC_MATCH_COMMENT: briefly explain why the primary topic is the best fit. - HOTSPOT_PAPER_TAGS: zero or more tags from this exact set only: daily_hot, new_frontier. - HOTSPOT_PAPER_COMMENT: briefly explain why the paper belongs in the daily hotspot paper feed when HOTSPOT_PAPER_TAGS is non-empty; otherwise use an empty string. - Use HOTSPOT_PAPER_TAGS sparingly. Most papers should return []. - daily_hot means the paper feels broadly important to the day and belongs in the daily hotspot paper section even if it is not part of the personalized foundational reading list. - new_frontier means the paper appears to open a genuinely new direction, paradigm, or field, even if the work is still early. - Do not output markdown, code fences, or any extra text.

Scoring Criteria

Relevance and Novelty are independent axes. Score both from 1 to 10.

Relevance Scoring

• 9-10: directly centered on the target foundational topics; highest when the core contribution is clearly within them.
• 7-8: substantially related, but partly peripheral or focused on a narrower aspect.
• 5-6: touches the target topics, but the main contribution is elsewhere.
• 3-4: largely outside the target topics, often application-focused or domain-specific.
• 1-2: unrelated.

Important: Broad frontier relevance, major launch status, or daily buzz is not enough for a high Relevance score here. Those cases belong in the hotspot digest unless the paper strongly matches the specialized paper topics.

Novelty Scoring

• 9-10: new paradigm, theory, or major methodological breakthrough.
• 7-8: substantial methodological advance or strong new insight.
• 5-6: meaningful but incremental extension or refinement.
• 3-4: minor, narrow, or mostly engineering or domain-specific improvement.
• 1-2: little originality; mainly standard application of existing methods.

Topic Registry

Use exactly one PRIMARY_TOPIC_ID chosen from the stable topic IDs below. - moe_training: MoE Training - Mixture-of-Experts training: routing, load balancing, expert granularity and shared experts, training stability, MoE parallelism and communication, and MoE kernels. - training_systems: Large-Scale Training Systems and Efficiency - Large-scale training systems and efficiency: distributed training algorithms, optimizers, and communication. - architecture_training: Architecture and Training Dynamics - Core architectural or computational mechanisms, dynamic computation, and training-stability dynamics. - efficiency_scaling: Efficiency, Compression, and Large-Scale Training - Compression, sparsity, memory or cache efficiency, and large-scale training systems that materially change cost or behavior.

Papers

[PAPER LIST HERE]

Relevant Topics

This is a training-side feed. The centre of gravity is Mixture-of-Experts training and the large-scale training systems it sits on; architecture and efficiency work is kept when it bears on how a large model is trained or what it costs to train.

Keep a paper when its CORE CONTRIBUTION falls in one of the four topics below. Filter it when the topic is merely the setting for a downstream application, a benchmark, or a product result.

1. MoE Training (primary) - Keep: expert routing and gating mechanisms; load balancing, with or without auxiliary losses; expert granularity, fine-grained and shared experts; MoE training stability (loss spikes, router collapse, z-loss and related fixes); expert and hybrid parallelism; all-to-all dispatch/combine and the communication schedules MoE needs; MoE kernels and grouped-GEMM style implementations; expert capacity, token dropping, and upcycling dense checkpoints into MoE. - Filter: papers that merely train on top of a MoE model without a new routing, balancing, stability, parallelism, or kernel contribution; "mixture of experts" in the classical ensemble or recommender sense.

2. Large-Scale Training Systems - Keep: distributed training algorithms (data, tensor, pipeline, sequence, and expert parallelism, FSDP and ZeRO-style sharding); optimizers and preconditioners for large-scale pretraining; communication algorithms, overlap, and collective design; throughput and MFU work that materially changes what a training run costs or whether it converges; scaling-law work that informs how such runs are configured. - Filter: routine infrastructure tuning with no new algorithmic or systems idea.

3. Architecture and Training Dynamics - Keep: new or analysed architectural mechanisms (attention variants, normalization and residual design, state-space and recurrent sequence modelling, dynamic or modular computation); optimisation and training-dynamics analysis that explains why large models train the way they do. - Filter: applying an existing architecture to a new task or benchmark with no mechanistic insight.

4. Efficiency and Compression - Keep: quantization, sparsity, pruning, low-rank adaptation, KV-cache and memory-efficient designs, and other work that materially changes the cost of running or training a large model, especially when the mechanism is new rather than a tuned variant. - Filter: straightforward application of standard efficiency methods, and deployment or serving engineering with no new idea.

Leave these to the hotspot digest unless the core contribution clearly falls in one of the four topics above: - major model or product releases - agent, tooling, and RAG launches - benchmark, leaderboard, or evaluation-only papers - representation-learning theory, interpretability, and feature-formation analysis - memory mechanisms and agent memory systems - world models, exploration, and reinforcement learning - alignment and post-training (RLHF, DPO, GRPO, RFT, instruction tuning) - downstream applications in medical imaging, segmentation, 3D vision, video understanding, information retrieval, summarization, recommendation, machine translation, speech recognition, time series, knowledge graphs, and similar domains