Paper Detail
Guy Hadad, Haggai Roitman, Moshe Eliasof, Bracha Shapira
Different optimizers impose different geometries on parameter updates. Adaptive methods such as Adam and AdamW use coordinate-wise updates, while matrix-aware optimizers such as Muon utilize weight-matrix structure through orthogonalized updates. For Graph Neural Networks (GNNs), however, the effective geometry of its weight matrices is shaped not only by its learned channel-mixing weights, but also by the graph-propagated representations it acts on. Thus, even if an update is well normalized as a matrix, its effect on the layer output can be amplified when applied to graph-propagated features. To address this gap, we introduce GraphMuon, a graph-aware optimizer that adapts Muon's matrix updates to the covariance structure of graph-propagated layer inputs. GraphMuon performs orthogonalization in a metric defined by these propagated representations and normalizes step sizes according to the update's induced effect on the layer representation. We show that theoretically, vanilla Muon can suffer from graph-induced update amplification, while our GraphMuon can mitigate this effect. As such, our GraphMuon yields a graph-metric analogue of Muon's polar update step. Empirically, GraphMuon achieves 1.32times-2.42times validation-loss time-to-target speedups over AdamW on three large-scale OGB benchmarks. Across 15 dataset-architecture configurations spanning node classification, inductive graph classification, and graph regression, it achieves lower time-to-target in 13 cases, while maintaining competitive or superior predictive performance. Code is available at https://github.com/guy-hadad/GraphMuon.
No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.
No ranking explanation is available yet.
No tags.
@misc{hadad2026module,
title = {Module-Aware Optimization for Graph Neural Networks},
author = {Guy Hadad and Haggai Roitman and Moshe Eliasof and Bracha Shapira},
year = {2026},
abstract = {Different optimizers impose different geometries on parameter updates. Adaptive methods such as Adam and AdamW use coordinate-wise updates, while matrix-aware optimizers such as Muon utilize weight-matrix structure through orthogonalized updates. For Graph Neural Networks (GNNs), however, the effective geometry of its weight matrices is shaped not only by its learned channel-mixing weights, but also by the graph-propagated representations it acts on. Thus, even if an update is well normalized as},
url = {https://huggingface.co/papers/2611.08379},
keywords = {adaptive optimizers, Adam, AdamW, coordinate-wise updates, matrix-aware optimizers, Muon, weight-matrix structure, orthogonalized updates, Graph Neural Networks, GNNs, channel-mixing weights, graph-propagated representations, graph-propagated features, GraphMuon, graph-aware optimizer, covariance structure, graph-propagated layer inputs, orthogonalization, graph-induced update amplification, graph-metric analogue, polar update step, validation-loss time-to-target, OGB benchmarks, node classification, inductive graph classification, graph regression, code available, huggingface daily},
eprint = {2611.08379},
archiveprefix = {arXiv},
}
{}