Paper Detail

Module-Aware Optimization for Graph Neural Networks

Guy Hadad, Haggai Roitman, Moshe Eliasof, Bracha Shapira

huggingface Score 5.4

Published 2026-11-05 · First seen 2026-11-12

General AI

Abstract

Different optimizers impose different geometries on parameter updates. Adaptive methods such as Adam and AdamW use coordinate-wise updates, while matrix-aware optimizers such as Muon utilize weight-matrix structure through orthogonalized updates. For Graph Neural Networks (GNNs), however, the effective geometry of its weight matrices is shaped not only by its learned channel-mixing weights, but also by the graph-propagated representations it acts on. Thus, even if an update is well normalized as a matrix, its effect on the layer output can be amplified when applied to graph-propagated features. To address this gap, we introduce GraphMuon, a graph-aware optimizer that adapts Muon's matrix updates to the covariance structure of graph-propagated layer inputs. GraphMuon performs orthogonalization in a metric defined by these propagated representations and normalizes step sizes according to the update's induced effect on the layer representation. We show that theoretically, vanilla Muon can suffer from graph-induced update amplification, while our GraphMuon can mitigate this effect. As such, our GraphMuon yields a graph-metric analogue of Muon's polar update step. Empirically, GraphMuon achieves 1.32times-2.42times validation-loss time-to-target speedups over AdamW on three large-scale OGB benchmarks. Across 15 dataset-architecture configurations spanning node classification, inductive graph classification, and graph regression, it achieves lower time-to-target in 13 cases, while maintaining competitive or superior predictive performance. Code is available at https://github.com/guy-hadad/GraphMuon.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
later
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@misc{hadad2026module,
  title = {Module-Aware Optimization for Graph Neural Networks},
  author = {Guy Hadad and Haggai Roitman and Moshe Eliasof and Bracha Shapira},
  year = {2026},
  abstract = {Different optimizers impose different geometries on parameter updates. Adaptive methods such as Adam and AdamW use coordinate-wise updates, while matrix-aware optimizers such as Muon utilize weight-matrix structure through orthogonalized updates. For Graph Neural Networks (GNNs), however, the effective geometry of its weight matrices is shaped not only by its learned channel-mixing weights, but also by the graph-propagated representations it acts on. Thus, even if an update is well normalized as},
  url = {https://huggingface.co/papers/2611.08379},
  keywords = {adaptive optimizers, Adam, AdamW, coordinate-wise updates, matrix-aware optimizers, Muon, weight-matrix structure, orthogonalized updates, Graph Neural Networks, GNNs, channel-mixing weights, graph-propagated representations, graph-propagated features, GraphMuon, graph-aware optimizer, covariance structure, graph-propagated layer inputs, orthogonalization, graph-induced update amplification, graph-metric analogue, polar update step, validation-loss time-to-target, OGB benchmarks, node classification, inductive graph classification, graph regression, code available, huggingface daily},
  eprint = {2611.08379},
  archiveprefix = {arXiv},
}

Metadata

{}