Paper Detail

PACER: GPU Excursion Management for ML Workloads

Rutwik Jain, Nathaniel Morris, Yiwei Jiang, Shidhartha Das, Gary Kao, Indrani Paul, Shivaram Venkataraman, Srilatha Manne, Matthew D. Sinclair

arxiv Score 7.8

Published 2026-11-12 · First seen 2026-11-13

General AI

Abstract

Variability is an inherent characteristic of modern GPU systems, arising from a combination of process variation, dynamic environmental conditions, and power management decisions. Prior work has characterized inter-GPU variability and found consistent performance differences across GPUs. However, we observe that modern workloads, particularly large language models (LLMs), exhibit significant intra-GPU variability. Their short-running kernels interact poorly with the coarse granularity of dynamic power management mechanisms, leading to unpredictable performance. A key source of this intra-GPU transient variability is power excursions, also referred to as electrical design power peaks (EDP$_{p}$), where power temporarily exceeds the GPU's rated power limits. While power management mechanisms are designed to ensure reliable operation despite the presence of EDP$_{p}$ sustained power peaks cause throttling that detrimentally impacts overall application performance. To the best of our knowledge, our work is the first to characterize the magnitude and frequency of EDP$_{p}$, showing that frequent, significant power overshoots are especially common in ML workloads, which combine high power draw, short kernels, and rapidly changing compute demands. To address this, we design and leverage hardware and firmware mechanisms to provide consistent power management behavior that reduces intra-GPU variability due to EDP$_{p}$ and improves overall performance. We evaluate our techniques across a range of popular workloads. Our approach improves performance by 3-12% over state-of-the-art GPU power management and comes within 3% of an oracular solution.

Workflow Status

Review status
pending
Role
unreviewed
Read priority
soon
Vote
Not set.
Saved
no
Collections
Not filed yet.
Next action
Not filled yet.

Reading Brief

No structured notes yet. Add `summary_sections`, `why_relevant`, `claim_impact`, or `next_action` in `papers.jsonl` to enrich this view.

Why It Surfaced

No ranking explanation is available yet.

Tags

No tags.

BibTeX

@article{jain2026pacer,
  title = {PACER: GPU Excursion Management for ML Workloads},
  author = {Rutwik Jain and Nathaniel Morris and Yiwei Jiang and Shidhartha Das and Gary Kao and Indrani Paul and Shivaram Venkataraman and Srilatha Manne and Matthew D. Sinclair},
  year = {2026},
  abstract = {Variability is an inherent characteristic of modern GPU systems, arising from a combination of process variation, dynamic environmental conditions, and power management decisions. Prior work has characterized inter-GPU variability and found consistent performance differences across GPUs. However, we observe that modern workloads, particularly large language models (LLMs), exhibit significant intra-GPU variability. Their short-running kernels interact poorly with the coarse granularity of dynamic},
  url = {https://arxiv.org/abs/2611.17206},
  keywords = {cs.DC},
  eprint = {2611.17206},
  archiveprefix = {arXiv},
}

Metadata

{}