Reinforcement Learning for Energy-Performance Co-optimization in HPC Compute Nodes

Abstract

The global energy footprint of high-performance computing (HPC) and AI infrastructure presents an urgent challenge: data centers consumed approximately 415~TWh of electricity in 2024 and are projected to nearly double to 945~TWh by 2030. At the compute-node level, the mismatch between processor speed and memory subsystem latency creates temporally and spatially uneven power consumption that static power budgets cannot address efficiently. Existing power management approaches are either hardware-specific, requiring per-platform characterization, or application-specific, requiring re-parameterization for each workload - neither is suited for the heterogeneous, multi-tenant HPC environments where hundreds of distinct applications share the same infrastructure. This dissertation develops a progression of reinforcement learning (RL) frameworks for adaptive, application-agnostic energy management in HPC, and extends the scope to dynamic resource management across the edge-to-HPC computing continuum. Our first contribution is an online RL controller using Proximal Policy Optimization (PPO) trained on a lightweight mathematical surrogate model of compute-node behavior, achieving a 21% reduction in energy consumption with only a 6.5% performance penalty on memory-bandwidth-intensive workloads. The second contribution eliminates the surrogate model requirement entirely through offline RL: a controller trained on pre-collected execution traces from multiple benchmark applications achieves 20.34% average energy savings with 7.4% average performance degradation across twelve benchmarks, without any live interaction with the target system during training. The third contribution is the Decay-Driven Multi-Objective Reinforcement Learning (DDMORL) framework, which enables a single offline-trained controller to cover the entire energy-performance trade-off surface through a user-specified degradation tolerance parameter. DDMORL achieves 31--39% energy savings on memory-bandwidth-bound workloads and 20--31% on compute-bound workloads, placing control of the energy-performance trade-off directly in the hands of the user without manual tuning or multiple trained agents. The fourth contribution, the Charon framework, addresses dynamic resource management across the edge-to-HPC computing continuum. Charon arbitrates edge computational resources through a closed-loop proportional-derivative controller and adaptive AI model quantization, demonstrating real-time processing of streaming synchrotron beamline data at a throughput target of 1{,}000 frames per second. Together, this dissertation demonstrate that learning-based adaptive control is a practical and generalizable framework for energy-efficient scientific computing, from individual HPC compute nodes to distributed edge-to-HPC workflows.