Adaptive and Scalable AI-Driven Decision-Making for Cyber-Physical Systems

Abstract

Learning enabled cyber physical systems (CPS) increasingly integrate AI driven policies to close decision loops, from self driving vehicles and aerial robots to dexterous manipulators. Although such approaches can tackle complex settings where classic model based techniques are limited, they face challenges as operating conditions evolve and as dimensionality and uncertainty increase. Among AI driven decision methods, sequence modeling for RL now stands out by learning strong priors over action sequences, but without an explicit planner it can chase lucky trajectories and breaks down under stochastic dynamics. Planning with learned models under uncertainty provides promising results, yet scaling to high dimensional continuous action spaces remains difficult. Risk averse planning for nonstationary dynamics provides assurance, but often stays too conservative once parts of the environment are understood. This research argues that reliable decision making in such settings needs three capabilities: (i) mechanisms to stay safe and adapt as operating conditions change; (ii) a learned lookahead model that scales to high dimensional, stochastic, continuous control; and (iii) a planner that reasons about uncertainty and uses it efficiently in decisions. We make these requirements concrete through five contributions. First, we introduce a runtime assurance framework that hands control between a high-performance learning-based controller and a verified safety controller so behavior remains predictable and safe as conditions shift. Second, we develop an online learning and planning framework that begins risk-averse, gathers information safely, and becomes reward-seeking as the agent adapts, by using separate estimates of epistemic and aleatoric uncertainty. Third, to make planning scalable in high-dimensional continuous control, we learn lookahead models that discretize the action space into a compact set of state-conditioned latent macro-actions and model future outcomes with sequence modeling. Fourth, we develop planners that leverage these learned models for efficient planning and decision making in both fully observable and partially observable settings through in-context temporal abstraction. Fifth, we introduce ARGO, an online model-based reinforcement learning framework that treats model uncertainty asymmetrically: an actor-centered optimistic planner expands replay coverage during data collection, while support-regularized local policy improvement constructs actor updates anchored in replay-supported, conservatively evaluated actions, improving sample efficiency over strong world-model baselines. Together, these five contributions outline a path toward model-based planning for CPS that is safe, uncertainty-aware, and scalable, while remaining adaptive under nonstationary conditions.