Mastering the Twin–Game: Hierarchical Reinforcement Learning in a Digital Twin Sandbox for Adaptive Urban Healthcare Optimization—A Case Study of Wuhan

作者：Yuxuan Hu, Shaohua Wang, Haojian Liang · 发表于：ISPRS International Journal of Geo-Information · 年份：2026 · DOI：10.3390/ijgi15060273 · 被引用次数：2 · 研究领域：Healthcare Operations and Scheduling Optimization、Reinforcement Learning in Robotics、Advanced Technologies in Various Fields

Urban healthcare systems are fundamentally constrained by the mismatch between static resource configurations and dynamically evolving patient demand. Under the tiered healthcare system, traditional static planning methods struggle to capture the complexity and randomness of patient flows. While recent reinforcement learning (RL) approaches enable adaptive decision-making, they suffer from dimensionality explosion and unstable convergence due to massive action spaces and delayed spatiotemporal credit assignment in city-scale environments. To address this gap, we propose Twin–Game: a digital twin-driven hierarchical reinforcement learning (HRL) framework that formulates adaptive healthcare resource optimization as a “Twin Game” between a simulation-based game environment (Strategic Sandbox) and a hierarchical decision policy. First, we construct the “first twin”—an offline digital twin that serves as the Strategic Sandbox parameterized with Wuhan’s observed facility, population, and transportation data, while patient arrivals and disease profiles are generated synthetically under documented assumptions because individual-level clinical flow data are not publicly available. This environment integrates a dynamic gravity model with a two-way referral mechanism to represent the nonlinear coupling between hospital attractiveness, crowding levels, and patient choice behaviors. Second, we build the “second twin”—an Option-based HRL policy. The Manager (Macro-level Strategic Layer) us...