On-demand and Parallel Checkpoint/Restore for GPU Applications

作者：Yanning Yang, Dong Du, Haitao Song, Yubin Xia · 年份：2024 · DOI：10.1145/3698038.3698510 · 被引用次数：8 · 研究领域：Parallel Computing and Optimization Techniques、Advanced Data Storage Technologies、Distributed and Parallel Computing Systems

Leveraging serverless computing for cloud-based machine learning services is on the rise, promising cost-efficiency and flexibility are crucial for ML applications relying on high-performance GPUs and substantial memory. However, despite modern serverless platforms handling diverse devices like GPUs seamlessly on a pay-as-you-go basis, a longstanding challenge remains: startup latency, a well-studied issue when serverless is CPU-centric. For example, initializing GPU apps with minor GPU models, like MobileNet, demands several seconds. For more intricate models such as GPT-2, startup latency can escalate to around 10 seconds, vastly overshadowing the short computation time for GPU-based inference. Prior solutions tailored for CPU serverless setups, like fork() and Checkpoint/Restore, cannot be directly and effectively applied due to differences between CPUs and GPUs.