Process Control in Semiconductor Manufacturing Based on Deep Distributional Soft Actor-Critic Reinforcement Learning

作者：Bangxu Liu, Dewen Zhao, Xinchun Lu, Yuhong Liu · 发表于：IEEE Transactions on Semiconductor Manufacturing · 年份：2025 · DOI：10.1109/tsm.2025.3539223 · 被引用次数：11 · 研究领域：Digital Transformation in Industry

The quality of semiconductor fabrication processes is typically degraded by variations in the manufacturing environment, which can be suppressed by run-to-run (R2R) control schemes. The performance of controlling systems to the produce process which is always highly complex and nonlinear physical model thus is strongly associated with the controlling strategy. However, previous works focusing on less complex semiconductor fabrication processes or linear controlling strategy are both hard to extend the application scenario. A novel structure for a R2R control system based on a distributed form of deep reinforcement learning (DRL), namely, distributional soft actor-critic (DSAC) DRL with twin-value distribution learning, is proposed for multizone pressure control in the chemical mechanical planarization (CMP) process, which is one of the most crucial manufacturing processes for the fabrication of ultra-large integrated circuits (ICs). In addition, several optimization algorithms for DRL, such as twin value distribution learning, are applied, further improving DSAC DRL to enhance the control performance. Compared with other reinforcement learning (RL)-based controllers, the proposed RL control policy achieves better control performance when tested using a multizone CMP virtual metrology (VM) model based on long short-term memory (LSTM) and one-dimensional convolutional neural network (1DCNN) architectures. This deep neural network (DNN) VM model, which is applied for the first t...