Mol-AE: Auto-Encoder Based Molecular Representation Learning With 3D Cloze Test Objective

作者：Junwei Yang, Kangjie Zheng, Siyu Long, Zaiqing Nie, Ming Zhang, Xinyu Dai, Wei‐Ying Ma, Hao Zhou · 发表于：bioRxiv (Cold Spring Harbor Laboratory) · 年份：2024 · DOI：10.1101/2024.04.13.589331 · 被引用次数：10 · 研究领域：Machine Learning in Materials Science、Computational Drug Discovery Methods、Machine Learning in Bioinformatics

Abstract 3D molecular representation learning has gained tremendous interest and achieved promising performance in various downstream tasks. A series of recent approaches follow a prevalent framework: an encoder-only model coupled with a coordinate denoising objective. However, through a series of analytical experiments, we prove that the encoderonly model with coordinate denoising objective exhibits inconsistency between pre-training and downstream objectives, as well as issues with disrupted atomic identifiers. To address these two issues, we propose M ol -AE for molecular representation learning, an auto-encoder model using positional encoding as atomic identifiers. We also propose a new training objective named 3D Cloze Test to make the model learn better atom spatial relationships from real molecular substructures. Empirical results demonstrate that M ol -AE achieves a large margin performance gain compared to the current state-of-the-art 3D molecular modeling approach. The source codes of M ol -AE are publicly available at https://github.com/yjwtheonly/MolAE .