No-Regret Non-Convex Online Meta-Learning

作者：Zhenxun Zhuang, Yunlong Wang, Kezi Yu, Songtao Lu · 年份：2020 · DOI：10.1109/icassp40776.2020.9054002 · 被引用次数：8 · 研究领域：Domain Adaptation and Few-Shot Learning、Multimodal Machine Learning Applications、Advanced Bandit Algorithms Research

The online meta-learning framework is designed for the continual lifelong learning setting. It bridges two fields: meta-learning which tries to extract prior knowledge from past tasks for fast learning of future tasks, and online-learning which tackles the sequential setting where problems are revealed one by one. In this paper, we generalize the original framework from convex to non-convex setting, and introduce the local regret as the alternative performance measure. We then apply this framework to stochastic settings, and show theoretically that it enjoys a logarithmic local regret, and is robust to any hyperparameter initialization. The empirical test on a real-world task demonstrates its superiority compared with traditional methods.