Information Theoretic Meta Learning with Gaussian Processes

作者：Michalis K. Titsias, Ruiz, Francisco J. R., Sotirios Nikoloutsopoulos, Alexandre Galashov · 发表于：arXiv (Cornell University) · 年份：2020 · DOI：10.48550/arxiv.2009.03228 · 被引用次数：5 · 研究领域：Domain Adaptation and Few-Shot Learning、Gaussian Processes and Bayesian Inference、Machine Learning and Data Classification

We formulate meta learning using information theoretic concepts; namely, mutual information and the information bottleneck. The idea is to learn a stochastic representation or encoding of the task description, given by a training set, that is highly informative about predicting the validation set. By making use of variational approximations to the mutual information, we derive a general and tractable framework for meta learning. This framework unifies existing gradient-based algorithms and also allows us to derive new algorithms. In particular, we develop a memory-based algorithm that uses Gaussian processes to obtain non-parametric encoding representations. We demonstrate our method on a few-shot regression problem and on four few-shot classification problems, obtaining competitive accuracy when compared to existing baselines.