Deep Bayesian Bandits: Exploring in Online Personalized Recommendations

作者：Dalin Guo, Sofia Ira Ktena, Pranay Kumar Myana, Ferenc Huszár, Wenzhe Shi, Alykhan Tejani, Michael Kneier, Sourav Das · 年份：2020 · DOI：10.1145/3383313.3412214 · 被引用次数：30 · 研究领域：Advanced Bandit Algorithms Research、Recommender Systems and Techniques、Smart Grid Energy Management

Recommender systems trained in a continuous learning fashion are plagued by the feedback loop problem, also known as algorithmic bias. This causes a newly trained model to act greedily and favor items that have already been engaged by users. This behavior is particularly harmful in personalised ads recommendations, as it can also cause new campaigns to remain unexplored. Exploration aims to address this limitation by providing new information about the environment, which encompasses user preference, and can lead to higher long-term reward. In this work, we formulate a display advertising recommender as a contextual bandit and implement exploration techniques that require sampling from the posterior distribution of click-through-rates in a computationally tractable manner. Traditional large-scale deep learning models do not provide uncertainty estimates by default. We approximate these uncertainty measurements of the predictions by employing a bootstrapped model with multiple heads and dropout units. We benchmark a number of different models in an offline simulation environment using a publicly available dataset of user-ads engagements. We test our proposed deep Bayesian bandits algorithm in the offline simulation and online AB setting with large-scale production traffic, where we demonstrate a positive gain of our exploration model.