Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibration

作者：Christian Tomani, D. Cremers, F. Buettner · 发表于：European Conference on Computer Vision · 年份：2021 · DOI：10.1007/978-3-031-19778-9_32 · 被引用次数：62 · 研究领域：Computer Science

. We address the problem of uncertainty calibration and introduce a novel calibration method, Parametrized Temperature Scaling (PTS). Standard deep neural networks typically yield uncalibrated predictions, which can be transformed into calibrated confidence scores using post-hoc calibration methods. In this contribution, we demonstrate that the performance of accuracy-preserving state-of-the-art post-hoc calibrators is limited by their intrinsic expressive power. We generalize temperature scaling by computing prediction-specific temperatures, parameterized by a neural network. We show with extensive experiments that our novel accuracy-preserving approach consistently outperforms existing algorithms across a large number of model architectures, datasets and metrics. 4 calibration map, that transforms all uncalibrated predictions of a neural network into calibrated predictions in the same manner without leveraging information from individual predictions.Wehypothesize that the performance of temperature-scaling based post-hoc calibration models is intrinsically limited by their expressive power, which stems from a lack of modeling a prediction-specific transformation. We show that our prediction-specific temperatures are indeed different for each model; in fact, they vary over a wide range of values, which is in stark contrast to only 1 or 3 temperatures for temperature scaling or ensemble temperature scaling and indicates that temperatures calculated based on each prediction separat...