Probing for Understanding of English Verb Classes and Alternations in Large Pre-trained Language Models

作者：David Yi, James V. Bruno, Jiayu Han, Peter Zukerman, Shane Steinert‐Threlkeld · 发表于：arXiv (Cornell University) · 年份：2022 · DOI：10.48550/arxiv.2209.04811 · 研究领域：Natural Language Processing Techniques、Topic Modeling、Text Readability and Simplification

We investigate the extent to which verb alternation classes, as described by Levin (1993), are encoded in the embeddings of Large Pre-trained Language Models (PLMs) such as BERT, RoBERTa, ELECTRA, and DeBERTa using selectively constructed diagnostic classifiers for word and sentence-level prediction tasks. We follow and expand upon the experiments of Kann et al. (2019), which aim to probe whether static embeddings encode frame-selectional properties of verbs. At both the word and sentence level, we find that contextual embeddings from PLMs not only outperform non-contextual embeddings, but achieve astonishingly high accuracies on tasks across most alternation classes. Additionally, we find evidence that the middle-to-upper layers of PLMs achieve better performance on average than the lower layers across all probing tasks.