What's wrong with rating scales? Psychology's replication and confidence crisis cannot be solved without transparency in data generation

作者：Jana Uher · 发表于：Social and Personality Psychology Compass · 年份：2023 · DOI：10.1111/spc3.12740 · 被引用次数：25 · 研究领域：Social Representations and Identity、Academic and Historical Perspectives in Psychology、Child and Animal Learning Development

Abstract Quantitative explorations of behaviour, psyche and society are common in psychology. This requires methods that justify the attribution of results to the measurands (the entities to be measured, e.g., in individuals) and that make the results' quantitative meaning publicly interpretable (e.g., for decision making). Do rating scales—psychology's primary methods to generate numerical data—meet these criteria? This article summarises selected epistemological and methodological problems of rating scales that arise, amongst others, from the intricacies of language‐based methods and from psychologists' challenges to distinguish their study phenomena from their means of exploring these phenomena. Failure to make this logical distinction entails that disparate scientific activities are conflated, thereby distorting scientific concepts and procedures. Rating scales promote such conflations because they serve both as description of the empirical study system (e.g., behaviours) and as symbolic study system (e.g., data variables), leaving the interpretation of each system and the mapping relations between them to raters' intuitive decisions. Verbal scales, however, have broad semantic fields of meanings, which are context‐sensitive and therefore interpreted differently, and which cannot logically match the quantitative meaning commonly ascribed to the numerical scores derived from them. The ease of using verbal descriptions as means of exploration drew psychologists' attention t...