A Revised Index of Interrater Agreement for Multi-Item Ratings of a Single Target

作者：Michael K. Lindell, Christina J. Brandt, David J. Whitney · 发表于：Applied Psychological Measurement · 年份：1999 · DOI：10.1177/01466219922031257 · 被引用次数：255 · 研究领域：Advanced Statistical Methods and Models、Reliability and Agreement in Measurement、Multi-Criteria Decision Making

The commonly used form of r wg. (J) can display irregular behavior, so four variants of this index were examined. An alternative index, r* wg. J , is recommended. This index is an inverse linear function of the ratio of the average obtained variance to the variance of uniformly distributed random error. r* wg.J is superficially similar to Cronbach’s α, but careful examination confirms that r* wg.J is an index of agreement, not reliability. Based on an examination of the small-sample behavior of r wg and r* wg.J , sample sizes of 10 or more raters are recommended.