MEASURING AGREEMENT WHEN TWO OBSERVERS CLASSIFY PEOPLE INTO CATEGORIES NOT DEFINED IN ADVANCE

作者：Robert L. Brennan, Richard J. Light · 发表于：British Journal of Mathematical and Statistical Psychology · 年份：1974 · DOI：10.1111/j.2044-8317.1974.tb00535.x · 被引用次数：72 · 研究领域：Reliability and Agreement in Measurement、Advanced Statistical Methods and Models、Multi-Criteria Decision Making

Basic to many psychological investigations is the question of agreement between observers who independently categorize people. Several recent studies have proposed measures of agreement when a set of nominal scale categories has been predefined and imposed on two observers. This study, in contrast, develops a measure of agreement for settings where observers independently define their own categories. Thus it is possible for observers to delineate different numbers of categories, with different names. Computational formulae for the mean and variance of the proposed measure of agreement are given; further, a statistic with a large‐sample normal distribution is suggested for testing the null hypothesis of random agreement. A computer‐based comparison of the large‐sample approximation with the exact distribution of the test statistic shows a generally good fit, even for moderate sample sizes. Finally, a worked example involving two psychologists' classifications of children illustrates the computations.