经常有人问我，“松哥，我有2个指标，高度相关，就是一致吧！”，其实不是的，相关只代表两个指标的走势一致（趋势一致），不代表数据本身一致。正如你买了指数基金，当大盘涨的时候，你也会涨，但涨的幅度与大盘是不一致的。
精鼎41期SPSS实战训练营：2018月5月25-2018年5月27日，中国北京
1、问题发现
如下表，2位评价者对同一批受试对象10人进行评分。
表1
Subject | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
Judge 1 | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 |
Judge 2 | 9 | 10 | 8 | 7 | 5 | 6 | 4 | 3 | 1 | 2 |
评分结果进行相关分析，得到Pearson 相关系数 r = .964.
而对下表的judge4和judge5而言，进行相关系数计算， r = .964. 结果可见，两表数据的相关系数是一样的，可是我们细心一点看两表的数据，差异可是大了去了。表2数据的一致性（agreement）可差了去了。
表2
Subject | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
Judge 4 | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 |
Judge 5 | 90 | 100 | 80 | 70 | 50 | 60 | 40 | 30 | 10 | 20 |
2、问题解决
关于一致性的问题，根据资料的类型，分为计数资料、等级资料和计量资料，一致性分析方法是不一样的。松哥即将出版的新书《统计思维与SPSS实战》会有详细讲解。对于本例为计量资料，可以采用ICC（组内相关系数）进行分析。
计算2表的ICC，发现表1的ICC=0.9672，表2的ICC=0.0535.
愿意阅读英文原文的，请继续往下浏览。
拓展阅读：
Intra-class Correlation Coefficient
Psychologists commonly measure various characteristics by having a rater assign scores to observed people, or events. When using such a measurement technique, it is desirable to measure the extent to which two or more raters agree when rating the same set of things. This can be treated as a sort of reliability statistic for the measurement procedure.
Continuous Ratings, Two Judges
For example, suppose that we have two judges rating the aggressiveness of each of a group of children on a playground. If the judges agree with one another, then there should be a high correlation between the ratings given by the one judge and those given by the other. Accordingly, one thing we can do to assess inter-rater agreement is to correlate the two judges' ratings. Consider the following ratings (they also happen to be ranks) of ten subjects:
Subject | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
Judge 1 | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 |
Judge 2 | 9 | 10 | 8 | 7 | 5 | 6 | 4 | 3 | 1 | 2 |
Here is the dialog window in SPSS. Click on Analyze, Correlate, Bivariate:
The Pearson correlation is impressive, r = .964. If our scores are ranks or we can justify converting them to ranks, we can compute the Spearman correlation coefficient or Kendall's tau. For these data Spearman rho is .964 and Kendall's tau is .867.
We must, however, consider the fact that two judges scores could be highly correlated with one another but show little agreement. Consider the following data:
Subject | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
Judge 4 | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 |
Judge 5 | 90 | 100 | 80 | 70 | 50 | 60 | 40 | 30 | 10 | 20 |
The correlations between judges 4 and 5 are identical to those between 1 and 2, but judges 4 and 5 obviously do not agree with one another well. Judges 4 and 5 agree on the ordering of the children with respect to their aggressiveness, but not on the overall amount of aggressiveness shown by the children.
One solution to this problem is to compute the intraclass correlation coefficient. For the data above, the intraclass correlation coefficient between Judges 1 and 2 is .9672 while that between Judges 4 and 5 is .0535.
What if we have more than two judges, as below? We could compute Pearson r, Spearman rho, or Kendall tau for each pair of judges and then average those coefficients, but we still would have the problem of high coefficients when the judges agree on ordering but not on magnitude. We can, however, compute the intraclass correlation coefficient when there are more than two judges. For the data from three judges below, the intraclass correlation coefficient is .8821.
Subject | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
Judge 1 | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 |
Judge 2 | 9 | 10 | 8 | 7 | 5 | 6 | 4 | 3 | 1 | 2 |
Judge 3 | 8 | 7 | 10 | 9 | 6 | 3 | 4 | 5 | 2 | 1 |
The intraclass correlation coefficient is an index of the reliability of the ratings for a typical, single judge. We employ it when we are going to collect most of our data using only one judge at a time, but we have used two or (preferably) more judges on a subset of the data for purposes of estimating inter-rater reliability. SPSS calls this statistic the single measure intraclass correlation.
The Intraclass Correlation Coefficient
Click Analyze, Scale, Reliability Analysis.
Scoot all three judges into the Items box.
Click Statistics. Ask for an Intraclass correlation coefficient, Two-Way Random model, Type = Absolute Agreement.
Continue, OK.
Here is the output. You are looking for the intraclass correlation coefficient, which I have bolded.
****** Method 1 (space saver) will be used for this analysis ******
Intraclass Correlation Coefficient
Two-way Random Effect Model (Absolute Agreement Definition):
People and Measure Effect Random
Single Measure Intraclass Correlation = .6961*
95.00% C.I.: Lower = .0558 Upper = .9604
F = 214.0000 DF = (4, 8.0) Sig. = .0000 (Test Value = .0000 )
Average Measure Intraclass Correlation = .8730
95.00% C.I.: Lower = .1480 Upper = .9864
F = 214.0000 DF = (4, 8.0) Sig. = .0000 (Test Value = .0000 )
*: Notice that the same estimator is used whether the interaction effect
is present or not.
联系客服
