Cohen's kappa (agreement between two raters)
Measure how well two raters or two tests agree beyond chance, from a 2×2 table of their ratings.
Your data does not go to our server. This tool runs in your browser; what you type or open never leaves your device.
This calculator needs JavaScript, because it calculates in your browser and sends nothing. Turn JavaScript on to use it. How it is calculated is explained below.
- Kappa (confidence interval)
- Observed agreement
- Agreement expected by chance
- Standard error
How it is calculated
Observed agreement is the share of cases both raters classify the same way; expected agreement is what two independent raters with the same ratings would reach by chance.
The standard error is the common large-sample approximation, so treat the interval of a small table as rough.
κ = (pₒ − pₑ) / (1 − pₑ)SE(κ) ≈ √[pₒ(1 − pₒ) / (n × (1 − pₑ)²)]
Worked example: Counts 20, 5, 10 and 15: observed agreement 70 %, expected 50 %, kappa 0.400 (fair).
Sources
- Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960.
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977.
These calculators support research; they do not replace the advice of a statistician or a clinician.
Questions
What is a good kappa?
Landis and Koch call 0.61 to 0.80 substantial and above 0.80 almost perfect agreement. These bands are a convention, not a rule, and kappa also depends on how common the finding is.