Paul MM van Haard - 1 year ago 193

R Question

I should like to calculate - by bootstrapping Krippendorff's Alpha outcomes - a 95% confidence interval for Krippendorff's Alpha coefficient of Raters Reliability using R package irr.

Let's use "C data from Krippendorff" in the package irr and the R script for calculating Krippendorff's Alpha once:

`# the "C" data from Krippendorff`

#rater per row; rated subject per column; NAs allowed

library(irr)

nmm<-matrix(c(1,1,NA,1,2,2,3,2,3,3,3,3,3,3,3,3,2,2,2,2,1,2,3,4,4,4,4,4,

1,1,2,1,2,2,2,2,NA,5,5,5,NA,NA,1,1,NA,NA,3,NA),nrow=4)

kripp.alpha(nmm,"ordinal")

Answer Source

You can use the `boot`

function from the `boot`

package to bootstrap values. Here I'll bootstrap the set of subjects but keep the raters fixed:

```
library(boot)
library(irr)
ka <- function(data, indices) kripp.alpha(nmm[,indices], "ordinal")$value
b <- boot(seq(ncol(nmm)), ka, 1000)
```

Now you can use the `boot.ci`

function to compute a 95% confidence interval for the bootstrapped value; I'll use the percentile confidence interval, but others are available (check out `?boot.ci`

):

```
boot.ci(b, type="perc")
# BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS
# Based on 1000 bootstrap replicates
#
# CALL :
# boot.ci(boot.out = b, type = "perc")
#
# Intervals :
# Level Percentile
# 95% ( 0.4297, 1.0000 )
# Calculations and Intervals on Original Scale
```