If you subset your original dataframe then you can plot each of them separately.
Lets say you split it every 20th rows. You can plot it using:
boxplot(DF[1:20,1]~DF[1:20,2],main="Boxplot 1", ylab="Reaction time", xlab="Number of participants", ylim=c(0,1000), las=1)
Where your dataframe is
"DF" and by using
DF[1:20,1] you're subsetting the first 20th rows of your dataframe and selecting the first column to plot agains the second column of the first 20th rows (