I have a number of lengthy jobs that I want to parallelize with foreach-dopar so that each thread works on a job independent of others. I want to track the status of each thread (some may fail while others succeed) by writing to a log file using
cl = makeCluster(2, type="SOCK")
dl = file("runlog.Rout", open="wt")
sink(dl, type="output", append=TRUE)
sink(dl, type="message", append=TRUE)
dump <- foreach(i=1:5,
.errorhandling = "stop",
beg.time = Sys.time()
cat(as.character(beg.time), " I am running....\n", file="mylog.txt")
# do something here.....
end.time = Sys.time()
del.tm = difftime(end.time, beg.time, units="mins")
cat("....saving output to file......\n\n", file="mylog.txt")
save(del.tm, file = paste("I:/Rhome/H", i, ".RData", sep=""))
....saving output to file......
Although I don't really trust having multiple processes write to the same file, you may have success by using the
cat("...\n", file="mylog.txt", append=TRUE)
Without setting this option,
cat will overwrite the previous contents of "mylog.txt" each time it's called.
For other approaches, see my answer here.