daroczig daroczig - 25 days ago 7
R Question

Memory leak with closed connections

I have an R application interacting with a Java daemon via

stdin
and
stdout
in an infinite loop, which seems to have some memory leaks. Simplified R app:

while (TRUE) {
con <- file('stdin', open = 'r', blocking = TRUE)
line <- scan(con, what = character(0), nlines = 1, quiet = TRUE)
close(con)
}


This loop ends up using more and more RAM, and even if I manually
gc()
after the
close(con)
call, the memory footprint seems to be OK for a while, but eventually grows forever.

A basic script to confirm this:

Rscript --vanilla -e "while(TRUE)cat(runif(1),'\n')" | Rscript --vanilla -e "cat(Sys.getpid(), '\n');while (TRUE) {con <- file('stdin', open = 'r', blocking = TRUE);line <- scan(con, what = character(0), nlines = 1, quiet = TRUE);close(con);gc()}"


This will start two R processes: one writing to
stdout
and the other reading from
stdin
connected with a pipe (and the second printing the
pid
so that you can monitor the related memory usage):

enter image description here

I'm not sure what I'm doing wrong, but would love to stop this memory leak so any help is highly appreciated.

Answer

Indeed (about reading about this on R-devel); notably, the memory leak has now been plugged in the development version of R, thanks to a patch by Gabor Csardi.