pachamaltese pachamaltese - 1 year ago 64
R Question

R create a inefficient loop in purpose to avoid bothering other server users

I have to process a dataset in my server, and using different parameters.

This a dummy example of what I'm doing

if (!require("pacman")) install.packages("pacman")

mtcars_experiments = dbConnect(RSQLite::SQLite(), "mtcars_experiments.sqlite")

for(a in -1:1) {
for(b in -1:1) {
for(c in -1:1) {
mtcars_experiment = mtcars %>%
mutate(my_col = mpg^a + cyl^b + disp^c)

dbWriteTable(mtcars_experiments, paste("mtcars_experiment",a,b,c, sep = "_"), mtcars_experiment)

I know that for loops sometimes are inefficient but in my case I don't want to compute this at maximum speed.

I'm trying to find a way that is not so slow but also not so fast, because if I use too much resources with parallelization other users of the server will have problems running their own code.

What can I do in this case? How can I do a bounded parallelization or alike?

Thanks !

thc thc
Answer Source

Two possibilities:

1) Add Sys.sleep(1) after every iteration. This consumes no resources, and does nothing 1 second after every iteration.

2) Lower the priority of the process. In ubuntu, you can do this by renice 20 PROCESS_ID (20 is the lowest priority).

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download