Alex W Alex W - 3 months ago 23
R Question

parallel::clusterExport - non exported library/package function

similar to this question

But how do I

clusterExport
a package's non-exported function to a cluster? For some reason this passed my tests before submitting to CRAN but isn't working in production. Obviously, I want to fix and resubmit to CRAN.

library(imputeMulti)
library(parallel)
imputeMulti:::count_compare # function to be exported

nnodes <- 2L
cl <- parallel::makeCluster(nnodes)

parallel::clusterExport(cl, varlist= c("count_compare")) # fails -- but initially passed tests
parallel::clusterExport(cl, varlist= c("count_compare"), envir= as.environment("package:imputeMulti")) # also fails


I'm using cluster export to avoid the CRAN/R CMD check note "use of
:::
in package". Obviously, I could export
count_compare
, but that's not a desirable choice.

Any help appreciated!

adding tests information:



devtools::test("imputeMulti", "count_levels")
Loading imputeMulti
Testing imputeMulti
int- count_levels works: ...............................

DONE ===========================================================================================================================================

Answer

Based on my tests and work, I do not see away to use parallel::clusterExport on a non-exported library function.

The following works, but results in 1 R CMD check note:

R CMD check results
0 errors | 0 warnings | 1 note
checking dependencies in R code ... NOTE
There are ::: calls to the package's namespace in its code. A package almost never needs to use ::: for its own objects: 'count_compare'

count_compare <- imputeMulti:::count_compare 
parallel::clusterExport(cl, varlist= c("count_compare"), envir= 1)

Perhaps one of the developers of library(testthat) can provide a solution / update regarding the problem with the testthat framework not catching this. Based off of Hadley's R-Journal article (pg7 using journal numbering), I'm guessing this has to do with how environments are used for testing. But, that's just a guess.

Note: Hadley has confirmed this is the reason for the testthat miss via email correspondence.