Does anyone have any wisdom on workflows for data analysis related to custom report writing? The use-case is basically this:
persondata.RData : ImportData.R ../../DATA/ss07por.csv Functions.R
$R --slave -f ImportData.R
persondata.Munged.RData : MungeData.R persondata.RData Functions.R
$R --slave -f MungeData.R
report.txt: TabulateAndGraph.R persondata.Munged.RData Functions.R
$R --slave -f TabulateAndGraph.R > report.txt
I generally break my projects into 4 pieces:
load.R: Takes care of loading in all the data required. Typically this is a short file, reading in data from files, URLs and/or ODBC. Depending on the project at this point I'll either write out the workspace using
save() or just keep things in memory for the next step.
clean.R: This is where all the ugly stuff lives - taking care of missing values, merging data frames, handling outliers.
func.R: Contains all of the functions needed to perform the actual analysis.
source()'ing this file should have no side effects other than loading up the function definitions. This means that you can modify this file and reload it without having to go back an repeat steps 1 & 2 which can take a long time to run for large data sets.
do.R: Calls the functions defined in func.R to perform the analysis and produce charts and tables.
The main motivation for this set up is for working with large data whereby you don't want to have to reload the data each time you make a change to a subsequent step. Also, keeping my code compartmentalized like this means I can come back to a long forgotten project and quickly read load.R and work out what data I need to update, and then look at do.R to work out what analysis was performed.