NEWS.RmdFirst release TBD The first release of gcamfaostat +1.0.0 includes the data generated for the Global Change Analysis Model +v7.0 GCAM +v7.0. The source data downloaded from FAOSTAT is archived at a Zenodo repository.
+usermod_vignette.RmdUsers may want to change the default gcamdata behavior by either +modifying input assumptions or changing intermediate chunks. They can +now write a “user modification” chunk that can be “plugged in” to the +data system. This new chunk can modify any objects that are used or +created in gcamdata and pass the modified object to all dependent +chunks.
+User-modification chunks have a format similar to other data chunks
+in gcamdata, except that instead of producing a new output, it returns a
+modified data object that replaces the original data object in the data
+system. These new chunks can be added to driver_drake() or
+driver() using the new arguments
+user_modifications and xml_suffix, which tell
+gcamdata which modification function to use and what suffix to add to
+all impacted downstream xmls.
Below we show an example user-modification chunk to change a +shareweight in an input csv file.
+Here we load in two csv files, “energy/A322.subsector_shrwt.csv” and
+“common/GCAM_region_names.csv”. We modify A322.subsector_shrwt, so we
+list it under driver.DECLARE_MODIFY, but do not modify
+GCAM_region_names, so it is listed under
+driver.DECLARE_INPUTS. Then, we set the shareweight column
+of the first row of A322.subsector_shrwt to NEW.SHWT.
+Finally, we use a new return_modified() function to return
+the modified A322.subsector_shrwt (note that we have to include the path
+for input files).
+usermod_fert <- function(command, ...) {
+ if(command == driver.DECLARE_MODIFY) {
+ return(c(FILE = "energy/A322.subsector_shrwt"))
+ } else if(command == driver.DECLARE_INPUTS) {
+ # In addition to the objects users want to modify we can also ask for any other
+ # inputs we need to do our operations but won't be modified
+ return(c(FILE = "common/GCAM_region_names"))
+ } else if(command == driver.MAKE) {
+ all_data <- list(...)[[1]]
+ GCAM_region_names <- get_data(all_data, "common/GCAM_region_names")
+ A322.subsector_shrwt <- get_data(all_data, "energy/A322.subsector_shrwt")
+
+ # Users could also read in additional files that exist outside of the data system
+ # They should do that manually instead of through the driver.DECLARE_INPUTS so as to
+ # avoid mixing user's custom files with Core files
+ # A23.globaltech_eff.mine <- read_csv("/path/to/my/custom/A23.globaltech_eff_with_random_changes.csv")
+
+ # Make some changes...
+ A322.subsector_shrwt <- A322.subsector_shrwt %>%
+ mutate(share.weight = as.double(share.weight),
+ year = as.integer(year))
+ A322.subsector_shrwt[1,"share.weight"] <- NEW.SHWT
+
+ # NOTE: we have to match the original object name we asked for in driver.DECLARE_MODIFY,
+ # which means including the file path for input files
+ # i.e. "energy/A322.subsector_shrwt" not "A322.subsector_shrwt"
+ # Other objects can be listed out just like for `return_data`
+ return_modified("energy/A322.subsector_shrwt" = A322.subsector_shrwt)
+
+ } else {
+ stop("Unknown command")
+ }
+}To include our modification, we include this new chunk in our call to
+driver_drake() and also include a suffix to append to any
+affected objects (currently mandatory to include suffix).
Because we used the constant NEW.SHWT to assign the new
+value in our function, we first need to set it here.
+NEW.SHWT <- 0.5
+
+driver_drake(user_modifications = c("usermod_fert"),
+ xml_suffix = "__1") # output xml will be saved as ORIGINALNAME_001.xmlWe can also generate multiple modified xmls using
+driver_drake(). To do this, we simply need to change the
+value of NEW.SHWT and ensure that each different value is
+associated with a different xml_suffix. As well, we need to
+clear the usermod_fert object from drake’s cache using
+drake::clean() as drake is not aware of the change to
+NEW.SHWT. If you do not include this call, drake may assume
+that all downstream objects/xmls do not need to be updated.
+# Multiple shareweights to use
+shareweights <- seq(0.2, 1, 0.1)
+
+for (i in 1:length(shareweights)){
+ drake::clean(list="usermod_fert") # Ensures that drakes knows to run usermod_fert
+
+ NEW.SHWT <- shareweights[i]
+
+ driver_drake(user_modifications = c("usermod_fert"),
+ xml_suffix = paste0("__", i))
+}Zhao X, Chepeliev M, Patel P, Narayan K, Wise M (2023). gcamfaostat: Prepare, process, and synthesize FAOSTAT data for global agroeconomic and multisector dynamic modeling. -R package version 1.1. +R package version 1.0.0.
@Manual{,
title = {gcamfaostat: Prepare, process, and synthesize FAOSTAT data for global agroeconomic and multisector dynamic modeling},
author = {Xin Zhao and Maksym Chepeliev and Pralit Patel and Kanishka Narayan and Marshall Wise},
year = {2023},
- note = {R package version 1.1},
+ note = {R package version 1.0.0},
}
diff --git a/docs/index.html b/docs/index.html
index 64287eca2..84c8e9e43 100644
--- a/docs/index.html
+++ b/docs/index.html
@@ -33,7 +33,7 @@
@@ -45,16 +45,36 @@
+Functions in this R package (gcamdata-faostat) download, clean, processe, connect, and visualize data from FAOSTAT for global economic and integrated assessment modeling. The package is built based on the existing gcamdata package structure for consistency, transparency, and traceability.
+gcamfaostat is an R package to prepare, process, and synthesize FAOSTAT data for global agroeconomic and multisector dynamic modeling. The Food and Agriculture Organization Statistical Database (FAOSTAT) provdes open access data on country-level agricultural production, trade, food, nutrients, prices, land use, etc, servering as the most important data source for global agroeconomic and multisector dynamic models. gcamfaostat aims to shorten the distance between the FAOSTAT raw data to economic modeling.
+gcamfaostat is built based on an existing R package, gcamdata, which has similar functions togcamfaostat though gcamdata includes broader aspects of data inputs and is designed for the global multisector dynamic model GCAM. gcamfaostat utilizes the robust, reproducible and transparent data processing systems built in gcamdata. The two packages are consistent, while gcamfaostat focuses on agroeconomic data processing and can provide input data for gcamdata (and thus GCAM) and other models that rely on FAOSTAT data.
The goals of this version are: (1) Check FAOSTAT data updates and download necessary datasets (2) Develop a new method of primary equivalent aggregation to aggregate supply-utilization-accounting (SUA) data for items along the supply chain (e.g., wheat flour, bran, and germ to wheat-primary-equivalent). The method preserves balance across space (trade balance), time (storage carryover), supply-utilization, and the combination of these dimensions with minimal adjustments. (3) Apply the new method of primary equivalent aggregation to aggregating FAO ~500 SUA (SCL) items to ~100 primary equivalent items in FAO Food Balance Sheet (FBS). (4) Compare the balanced data compiled using different methods and visualize the difference.
+The package is documented in the online manual
+
+install.packages("devtools")
+devtools::install_github("realxinzhao/gcamfaostat")Open the gcamfaostat.Rproj file in the gcamfaostat folder. RStudio should open the project.
To load the gcamdata package, enter:
{r eval = FALSE} devtools::load_all()
There are two ways to run the driver: 1. {r eval = FALSE} driver_drake() driver_drake() runs the driver and stores the outputs in a hidden cache. When you run driver_drake() again it will skip steps that are up-to-date. This is useful if you will be adjusting the data inputs and code and running the data system multiple times. For this reason, we almost always recommend using driver_drake(). More details can be found in the vignette.
{r eval = FALSE} driver() See the documentation for more options when running the driver, such as what outputs to generate or when to stop.Users can specify the output directory (DIR_OUTPUT_CSV) that stores the output csv files in constants.R. The default directory is outputs/CSV. The the file will be exported when OUTPUT_Export_CSV == TRUE (an option in constants.R).
+Users can also make use of the functions to trace the processing by step, whendriver_drake() is employed.
Copyright 2019 Battelle Memorial Institute; see the LICENSE file.
+