Error when reading database

[weilinleee]

I try to follow the case tutorial and read database IDBNexus downloaded from Github, but there is always an error saying that "Database does not exist or is invalid." ( I am pretty sure the path is correct.) I also try to read GCAM database provided by my colleague, but the error comes out all the time.

[mengqi-z]

Hi @weilinleee, there is an issue with rgcam if the working directory is the same as the GCAM database directory, it does not recognize the database. I updated the Case Tutorial to avoid this issue. Please try the updated tutorial code in example section 3 and see if it works.

[weilinleee]

Hi @mengqi-z, thanks for the update. However, the error is coming out again. I also try to set a working directory that is totally different from database directory, but is does not work either.

[mengqi-z]

Hi @weilinleee , would you please check if those GCAM .basex files are directly within this path: /Users/winnielee/Desktop/PLUTUS/IDBNexus? I figured sometimes unzipping the .zip file will create double-level folders where your .basex file may be located in /Users/winnielee/Desktop/PLUTUS/IDBNexus/IDBNexus. The path_to_gcamdatabase needs to be the path that directly holds those .basex files.

Also, I see that you are running in code chunks. Some encountered problems when using code chunks to run plutus in R markdown. I would suggest running the code in a regular R script.

[CarlaCamposM]

Hi @Mengqi,

I run the following code:

Load required packages

library(plutus)
library(dplyr)

Set your directory based on the example dataset location

workdir <- 'C:/Users/carla/Desktop/GCAM53'

provide path to the desired GCAM database folder that holds .basex files.

path_to_gcamdatabase <- file.path(workdir, 'IDBNexus_gcam5p3_HadGEM2-ES_rcp8p5')

Use plutus::gcamInvest to calculate stranded assets and electricity investments

invest <- plutus::gcamInvest(gcamdatabase = path_to_gcamdatabase,
dataProjFile = file.path(workdir, 'outputs', 'dataProj.proj'),
dirOutputs = file.path(workdir, 'outputs'),
reReadData = T,
# gcamdataFile = path_to_gcamdataFile, # Use this argument if any data files has been updated
scenOrigNames = c('Reference'),
regionsSelect = c('USA', 'China'),
saveData = T)

I got a similar error to @weilinleee when reading the IDBNexus database. I have double checked that the GCAM .basex files are directly within the following path:

C:/Users/carla/Desktop/GCAM53/IDBNexus_gcam5p3_HadGEM2-ES_rcp8p5

I am not running the code in chunks. Below is the error I got in the test.txt file that appeared after running the function:

##########################################################################################
Error in rgcam::localDBConn(gcamdatabasePath, gcamdatabaseName) :
Database does not exist or is invalid: C:/Users/carla/Desktop/GCAM53/IDBNexus_gcam5p3_HadGEM2-ES_rcp8p5
In addition: Warning message:

The following named parsers don't match the column names: name, date, version
[##########################################################################################

On the other hand, the function does not generate a dataProj.proj.

I would appreciate if you could help me to solve this issue. Thanks.

[mengqi-z]

Hi @CarlaCamposM,

I am sorry that you are experiencing the same problem related to rgcam, a package we use in plutus to extract data from GCAM. Currently, I am not sure how to fix this besides setting the working directory differently from the location of the dataset, which works for me. I created an issue in rgcam Github about this problem and I am still waiting for their solution.

In the meanwhile, if you would like to explore outputs from plutus, you can also use the built-in example dataset plutus::exampleGCAMproj as input to run the function. Here is an example code you could try. More details can be found Here.

library(plutus)

invest <- plutus::gcamInvest(dataProjFile = plutus::exampleGCAMproj,
                             scenOrigNames = c('Reference', 'Impacts', 'Policy'),
                             scenNewNames = c('Reference', 'Climate Impacts', 'Climate Policy'),
                             regionsSelect = c('USA', 'China'))

[nurf004]

Hi @mengqi-z
I just found your article regarding Plutus.
I have followed the tutorial using my database. However, I found an error when running the invest function.

invest <- plutus::gcamInvest(gcamdatabase = path_to_gcamdatabase,

•                          dataProjFile = file.path(workdir, 'outputs', 'dataProj.proj'),
  

•                          dirOutputs = file.path(workdir, 'outputs'),
  

•                          reReadData = TRUE,
  

•                          #gcamdataFile = path_to_gcamdataFile, #Use this argument if any data files has been updated
  

•                          scenOrigNames = c('Reference'),
  

•                          regionsSelect = c('Indonesia'),
  

•                          saveData = TRUE)
  

[1] "Connecting to GCAM database provided C:/Users/nurfi/Desktop/GCAM/output//database_basexdb..."
[1] "Will save GCAM data to C:/Users/nurfi/Desktop/GCAM/output/outputs/C:/Users/nurfi/Desktop/GCAM/output/outputs/dataProj.proj..."
[1] "Reading queries from queryFile created: C:/Users/nurfi/Desktop/GCAM/output/outputs/readGCAM/subSetQueries.xml."
[1] "Database scenarios: Reference, Socioeconomic_update, Policy1, Policy2"
[1] "All scenarios in data available: Reference, Socioeconomic_update, Policy1, Policy2"
[1] "scenOrigNames available in scenarios are :Reference"
[1] "Other scenarios not selected are :Socioeconomic_update, Policy1, Policy2"
0sDatabase scenarios: Reference, Socioeconomic_update, Policy1, Policy2 0s
Scenario =2021-17-12T10:02:43+09:00" does not exist in this project. Creating. 0s
Scenario 2070 does not exist in this project. Creating.
Scenario Africa_Eastern does not exist in this project. Creating.
Scenario Africa_Northern does not exist in this project. Creating.
Scenario ar=2020 does not exist in this project. Creating.
Scenario biomass does not exist in this project. Creating.
Scenario Ref does not exist in this project. Creating.
Scenario Refe does not exist in this project. Creating.
Scenario Refer does not exist in this project. Creating.
Scenario Referen does not exist in this project. Creating.
Scenario Reference does not exist in this project. Creating.
Scenario Reference,da does not exist in this project. Creating.
Scenario year=2045" does not exist in this project. Creating.
Scenario year=2070" does not exist in this project. Creating.
Scenario R does not exist in this project. Creating. 0s
Scenario Re does not exist in this project. Creating.
Scenario Reference,date does not exist in this project. Creating.
[1] "Calculating electricity investment for scenario: =2021-17-12T10:02:43+09:00""
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
[1] "Calculating electricity investment for scenario: 2070"
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
[1] "Calculating electricity investment for scenario: Africa_Eastern"
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
[1] "Calculating electricity investment for scenario: Africa_Northern"
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
[1] "Calculating electricity investment for scenario: ar=2020"
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
[1] "Calculating electricity investment for scenario: biomass"
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
[1] "Calculating electricity investment for scenario: Ref"
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
[1] "Calculating electricity investment for scenario: Refe"
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
[1] "Calculating electricity investment for scenario: Refer"
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
[1] "Calculating electricity investment for scenario: Referen"
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
[1] "Calculating electricity investment for scenario: Reference"
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
[1] "Calculating electricity investment for scenario: Reference,da"
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
[1] "Calculating electricity investment for scenario: year=2045""
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
[1] "Calculating electricity investment for scenario: year=2070""
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
[1] "Calculating electricity investment for scenario: R"
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
[1] "Calculating electricity investment for scenario: Re"
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
[1] "Calculating electricity investment for scenario: Reference,date"
[1] "---------------------------------------"
[1] "Using default data and assumptions ..."
[1] "---------------------------------------"
Error in firstYrLoc:length(hydro_energy) : NA/NaN argument
In addition: Warning messages:
1: In sink() : no sink to remove
2: One or more parsing issues, see problems() for details
3: 16 failed to parse.
4: One or more parsing issues, see problems() for details
5: 2 failed to parse.
6: Problem with mutate() column Year.
i Year = as.numeric(Year).
i NAs introduced by coercion
7: Expected 2 pieces. Missing pieces filled with NA in 66 rows [18, 19, 164, 853, 854, 999, 1688, 1689, 1834, 2523, 2524, 2669, 3358, 3359, 3504, 4193, 4194, 4339, 5028, 5029, ...].

From this, there are dataProj file and readGCAM folder in the designated folder.
I use GCAM 5.4.
I would appreciate if you could help me to solve this issue. Thank you very much.

[mengqi-z]

Hi @nurf-004,

I think it is an issue related to rgcam, an R package built to extract GCAM output database and it is used in plutus. Anyway, your errors might be caused by the version of readr in your R library. Please install readr 1.4.0 and see if it solves your problem.

More details related to this issue for rgcam can be found here.

Mengqi

[nurf004]

Hi @mengqi-z

Thank you for your kind response. I have solved this issue finally. May I ask you some questions and your confirmation. First, I'm wondering the calculation process of annual electricity capacity installations and annual premature retirements. I'm trying to understand how the Plutus works when I look at the elec gen by gen tech and cooling and vintage in the ModelInterface (how do we calculate based on such information?). Second, if we want to modify the parameters (lifetime and steepness, and capacity factors), should we do it first before we run the GCAM (the input files, assuming we used given assumptions in GCAM) to get the datasets or after we generate the datasets (you mention: # gcamdataFile = path_to_gcamdataFile, # Use this argument if any data files has been updated). Maybe I misunderstood regarding the statement. Last, is there any differences between Class1 and Class2? The explanation is not clear enough for me.

Thank you very much for your help!

[mengqi-z]

Hi @nurf004 ,

Thank you for your questions.

1. Please refer to Binsted et al., 2020 for the methodology of calculating new capacity installation and retirement.

2. To use customized parameters:

• You would want to modify parameters in the corresponding GCAM input files (under gcamdata folder). Please refer to this Table for detailed information on specific file for data and assumptions used in plutus.
• Run GCAM with modified input files, and get the updated GCAM output database.
• When using plutus, link to the gcamdata folder that has your updated files by setting the gcamdataFile = "gcam-v6.0/input/gcamdata", for example. Also use your GCAM output database from above step.

3. Class 1 and Class 2 are different details of data. Class 2 aggregates all Class 1 categories, which are technologies in this case, under each scenario, region, parameter, year. For example, for output parameter elecNewCapGW, class 1 is the new installation for each technology. Class 2 will be the total new installation of all the technologies together.

[nurf004]

Hi @mengqi-z

Noted. Thank you very much for answering my questions.