Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more information about sources to data catalog #370

Closed
wants to merge 7 commits into from
Closed

Add more information about sources to data catalog #370

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
c9225d4
update:new_columns and datasource_info
mfonsecaOEF Mar 11, 2024
56f6128
update: column descriptions
mfonsecaOEF Mar 11, 2024
824ff64
add:new_columns
mfonsecaOEF Mar 11, 2024
bf867fb
update:table content
mfonsecaOEF Mar 11, 2024
f812b64
drop:description_column
mfonsecaOEF Mar 11, 2024
9dbe962
drop:description_column
mfonsecaOEF Mar 11, 2024
ebfe113
Merge branch 'develop' into datacatalog_update_v2
mfonsecaOEF Mar 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions global-api/importer/datasource_seeder/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,12 @@ psql -U ccglobal -d ccglobal -f import_datasource_seeder.sql

- `datasource_id`: unique UUID for the datasource
- `publisher_id`: abbreviation of the original name of the datasource (eg: EPA)
- `name`: complete name of the datasource (eg: Environmental Protection Agency)
- `description`: brief description of the dataset from the datasource
- `datasource_name`: complete name of the datasource (eg: Environmental Protection Agency)
- `dataset_name`: dataset name assign after transformation phase (eg. Manufacturing Industries and Construction Direct Emitters reported in the Greenhouse Gas Reporting Program)
- `dataset_description`: brief description of the dataset from the datasource
- `source_type`: should always be set to `third_party` for the global API. This is set to `user` for user-supplied data.
- `access_type`: the type of access the data (eg: free, paid, etc)
- `URL`: the link where the data was accessed
- `dataset_url`: the link where the data was accessed
- `geographical_location`: it contains the geographical coverage of the data (eg: AR for Argentina data, EARTH for international datasources with global coverage)
- `start_year`: data start date
- `end_year`: data end date
Expand All @@ -26,9 +27,12 @@ psql -U ccglobal -d ccglobal -f import_datasource_seeder.sql
- `language`: original language of the dataset
- `accessibility`: if this data source can be publicly accessed or requires special access privileges (e.g. `public`, `private`, `paid`)
- `data_quality`: quality of the data based on the GPC classification (low, medium, high)
- `notes`: extra information about the dataset (eg: brief description of the methodology applied to obtain emission values when the raw data is activity data)
- `notes`: extra information about the dataset
- `units`: units of the emissions
- `methodology_url`: link to the methodology on how the data was obtained by the source
- `methodology_description`: description of the methodology on how the data was obtained by the source
- `methodology_url`: link to the methodology on how the data was obtained by the source (when it is available)
- `transformation_description`: the type of transformation that has been done to the raw data, the assumptions that were made, emission factors that were applied, etc
- `retrieval_method`: method used by CityCatalyst to access the data from the database
- `api_endpoint`: Global API endpoint to access the data
- `gpc_reference_number`: Sub-sector reference number from the Global Protocol for Community-Scale Greenhouse Gas Emission Inventories (GPC)
- `gpc_reference_number`: sub-sector reference number from the Global Protocol for Community-Scale Greenhouse Gas Emission Inventories (GPC)
- `scope`: the scope for which it includes data within the subsector
Loading
Loading