Prepare and publish datasets

[gubuntu]

We need to prepare datasets on the hazard and exposure side to populate the Labs GeoNode

Labs Geonode (http://45.55.174.20/) has many Hazard datasets but no Exposure datasets.

We need the following layer prepared, styled and published on staging (related to #95):

Exposure

• global population (WorldPop)
• global roads (OSM)
• LandScan (WB has a licence so we can use it, discuss with @cgiovando)

Hazard

• see Cristiano's comment below

Layer and analysis status is at https://docs.google.com/spreadsheets/d/1Nni6SpmTwgsOgmG2dUNj47wGzUGGHkMwT6T_7eG9s4Q/edit?usp=sharing

[NyakudyaA]

Populated metadata for population data downloaded from http://sedac.ciesin.columbia.edu/data/sets/browse and http://www.worldpop.org.uk/data/data_sources/.

Tried uploading the population data. There is an upload error. I think the file size is limiting geonode upload. We need to fix this so that users are able to upload any file size.

Still busy with getting global roads. Doing this as part of kartoza/gfdrr_oondra#62 (comment) As oondra was already using docker-osm

[NyakudyaA]

kartoza/geonode#193 (error uploading large rasters)

[gubuntu]

@NyakudyaA please make the data and metadata available in a ready-to-publish form (e.g. via a resilio share) and put instructions here for exactly how to publish them on the Labs Geonode platform so that @fvanderbiest @fjacon @stufraser1 can easily publish them when they are ready.

[NyakudyaA]

I have prepared the datasets with full metadata. The datasets are available in a resilio (btsync) share using the key BI4KJS6V75VQ22FJKGGDNNN3HCU52QES3.

• Sync the data to your local machine
• Copy the synced data folder into the django container: docker cp geosafe_data container_name:/tmp/
• Exec into the django container and import the data using the management commands to register the layers with geonode
• Run python manage.py importlayers -m data_directory Where data_directory is the path containing the data. In this case it will be /tmp/geosafe_data

[cgiovando]

Here are the datasets to test and load for the upcoming demo:

Hazard

Earthquake (mmi units):

• Tanzania: 2016 M5.9 event shakemap --> shapefile download
• Nepal: 2015 M 7.8 event shakemap --> shapefile download
• Global shakemap API directly from USGS service <-- Can we use this?

Flood (water depth in mm):

• Madagascar: Historical events --> MaxDepth as point shapefile
• Madagascar: 250-year tropical cyclone flood event --> ESRI grid format
• Malawi: Dartmouth Flood Observatory (DFO) flood layer
• Malawi: Radarsat flooded areas by DLR as of Jan 2015 layer

Exposure

Population:

• Madagascar: Facebook/CIESIN High Resolution Settlement Layer in GeoTIFF format (band 1, number of persons estimated to live in each 30m pixel)
• Global: country subsets for MG, TZ and NP from WorldPop API or GeoTIFF download
• Global: country subsets for MG, TZ and NP from GHSL Population Grid
• Global: country subsets for MG, TZ and NP from Landscan (to come): approximately 1 km resolution representing an ambient population (average over 24 hours).

Buildings:

• OSM subsets for MG, TZ, and NP from GeoFabrik

Roads:

• Global Roads Open Access Data Set (gROADS), v1 (1980 – 2010)
• OSM subsets for MG, TZ, and NP from GeoFabrik

[gubuntu]

@nyakudya for the OSM roads and buildings I think we should try to use the docker-osm sets you've prepared, on a per country basis until we are sure GeoSAFE can run on subsets.

And I know we used Worldpop tifs for OONDRA but work has been done on using the API so let's try that.

[NyakudyaA]

Global Roads do not have a class that can be used to assign keywords in Inasafe. I will use Natural earth in the meantime

[cgiovando]

I will use Natural earth in the meantime

Please use OSM instead. Thanks

[gubuntu]

large layers currently have to be loaded via backend management commands. kartoza/geonode#52 is addressing the limitation on uploading large layers through the UI.

[NyakudyaA]

@cgiovando Any update about the Landscan data.

[cgiovando]

Any update about the Landscan data.

@vdeparday can you comment and confirm that the license is compatible? Thanks!

[gubuntu]

all available layers are loaded with metadata populated at http://geonode.kartoza.com

[NyakudyaA]

@cgiovando what are the units for tropical cyclone. I have looked at the data source and no units are listed.