Curate NCBI geolocation metadata

[anna-parker]

This curation was also requested by users.

NCBI’s geolocation field is of the format: country:subdivision according to their documentation it should be possible to further split subdivision into geoLocAdmin1, geoLocAdmin2. However, for a large number of samples the division field does not follow this format. Currently we map country to our geoLocCountry field and the entire division field into geoLocAdmin1. We would like to be able to at least split this into geoLocAdmin1 and geoLocAdmin2.

Discussion

From discussion in group these changes should actually be made by by a curator bot
https://docs.google.com/document/d/1TQiE66Hk6WjgkMvvMMhu8uMSEhHFP2AwINMAIsyODCw/edit

Proof of concept of suggestions in ingest

#3015
#3026

[corneliusroemer]

The most long term high level decision is on what to normalize to. Ideally we normalize to an established gazetteer that unambiguously allows pulling things like coordinates, and potentially other metadata for free. This means we'd possibly actually want to normalize to an id, instead of our current four location fields (these would semi-automatically derive from the entity pointed to).

Another key decision is what we mean by admin level 1/2 and city for each country.

A target gazetter should be open, allow dump download (to be free of rate limits), have stability track record, be maintained.

Geonames is one option, OpenStreetmap another.

The actual how to should be an implementation detail, which isn't to devalue it, just to beware not to let convenient implementation drive high level permanent decisions.