Skip to content
Craig Willis edited this page Dec 5, 2016 · 3 revisions

System overview

Overview diagram

System overview diagram

Software components

Component Description
terraref.org Primary website, maintained via Github pages
Clowder Primary user interface
Clowder-dev Development instance
BETYdb Database for experimental context, agronomic management, and trait data
BETYdb-dev Development instance
Globus Primary data transfer and access
iRODS iCommands for data transfer to Cyverse?
Globus monitor Process to monitor status of Globus transfers and ingest data to Clowder
Gantry cache monitoring scripts Scripts to monitor the Gantry cache, validate transfers, produce reports
Nagios/checkmk External monitoring/alert system
Extractors Primary data processing mechanism in compute pipeline
CyVerse External system used to store genetics data?
CoGe Genetics pipeline endpoint
ROGER NSF MRI system with block storage, compute, cloud (openstack), gridftp capabilities

Clowder

Clowder is used for the following:

  • Metadata storage (primary data stored on filesystem)
  • Divice/sensor and calibration data
  • Search: 46
  • Tool server: 204
  • Access control
  • API
  • Extractor pipeline
  • PostGIS/Geostreams API
  • Launching batch extractors: 195

BETYdb

BETYdb is used for the following:

  • Agronomic metadata: 57
  • Field measurements: 46, 61,
  • Site-level metadata: 7
  • Experimental design: 97
  • Trait data: 193, 33
  • Site/treatment/genotypes: 54
  • Plot-level data: 186, 167
  • API access: 9