TERRA-REF Breeder's API (BRAPI)

Implementation of the BRAPI standard for TERRA-REF instance of the BETYdb database (terraref.org/bety).

Mappings from BETY to BRAPI models

BRAPI	BETY	Notes
/calls	generated
/events	managements
/locations	sitegroups	lat/lon computed from sites part of sitegroup
/seasons	experiments	season = month of start_date, year of start_date
/germplasm	cultivars.
/observations	traits
/variables	variables

Currently implemented endpoints & parameters

See Swagger documentation for more details

  /calls	
  	dataType
  	paging
  /commoncropnames
  	paging				
  /events
    date
    eventDbId
    eventDescription
    eventParameters
    eventType
    observationUnitDbIds
    studyDbId
  /locations
  	locationType - Unavailable
  	paging
  /locations/{locationDbId}
  /trials
  	commonCropName	
  	programDbId		
  	locationDbId - Not supported
  	active - Unavailable
  	sorting	- Not supported
  	paging
  /programs
  	commonCropName
  	programName	
  	abbreviation
  	paging		
  /seasons
  	seasonDbId	
  	season	
  	year	
  	paging	
  /studies
  	commonCropName - Not supported
  	studyTypeDbId - Unavailable
  	programDbId	- Not supported
  	locationDbId
  	seasonDbId
  	trialDbId - Not supported
  	studyDBId
  	active - Unavailable
  	sorting
  	paging
  /studies/{studyDbId}			
  /studies/{studyDbId}/germplasm
  	paging				
  /studies/{studyDbId}/layouts
  	paging				
  /observationunits
  	germplasmDbId		
  	observationVariableDbId	
  	studyDbId	
  	locationDbId	
  	trialDbId - Not supported
  	programDbId - Not supported
  	seasonDbId
  	observationLevel - Unavailable
  	observationTimeStampRangeStart
  	observationTimeStampRangeEnd
  	paging
  /germplasm
  	germplasmPUI			
  	germplasmDbId			
  	germplasmName		
  	commonCropName		
  	paging
  /variables
  	name
  	observationVariableName
  	observationVariableDbID
  	scale
    

How to set up a development environment.

We assume you have the following installed:

• Docker (to run BETYdb)
• Postman (to test out APIs)
• PyCharm (for development)

Run a development instance of BETYdb

The following commands can be used to initialize the database to be used with BRAPI development. This will fetch public data from TERRA-REF database (id=6). To get the latest data you can call the sync command.

docker-compose up -d postgres
docker-compose run --rm bety initialize
docker-compose run --rm bety sync

Note: If you get an error about port 5432 already in use, you likely need to stop another instance of postgres (locally or as docker).

Set up PyCharm

Now that you have BETYdb running on port 5432, you will be able to begin developing the server. These instructions assume that you are using PyCharm and have a project open to your brapi directory.

• Go to run --> edit configurations
• Set the "name" field to "server"
• Set the 'script path' to <localpath>/brapi/server.py
• In the top right of PyCharm, select the server configuration and click the green triangle to run the server
• In Postman test that the API exists by navigating to http://localhost:5000/brapi/v1/calls

How to evaluate changes in an endpoint

• modify the file named api/<endpoint>.py
• click the 'reload' button in PyCharm to restart the server
• reload the endpoint in Postman and determine if requested changes are present.

How to add an endpoint

You will need add a new file under the api folder called with the name of the endpoint, for example the /locations endpoint will be defined in a file called api/locations.py. In that file you will need a search function to handle the call to the GET endpoint. If the function is not implemented the url will return the actual name of the file and the function name.

Showing data in BETY database

You can use the following command to start a docker container that is connected to the database allowing you to browse the database.

docker-compose up -d bety

To enable the guest user you can run the following sql query when connected to the database:

INSERT INTO users (login, name, email, crypted_password, salt, city, state_prov, postal_code, country, area, access_level, page_access_level, created_at, updated_at, apikey, remember_token, remember_token_expires_at)
 VALUES ('guestuser', 'guestuser', '[email protected]', '994363a949b6486fc7ea54bf40335127f5413318', 'bety', 'Urbana', 'IL', '61801', 'USA', '', 4, 4, NOW(), NOW(), NULL, NULL, NULL);

Testing your additions

Before you commit your new endpoints or any new code, please make sure that you check it with BRAVA. You can do this by running the brava container

docker-compose up -d brava

At this point you can connect to http://localhost:8080/ to see the Brava UI.

Next switch to 'Test your own' and change the URL. To launch the BrAPI:

docker-compose up -d brapi

If you run the brapi server as a docker container in docker-compose you can use http://brapi:5000/brapi/v1 as the URL. If you run the server on your machine you can either use http://host.docker.internal:5000/brapi/v1 or you can use http://<ipaddress>:5000/brapi/v1.

Configuring and Augmenting the TERRA REF database

Contributed Data

This repository provides the canonical reference for data that is outside of the scope of databases used in the TERRA REF program. Such data can be found in the /contrib/ folder.

Genomics data in contrib/genomics is in a set of CSVs that were previously only available in the experimental design section of the TERRA REF documentation. These files provide metadata that describe the germplasm used in the sorghum trials, and were originally prepared by Noah Fahlgren.

The data in these files could be inserted into the BETYdb attributes table.

The data in contrib/ontologies/variables_annotations.csv contain sorghum phenotypes, descriptions, ontologyReference from the Plant Trait Ontology, and the persistent URL, which were annotated by Laurel Cooper.

The file contrib/ontologies/ontology_reference.json contains metadata for the Plant Trait Ontology.

Setup Locations

For the locations endpoint to work correctly we grouped multiple sites together in a sitegroup. The commands in contrib/locations_setup.sql were used to associate sites with the sitegroups. (This has been done and is now part of the TERRA-REF dump).

Errors

wrong password for database or process already using a port: the port numbers for the databases (5432) specified in docker-compose.yml needs to match the one in database.py