Prototype OBIS-ENV-Data package for HABDAC IFCB data products from Santa Cruz Municipal Wharf

Prepared for 2024 OBIS IOOS Marine Biological Data Mobilization Workshop.

Participants: Frederick Bahr (CeNCOOS) and Stace Beaulieu (WHOI), with pre- and post-meetings with Patrick Daniel (UCSC)

This is a prototype for testing purposes only. A protocol is being developed to determine if and when appropriate to submit products from automated classification to OBIS.

Sponsored by NOAA PCMHAB20 project “Harmful Algal Bloom Community Technology Accelerator”

Background

This is a repository for CeNCOOS at the 2024 OBIS IOOS Marine Biological Data Mobilization Workshop. The plan is to take work done by Axiom and Ian Brunjes (SCCOOS) from previous workshops and create a full workflow to an OBIS-ENV-Data ready output. Code has already been created to generate Event and Occurrence tables. The plan is to supplement these with the EMoF table following best practices (http://dx.doi.org/10.25607/OBP-1955). Pre-workshop coding is being done to translate Ian's magnificent work from R to Python.

How to Run

Setting up the environment.yml in conda:
conda env create -f environment.yml
conda activate obis_ifcb

Input Files

• IFCB data products and metadata are acquired using IFCB Dashboard API
• Also, in data folder: class_names_matched_to_WoRMS, class_thresholds, DwC_matching_to_IFCB

Workflow

The prototype workflow is run in a notebook (see notebooks folder).

1. Load class labels and call WoRMS API

2. Load class specific thresholds

3. Load helper functions

4. Call the dashboard API

5. Get available datasets from the dashboard

6. Get bin files within the date range

7. Get bin metadata file

8. Check if the bin file has an autoclass file on the dashboard and load the file into a pandas dataframe

9. Get the ml_analyzed from the IFCB database for each bin

10. Get image feature information

11. Select top class for each roi

12. Generate a summary table

13. Generate event table

14. Generate the occurrence table

15. Generate Extended Measurement of Fact Table including Pelagic Size Structure Database (PSSdbb) biovolume

Output Files

• In data folder: ifcb_event.csv, ifcb_occurrence.csv, ifcb_emof.csv