Python scripts to ingestion HAB sampling information from email Vicky is excited about this!
The e-mail scraping code is read_HAB_email_to_post.py This code has been modified to remove some of the specific urls and usernames that are specific to my implementation. In the code it is noted where you need to place your specific url and username and password.
The code was developed using a Zimbra e-mail browser. Some of these details may be different for your implementation.
The code uses find_plankton_word.py to find the plankton word in the e-mail payload.
The code also uses find_word_val.py to find some non-plankton values that might be important.
HABdate_to_ordinal.py converts the various date formats to python ordinal dates for plotting.
An ancillary file plankton_names.txt is a list of names that have been encountered so far in the e-mails. It does not mean that other names might arrise in the e-mails. It is alphabetized for easy addition of names.
The python regular expression module is used extensively to find the index of the data in the e-mail payloads.
A pickle file of the dataframes is generated for use by a NetCDF writer. The NetCDF writer was created to try and meet Darwin Core and OBIS standards (though it is probably missing a few things at the moment).
writehab.py is code to start to write data as Darwin Core/OBIS data. This code uses pyworms to load some data.