Beta Testing

[matinnuhamunada]

[UPDATE] - New Database Structure

We have finished the BGCFlow run for the G1034 dataset. Unfortunately, two genomes were dropped because of an issue with the annotation (inconsistent exon ordering for features):

• NBC_01310 (NBC_0131000000000_213589.current.gb)
• NBC_01080 (NBC_0108000000000_76298.current.gb).

Therefore, currently we only have result ready for the 1032 genomes (~/net/o-drive/CFB-S-NewBioactiveCompounds/11_G1000/bgcflow/G1034_20240208/data/processed/G1032_20240208).

The new database structure now host two data schema, one for BGCFlow and another for antiSMASH. In the SQL query, this ca be fetched by explicitly mentioning the schema, for example bgcflow.cdss and antismash.cdss are two different tables. See at the bottom on how it looks like in DBeaver.

For lab members who want to try ChatBGC using the G1032 dataset, follow this step:

Get access to OpenAI API

To get the OpenAI API key, you can ask @matinnuhamunada for a key or use your own API.

Then, run this on your terminal

OPENAI_API_KEY="<change this to your API Key>"

Installing ChatBGC

Assuming you are using one of our shared machines and have python installed, follow this steps:

# Create new folder to set up duckdb and vector database
mkdir chatbgc
cd chatbgc

# Copy BGCFlow run result for G1032 dataset (using of our shared machine)
cp ~/net/o-drive/CFB-S-NewBioactiveCompounds/11_G1000/bgcflow/G1034_20240208/data/processed/G1032_20240208/dbt/antiSMASH_7.1.0/dbt_bgcflow.duckdb G1032_dbt_bgcflow.duckdb -n

# Create python environment and install ChatBGC
python3 -m venv chatbgc_env
source chatbgc_env/bin/activate
python3 -m pip install --upgrade pip
pip install git+https://github.com/NBChub/chatBGC.git

# Setup variable environment / secrets
touch .env
echo "export OPENAI_API_KEY=$OPENAI_API_KEY" > .env
source .env

Running ChatBGC

Training (do this once!)

# Train ChatBGC (Do it once)
chatbgc train --llm_type openai_chat --model gpt-4o G1032_dbt_bgcflow.duckdb

Running the chatbot

# Run ChatBGC
chatbgc run --llm_type openai_chat --model gpt-4o G1032_dbt_bgcflow.duckdb

Run using streamlit (experimental)

# Run ChatBGC
chatbgc run_streamlit --llm_type openai_chat --model gpt-4o G1032_dbt_bgcflow.duckdb

Manually exploring the database

If you want to explore the database generated manually, the easiest way is to install DBeaver

[Sam-Will]

• Would be good to ask follow-up questions
• Is ARTS included in this dataset?
  "The provided context does not include information about ARTS or duplicated core genes. Therefore, the SQL query cannot be generated based on the given context."
• It would be good to get more metadata even if its not asked for:
  i.e. "Do we have any kirromycin gene clusters?" gives a list regions
  "Are they're any Kirromycin gene clusters in the data? IF so give me the regions, % identity to abyssomicin C, taxa of the producting strain" provides a very impressive summary of this information, I think producing this more extensive data is more useful than the first one, even if the question asked is simple
  i.e. What is the highest percent identity to Abyssomicin C found in the data? Give 63%
  What would be more useful is taxa, region, etc

[matinnuhamunada]

• Unfortunately. I haven't incorporate ARTS in the database schema, but already have a PR on that.
• I think in the example question-sql pair, we can give it a better example where it also grabs relevant metadata so it can explain things better
• Adding the history of the chat data is something the community is currently working on.
• @Sam-Will if you're curious on the knowledge corpus it was trained, you can find it here:
  https://github.com/NBChub/chatBGC/tree/main/chatbgc/data
• As you can see this is a minimal training data, especially the sql-pair question
• Will need to improve this part using the experience of the people in the group

[drboothtj]

Super impressed with the ease of set up! But I managed to break it with the first question ;) I asked how many streptomyces in the dataset and it ran this:

SELECT COUNT(*) FROM genomes g JOIN taxa t ON g.tax_id = t.tax_id WHERE t.genus = 'Streptomyces';

Which I think is correct, but because it using the GTDB taxonomy, I think the search should be 'g__Streptomyces'. What do you think?

[drboothtj]

This works.

SELECT COUNT(*) 
FROM genomes g 
JOIN taxa t ON g.tax_id = t.tax_id 
WHERE t.genus = 'g__Streptomyces';

Very useful that you get the SQL though because it made it very easy to fix. Very exciting stuff! :)

[matinnuhamunada]

Thanks for the feedback, there are two features that might solve this issue and can be implemented soon:

• use WHERE t.genus ILIKE '%Streptomyces%' instead of WHERE t.genus = 'Streptomyces' for flexibility. I can add more of this example in the RAG training.
• use a mention function in the chat, such as if you type a certain character, for example @, it will look up through available glossary/vocabularies in the dataset.