Ganymede-Bio · bensonlee5 · Sep 13, 2024 · Sep 13, 2024 · Sep 13, 2024 · Sep 13, 2024
diff --git a/.gitmodules b/.gitmodules
@@ -7,6 +7,3 @@
 [submodule "pylib"]
 	path = pylib
 	url = [email protected]:Ganymede-Bio/pylib.git
-[submodule "api-server"]
-	path = api-server
-	url = https://github.com/Ganymede-Bio/api-server.git
diff --git a/.wordlist.txt b/.wordlist.txt
@@ -284,4 +284,8 @@ VPN
 EOL
 PowerShell
 netcat
-MockGanymedeContext
+Webhook
+webhook
+AIA
+CDF
+MockGanymedeContext
diff --git a/api-server b/api-server
diff --git a/core-dev-operators b/core-dev-operators
diff --git a/core-operators b/core-operators
diff --git a/docs/nodes/Analysis/Branch_Python.md b/docs/nodes/Analysis/Branch_Python.md
@@ -53,4 +53,4 @@ is written to the output bucket.
 ### Notes
 
 Files can also be retrieved and processed using the list_files and retrieve_files functions.
-Documentation on these functions can be found at https://docs.ganymede.bio/sdk/GanymedeClass
+Documentation on these functions can be found at https://docs.ganymede.bio/sdk/GanymedeClass
diff --git a/docs/nodes/Analysis/Python.md b/docs/nodes/Analysis/Python.md
@@ -47,4 +47,4 @@ is written to the output bucket.
 ### Notes
 
 Files can also be retrieved and processed using the list_files and retrieve_files functions.
-Documentation on these functions can be found at https://docs.ganymede.bio/sdk/GanymedeClass
+Documentation on these functions can be found at https://docs.ganymede.bio/sdk/GanymedeClass
diff --git a/docs/nodes/Analysis/Transform_SQL.md b/docs/nodes/Analysis/Transform_SQL.md
@@ -17,4 +17,4 @@ Runs SQL query on data lake
 
 ### Notes
 
-In the execute function, returning NodeReturn(tables_to_upload=\{'sql_result': df\}) would render the DataFrame df in the Flow Editor if Table Head visualization is enabled.
+In the execute function, returning NodeReturn(tables_to_upload=\{'sql_result': df\}) would render the DataFrame df in the Flow Editor if Table Head visualization is enabled.
diff --git a/docs/nodes/Analysis/Trigger_Python.md b/docs/nodes/Analysis/Trigger_Python.md
@@ -33,4 +33,4 @@ Processes data, then triggers downstream flow
 ### Notes
 
 The ID for the triggered flow can be found by navigating to the flow in the Ganymede UI
-and clicking on Manage.  The ID is an immutable string found on the Configuration panel.
+and clicking on Manage.  The ID is an immutable string found on the Configuration panel.
diff --git a/docs/nodes/App/APINode.md b/docs/nodes/App/APINode.md
@@ -17,14 +17,15 @@ Demonstrates access of generic API
 ### Parameters
 
 - **ganymede_context** : `GanymedeContext`
-    - Ganymede context variable, which stores flow run metadata
+  - Ganymede context variable, which stores flow run metadata
 
 ### Returns
 
 `NodeReturn`
   Object containing data to store in data lake and/or file storage.  NodeReturn object takes
   2 parameters:
-  - **tables_to_upload**: `dict[str, pd.DataFrame]`
-    keys are table names, values are pandas DataFrames to upload
-  - **files_to_upload**: `dict[str, bytes]`
-    keys are file names, values are file data to upload
+
+- **tables_to_upload**: `dict[str, pd.DataFrame]`
+  keys are table names, values are pandas DataFrames to upload
+- **files_to_upload**: `dict[str, bytes]`
+  keys are file names, values are file data to upload
diff --git a/docs/nodes/App/AirtableExport.md b/docs/nodes/App/AirtableExport.md
@@ -22,6 +22,7 @@ Reads table(s) from Ganymede data lake and uploads to Airtable.
 ### Notes
 
 Prior to usage, the following secrets must be configured in your Ganymede environment:
+
 - **airtable_api_key**: API key from Airtable
 
 Secrets can be configured by clicking on your username in the upper-right hand of the Ganymede
@@ -35,9 +36,9 @@ Process tabular data from user-defined SQL query, writing results to Airtable
 ### Parameters
 
 - **df_sql_result** : `pd.DataFrame | list[pd.DataFrame]`
-    - Table(s) or list of tables retrieved from user-defined SQL query
+  - Table(s) or list of tables retrieved from user-defined SQL query
 - **ganymede_context** : `GanymedeContext`
-    - Ganymede context variable, which stores flow run metadata
+  - Ganymede context variable, which stores flow run metadata
 
 ### Returns
 

diff --git a/docs/nodes/App/AirtableImport.md b/docs/nodes/App/AirtableImport.md
@@ -22,6 +22,7 @@ Retrieves Airtable table and uploads data to Ganymede data lake.
 ### Notes
 
 Prior to usage, the following secrets must be configured in your Ganymede environment:
+
 - **airtable_api_key**: API key from Airtable
 
 Secrets can be configured by clicking on your username in the upper-right hand of the Ganymede

diff --git a/docs/nodes/App/Airtable_Export.md b/docs/nodes/App/Airtable_Export.md
@@ -22,6 +22,7 @@ Reads table(s) from Ganymede data lake and uploads to Airtable.
 ### Notes
 
 Prior to usage, the following secrets must be configured in your Ganymede environment:
+
 - **airtable_api_key**: API key from Airtable
 
 Secrets can be configured by clicking on your username in the upper-right hand of the Ganymede
@@ -35,11 +36,11 @@ Process tabular data from user-defined SQL query, writing results to Airtable
 ### Parameters
 
 - **df_sql_result** : `pd.DataFrame | list[pd.DataFrame]`
-    - Table(s) or list of tables retrieved from user-defined SQL query
+  - Table(s) or list of tables retrieved from user-defined SQL query
 - **ganymede_context** : `GanymedeContext`
-    - Ganymede context variable, which stores flow run metadata
+  - Ganymede context variable, which stores flow run metadata
 
 ### Returns
 
 `NodeReturn`
-  Object containing data to store in data lake and/or file storage
+  Object containing data to store in data lake and/or file storage
diff --git a/docs/nodes/App/Airtable_Import.md b/docs/nodes/App/Airtable_Import.md
@@ -22,6 +22,7 @@ Retrieves Airtable table and uploads data to Ganymede data lake.
 ### Notes
 
 Prior to usage, the following secrets must be configured in your Ganymede environment:
+
 - **airtable_api_key**: API key from Airtable
 
 Secrets can be configured by clicking on your username in the upper-right hand of the Ganymede

diff --git a/docs/nodes/App/Azure_Query.md b/docs/nodes/App/Azure_Query.md
@@ -22,6 +22,7 @@ Query Azure SQL database, process data in Python, and upload results to Ganymede
 ### Notes
 
 Prior to usage, the following secrets must be configured in your Ganymede environment:
+
 - **azure_client_id**: App client ID
 - **azure_object_id**: App object ID
 - **azure_tenant_id**: App tenant ID
@@ -61,4 +62,4 @@ Process tabular data from user-defined SQL query, writing results back to data l
 ### Returns
 
 `NodeReturn`
-  Object containing data to store in data lake and/or file storage
+  Object containing data to store in data lake and/or file storage
diff --git a/docs/nodes/App/Azure_Read.md b/docs/nodes/App/Azure_Read.md
@@ -24,6 +24,7 @@ Read files from Azure Blob Storage into data lake.
 ### Notes
 
 Prior to usage, the following secrets must be configured in your Ganymede environment:
+
 - **azure_client_id**: App client ID
 - **azure_tenant_id**: App tenant ID
 - **azure_subscription_id**: Azure subscription ID
@@ -39,5 +40,5 @@ An example configuration is shown below:
 
 - **azure_storage_uri**: https://abc123.blob.core.windows.net
 - **src_azure_container_name**: abc123
-- **src_azure_blob_name**: "run_results/filename.parquet"
-- **dest_blob_name**: filename.parquet
+- **src_azure_blob_name**: `"run_results/filename.parquet"`
+- **dest_blob_name**: filename.parquet
diff --git a/docs/nodes/App/Azure_Read_Multi.md b/docs/nodes/App/Azure_Read_Multi.md
@@ -20,6 +20,7 @@ Read all files from a Azure Blob Storage container into Ganymede.
 ### Notes
 
 Prior to usage, the following secrets must be configured in your Ganymede environment:
+
 - **azure_client_id**: App client ID
 - **azure_tenant_id**: App tenant ID
 - **azure_subscription_id**: Azure subscription ID
@@ -34,4 +35,4 @@ assistance, please don't hesitate to reach out to Ganymede.
 An example configuration is shown below:
 
 - **azure_storage_uri**: https://abc123.blob.core.windows.net
-- **src_azure_container_name**: abc123
+- **src_azure_container_name**: abc123
diff --git a/docs/nodes/App/Azure_Write.md b/docs/nodes/App/Azure_Write.md
@@ -24,6 +24,7 @@ Write files from data lake to Azure Blob Storage.
 ### Notes
 
 Prior to usage, the following secrets must be configured in your Ganymede environment:
+
 - **azure_client_id**: App client ID
 - **azure_tenant_id**: App tenant ID
 - **azure_subscription_id**: Azure subscription ID
@@ -40,4 +41,4 @@ An example configuration is shown below:
 - **azure_storage_uri**: https://abc123.blob.core.windows.net
 - **src_object_name**: filename.parquet
 - **dest_azure_container_Name**: abc123
-- **dest_azure_blob_name**: "\{\{dag.dag_id\}\}/filename.parquet"
+- **dest_azure_blob_name**: \{dag.dag_id\}/filename.parquet
diff --git a/docs/nodes/App/Benchling_Api.md b/docs/nodes/App/Benchling_Api.md
@@ -25,6 +25,7 @@ in the relevant Benchling tenant.
 
 Prior to usage, the secrets below must be configured in your Ganymede environment.  More information
 on acquiring these credentials can be found [here](https://docs.benchling.com/docs/authentication).
+
 - **benchling_url**: URL for Benchling tenant; has form https://\<tenant\>.benchling.com
 - **benchling_app_client_id**: Client_ID; has form xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
 - **benchling_client_secret**: Client_Secret; has form cs_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
@@ -43,9 +44,9 @@ conjunction with the Input_Param node.
 ### Parameters
 
 - **benchling_context** : `BenchlingContext`
-    - Benchling context variable, which stores Benchling connection information
+  - Benchling context variable, which stores Benchling connection information
 - **ganymede_context** : `GanymedeContext`
-    - Ganymede context variable, which stores flow run metadata
+  - Ganymede context variable, which stores flow run metadata
 
 ### Returns
 
@@ -54,4 +55,4 @@ conjunction with the Input_Param node.
 
 ### Notes
 
-Documentation for the Benchling API can be found here: https://benchling.com/api/reference
+Documentation for the Benchling API can be found here: https://benchling.com/api/reference
diff --git a/docs/nodes/App/Benchling_Event.md b/docs/nodes/App/Benchling_Event.md
@@ -33,6 +33,7 @@ in the relevant Benchling tenant.
 
 Prior to usage, the secrets below must be configured in your Ganymede environment.  More information
 on acquiring these credentials can be found [here](https://docs.benchling.com/docs/authentication).
+
 - **benchling_url**: URL for Benchling tenant; has form https://\<tenant\>.benchling.com
 - **benchling_app_client_id**: Client_ID; has form xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
 - **benchling_client_secret**: Client_Secret; has form cs_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
@@ -55,13 +56,13 @@ Calls Benchling to get data for flow to trigger
 ### Parameters
 
 - **object_id** : `str`
-    - ID used in Benchling API (e.g. - entity schema ID or notebook entry ID)
+  - ID used in Benchling API (e.g. - entity schema ID or notebook entry ID)
 - **event_type** : `str`
-    - Type of Benchling event (e.g. - v2.entity.registered)
+  - Type of Benchling event (e.g. - v2.entity.registered)
 - **benchling_context** : `BenchlingContext`
-    - Benchling context variable, which stores Benchling connection information
+  - Benchling context variable, which stores Benchling connection information
 - **ganymede_context** : `GanymedeContext`
-    - Ganymede context variable, which stores flow run metadata
+  - Ganymede context variable, which stores flow run metadata
 
 ### Returns
 
@@ -75,6 +76,7 @@ Documentation for the Benchling API can be found here: https://benchling.com/api
 Reference for Benchling events can be found here: https://docs.benchling.com/docs/events-reference
 
 benchling_context is an object of class BenchlingContext, consisting of
+
 - **conn**: Benchling connection
 - **run_tag**: Benchling ID associated with run tag
-- **display_tag**: Displayed value of Benchling run tag
+- **display_tag**: Displayed value of Benchling run tag
diff --git a/docs/nodes/App/Benchling_Read.md b/docs/nodes/App/Benchling_Read.md
@@ -32,6 +32,7 @@ in the relevant Benchling tenant.
 
 Prior to usage, the secrets below must be configured in your Ganymede environment.  More information
 on acquiring these credentials can be found [here](https://docs.benchling.com/docs/authentication).
+
 - **benchling_url**: URL for Benchling tenant; has form https://\<tenant\>.benchling.com
 - **benchling_app_client_id**: Client_ID; has form xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
 - **benchling_client_secret**: Client_Secret; has form cs_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
@@ -50,9 +51,9 @@ conjunction with the Benchling_Tag node.
 ### Parameters
 
 - **benchling_context** : `BenchlingContext`
-    - Benchling context variable, which stores Benchling connection information
+  - Benchling context variable, which stores Benchling connection information
 - **ganymede_context** : `GanymedeContext`
-    - Ganymede context variable, which stores flow run metadata
+  - Ganymede context variable, which stores flow run metadata
 
 ### Returns
 
@@ -65,6 +66,7 @@ This template shows how a target plate could be pulled via the Benchling object
 Documentation for the Benchling API can be found here: https://benchling.com/api/reference
 
 benchling_context is an object of class BenchlingContext, consisting of
+
 - **conn**: Benchling connection
 - **run_tag**: Benchling ID associated with run tag
-- **display_tag**: Displayed value of Benchling run tag
+- **display_tag**: Displayed value of Benchling run tag
diff --git a/docs/nodes/App/Benchling_Read_Object.md b/docs/nodes/App/Benchling_Read_Object.md
@@ -31,6 +31,7 @@ in the relevant Benchling tenant.
 
 Prior to usage, the secrets below must be configured in your Ganymede environment.  More information
 on acquiring these credentials can be found [here](https://docs.benchling.com/docs/authentication).
+
 - **benchling_url**: URL for Benchling tenant; has form https://\<tenant\>.benchling.com
 - **benchling_app_client_id**: Client_ID; has form xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
 - **benchling_client_secret**: Client_Secret; has form cs_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
@@ -49,9 +50,9 @@ conjunction with the Input_Param node.
 ### Parameters
 
 - **benchling_context** : `BenchlingContext`
-    - Benchling context variable, which stores Benchling connection information
+  - Benchling context variable, which stores Benchling connection information
 - **ganymede_context** : `GanymedeContext`
-    - Ganymede context variable, which stores flow run metadata
+  - Ganymede context variable, which stores flow run metadata
 
 ### Returns
 
@@ -64,6 +65,7 @@ This template shows how a Benchling entry could be accessed via the Benchling ob
 Documentation for the Benchling API can be found here: https://benchling.com/api/reference
 
 benchling_context is an object of class BenchlingContext, consisting of
+
 - **conn**: Benchling connection
 - **run_tag**: Benchling ID associated with run tag
-- **display_tag**: Displayed value of Benchling run tag
+- **display_tag**: Displayed value of Benchling run tag
diff --git a/docs/nodes/App/Benchling_Warehouse_Query.md b/docs/nodes/App/Benchling_Warehouse_Query.md
@@ -13,6 +13,7 @@ Query Benchling Warehouse Postgres database, process data in Python, and upload
 ### Notes
 
 Prior to usage, the following secrets must be configured in your Ganymede environment:
+
 - **benchling_postgres_host**: Host for Benchling warehouse database.  Of the form: postgres-warehouse.\<tenant\>.benchling.com
 - **benchling_postgres_username**: Username for Benchling warehouse database.  Note that this is not the same as your Benchling username.
 - **benchling_postgres_password**: Password for Benchling warehouse database.  Note that this is not the same as your Benchling password.
@@ -33,11 +34,11 @@ Process tabular data from user-defined SQL query, writing results back to data l
 ### Parameters
 
 - **df_sql_result** : `pd.DataFrame | list[pd.DataFrame]`
-    - Table(s) or list of tables retrieved from user-defined SQL query
+- Table(s) or list of tables retrieved from user-defined SQL query
 - **ganymede_context** : `GanymedeContext`
-    - Ganymede context variable, which stores flow run metadata
+- Ganymede context variable, which stores flow run metadata
 
 ### Returns
 
 `NodeReturn`
-  Object containing data to store in data lake and/or file storage
+  Object containing data to store in data lake and/or file storage
diff --git a/docs/nodes/App/Benchling_Warehouse_Sync.md b/docs/nodes/App/Benchling_Warehouse_Sync.md
@@ -36,7 +36,7 @@ assistance, please don't hesitate to reach out to Ganymede.
 
 ## User-Defined Python
 
-This node helps to sync tables from Benchling warehouse to Ganymede data lake. A dataframe is
+This node helps to sync tables from Benchling warehouse to Ganymede datalake. A dataframe is
 constructed with the following columns
 
 - schema: str
@@ -52,9 +52,9 @@ Ganymede.
 ### Parameters
 
 - **ganymede_context** : `GanymedeContext`
-    - Ganymede context variable, which stores flow run metadata
+  - Ganymede context variable, which stores flow run metadata
 
 ### Returns
 
 `pd.DataFrame`
-  DataFrame containing schema, from_benchling_table, and to_ganymede_table
+  DataFrame containing schema, from_benchling_table, and to_ganymede_table
diff --git a/docs/nodes/App/Benchling_Write.md b/docs/nodes/App/Benchling_Write.md
@@ -23,6 +23,7 @@ in the relevant Benchling tenant.
 
 Prior to usage, the secrets below must be configured in your Ganymede environment.  More information
 on acquiring these credentials can be found [here](https://docs.benchling.com/docs/authentication).
+
 - **benchling_url**: URL for Benchling tenant; has form https://\<tenant\>.benchling.com
 - **benchling_app_client_id**: Client_ID; has form xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
 - **benchling_client_secret**: Client_Secret; has form cs_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
@@ -38,13 +39,13 @@ Processes input DataFrame for upload to Benchling
 ### Parameters
 
 - **df** : `pd.DataFrame`
-    - Tabular result of user-defined SQL query
+  - Tabular result of user-defined SQL query
 - **benchling_context** : `BenchlingContext`
-    - Benchling context variable, which stores Benchling connection information
+  - Benchling context variable, which stores Benchling connection information
 - **ganymede_context** : `GanymedeContext`
-    - Ganymede context variable, which stores flow run metadata
+  - Ganymede context variable, which stores flow run metadata
 
 ### Returns
 
 `NodeReturn`
-  Object containing data to store in data lake and/or blob storage
+  Object containing data to store in data lake and/or blob storage
Original file line number	Diff line number	Diff line change
Expand Up		@@ -17,4 +17,4 @@ Runs SQL query on data lake

		### Notes

		In the execute function, returning NodeReturn(tables_to_upload=\{'sql_result': df\}) would render the DataFrame df in the Flow Editor if Table Head visualization is enabled.
		In the execute function, returning NodeReturn(tables_to_upload=\{'sql_result': df\}) would render the DataFrame df in the Flow Editor if Table Head visualization is enabled.