Skip to content

Commit

Permalink
clarify agent kwargs types + upd s3 read doc on types
Browse files Browse the repository at this point in the history
  • Loading branch information
bensonlee5 committed Oct 11, 2024
1 parent f020770 commit 5353f10
Show file tree
Hide file tree
Showing 5 changed files with 33 additions and 8 deletions.
2 changes: 1 addition & 1 deletion core-dev-operators
2 changes: 1 addition & 1 deletion core-operators
7 changes: 4 additions & 3 deletions docs/app/agents/Agent.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -462,12 +462,13 @@ More generally, Variables and Labels can be flexibly used to control Connection

And [modified in the corresponding Connections panel in the Ganymede web app after installation, for Agent versions 4.5+](AgentMonitoring#monitoring-agent-connections).


Variables and labels can be referenced in your user-defined code by extracting the values from `kwargs` in the _execute_ function:
Variables and labels can be referenced in your user-defined code by extracting the values from `kwargs` in the _execute_ function.

```python
# Labels comes back as a list of strings
labels = kwargs.get('labels', [])
# -- OR --

# All variables retrieved have a dict[str, str] type
variables = kwargs.get('vars', {})

# Example: Retrieve the `input_path` variable for use in Agent code
Expand Down
14 changes: 11 additions & 3 deletions docs/nodes/App/S3_Read.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,17 @@ Specify list of S3 keys to upload to GCS

### Returns

`dict[str, str]`
Dictionary specifying files to upload to GCS. The keys are the keys in S3 and the values
are the file names to use in Ganymede
`dict[str, str] | NodeReturn`
If dict, files to upload to GCS. The keys are the keys in S3 and the values
are the file names to use in Ganymede.

If NodeReturn, object containing data to store in data lake and/or file storage.
NodeReturn object takes 2 parameters:

- tables_to_upload: dict[str, pd.DataFrame]
keys are table names, values are pandas DataFrames to upload
- files_to_upload: dict[str, bytes]
keys are file names, values are file data to upload

### Examples

Expand Down
16 changes: 16 additions & 0 deletions src/css/custom.css
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,22 @@
color: var(--ifm-color-gray-light);
}

.purple_alt_button {
border: 1px;
display: inline-block;
border-radius: 2px;
padding-left: 20px;
padding-right: 20px;
padding-top: 20px;
padding-bottom: 20px;
border-style: solid;
vertical-align: text-middle;
background-color: #736bdd;
stroke-width: 0.1px;
color: white;
}


.button.purple_button_dashboard {
border: 0px;
padding-left: 6px;
Expand Down

0 comments on commit 5353f10

Please sign in to comment.