Skip to content

Commit

Permalink
feat: explore select (#75)
Browse files Browse the repository at this point in the history
* initial new layout

* add the title and examples

* reset the chat, submit the query

* start reconnecting everything

* fix the message display order

* improve the prompt input

* remove unused-pages

* improve layout

* remove un-used files

* improve message styling

* Improve the spinner

* improve UX

* fix summarization

* load only when ready

* connect chat to history

* persist the redux store

* clear history

* don't persist dimension and measures

* use the data toggle

* Docs(updating readme with new ui/ux gif)

* feat: multi explore support

---------

Co-authored-by: Jawad Laraqui <[email protected]>
Co-authored-by: Luka Fontanilla <[email protected]>
  • Loading branch information
3 people authored Aug 12, 2024
1 parent c986fe3 commit 7cc6138
Show file tree
Hide file tree
Showing 16 changed files with 457 additions and 176 deletions.
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
# Changelog
## v3.1

### Added
- Explore dropdown for asking questions against different explores
- only loads explores that exist in examples table on bigquery end
- extension frontend picks up explores without redeployment of app
- Dynamic explore samples loaded from samples table in BigQuery
- terraform update to deploy samples table
- bigquery loader script update to upload samples per explore

## v3.0

### Added
Expand Down
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,14 @@ Additionally, the extension provides:
- Flexible Deployment Options
- Multi-turn
- Insight Summarization
- Dynamic Explore Selection

### Technologies Used
#### Frontend
- [React](https://reactjs.org/)
- [TypeScript](https://www.typescriptlang.org/)
- [Webpack](https://webpack.js.org/).
- [Styled components](https://www.styled-components.com/docs)
- [Tailwind CSS](https://tailwindcss.com/)

#### Looker
- [Looker Extension SDK](https://github.com/looker-open-source/sdk-codegen/tree/main/packages/extension-sdk-react)
Expand All @@ -37,11 +38,11 @@ Additionally, the extension provides:

## Setup

Getting started involves:
Getting started involves (*in this order*):

- Frontend Setup - setup Looker Extension Framework Applications by following [these instructions](./explore-assistant-extension/README.md).
- Backend Setup - setup the GCP backend for communicating with the Vertex API [using these instructions.](./explore-assistant-backend/README.md)
- Example generation - generate a list of examples and upload them to BigQuery [using these instructions.](./explore-assistant-examples/README.md)
- Frontend Setup - setup Looker Extension Framework Applications by following [these instructions](./explore-assistant-extension/README.md).

The local cloud function backend and example generation require some python packages. It is recommended to create a python virtual environment and install the dependencies:

Expand Down
26 changes: 26 additions & 0 deletions explore-assistant-backend/terraform/bigquery_examples.tf
Original file line number Diff line number Diff line change
Expand Up @@ -67,3 +67,29 @@ resource "google_bigquery_job" "create_explore_assistant_refinement_examples_tab
ignore_changes = [query, job_id]
}
}

resource "google_bigquery_job" "create_explore_assistant_samples_table" {
job_id = "create_explore_assistant_samples_table-${formatdate("YYYYMMDDhhmmss", timestamp())}"
query {
query = <<EOF
CREATE OR REPLACE TABLE `${google_bigquery_dataset.dataset.dataset_id}.explore_assistant_samples` (
explore_id STRING OPTIONS (description = 'Explore id of the explore to pull examples for in a format of -> lookml_model:lookml_explore'),
samples STRING OPTIONS (description = 'Samples for Explore Assistant Samples displayed in UI. JSON document with listed samples with category, prompt and color keys.')
)
EOF
create_disposition = ""
write_disposition = ""
allow_large_results = false
flatten_results = false
maximum_billing_tier = 0
schema_update_options = []
use_legacy_sql = false
}

location = var.deployment_region
depends_on = [time_sleep.wait_after_apis_activate]

lifecycle {
ignore_changes = [query, job_id]
}
}
10 changes: 8 additions & 2 deletions explore-assistant-examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,14 @@ python load_examples.py --project_id YOUR_PROJECT_ID --explore_id YOUR_EXPLORE_I

Load the refinement examples
```bash
python load_examples.py --project_id YOUR_PROJECT_ID --explore_id YOUR_EXPLORE_ID --table_id explore_assistant_refinement_examples --json_file refinement_examples.json
```
python load_examples.py --project_id YOUR_PROJECT_ID --explore_id YOUR_EXPLORE_ID --table_id explore_assistant_refinement_examples --json_file refinement_examples.json
```

Load the samples

```bash
python load_examples.py --project_id YOUR_PROJECT_ID --explore_id YOUR_EXPLORE_ID --table_id explore_assistant_samples --column_name samples --json_file samples.json

### Description

This Python script is designed to manage data uploads from a JSON file into a Google BigQuery table, particularly focusing on scenarios where specific entries identified by an `explore_id` need to be refreshed or updated in the dataset.
Expand Down
7 changes: 4 additions & 3 deletions explore-assistant-examples/load_examples.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ def parse_arguments():
parser.add_argument('--project_id', type=str, required=True, help='Google Cloud project ID')
parser.add_argument('--dataset_id', type=str, help='BigQuery dataset ID', default='explore_assistant')
parser.add_argument('--table_id', type=str, help='BigQuery table ID', default='explore_assistant_examples')
parser.add_argument('--column_name', type=str, help='Column name, if different than "examples"', default='examples')
parser.add_argument('--explore_id', type=str, required=True, help='The name of the explore in the model:explore_name format')
parser.add_argument('--json_file', type=str, help='Path to the JSON file containing the data', default='examples.json')
return parser.parse_args()
Expand Down Expand Up @@ -35,14 +36,14 @@ def load_data_from_file(json_file_path):
with open(json_file_path, 'r') as file:
return json.load(file)

def insert_data_into_bigquery(client, dataset_id, table_id, explore_id, data):
def insert_data_into_bigquery(client, dataset_id, table_id, column_name, explore_id, data):
"""Insert data into BigQuery using a SQL INSERT statement."""
# Convert the data to a JSON string
data_json = json.dumps(data)

# Create a BigQuery SQL INSERT statement
insert_query = f"""
INSERT INTO `{dataset_id}.{table_id}` (explore_id, examples)
INSERT INTO `{dataset_id}.{table_id}` (explore_id, `{column_name}`)
VALUES (@explore_id, @examples)
"""

Expand Down Expand Up @@ -73,7 +74,7 @@ def main():

# load data from file and insert into BigQuery
data = load_data_from_file(args.json_file)
insert_data_into_bigquery(client, args.dataset_id, args.table_id, args.explore_id, data)
insert_data_into_bigquery(client, args.dataset_id, args.table_id, args.column_name, args.explore_id, data)

if __name__ == '__main__':
main()
17 changes: 17 additions & 0 deletions explore-assistant-examples/samples.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
[
{
"category": "Cohorting",
"prompt": "Count of Users by first purchase date",
"color": "green"
},
{
"category": "Audience Building",
"prompt":"Users who have purchased more than 100 dollars worth of Calvin Klein products and have purchased in the last 30 days",
"color": "blue"
},
{
"category": "Period Comparison",
"prompt": "Total revenue by category this year compared to last year in a line chart with year pivoted",
"color": "red"
}
]
36 changes: 18 additions & 18 deletions explore-assistant-extension/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 10 additions & 4 deletions explore-assistant-extension/src/App.tsx
Original file line number Diff line number Diff line change
@@ -1,24 +1,30 @@
import React, { useEffect } from 'react'
import { hot } from 'react-hot-loader/root'
import { Route, Switch, Redirect } from 'react-router-dom'
import { useDispatch } from 'react-redux'
import { useDispatch, useSelector } from 'react-redux'
import {
setExploreId,
setExploreName,
setModelName,
} from './slices/assistantSlice'
import { RootState } from './store'
import { useLookerFields } from './hooks/useLookerFields'
import { useBigQueryExamples } from './hooks/useBigQueryExamples'
import AgentPage from './pages/AgentPage'

const ExploreApp = () => {
const dispatch = useDispatch()
const {
exploreName,
modelName,
exploreId,
} = useSelector((state: RootState) => state.assistant)
const LOOKER_EXPLORE_ID =
`${process.env.LOOKER_MODEL}/${process.env.LOOKER_EXPLORE}` || ''
exploreId || `${process.env.LOOKER_MODEL}/${process.env.LOOKER_EXPLORE}` || ''
useEffect(() => {
dispatch(setExploreId(LOOKER_EXPLORE_ID))
dispatch(setModelName(process.env.LOOKER_MODEL || ''))
dispatch(setExploreName(process.env.LOOKER_EXPLORE || ''))
dispatch(setModelName(modelName || process.env.LOOKER_MODEL || ''))
dispatch(setExploreName(exploreName || process.env.LOOKER_EXPLORE || ''))
}, [])


Expand Down
77 changes: 44 additions & 33 deletions explore-assistant-extension/src/components/SamplePrompts.tsx
Original file line number Diff line number Diff line change
@@ -1,47 +1,58 @@
import React from 'react'
import { useDispatch } from 'react-redux'
import { useDispatch, useSelector } from 'react-redux'
import { resetChat, setIsChatMode, setQuery } from '../slices/assistantSlice'
import { RootState } from '../store'

const SamplePrompts = () => {
const dispatch = useDispatch()
const categorizedPrompts = [
{
category: 'Cohorting',
prompt: 'Count of Users by first purchase date',
},
{
category: 'Audience Building',
prompt:
'Users who have purchased more than 100 dollars worth of Calvin Klein products and have purchased in the last 30 days',
},
{
category: 'Period Comparison',
prompt:
'Total revenue by category this year compared to last year in a line chart with year pivoted',
},
]
const {
examples,
exploreId,
} = useSelector((state: RootState) => state.assistant)
const [samplesLoaded, setSamplesLoaded] = React.useState(false)

React.useEffect(() => {
if(examples.exploreSamples.length > 0) {
setSamplesLoaded(true)
console.log(examples.exploreSamples)
}
},[examples.exploreSamples])

const handleSubmit = (prompt: string) => {
dispatch(resetChat())
dispatch(setQuery(prompt))
dispatch(setIsChatMode(true))
}
return (
<div className="flex flex-wrap max-w-5xl">
{categorizedPrompts.map((item, index: number) => (
<div
className="flex flex-col w-56 bg-gray-200/50 hover:bg-gray-200 rounded-lg cursor-pointer text-sm p-4 m-2"
key={index}
onClick={() => {
handleSubmit(item.prompt)
}}
>
<div className="flex-grow font-light line-camp-5">{item.prompt}</div>
<div className="mt-2 font-semibold justify-end">{item.category}</div>
</div>
))}
</div>
)

if(samplesLoaded) {
const categorizedPrompts = JSON.parse(
examples.exploreSamples.filter(
explore => explore.explore_id === exploreId.replace("/",":")
)
?.[0]
?.['samples'] ?? '[]'
)

return (
<div className="flex flex-wrap max-w-5xl">
{categorizedPrompts.map((item, index: number) => (
<div
className="flex flex-col w-56 bg-gray-200/50 hover:bg-gray-200 rounded-lg cursor-pointer text-sm p-4 m-2"
key={index}
onClick={() => {
handleSubmit(item.prompt)
}}
>
<div className="flex-grow font-light line-camp-5">{item.prompt}</div>
<div className="mt-2 font-semibold justify-end">{item.category}</div>
</div>
))
}
</div>
)
} else {
return <></>
}
}

export default SamplePrompts
Loading

0 comments on commit 7cc6138

Please sign in to comment.