Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: leapfrogai-ui registry1 flavor integration and uds tasks refactor #1178

Draft
wants to merge 24 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
600bb6c
UI package refactors, task additions and refactor
justinthelaw Oct 2, 2024
3119efe
prevent badge workflow from drafts
justinthelaw Oct 2, 2024
ff79754
add UI to weekly registry1 workflow
justinthelaw Oct 2, 2024
8eaf672
setup tasks for k3d cluster and test user
justinthelaw Oct 2, 2024
2a091e5
setup tasks for k3d cluster and test user, pt.2
justinthelaw Oct 2, 2024
c04ea21
creates for all Zarf packages and images
justinthelaw Oct 2, 2024
8e4c033
WIP deploy tasks
justinthelaw Oct 2, 2024
c9505a9
WIP deploy tasks
justinthelaw Oct 3, 2024
3ce634e
Merge remote-tracking branch 'origin/main' into 736-feat-leapfrogai-u…
justinthelaw Oct 3, 2024
6068be7
WIP deploy tasks, merge with main
justinthelaw Oct 3, 2024
86103bf
registry1 UI 0.13.1 working
justinthelaw Oct 3, 2024
3cc4b42
Merge remote-tracking branch 'origin/main' into 736-feat-leapfrogai-u…
justinthelaw Oct 4, 2024
aef6cd3
cyclic loop workaround
justinthelaw Oct 4, 2024
badcabf
upstream zarfConfig contribution
justinthelaw Oct 4, 2024
ea1ff6d
undo formatting
justinthelaw Oct 4, 2024
32e2558
Merge remote-tracking branch 'origin/main' into 736-feat-leapfrogai-u…
justinthelaw Oct 4, 2024
4fb8de8
upstream zarfConfig contribution, implemented
justinthelaw Oct 4, 2024
0d9076e
streamline logging, minor fixes
justinthelaw Oct 4, 2024
d0e25fd
Merge branch 'main' into 736-feat-leapfrogai-ui-registry1-flavor
justinthelaw Oct 4, 2024
7206738
re-align to upstream create and deploy
justinthelaw Oct 4, 2024
c9a0499
WIP docs updates, add cleans
justinthelaw Oct 4, 2024
f3c917c
remove temp UI registry1 workaround
justinthelaw Oct 7, 2024
2f01de4
clean-up Make targets that already exist
justinthelaw Oct 7, 2024
6366d8f
fix pytest
justinthelaw Oct 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/e2e-playwright.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@ jobs:
rm packages/ui/zarf-package-leapfrogai-ui-amd64-e2e-test.tar.zst

# Run the playwright UI tests using the deployed Supabase endpoint and upload report as an artifact
# Note - workflow doesn't need teardown and causes import issues
# NOTE: the workflow doesn't need `global.teardown.ts` and it also causes import issues
- name: UI/API/Supabase E2E Playwright Tests
run: |
cp src/leapfrogai_ui/.env.example src/leapfrogai_ui/.env
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/nightly-snapshot-release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@ on:
workflow_dispatch: # trigger manually as needed
pull_request:
types:
- opened # default trigger
justinthelaw marked this conversation as resolved.
Show resolved Hide resolved
- reopened # default trigger
- synchronize # default trigger
- ready_for_review # don't run on draft PRs
- milestoned # allows us to trigger on bot PRs
- opened # default trigger
- reopened # default trigger
- synchronize # default trigger
- ready_for_review # don't run on draft PRs
- milestoned # allows us to trigger on bot PRs
paths:
- .github/workflows/nightly-snapshot-release.yaml

Expand Down
1 change: 1 addition & 0 deletions .github/workflows/nightly-uds-badge-verification.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ jobs:
uds-badge-verification:
runs-on: ai-ubuntu-big-boy-8-core
name: nightly_uds_badge_verification
if: ${{ !github.event.pull_request.draft }}

steps:
- name: Checkout Repo
Expand Down
18 changes: 18 additions & 0 deletions .github/workflows/weekly-registry1-flavor-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -75,11 +75,18 @@ jobs:

- name: Mutation of the Zarf Packages
run: |
# API
uds zarf tools yq -i '
.components[].images[0] |= sub(":v[0-9\.]+$", ":v${{ steps.get_version.outputs.LFAI_VERSION }}")
' packages/api/zarf.yaml
uds zarf tools yq -i '.api.image.tag = "v${{ steps.get_version.outputs.LFAI_VERSION }}"' packages/api/values/registry1-values.yaml

# UI
uds zarf tools yq -i '
.components[].images[0] |= sub(":v[0-9\.]+$", ":v${{ steps.get_version.outputs.LFAI_VERSION }}")
' packages/ui/zarf.yaml
uds zarf tools yq -i '.ui.image.tag = "v${{ steps.get_version.outputs.LFAI_VERSION }}"' packages/api/values/registry1-values.yaml

- name: Print the Modified Zarf Packages
run: |
cat packages/api/zarf.yaml
Expand All @@ -97,10 +104,16 @@ jobs:

uds zarf tools yq -i '.packages[].ref |= sub("^[^ ]+-upstream$", "${{ steps.get_version.outputs.LFAI_VERSION }}-upstream")' bundles/latest/cpu/uds-bundle.yaml

# API
uds zarf tools yq -i '.packages[1] |= del(.repository)' bundles/latest/cpu/uds-bundle.yaml
uds zarf tools yq -i '.packages[1] |= .ref = "${{ steps.get_version.outputs.LFAI_VERSION }}"' bundles/latest/cpu/uds-bundle.yaml
uds zarf tools yq -i '.packages[1] |= .path = "../../../"' bundles/latest/cpu/uds-bundle.yaml

# UI
uds zarf tools yq -i '.packages[5] |= del(.repository)' bundles/latest/cpu/uds-bundle.yaml
uds zarf tools yq -i '.packages[5] |= .ref = "${{ steps.get_version.outputs.LFAI_VERSION }}"' bundles/latest/cpu/uds-bundle.yaml
uds zarf tools yq -i '.packages[5] |= .path = "../../../"' bundles/latest/cpu/uds-bundle.yaml

- name: Print the Modified UDS Bundle
run: |
cat bundles/latest/cpu/uds-config.yaml
Expand Down Expand Up @@ -152,6 +165,8 @@ jobs:
npm --prefix src/leapfrogai_ui ci
npx --prefix src/leapfrogai_ui playwright install

# NOTE: the workflow doesn't need `global.teardown.ts` and it also causes import issues
# TODO: some tests on main for `api-keys.test.ts` are failing in 0.13.1, re-enable post-0.13.1
- name: Run Playwright E2E Tests
env:
SERVICE_ROLE_KEY: ${{ steps.generate_secrets.outputs.SERVICE_KEY }}
Expand All @@ -165,6 +180,9 @@ jobs:
mkdir -p playwright/auth
touch playwright/auth.user.json

rm src/leapfrogai_ui/tests/api-keys.test.ts
rm src/leapfrogai_ui/tests/global.teardown.ts

SERVICE_ROLE_KEY=$SERVICE_ROLE_KEY TEST_ENV=CI USERNAME=doug PASSWORD=$FAKE_E2E_USER_PASSWORD PUBLIC_SUPABASE_ANON_KEY=$ANON_KEY DEFAULT_MODEL=llama-cpp-python npm --prefix src/leapfrogai_ui run test:integration:ci

- name: Archive Playwright Report
Expand Down
34 changes: 34 additions & 0 deletions packages/ui/common/zarf.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/defenseunicorns/uds-cli/v0.14.0/zarf.schema.json

kind: ZarfPackageConfig
metadata:
description: "LeapfrogAI UI common"
name: leapfrogai-ui-common
version: "###ZARF_PKG_TMPL_IMAGE_VERSION###"

components:
- name: leapfrogai-ui
description: "A UI for LeapfrogAI"
required: true
charts:
- name: leapfrogai-ui
namespace: leapfrogai
localPath: ../chart
# x-release-please-start-version
version: 0.13.1
# x-release-please-end
actions:
onDeploy:
after:
- wait:
cluster:
kind: Job
name: leapfrogai-ui-migrations-###ZARF_PKG_TMPL_IMAGE_VERSION###
namespace: leapfrogai
condition: complete
- wait:
cluster:
kind: Deployment
name: leapfrogai-ui
namespace: leapfrogai
condition: Available
48 changes: 48 additions & 0 deletions packages/ui/values/registry1-values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
image:
repository: "registry1.dso.mil/ironbank/opensource/defenseunicorns/leapfrogai/ui"
tag: v###ZARF_CONST_IMAGE_VERSION###

env:
- name: LEAPFROGAI_API_BASE_URL
value: "###ZARF_VAR_LEAPFROGAI_API_BASE_URL###"
- name: ORIGIN
value: "https://###ZARF_VAR_SUBDOMAIN###.###ZARF_VAR_DOMAIN###"
- name: DEFAULT_MODEL
value: "###ZARF_VAR_MODEL###"
- name: DEFAULT_SYSTEM_PROMPT
value: "###ZARF_VAR_SYSTEM_PROMPT###"
- name: DEFAULT_TEMPERATURE
value: "###ZARF_VAR_TEMPERATURE###"
- name: OPENAI_API_KEY
value: "###ZARF_VAR_OPENAI_API_KEY###"
- name: PUBLIC_SUPABASE_URL
value: "https://supabase-kong.###ZARF_VAR_DOMAIN###"
- name: PUBLIC_DISABLE_KEYCLOAK
value: "###ZARF_VAR_DISABLE_KEYCLOAK###"
- name: PUBLIC_MESSAGE_LENGTH_LIMIT
value: "10000"
- name: SUPABASE_AUTH_EXTERNAL_KEYCLOAK_URL
value: "https://sso.uds.dev/realms/uds"
- name: SUPABASE_AUTH_KEYCLOAK_CLIENT_ID
valueFrom:
secretKeyRef:
name: sso-client-uds-supabase
key: clientId
- name: SUPABASE_AUTH_KEYCLOAK_SECRET
valueFrom:
secretKeyRef:
name: sso-client-uds-supabase
key: secret

extraEnv:
# Only required when Supabase is hosted via PaaS outside of the cluster
supabaseAnonKey: "###ZARF_VAR_SUPABASE_ANON_KEY###"
# UDS package CR hosting subdomain (e.g. `ai` will produce a VirtualService with `ai.uds.dev`)
host: "###ZARF_VAR_SUBDOMAIN###"

migration:
image:
# TODO: replace with Ironbank image once hardened: registry1.dso.mil/ironbank/opensource/defenseunicorns/leapfrogai/ui/migrations
repository: ghcr.io/defenseunicorns/leapfrogai/ui-migrations
tag: "###ZARF_CONST_IMAGE_VERSION###"
imagePullPolicy: Always
6 changes: 6 additions & 0 deletions packages/ui/zarf-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package:
create:
set:
# x-release-please-start-version
image_version: "0.13.1"
# x-release-please-end
35 changes: 20 additions & 15 deletions packages/ui/zarf.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -63,9 +63,12 @@ variables:

components:
- name: leapfrogai-ui
description: "A UI for LeapfrogAI"
required: true
only:
flavor: upstream
import:
path: common
charts:
- name: leapfrogai-ui
namespace: leapfrogai
Expand All @@ -78,18 +81,20 @@ components:
images:
- ghcr.io/defenseunicorns/leapfrogai/leapfrogai-ui:###ZARF_PKG_TMPL_IMAGE_VERSION###
- ghcr.io/defenseunicorns/leapfrogai/ui-migrations:###ZARF_PKG_TMPL_IMAGE_VERSION###
actions:
onDeploy:
after:
- wait:
cluster:
kind: Job
name: leapfrogai-ui-migrations-###ZARF_PKG_TMPL_IMAGE_VERSION###
namespace: leapfrogai
condition: complete
- wait:
cluster:
kind: Deployment
name: leapfrogai-ui
namespace: leapfrogai
condition: Available

- name: leapfrogai-ui
description: "A Python API that shadows the OpenAI API specification"
only:
flavor: registry1
required: true
import:
path: common
charts:
- name: leapfrogai-ui
namespace: leapfrogai
valuesFiles:
- "values/registry1-values.yaml"
images:
- "registry1.dso.mil/ironbank/opensource/defenseunicorns/leapfrogai/ui:v###ZARF_PKG_TMPL_IMAGE_VERSION###"
# TODO: replace with Ironbank image once hardened: registry1.dso.mil/ironbank/opensource/defenseunicorns/leapfrogai/ui/migrations
- "ghcr.io/defenseunicorns/leapfrogai/ui-migrations:###ZARF_PKG_TMPL_IMAGE_VERSION###"
Loading