Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing pr #17

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions .github/workflows/feast_apply_aws.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,19 +13,28 @@ jobs:
id: setup-python
uses: actions/setup-python@v2
with:
python-version: "3.7"
python-version: "3.8"
architecture: x64
- name: Set up AWS SDK
- name: Set up AWS SDK w/ Secrets
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-west-2

# Upgrade pip
- uses: actions/checkout@v2
- name: Upgrade pip
run: pip install --upgrade pip
- name: Upgrade setuptools
run: pip install --upgrade setuptools
- name: Install wheel
run: pip install wheel

# Run `feast apply`
- uses: actions/checkout@v2
- name: Install feast
run: pip install "feast[aws]"
run: pip install --no-build-isolation --force-reinstall "feast[aws]"
- name: Run feast apply
env:
FEAST_USAGE: "False"
Expand Down
37 changes: 0 additions & 37 deletions .github/workflows/feast_apply_gcp.yml

This file was deleted.

48 changes: 0 additions & 48 deletions .github/workflows/feast_plan_gcp.yml

This file was deleted.

2 changes: 1 addition & 1 deletion module_0/client_aws/feature_store.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
project: feast_demo_aws
provider: aws
registry: s3://feast-workshop-danny/registry.pb # TODO: Replace with your bucket
registry: s3://feast-repo-gyerli/feast-workshop-module-0/registry.pb # TODO: Replace with your bucket
online_store: null
offline_store:
type: file
Expand Down
4 changes: 2 additions & 2 deletions module_0/feature_repo_aws/data_sources.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@

driver_stats = FileSource(
name="driver_stats_source",
path="s3://feast-workshop-danny/driver_stats.parquet", # TODO: Replace with your bucket
path="s3://feast-workshop-gyerli/feast-workshop-module-0/driver_stats.parquet", # TODO: Replace with your bucket
s3_endpoint_override="http://s3.us-west-2.amazonaws.com", # Needed since s3fs defaults to us-east-1
timestamp_field="event_timestamp",
created_timestamp_column="created",
description="A table describing the stats of a driver based on hourly logs",
owner="[email protected]",
owner="[email protected]"
)
4 changes: 2 additions & 2 deletions module_0/feature_repo_aws/entities.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,5 @@
name="driver",
join_keys=["driver_id"],
value_type=ValueType.INT64,
description="driver id",
)
description="driver id"
)
6 changes: 4 additions & 2 deletions module_0/feature_repo_aws/feature_services.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,11 @@
feature_service = FeatureService(
name="model_v1",
features=[driver_hourly_stats_view[["conv_rate"]]],
owner="[email protected]",
owner="[email protected]",
)

feature_service_2 = FeatureService(
name="model_v2", features=[driver_hourly_stats_view], owner="[email protected]",
name="model_v2",
features=[driver_hourly_stats_view],
owner="[email protected]",
)
2 changes: 1 addition & 1 deletion module_0/feature_repo_aws/feature_store.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
project: feast_demo_aws
provider: aws
registry: s3://feast-workshop-danny/registry.pb # TODO: Replace with your bucket
registry: s3://feast-repo-gyerli/feast-workshop-module-0/registry.pb # TODO: Replace with your bucket
online_store: null
offline_store:
type: file
Expand Down
2 changes: 1 addition & 1 deletion module_0/feature_repo_aws/features.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
driver_hourly_stats_view = FeatureView(
name="driver_hourly_stats",
description="Hourly features",
entities=["driver"],
entities=[driver],
ttl=timedelta(seconds=8640000000),
schema=[
Field(name="conv_rate", dtype=Float32),
Expand Down
16 changes: 10 additions & 6 deletions module_0/infra/aws/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,21 @@ provider "aws" {
}

resource "aws_s3_bucket" "feast_bucket" {
bucket = "feast-workshop-${var.project_name}"
bucket = "feast-repo-gyerli"
force_destroy = true
}

resource "aws_s3_bucket_acl" "feast_bucket_acl" {
bucket = aws_s3_bucket.feast_bucket.bucket
acl = "private"
# resource "aws_s3_bucket_acl" "feast_bucket_acl" {
# bucket = aws_s3_bucket.feast_bucket.bucket
# }

resource "aws_s3_bucket_object" "feast_workshop_folder" {
bucket = aws_s3_bucket.feast_bucket.id
key = "feast-workshop-${var.project_name}/"
}

resource "aws_s3_object" "driver_stats_upload" {
bucket = aws_s3_bucket.feast_bucket.bucket
key = "driver_stats.parquet"
key = "feast-workshop-${var.project_name}/driver_stats.parquet"
source = "${path.module}/../driver_stats.parquet"
}
}
2 changes: 1 addition & 1 deletion module_0/infra/aws/variables.tf
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
variable "project_name" {
type = string
description = "The project identifier is used to uniquely namespace resources"
description = "The project identifier is used to uniquely namespace resources - gyerli"
}