Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiplatform #520

Merged
merged 28 commits into from
Aug 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
5be09c8
add multiplatform building to publish step
TShapinsky Jul 24, 2024
8a02b78
specify just worker for multi platform
TShapinsky Jul 24, 2024
34c6234
try to push from bake action
TShapinsky Jul 24, 2024
b495063
use push instead of load for multiplatform builds
TShapinsky Jul 25, 2024
cffe4e5
add metadata bake file for proper tagging
TShapinsky Jul 25, 2024
a1c330c
just add tags from metadata action
TShapinsky Jul 25, 2024
71f1740
try passing tags as string
TShapinsky Jul 25, 2024
92ca445
try string wrap
TShapinsky Jul 25, 2024
e97eddd
try fancy substitution
TShapinsky Jul 25, 2024
630d0dc
remove quotes
TShapinsky Jul 25, 2024
903debb
try injecting newline
TShapinsky Jul 25, 2024
9e0a2f1
try double quote instead of single quote
TShapinsky Jul 25, 2024
1f05a72
try brace expansion
TShapinsky Jul 25, 2024
1c54809
try prepending image to tag
TShapinsky Jul 25, 2024
0a8f4e7
try running bake without action
TShapinsky Jul 25, 2024
c41dadb
split to multiple sets
TShapinsky Jul 25, 2024
e56fc50
add full path to tag
TShapinsky Jul 25, 2024
7fd4609
fix expansion
TShapinsky Jul 25, 2024
32ee072
use correct format syntax
TShapinsky Jul 25, 2024
32aea0a
use proper seperator for tags
TShapinsky Jul 25, 2024
5b933d8
multiplatform for all containers
TShapinsky Jul 25, 2024
7b9f8e3
try exposing runtime in order to make caching work
TShapinsky Jul 25, 2024
d20ec23
uncomment ci
TShapinsky Jul 25, 2024
e9f2984
remove python dependencies that now live elsewhere
TShapinsky Jul 26, 2024
2d846f4
slim down worker image
TShapinsky Jul 26, 2024
6c09afb
ununcomment scipy for influxdb
TShapinsky Jul 26, 2024
5a469b0
update to dependencies 3.1.0
TShapinsky Aug 5, 2024
ae7ec96
remove comment in toml
TShapinsky Aug 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 25 additions & 22 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -255,43 +255,46 @@ jobs:
uses: actions/checkout@v4

- name: Log into container registry
uses: docker/login-action@v2
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Set up QEMU
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can i get a quick update on what QEMU is doing?

uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Docker meta
uses: docker/metadata-action@v4
uses: docker/metadata-action@v5
id: meta
with:
images: |
${{ env.REGISTRY_BASE_URI }}/worker
${{ env.REGISTRY_BASE_URI }}/web
${{ env.REGISTRY_BASE_URI }}/grafana
tags: |
type=ref,event=branch
type=sha
type=semver,pattern={{version}},enable=${{ github.event_name == 'release' }}
type=semver,pattern={{major}},enable=${{ github.event_name == 'release' }}
type=semver,pattern={{major}}.{{minor}},enable=${{ github.event_name == 'release' }}
type=edge

- name: Build
uses: docker/bake-action@v4
with:
files: |
docker-compose.yml
docker-compose.historian.yml
set: |
web.cache-from=type=gha,scope=web:${{ github.ref_name }}
worker.cache-from=type=gha,scope=worker:${{ github.ref_name }}
grafana.cache-from=type=gha,scope=grafana:${{ github.ref_name }}
web.cache-to=type=gha,mode=max,scope=web:${{ github.ref_name }}
worker.cache-to=type=gha,mode=max,scope=worker:${{ github.ref_name }}
grafana.cache-to=type=gha,mode=max,scope=grafana:${{ github.ref_name }}
load: true
- name: Expose GitHub Runtime
uses: crazy-max/ghaction-github-runtime@v3

- name: Tag and push images
run: ./.github/workflows/tag_push_docker.sh
- name: Build Push
run: |
/usr/bin/docker buildx bake \
--file docker-compose.yml \
--file docker-compose.historian.yml \
--set *.platform=linux/arm64,linux/amd64 \
--set web.cache-from=type=gha,scope=web:${{ github.ref_name }} \
--set worker.cache-from=type=gha,scope=worker:${{ github.ref_name }} \
--set grafana.cache-from=type=gha,scope=grafana:${{ github.ref_name }} \
--set web.cache-to=type=gha,mode=max,scope=web:${{ github.ref_name }} \
--set worker.cache-to=type=gha,mode=max,scope=worker:${{ github.ref_name }} \
--set grafana.cache-to=type=gha,mode=max,scope=grafana:${{ github.ref_name }} \
--set worker.tags=${{ env.REGISTRY_BASE_URI }}/worker:${{ join(fromJson(steps.meta.outputs.json)['tags'],format(' --set worker.tags={0}/worker:', env.REGISTRY_BASE_URI)) }} \
--set web.tags=${{ env.REGISTRY_BASE_URI }}/web:${{ join(fromJson(steps.meta.outputs.json)['tags'],format(' --set web.tags={0}/web:', env.REGISTRY_BASE_URI)) }} \
--set grafana.tags=${{ env.REGISTRY_BASE_URI }}/grafana:${{ join(fromJson(steps.meta.outputs.json)['tags'],format(' --set grafana.tags={0}/grafana:', env.REGISTRY_BASE_URI)) }} \
--push
33 changes: 12 additions & 21 deletions alfalfa_worker/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,17 +1,13 @@
FROM ghcr.io/nrel/alfalfa-dependencies:3.0.0 AS base
FROM ghcr.io/nrel/alfalfa-dependencies:3.1.0 AS base

USER root

ENV HOME /alfalfa
ENV HOME=/alfalfa

WORKDIR $HOME
# Need to set the lang to use Python 3.8 with Poetry
ENV LANG C.UTF-8
ENV DEBIAN_FRONTEND noninteractive

# Install poetry
RUN pip3 install poetry==1.8.3
RUN poetry config virtualenvs.options.system-site-packages true
RUN pip3 install poetry==1.8.3 \
&& poetry config virtualenvs.options.system-site-packages true \
&& poetry config virtualenvs.create false

# Copy and install poetry-based dependencies
COPY pyproject.toml /alfalfa
Expand All @@ -21,23 +17,18 @@ COPY poetry.lock /alfalfa
RUN poetry install --only main

# Include the path to where alfalfa_worker is so that system calls can find it.
ENV PYTHONPATH="/alfalfa:${PYTHONPATH}"
ENV PYTHONPATH="${HOME}:${PYTHONPATH}"

COPY ./alfalfa_worker ${HOME}/alfalfa_worker

COPY ./alfalfa_worker /alfalfa/alfalfa_worker
RUN pip3.8 install virtualenv \
&& pip3.8 install \
scipy \
symfit

COPY ./deploy /alfalfa/deploy
COPY ./deploy/wait-for-it.sh /usr/local/wait-for-it.sh

# update the path to python, which is in the Poetry virtual environment.
# The path (mfknN3Ol) is generated by Poetry and should be updated if the
# install path changes.
#
# The dependency warning still exists, but it is a warning. Eventually this
# will be resolved when we move to a newer version of Ubuntu.
# See https://bugs.launchpad.net/ubuntu/+source/python-debian/+bug/1926870
# Error: RequestsDependencyWarning: urllib3 (1.26.13) or chardet (3.0.4) doesn't match a supported version!
# warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported
ENV PATH=$HOME/.cache/pypoetry/virtualenvs/alfalfa-mfknN3Ol-py3.8/bin:$PATH
WORKDIR $HOME/alfalfa_worker


Expand Down
2 changes: 1 addition & 1 deletion alfalfa_worker/jobs/openstudio/create_run.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ def exec(self):
# If there are requirements.txt files in the model create a python virtual environment and install packaged there
requirements = self.run.glob("**/requirements.txt")
if len(requirements) > 0:
check_output(["python", "-m", "venv", "--system-site-packages", "--symlinks", str(self.dir / '.venv')])
check_output(["python3.8", "-m", "venv", "--system-site-packages", "--symlinks", str(self.dir / '.venv')])
for requirements_file in requirements:
check_output([str(self.dir / '.venv' / 'bin' / 'pip'), "install", "-r", str(requirements_file)])

Expand Down
4 changes: 2 additions & 2 deletions alfalfa_worker/lib/testcase.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@

import numpy as np
from pyfmi import load_fmu
from scipy.integrate import trapz
from scipy.integrate import trapezoid

from alfalfa_worker.lib.data.data_manager import Data_Manager

Expand Down Expand Up @@ -289,7 +289,7 @@ def get_kpis(self) -> dict:
data = np.array(self.y_store[signal])
dT_heating = heat_setpoint - data
dT_heating[dT_heating < 0] = 0
tot_dis = tot_dis + trapz(dT_heating, self.y_store['time']) / 3600
tot_dis = tot_dis + trapezoid(dT_heating, self.y_store['time']) / 3600
# Store result in dictionary
kpis['comfort'] = tot_dis
else:
Expand Down
Loading