Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overwatcher #32

Open
wants to merge 118 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
118 commits
Select commit Hold shift + click to select a range
bc79513
Very basic structure for the overwatcher
albireox Mar 26, 2024
1f1fee7
Build docker images for all branches
albireox Mar 26, 2024
b7c801b
Restructure things and add EphemerisOverwatcher
albireox Mar 26, 2024
d8efd7d
Merge branch 'main' into overwatcher
albireox Mar 26, 2024
858b8d7
Update deps
albireox Mar 26, 2024
c3d922a
Improve twiligh flats recipe
albireox Mar 26, 2024
1ac1f2a
Remove script file
albireox Mar 27, 2024
1fc7883
Replace pandas with polars
albireox Mar 27, 2024
aa21314
More overwatcher changes
albireox Mar 27, 2024
0b2ca94
Add some extra logging
albireox Mar 27, 2024
b417d8e
Require dome to be open for twilight flats
albireox Mar 28, 2024
b2da1ad
Do startup in overwatcher observer when opening dome
albireox Mar 28, 2024
ec38189
Await last exposure when cancelling loop
albireox Mar 28, 2024
78f8244
Freeze import order
albireox Mar 28, 2024
a8d8e2a
Set allow_observations
albireox Mar 28, 2024
245e9f2
Remove old calibration recipes code
albireox Mar 28, 2024
7b6f2f6
Allow to set the configuration file
albireox Mar 28, 2024
ca04e5b
Some work on overwatcher calibrations
albireox Mar 28, 2024
5395c2d
Add gort_logo_slack.png
albireox Mar 28, 2024
da1a375
Significant improvements to overwatcher
albireox Mar 29, 2024
cc35d58
A bit more work on the calibrations
albireox Mar 29, 2024
42cc286
Merge branch 'main' into overwatcher
albireox Mar 29, 2024
42c6501
Use API to handle Slack mentions
albireox Mar 29, 2024
fdbd20e
More overwatcher work
albireox Apr 3, 2024
b821252
Merge branch 'main' into overwatcher
albireox Apr 8, 2024
100b088
Move config for external services to "services"
albireox Apr 8, 2024
3b0073d
Significant changes to calibrations scheduling
albireox Apr 9, 2024
a50c679
Remove websocket server to be replaced with Redis and lvmweb
albireox Apr 9, 2024
de76d3c
More removing of the websocket
albireox Apr 9, 2024
ee452ee
Initial work on overwatcher notifications and commands
albireox Apr 9, 2024
e1a925d
More pubsub
albireox Apr 9, 2024
856d77f
Merge branch 'main' into overwatcher
albireox Jul 9, 2024
91ff7f6
format overwatcher files
albireox Jul 9, 2024
d1b28e6
Improve redis-py typing
albireox Jul 9, 2024
b855ff3
Merge branch 'main' into overwatcher
albireox Jul 9, 2024
2dba3c5
Fix import order
albireox Jul 9, 2024
2b337ff
Style GORT
albireox Jul 9, 2024
078b926
Tweaks to TwilightFlats recipe
albireox Jul 9, 2024
4ac918d
Add back CLI overwatcher command
albireox Jul 9, 2024
2d26192
Add checking the LVM rain sensor
albireox Jul 10, 2024
14dbaf6
Initial warning if the overwatcher is not enabled
albireox Jul 10, 2024
3bf876b
Warn if enabled status changes
albireox Jul 10, 2024
dc48d7b
Improve messages
albireox Jul 10, 2024
395f375
Independently check wheather in start_observing()
albireox Jul 10, 2024
48a7e9e
Add open/close CLI commands
albireox Jul 10, 2024
de5ec14
Better grammar
albireox Jul 10, 2024
4a60209
Start flats 0.5 minutes after sunset
albireox Jul 10, 2024
a624cd2
Add more methods to ObserverStatus
albireox Jul 10, 2024
fc2741d
Merge branch 'main' into overwatcher
albireox Jul 27, 2024
b9ae4e9
Delete src/gort/pubsub.py
albireox Jul 27, 2024
5760318
Remove NotificationsMonitor task from NotificationsOverwatcher
albireox Jul 29, 2024
fdead21
Typo: cleanup_on_interrrupt -> cleanup_on_interrupt
albireox Jul 29, 2024
fc5891d
Set current GortObserve instance in Gort
albireox Jul 29, 2024
b3a6758
Add overwatcher_is_running function
albireox Jul 30, 2024
7b6dd8f
Add initial overwatcher actor
albireox Aug 5, 2024
94685c1
Add checks to prevent executing commands if Overwatcher is running
albireox Aug 5, 2024
08606cb
Use lvmapi Kubernetes routes
albireox Aug 5, 2024
e23b563
Update deps
albireox Aug 14, 2024
5e1c0fc
Add commands to enable/disable the overwatcher
albireox Aug 14, 2024
6222625
Return running status on overwatcher status command
albireox Aug 19, 2024
938fcac
Basic PubSub and notification scheme
albireox Aug 22, 2024
32ae0ea
Remove check_overwatcher_not_running
albireox Aug 22, 2024
163ef54
Add args and kwargs to recipe payload
albireox Aug 22, 2024
4f6aa9b
Add async/sync redis clients
albireox Aug 25, 2024
882e591
Events and notifications overwatcher modules
albireox Aug 25, 2024
fa0ad24
Refactor calibrations module
albireox Aug 25, 2024
3c897f9
Update deps
albireox Aug 25, 2024
46fa319
maskbits -> enums. Move ErrorCodes to enums and rename ErrorCode
albireox Aug 25, 2024
2228c34
Linting
albireox Aug 25, 2024
388aa83
Adjust twilight cals time ranges
albireox Aug 26, 2024
ef6e5be
Add command to reset calibrations
albireox Aug 26, 2024
23d2830
Add overwatcher dry-run mode
albireox Aug 27, 2024
327b34c
Move table names to the services section of the configuration file
albireox Aug 27, 2024
12be9f5
Hotfix to handle cases when exposures are not found after being taken
albireox Aug 27, 2024
6e434a0
Record exposure metadata to gortdb
albireox Aug 27, 2024
e560046
Make Exposure.write_to_db staticmethod
albireox Aug 27, 2024
f2a63c6
Simplify Exposure.write_to_db and remove non-JSON
albireox Aug 28, 2024
991da28
Await write_to_db with timeout
albireox Aug 28, 2024
052cbb3
Use SMJD for the DB MJD
albireox Aug 28, 2024
98a9f22
Report night, safe, and dry-run status in status command
albireox Aug 28, 2024
4b87859
Only use LVM rain sensor
albireox Aug 28, 2024
0f3b1d9
Fix checking weather risk
albireox Aug 28, 2024
977d1a1
Allow to set dry-mode in CLI
albireox Aug 28, 2024
a3e1c6a
Additional status attributes
albireox Aug 28, 2024
d0d218d
Fix cal not startng if not enable but dome at valid position
albireox Aug 28, 2024
4355997
Allow both after and min_start_time in calibration
albireox Aug 28, 2024
86094f1
Report close_dome_after
albireox Aug 28, 2024
817ea4f
Fix linting
albireox Aug 28, 2024
7e5979c
More handling min/max_start_time being exactly zero
albireox Aug 28, 2024
56d98c6
Prevent opening dome for cal in unsafe weather
albireox Aug 28, 2024
0a27391
Return calibration runnning if retrying
albireox Aug 28, 2024
7c99049
Add robustness to the PubSub library for IPython
albireox Sep 6, 2024
3733dad
Wrap sleep in try-except
albireox Sep 6, 2024
19d030a
Clean up pubsub reconnect code
albireox Sep 7, 2024
fa6c4c7
Add scheme for retries and delay in RemoteCommand
albireox Sep 7, 2024
6e9e2a5
Fix circular import
albireox Sep 7, 2024
57f707c
__current_retry -> _current_retry
albireox Sep 8, 2024
09a2b4d
Add Gort.restart_kubernetes_deployments method
albireox Sep 8, 2024
9e9dfdb
Migrate to using uv
albireox Sep 25, 2024
9c7c94a
Add observatory for tests
albireox Sep 25, 2024
88a8509
Disable overwatcher checks in emergency_close and shutdown
albireox Sep 26, 2024
0f6701f
Remove poetry lock
albireox Sep 26, 2024
64a6f8d
Fix issue disabling a tile
albireox Sep 26, 2024
e2c60ad
Remove unused parameter
albireox Sep 27, 2024
bbafbe2
Ensure that the kmirror task is awaited and errors propagated
albireox Sep 27, 2024
1e4018b
Deal with cases when the calibration is recorded as running but it is…
albireox Oct 2, 2024
9eff25c
Fix setting risk attribute
albireox Oct 2, 2024
5c14cff
Set calibrating state
albireox Oct 2, 2024
2e7ce43
Add warning
albireox Oct 2, 2024
5f7141a
Keep track of observer stages and reuse observer instance
albireox Oct 2, 2024
2dbcc4e
Improve enable/disable overwatcher logic
albireox Oct 2, 2024
a12706d
Fix CLI definition on pyproject.toml
albireox Oct 2, 2024
f8ca856
Focus telescope on first tile and every hour
albireox Oct 3, 2024
0982627
Add some delays to overwatcher modules
albireox Oct 3, 2024
86415fb
Improve observe loop cancellation and add observer stage events
albireox Oct 3, 2024
98aca6a
Use Gort.notify_event()
albireox Oct 3, 2024
d9f2ca0
Make sure dome is closed after calibration on error
albireox Oct 3, 2024
0da6448
Update deps
albireox Oct 3, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion .github/workflows/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,23 @@ name: Docker
on:
push:
branches:
- main
- '*'
- '*/*'
tags:
- '*'
paths-ignore:
- 'docs/**'

jobs:
docker:
name: Docker

runs-on: ubuntu-latest

env:
USER: sdss
APP: lvmgort

steps:
- name: Set docker tags
id: set-tags
Expand All @@ -29,19 +34,26 @@ jobs:
else
echo TAGS=$USER/$APP:${GITHUB_REF#refs/tags/} >> $GITHUB_OUTPUT
fi

- name: Show tags
run: echo ${{ steps.set-tags.outputs.TAGS }}

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Log in to registry
run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login ghcr.io -u $ --password-stdin

- name: Build and push
id: docker_build
uses: docker/build-push-action@v6
with:
push: true
provenance: false
tags: ghcr.io/${{ steps.set-tags.outputs.TAGS }}

- name: Image digest
run: echo ${{ steps.docker_build.outputs.digest }}
4 changes: 3 additions & 1 deletion .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ on:

jobs:
lint:
name: Lint

runs-on: ubuntu-latest

steps:
Expand All @@ -18,7 +20,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: 3.12
python-version: '3.12'
cache: 'pip'

- name: Lint with ruff
Expand Down
34 changes: 15 additions & 19 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,10 @@ on:
name: Create Release

jobs:
build:
name: Build wheels
runs-on: ubuntu-latest
release:
name: Build and publish

strategy:
matrix:
python-version: ['3.12']
runs-on: ubuntu-latest

permissions:
contents: write
Expand All @@ -23,26 +20,25 @@ jobs:
uses: actions/checkout@v4

- name: Create release
uses: softprops/action-gh-release@v2
uses: taiki-e/create-gh-release-action@v1
with:
title: GORT $version
changelog: CHANGELOG.md
token: ${{ secrets.GITHUB_TOKEN }}

- name: Install uv
uses: astral-sh/setup-uv@v3
with:
name: GORT ${{ github.ref_name }}
enable-cache: true

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
python -m pip install --upgrade pip setuptools wheel build

- name: Build wheels
run: |
pyproject-build -w
python-version: '3.12'

- name: Build source
- name: Build source and dist
run: |
pyproject-build -s
uv build

- name: Publish package distributions to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
21 changes: 13 additions & 8 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,9 @@ on:
- 'docs/**'

jobs:
build:
test:
name: Test

runs-on: ubuntu-latest

strategy:
Expand All @@ -18,7 +20,7 @@ jobs:
python-version: ['3.11', '3.12']

env:
OBSERVATORY: LCO
OBSERVATORY: 'LCO'

steps:
- uses: actions/checkout@v4
Expand All @@ -27,20 +29,23 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'

- name: Install uv
uses: astral-sh/setup-uv@v3
with:
enable-cache: true

- name: Install dependencies
run: |
pip install --upgrade wheel pip setuptools
pip install .
uv sync --no-dev

- name: Test with pytest
run: |
pip install pytest pytest-mock pytest-asyncio pytest-cov
pytest
uv pip install pytest pytest-mock pytest-asyncio pytest-cov
uv run pytest

# - name: Upload coverage to Codecov
# uses: codecov/codecov-action@v3
# uses: codecov/codecov-action@v4
# with:
# token: ${{ secrets.CODECOV_TOKEN }}
# file: ./coverage.xml
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
FROM python:3.11-slim-bookworm

MAINTAINER Jose Sanchez-Gallego, [email protected]

Check warning on line 3 in Dockerfile

View workflow job for this annotation

GitHub Actions / Docker

The MAINTAINER instruction is deprecated, use a label instead to define an image author

MaintainerDeprecated: Maintainer instruction is deprecated in favor of using label More info: https://docs.docker.com/go/dockerfile/rule/maintainer-deprecated/
LABEL org.opencontainers.image.source https://github.com/albireox/lvmgort

Check warning on line 4 in Dockerfile

View workflow job for this annotation

GitHub Actions / Docker

Legacy key/value format with whitespace separator should not be used

LegacyKeyValueFormat: "LABEL key=value" should be used instead of legacy "LABEL key value" format More info: https://docs.docker.com/go/dockerfile/rule/legacy-key-value-format/

WORKDIR /opt

Expand All @@ -11,4 +11,4 @@
RUN cd lvmgort && pip3 install .
RUN rm -Rf lvmgort

ENTRYPOINT lvmgort websocket start --debug
ENTRYPOINT lvmgort overwatcher

Check warning on line 14 in Dockerfile

View workflow job for this annotation

GitHub Actions / Docker

JSON arguments recommended for ENTRYPOINT/CMD to prevent unintended behavior related to OS signals

JSONArgsRecommended: JSON arguments recommended for ENTRYPOINT to prevent unintended behavior related to OS signals More info: https://docs.docker.com/go/dockerfile/rule/json-args-recommended/
Binary file added docs/sphinx/_static/gort_logo_slack.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
93 changes: 0 additions & 93 deletions docs/sphinx/calibrations.rst

This file was deleted.

1 change: 0 additions & 1 deletion docs/sphinx/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ Contents
observing
tiles
recipes
calibrations
configuration
troubleshooting
api
Expand Down
1 change: 0 additions & 1 deletion docs/sphinx/troubleshooting.rst
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,6 @@ If the deployment was not running you may see a message indicating that the depl
['local-path-provisioner',
'rabbitmq',
'lvmnps',
'gort-websocket',
'lvmieb',
'lvmtelemetry',
'restapi',
Expand Down
Loading