Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhacement/flexible handlers #13

Merged
merged 72 commits into from
May 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
fa7aa93
Add base default fields
oscgonfer Nov 21, 2023
63fb0cb
Modify device handling methods
oscgonfer Nov 22, 2023
992ecd8
Add new air quality blueprint
oscgonfer Nov 29, 2023
616a830
Test for new aq blueprint
oscgonfer Nov 29, 2023
eff0754
Improve alphasense processing and cleanup
oscgonfer Dec 2, 2023
8ccb2e4
Further updates in processing, connect handler
oscgonfer Dec 4, 2023
53f9d12
Small functions for printing management
oscgonfer Dec 4, 2023
5b323ef
Move names to different model
oscgonfer Dec 4, 2023
3e96e85
Fix names json
oscgonfer Dec 4, 2023
7e791e9
Fix names
oscgonfer Dec 18, 2023
7e117a3
Improvements on blueprint for consistency
oscgonfer Dec 18, 2023
9ce7a90
Update blueprint model
oscgonfer Jan 2, 2024
0212f6c
Move over to separate blueprints per source
oscgonfer Jan 2, 2024
c162812
Hardware test to api
oscgonfer Jan 2, 2024
3e4e3f8
Add water blueprint
oscgonfer Feb 12, 2024
aec14ec
Minor cosmetic fix
oscgonfer Feb 12, 2024
0f06593
Update gitignore
oscgonfer Feb 20, 2024
b508a9d
Add workflow for publish to pypi
oscgonfer Apr 7, 2024
448703e
Remove tasks
oscgonfer Apr 7, 2024
42fe99d
Move to logger
oscgonfer Apr 7, 2024
521a42a
Rename test/tools folder
oscgonfer Apr 7, 2024
3894985
Move device into single file
oscgonfer Apr 7, 2024
deab14b
Remove user api
oscgonfer Apr 7, 2024
add8817
Remove firmware names
oscgonfer Apr 7, 2024
46f8791
Add pydantic models
oscgonfer Apr 7, 2024
cdd52bb
Move test to file
oscgonfer Apr 7, 2024
5a04132
Remove NILU file
oscgonfer Apr 7, 2024
c241bac
Add find by field in utils and remove logs
oscgonfer Apr 7, 2024
dff12c8
Remove dedicated test load
oscgonfer Apr 7, 2024
3cb27ec
Missing logger transition
oscgonfer Apr 7, 2024
ce0507f
Add test checks and dispersion tools
oscgonfer Apr 7, 2024
bc3036b
Mostly remove api handlers
oscgonfer Apr 7, 2024
fa290ea
Rename file api
oscgonfer Apr 7, 2024
1f91c6f
Air and water blueprints
oscgonfer Apr 7, 2024
eeb800b
Models in init
oscgonfer Apr 7, 2024
552e4a6
Missing imports for device file
oscgonfer Apr 7, 2024
afdc2cd
Convert metrics and blueprints to models
oscgonfer Apr 7, 2024
7b79952
WIP config update
oscgonfer Apr 7, 2024
702ca9e
Rename utils to tools
oscgonfer Apr 7, 2024
95950bb
Add sc_air_file fixes
oscgonfer Apr 7, 2024
c5a3bcc
Remove old blueprints
oscgonfer Apr 7, 2024
31b5c4a
Remove source from blueprints
oscgonfer Apr 7, 2024
a5516d4
Update SCAS210099.json
oscgonfer Apr 9, 2024
3cf471b
Cleanup config and move dedicated params to specific files
oscgonfer Apr 29, 2024
6a3b9d3
Fix in config
oscgonfer Apr 30, 2024
e82c8c8
Add CSV handler
oscgonfer Apr 30, 2024
ebbacd3
Add options to test
oscgonfer Apr 30, 2024
3a6d68d
Missing device params
oscgonfer Apr 30, 2024
8c8d5ba
Logger information for device
oscgonfer Apr 30, 2024
0f97c58
Avoid paralelizing device loads
oscgonfer Apr 30, 2024
748021f
Update examples
oscgonfer May 3, 2024
c3a53c7
Remove connectors
oscgonfer May 3, 2024
3f7d54f
Allow any kind of param to be pased to device
oscgonfer May 3, 2024
4947a2b
Remove preprocess of sd-card (commented)
oscgonfer May 3, 2024
4debd70
Minor lint
oscgonfer May 3, 2024
9d0c3fa
WIP - file export
oscgonfer May 3, 2024
a19a8b5
WIP Zenodo upload
oscgonfer May 3, 2024
aa05dfc
Add MANIFEST tools
oscgonfer May 3, 2024
a987518
Make sure loaded is updated
oscgonfer May 8, 2024
d34a578
Avoid making a wrong process status if no postprocessing
oscgonfer May 8, 2024
1efcb79
Fix requirements and tests running issues
oscgonfer May 8, 2024
6cd010f
Relax matplotlib requirement, add test for latest version
oscgonfer May 9, 2024
a4a8a45
Fix iplot import and stats linregress
oscgonfer May 9, 2024
7ef82d0
Matplotlib plot styles fix and flask requirement
oscgonfer May 9, 2024
28000a4
Update pydantic version
oscgonfer May 9, 2024
a2806ca
Fix test tests
oscgonfer May 9, 2024
82a9335
Comment uplot test
oscgonfer May 9, 2024
c1e768e
Fix names for tests
oscgonfer May 9, 2024
c5aeeb9
Fix test names
oscgonfer May 9, 2024
f01f135
Update examples
oscgonfer May 9, 2024
c66753a
Remove makefile
oscgonfer May 9, 2024
15d672c
Bump to 1.0.0 version
oscgonfer May 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,11 @@ on:
branches: [ "master" ]

jobs:
build:
build-latest:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.8", "3.9"]
python-version: ["3.11"]
steps:
- uses: actions/checkout@v2
- name: Set up Python
Expand All @@ -37,31 +37,4 @@ jobs:
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
run: |
pytest
- name: Build package
run: |
python setup.py sdist bdist_wheel
- name: Archive artifacts
uses: actions/upload-artifact@v1
with:
name: scdata-pkg
path: dist
pypi-publish:
name: Upload release to PyPI
if: startsWith(github.ref, 'refs/tags/')
needs:
- build
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/scdata
permissions:
id-token: write
steps:
- name: Download all the dists
uses: actions/download-artifact@v3
with:
name: python-package-distributions
path: dist/
- name: Publish distribution 📦 to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
pytest tests/all
43 changes: 43 additions & 0 deletions .github/workflows/python-multiple-versions.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions

name: Python application

on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]

jobs:
build:
runs-on: "${{ matrix.os }}"
strategy:
fail-fast: true
matrix:
os: ["ubuntu-latest", "windows-latest", "macos-latest"]
python-version: ["3.9", "3.10", "3.11"]
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
cache: 'pip'
- name: Display Python version
run: python -c "import sys; print(sys.version)"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
pip install .
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
# flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
run: |
pytest tests/cross-version

39 changes: 39 additions & 0 deletions .github/workflows/python-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# This workflow will upload a Python Package using Twine when a release is created
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python#publishing-to-package-registries

# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.

name: Upload Python Package

on:
release:
types: [published]

permissions:
contents: read

jobs:
deploy:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v3
with:
python-version: '3.11'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install build
- name: Build package
run: python -m build
- name: Publish package
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
with:
user: __token__
password: ${{ secrets.PYPI_API_TOKEN }}
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -152,3 +152,9 @@ ENV/

# pytest
.pytest_cache/

tests/local

scripts/

.vscode
6 changes: 3 additions & 3 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@ include LICENSE
include setup.py
include setup.cfg
include scdata/test/export/templates/sc_template.html
include scdata/utils/interim/*.csv
include scdata/utils/zenodo_templates/*.*
include scdata/utils/uploads/*.*
include scdata/tools/interim/*.csv
include scdata/tools/zenodo_templates/*.*
include scdata/tools/uploads/*.*
20 changes: 0 additions & 20 deletions Makefile

This file was deleted.

13 changes: 0 additions & 13 deletions blueprints/base.json

This file was deleted.

74 changes: 0 additions & 74 deletions blueprints/csic_station.json

This file was deleted.

64 changes: 0 additions & 64 deletions blueprints/muv_station.json

This file was deleted.

66 changes: 0 additions & 66 deletions blueprints/nilu_lab.json

This file was deleted.

Loading
Loading