Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(framework) Move everything into a framework folder #4643

Draft
wants to merge 15 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
4 changes: 2 additions & 2 deletions .flake8
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[flake8]
max-line-length = 88
exclude = src/py/flwr/proto
exclude = framework/src/py/flwr/proto
ignore = E302,W503,E203
per-file-ignores =
src/py/flwr/server/strategy/*.py:E501
framework/src/py/flwr/server/strategy/*.py:E501
4 changes: 2 additions & 2 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@ README.md @jafermarq @tanertopal @danieljanes
/src/py/flwr/cli/new/templates @jafermarq @tanertopal @danieljanes

# Changelog
/doc/source/ref-changelog.md @jafermarq @tanertopal @danieljanes
/framework/docs/source/ref-changelog.md @jafermarq @tanertopal @danieljanes

# Translations
/doc/locales @charlesbvll @tanertopal @danieljanes
/framework/docs/locales @charlesbvll @tanertopal @danieljanes

# GitHub Actions and Workflows
/.github/workflows @Robert-Steiner @tanertopal @danieljanes
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/android-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
publish:
defaults:
run:
working-directory: src/kotlin
working-directory: framework/src/kotlin
name: Release build and publish
if: github.repository == 'adap/flower'
runs-on: ubuntu-22.04
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/baselines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
FILTER+=$(echo "$DIR: ${BASELINES_PATH}/**\n")
done < <(find baselines -maxdepth 1 \
-name ".*" -prune -o \
-path "baselines/doc" -prune -o \
-path "baselines/docs" -prune -o \
-path "baselines/dev" -prune -o \
-path "baselines/baseline_template" -prune -o \
-path "baselines/flwr_baselines" -prune -o \
Expand Down Expand Up @@ -82,8 +82,8 @@ jobs:

- name: Testing ${{ matrix.baseline }}
working-directory: baselines
run: ./dev/test-baseline.sh ${{ matrix.baseline }}
run: ./framework/dev/test-baseline.sh ${{ matrix.baseline }}

- name: Test Structure of ${{ matrix.baseline }}
working-directory: baselines
run: ./dev/test-baseline-structure.sh ${{ matrix.baseline }}
run: ./framework/dev/test-baseline-structure.sh ${{ matrix.baseline }}
2 changes: 1 addition & 1 deletion .github/workflows/cache-cleanup.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
gh extension install actions/gh-actions-cache

REPO=${{ github.repository }}
LATEST_KEY=pythonloc-${{ matrix.directory }}-${{ env.pythonLocation }}-${{ hashFiles(format('./e2e/{0}/pyproject.toml', matrix.directory)) }}
LATEST_KEY=pythonloc-${{ matrix.directory }}-${{ env.pythonLocation }}-${{ hashFiles(format('./framework/e2e/{0}/pyproject.toml', matrix.directory)) }}

echo "Fetching list of cache keys"
cacheKeys=$(gh actions-cache list -R $REPO | grep "${{ matrix.directory }}" | cut -f 1 )
Expand Down
14 changes: 7 additions & 7 deletions .github/workflows/cpp.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@ name: C++ SDK
on:
push:
branches: ['main']
paths: ['src/cc/flwr/**']
paths: ['framework/src/cc/flwr/**']
pull_request:
branches: ['main']
paths: ['src/cc/flwr/**']
paths: ['framework/src/cc/flwr/**']

jobs:
build_and_test:
Expand Down Expand Up @@ -37,30 +37,30 @@ jobs:

- name: Check source Formatting
run: |
find src/cc/flwr/src -name '*.cc' | xargs clang-format -i
find framework/src/cc/flwr/src -name '*.cc' | xargs clang-format -i
git diff --exit-code

- name: Check header Formatting
run: |
find src/cc/flwr/include -name '*.h' -not -path "src/cc/flwr/include/flwr/*" | xargs clang-format -i
find framework/src/cc/flwr/include -name '*.h' -not -path "framework/src/cc/flwr/include/flwr/*" | xargs clang-format -i
git diff --exit-code

- name: Build
run: |
mkdir -p build
cd build
cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ../src/cc/flwr
cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ../framework/src/cc/flwr
make

- name: Run clang-tidy
run: |
cd build
find ../src/cc/flwr/src -name '*.cc' | xargs clang-tidy
find ../framework/src/cc/flwr/src -name '*.cc' | xargs clang-tidy

- name: Run cppcheck
run: |
cd build
cppcheck --enable=all -I../src/cc/flwr/include ../src/cc/flwr/src
cppcheck --enable=all -I../framework/src/cc/flwr/include ../framework/src/cc/flwr/src

- name: End-to-end test
run: |
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/deprecated_baselines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,4 +33,4 @@ jobs:
run: |
python -m poetry install
- name: Lint + Test (isort/black/mypy/pylint/pytest)
run: ./dev/test.sh
run: ./framework/dev/test.sh
4 changes: 2 additions & 2 deletions .github/workflows/devtools.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,12 @@ on:
branches:
- main
paths:
- "src/py/flwr_tool/**"
- "framework/src/py/flwr_tool/**"
pull_request:
branches:
- main
paths:
- "src/py/flwr_tool/**"
- "framework/src/py/flwr_tool/**"

concurrency:
group: ${{ github.workflow }}-${{ github.ref == 'refs/heads/main' && github.run_id || github.event.pull_request.number || github.ref }}
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/docker-readme.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ on:
branches:
- 'main'
paths:
- 'src/docker/**/README.md'
- 'framework/src/docker/**/README.md'

jobs:
collect:
Expand All @@ -24,7 +24,7 @@ jobs:
list-files: "json"
filters: |
readme:
- added|modified: 'src/docker/**/README.md'
- added|modified: 'framework/src/docker/**/README.md'

update:
if: ${{ needs.collect.outputs.readme_files != '' && toJson(fromJson(needs.collect.outputs.readme_files)) != '[]' }}
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
with:
fetch-depth: 0
- name: Check copyright line
run: ./dev/test-copyright.sh
run: ./framework/dev/test-copyright.sh
- name: Bootstrap
uses: ./.github/actions/bootstrap
- name: Install pandoc
Expand All @@ -36,7 +36,7 @@ jobs:
cd datasets
python -m poetry install
- name: Build docs
run: ./dev/build-docs.sh ${{ github.ref == 'refs/heads/main' && github.repository == 'adap/flower' && !github.event.pull_request.head.repo.fork }}
run: ./framework/dev/build-docs.sh ${{ github.ref == 'refs/heads/main' && github.repository == 'adap/flower' && !github.event.pull_request.head.repo.fork }}
- name: Deploy docs
if: ${{ github.ref == 'refs/heads/main' && github.repository == 'adap/flower' && !github.event.pull_request.head.repo.fork }}
env:
Expand All @@ -45,7 +45,7 @@ jobs:
AWS_SECRET_ACCESS_KEY: ${{ secrets. AWS_SECRET_ACCESS_KEY }}
DOCS_BUCKET: flower.ai
run: |
aws s3 sync --delete --exclude ".*" --exclude "v/*" --cache-control "no-cache" ./doc/build/html/ s3://${{ env.DOCS_BUCKET }}/docs/framework
aws s3 sync --delete --exclude ".*" --exclude "v/*" --cache-control "no-cache" ./framework/docs/build/html/ s3://${{ env.DOCS_BUCKET }}/docs/framework
aws s3 sync --delete --exclude ".*" --exclude "v/*" --cache-control "no-cache" ./baselines/doc/build/html/ s3://${{ env.DOCS_BUCKET }}/docs/baselines
aws s3 sync --delete --exclude ".*" --exclude "v/*" --cache-control "no-cache" ./examples/doc/build/html/ s3://${{ env.DOCS_BUCKET }}/docs/examples
aws s3 sync --delete --exclude ".*" --exclude "v/*" --cache-control "no-cache" ./datasets/doc/build/html/ s3://${{ env.DOCS_BUCKET }}/docs/datasets
20 changes: 10 additions & 10 deletions .github/workflows/e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,9 @@ jobs:
- name: Install dependencies (mandatory only)
run: python -m poetry install
- name: Build wheel
run: ./dev/build.sh
run: ./framework/dev/build.sh
- name: Test wheel
run: ./dev/test-wheel.sh
run: ./framework/dev/test-wheel.sh
- name: Upload wheel
if: ${{ github.repository == 'adap/flower' && !github.event.pull_request.head.repo.fork && github.actor != 'dependabot[bot]' }}
id: upload
Expand Down Expand Up @@ -74,7 +74,7 @@ jobs:
${{ matrix.engine }}
defaults:
run:
working-directory: e2e/${{ matrix.directory }}
working-directory: framework/e2e/${{ matrix.directory }}
steps:
- uses: actions/checkout@v4
- name: Bootstrap
Expand Down Expand Up @@ -106,7 +106,7 @@ jobs:
${{ matrix.connection }} /
${{ matrix.authentication }} /
${{ matrix.engine }}
working-directory: e2e/${{ matrix.directory }}
working-directory: framework/e2e/${{ matrix.directory }}
run: ./../test_exec_api.sh "${{ matrix.connection }}" "${{ matrix.authentication}}" "${{ matrix.engine }}"

frameworks:
Expand Down Expand Up @@ -178,7 +178,7 @@ jobs:

defaults:
run:
working-directory: e2e/${{ matrix.directory }}
working-directory: framework/e2e/${{ matrix.directory }}

steps:
- uses: actions/checkout@v4
Expand All @@ -197,7 +197,7 @@ jobs:
uses: actions/cache/restore@v4
with:
path: ${{ env.pythonLocation }}
key: pythonloc-${{ runner.os }}-${{ matrix.directory }}-${{ env.pythonLocation }}-${{ hashFiles(format('./e2e/{0}/pyproject.toml', matrix.directory)) }}
key: pythonloc-${{ runner.os }}-${{ matrix.directory }}-${{ env.pythonLocation }}-${{ hashFiles(format('./framework/e2e/{0}/pyproject.toml', matrix.directory)) }}
- name: Install dependencies
run: python -m pip install --upgrade .
- name: Install Flower wheel from artifact store
Expand All @@ -209,7 +209,7 @@ jobs:
run: python -c "${{ matrix.dataset }}"
- name: Run edge client test
if: ${{ matrix.directory != 'e2e-bare-auth' }}
working-directory: e2e/${{ matrix.directory }}/${{ matrix.e2e }}
working-directory: framework/e2e/${{ matrix.directory }}/${{ matrix.e2e }}
run: ./../../test_legacy.sh "${{ matrix.directory }}"
- name: Run virtual client test
if: ${{ matrix.directory != 'e2e-bare-auth' }}
Expand Down Expand Up @@ -252,7 +252,7 @@ jobs:

defaults:
run:
working-directory: e2e/strategies
working-directory: framework/e2e/strategies

steps:
- uses: actions/checkout@v4
Expand Down Expand Up @@ -369,7 +369,7 @@ jobs:
${{ matrix.engine }}
defaults:
run:
working-directory: e2e/${{ matrix.directory }}
working-directory: framework/e2e/${{ matrix.directory }}
steps:
- uses: actions/checkout@v4
- name: Bootstrap
Expand Down Expand Up @@ -401,5 +401,5 @@ jobs:
${{ matrix.connection }} /
${{ matrix.authentication }} /
${{ matrix.engine }}
working-directory: e2e/${{ matrix.directory }}
working-directory: framework/e2e/${{ matrix.directory }}
run: ./../test_exec_api.sh "${{ matrix.connection }}" "${{ matrix.authentication}}" "${{ matrix.engine }}"
4 changes: 2 additions & 2 deletions .github/workflows/flower-swift_sync.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: Sync flower-swift
on:
push:
branches: ['main']
paths: ['src/swift/flwr/**']
paths: ['framework/src/swift/flwr/**']

concurrency:
group: ${{ github.workflow }}-${{ github.ref == 'refs/heads/main' && github.run_id || github.event.pull_request.number || github.ref }}
Expand All @@ -20,6 +20,6 @@ jobs:
env:
SSH_DEPLOY_KEY: ${{ secrets.FLOWER_SWIFT_SSH }}
with:
source-directory: 'src/swift/flwr'
source-directory: 'framework/src/swift/flwr'
destination-github-username: 'adap'
destination-repository-name: 'flower-swift'
2 changes: 1 addition & 1 deletion .github/workflows/framework-draft-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ jobs:

- name: Generate body
run: |
./dev/get-latest-changelog.sh > body.md
./framework/dev/get-latest-changelog.sh > body.md
cat body.md

- name: Release
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/framework-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ jobs:

- id: matrix
run: |
python dev/build-docker-image-matrix.py --flwr-version "${{ needs.publish.outputs.flwr-version }}" --matrix stable > matrix.json
python framework/dev/build-docker-image-matrix.py --flwr-version "${{ needs.publish.outputs.flwr-version }}" --matrix stable > matrix.json
echo "matrix=$(cat matrix.json)" >> $GITHUB_OUTPUT

build-base-images:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/framework.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,6 @@ jobs:
- name: Install dependencies (mandatory only)
run: python -m poetry install --all-extras
- name: Check if protos need recompilation
run: ./dev/check-protos.sh
run: ./framework/dev/check-protos.sh
- name: Lint + Test (isort/black/docformatter/mypy/pylint/flake8/pytest)
run: ./dev/test.sh
run: ./framework/dev/test.sh
2 changes: 1 addition & 1 deletion .github/workflows/pr_check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,4 @@ jobs:
python-version: 3.11
poetry-skip: 'true'
- name: Check PR title format
run: python ./dev/check_pr_title.py "${{ github.event.pull_request.title }}"
run: python ./framework/dev/check_pr_title.py "${{ github.event.pull_request.title }}"
4 changes: 2 additions & 2 deletions .github/workflows/release-nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
env:
PYPI_TOKEN: ${{ secrets.PYPI_TOKEN }}
run: |
RESULT=$(./dev/publish-nightly.sh)
RESULT=$(./framework/dev/publish-nightly.sh)
if [ "$RESULT" == "There were no commits in the last 24 hours." ]; then
echo "skip=true" >> $GITHUB_OUTPUT
fi
Expand All @@ -37,7 +37,7 @@ jobs:

NAME=$(poetry version | awk {'print $1'})
VERSION=$(poetry version -s)
python dev/build-docker-image-matrix.py --flwr-version "${VERSION}" --matrix nightly --flwr-package "${NAME}" > matrix.json
python framework/dev/build-docker-image-matrix.py --flwr-version "${VERSION}" --matrix nightly --flwr-package "${NAME}" > matrix.json
echo "matrix=$(cat matrix.json)" >> $GITHUB_OUTPUT

build-docker-base-images:
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/swift.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@ name: Swift
on:
push:
branches: ['main']
paths: ['src/swift/**']
paths: ['framework/src/swift/**']
pull_request:
branches: ['main']
paths: ['src/swift/**']
paths: ['framework/src/swift/**']

concurrency:
group: ${{ github.workflow }}-${{ github.ref == 'refs/heads/main' && github.run_id || github.event.pull_request.number || github.ref }}
Expand All @@ -16,7 +16,7 @@ jobs:
test:
defaults:
run:
working-directory: src/swift/flwr
working-directory: framework/src/swift/flwr
name: Test
runs-on: macos-14
steps:
Expand All @@ -36,7 +36,7 @@ jobs:
swift-version: 5.10
- uses: actions/checkout@v4
- name: Build docs
run: ./dev/build-swift-api-ref.sh
run: ./framework/dev/build-swift-api-ref.sh

deploy_docs:
needs: "build_docs"
Expand All @@ -53,4 +53,4 @@ jobs:
AWS_DEFAULT_REGION: ${{ secrets. AWS_DEFAULT_REGION }}
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets. AWS_SECRET_ACCESS_KEY }}
run: ./dev/deploy-swift-docs.sh
run: ./framework/dev/deploy-swift-docs.sh
4 changes: 2 additions & 2 deletions .github/workflows/update_translations.yml
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ jobs:

- name: Update text and translations for all locales
run: |
cd doc
cd framework/docs
make update-text
for langDir in locales/*; do
if [ -d "$langDir" ]; then
Expand All @@ -52,7 +52,7 @@ jobs:
run: |
git config --local user.email "41898282+github-actions[bot]@users.noreply.github.com"
git config --local user.name "github-actions[bot]"
git add doc/locales
git add framework/docs/locales
git commit -m "Update text and language files"
continue-on-error: true

Expand Down
Loading