Skip to content

0.22.0

Latest
Latest
Compare
Choose a tag to compare
@KFilippopolitis KFilippopolitis released this 22 Jul 11:19
· 3 commits to master since this release
0.22.0
fcb3188
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.

Changelog

0.22.0 - 22/7/2024

General System Refactoring

  • Renamed NODE to WORKER and restructured WORKER packages (#477)
    • Worker is packaged by feature, including exareme2/flower packages.
    • Fixed version incompatibility issues between pre-commit config and poetry.
    • Resolved a bug in Kubernetes templates.
  • System tables are loaded from SQLite (#488)
  • Refactored Task Handlers (#476)
    • Restructured the tests to align with the project structure. (#479)
  • Refactored Algorithm Specifications (#480)
    • Specifications are now loaded via JSON files.
    • Added type specifications to algorithms in preparation for integrating Flower.

Algorithms

  • Dev/mip 897/add log transformation (#487)
    • Added PCA with transformations

Flower Integration

  • Integrated Flower Federated Learning Framework (#478)

    • New Controller for Flower Execution:
      • Introduced a controller with modules for managing Flower workflow and algorithm execution.
    • Added Flower-Compatible Algorithms:
      • Logistic Regression with MIP Data: Utilizes MIP data with parameter customization.
      • Logistic Regression with MNIST Data: Tailored for MNIST data operations.
    • Robust Process Management Module:
      • Enhanced process control with functions for sending signals, checking status, managing zombie processes, and process termination with retry capabilities.
      • The controller uses this module to initiate, monitor, and safely terminate Flower execution processes, ensuring better oversight of Flower's client and server components.
      • Lowered local workers in production tests due to increased resource usage by Flower and controller.
      • Implemented process garbage collection at the start of each Flower algorithm.

  • Error Handling: (#482)

  • Dev/mip 902/flower logging (#486)

    • Added a Flower logger similar to the worker.

  • Retry mechanism for Flower client-server connections. (#484)

  • Implemented DataFrame filter for Flower inputdata processing. (#485)

  • Test Data Loaded on GlobalWorker (#489)

    • Integrated Mipdb into GlobalWorker for dataset monitoring by Worker Landscape Aggregator (WLA).
    • Added test datasets, excluded from Exareme2 flow.

  • Inputdata: Split datasets to training datasets and validation datasets. (#490)

  • Updated deployment logic:

    • Kubernetes: Ensure GlobalWorker deployment even in single-worker setups.
    • Flower: Configure server to run on GlobalWorker.
Assets 2
Loading