Skip to content

Releases: zenml-io/zenml

0.7.3

28 Apr 09:01
@schustmi schustmi
0.7.3
4d523f0
This commit was signed with the committer’s verified signature.
schustmi Michael Schuster
GPG key ID: C5884EDD066312AA
Verified
Learn about vigilant mode.
Compare
Choose a tag to compare
0.7.3

📊 Experiment Tracking Components

PR #530 adds a new stack component to ZenMLs ever-growing list: experiment_trackers allows users to configure your experiment tracking tools with ZenML. Examples of experiment tracking tools are Weights&Biases, mlflow, Neptune, amongst others.

Existing users might be confused, as ZenML has had MLflow and wandb support for a while now without such a component. However, this component allows uses more control over the configuration of MLflow and wandb with the new MLFlowExperimentTracker and
WandbExperimentTracker components. This allows these tools to work in more scenarios than the currently limiting local use-cases.

🔎 XGBoost and LightGBM support

XGBoost and LightGBM are one of the most widely used boosting algorithm libraries out there. This release adds materializers for native objects for each library.

Check out both examples here and PR's #544 and #538 for more details.

📂 Parameterized S3FS support to enable non-AWS S3 storage (minio, ceph)

A big complaint of the S3 Artifact Store integration was that it was hard to parameterize it in a way that it supports non-AWS S3 storage like minio and ceph. The latest release
made this super simple! When you want to register an S3ArtifactStore from the CLI, you can now pass in client_kwargs, config_kwargs or s3_additional_kwargs as a JSON string. For example:

zenml artifact-store register my_s3_store --type=s3 --path=s3://my_bucket \
    --client_kwargs='{"endpoint_url": "http://my-s3-endpoint"}'

See PR #532 for more details.

🧱 New CLI commands to update stack components

We added functionality to allow users to update stacks that already exist. This shows the basic workflow:

zenml orchestrator register local_orchestrator2 -t local
zenml stack update default -o local_orchestrator2
zenml stack describe default
zenml container-registry register local_registry --type=default --uri=localhost:5000
zenml container-registry update local --uri='somethingelse.com'
zenml container-registry rename local local2
zenml container-registry describe local2
zenml stack rename default new_default
zenml stack update new_default -c local2
zenml stack describe new_default
zenml stack remove-component -c

More details are in the CLI docs.
Users can add new stack components to a pre-existing stack, or they can modify
already-present stack components. They can also rename their stack and individual stack components.

🐛 Seldon Core authentication through ZenML secrets

The Seldon Core Model Deployer stack component was updated in this release to allow the configuration of ZenML secrets with credentials that authenticate Seldon to access the Artifact Store. The Seldon Core integration provides 3 different secret schemas for the 3 flavors of Artifact Store: AWS, GCP, and Azure, but custom secrets can be used as well. For more information on how to use this feature please refer to our Seldon Core deployment example.

Lastly, we had numerous other changes such as ensuring the PyTorch materializer works across all artifact stores
and the Kubeflow Metadata Store can be easily queried locally.

Detailed Changelog

New Contributors

Full Changelog: 0.7.2...0.7.3
Blog Post: https://blog.zenml.io/zero-seven-two-three-release/

Contributors

strickvl, stefannica, and 7 other contributors
Loading

0.7.2

14 Apr 14:44
@htahir1 htahir1
0.7.2
e882df3
Compare
Choose a tag to compare
0.7.2

0.7.2 is a minor release which quickly patches some bugs found in the last release to do with Seldon and Mlflow deployment.

This release also features initial versions of two amazing new community-led integrations: HuggingFace and Weights&Biases!

richard_socher_shoutout

Continuous model deployment with MLflow has been improved with ZenML 0.7.2. A new MLflow Model Deployer Stack component is now available and needs to be part of your stack to be able to deploy models:

zenml integration install mlflow
zenml model-deployer register mlflow --type=mlflow
zenml stack register local_with_mlflow -m default -a default -o default -d mlflow
zenml stack set local_with_mlflow

The MLflow Model Deployer is yet another addition to the list of Model Deployers available in ZenML. You can read more on deploying models to production with MLflow in our [Continuous Training and Deployment documentation section](https://docs.zenml.io/features/continuous training-and-deployment) and our MLflow deployment example.

What's Changed

New Contributors

  • @Ankur3107 made their first contribution to the HuggingFace integration in #467
  • @soumik12345 from Weights&Biases also made their first contribution to the wandb integration in #486!

Full Changelog: 0.7.1...0.7.2

Contributors

strickvl, stefannica, and 6 other contributors
Loading

0.7.1

11 Apr 19:11
@htahir1 htahir1
0.7.1
aac824b
Compare
Choose a tag to compare
0.7.1

0.7.1

Feast-Seldon

The release introduces the Seldon Core ZenML integration, featuring the Seldon Core Model Deployer and a Seldon Core standard model deployer step. The Model Deployer is a new type of stack component that enables you to develop continuous model deployment pipelines that train models and continuously deploy them to an external model serving tool, service or platform. You can read more on deploying models to production with Seldon Core in our Continuous Training and Deployment documentation section and our Seldon Core deployment example.

We also see two new integrations with Feast as ZenML's first feature store integration. Feature stores allow data teams to serve data via an offline store and an online low-latency store where data is kept in sync between the two. It also offers a centralized registry where features (and feature schemas) are stored for use within a team or wider organization. ZenML now supports connecting to a Redis-backed Feast feature store as a stack component integration. Check out the full example to see it in action!

0.7.1 also brings an addition to ZenML training library integrations with NeuralProphet. Check out the new example for more details, and the docs for more further detail on all new features!

What's Changed

Contributors

strickvl, stefannica, and 7 other contributors
Loading

0.7.0

28 Mar 14:44
@stefannica stefannica
0.7.0
5b9449a
Compare
Choose a tag to compare
0.7.0

With ZenML 0.7.0, a lot has been revamped under the hood about how things are stored. Importantly what this means is that ZenML now has system-wide profiles that let you register stacks to share across several of your projects! If you still want to manage your stacks for each project folder individually, profiles still let you do that as well.

Most projects of any complexity will require passwords or tokens to access data and infrastructure, and for this purpose ZenML 0.7.0 introduces the Secrets Manager stack component to seamlessly pass around these values to your steps. Our AWS integration also allows you to use AWS Secrets Manager as a backend to handle all your secret persistence needs.

Finally, in addition to the new AzureML and Sagemaker Step Operators that version 0.6.3 brought, this release also adds the ability to run individual steps on GCP's Vertex AI.

Beyond this, some smaller bugfixes and documentation changes combine to make ZenML 0.7.0 a more pleasant user experience.

What's Changed

New Contributors

Full Changelog: 0.6.3...0.7.0rc

Contributors

strickvl, stefannica, and 5 other contributors
Loading

0.6.3

14 Mar 14:20
@htahir1 htahir1
0.6.3
4117bca
Compare
Choose a tag to compare
0.6.3

0.6.3

ZenInTheClouds_3

With ZenML 0.6.3, you can now run your ZenML steps on Sagemaker and AzureML! It's normal to have certain steps that require specific hardware on which to run model training, for example, and this latest release gives you the power to switch out hardware for individual steps to support this.

We added a new Tensorboard visualisation that you can make use of when using our Kubeflow Pipelines integration. We handle the background processes needed to spin up this interactive web interface that you can use to visualise your model's performance over time.

Behind the scenes we gave our integration testing suite a massive upgrade, fixed a number of smaller bugs and made documentation updates. For a detailed look at what's changed, see below:

What's Changed

New Contributors

Contributors

strickvl, stefannica, and 8 other contributors
Loading

0.6.2

23 Feb 13:23
@stefannica stefannica
0.6.2
86d290e
Compare
Choose a tag to compare
0.6.2

ZenML0-6-2

ZenML 0.6.2 brings you the ability to serve models using MLflow deployments as well as an updated CLI interface! For a real continuous deployment cycle, we know that ZenML pipelines should be able to handle everything — from pre-processing to training to serving to monitoring and then potentially re-training and re-serving. The interfaces we created in this release are the foundation on which all of this will build.

We also improved how you interact with ZenML through the CLI. Everything looks so much smarter and readable now with the popular rich library integrated into our dependencies.

Smaller changes that you'll notice include updates to our cloud integrations and bug fixes for Windows users. For a detailed look at what's changed, see below.

What's Changed

Contributors

strickvl, stefannica, and 8 other contributors
Loading

0.6.1

07 Feb 14:41
@htahir1 htahir1
0.6.1
608b6cd
Compare
Choose a tag to compare
0.6.1

0.6.1

ZenML 0.6.1 is out and it's all about the cloud ☁️! We have improved AWS integration and a brand-new Azure integration! Run your pipelines on AWS and Azure now and let us know how it went on our Slack. To learn more, check out the new documentation page we just included to guide you in deploying your pipelines to AWS, GCP and/or Azure.

Smaller changes that you'll notice include much-awaited updates and fixes, including the first iterations of scheduling pipelines and tracking more reproducibility-relevant data in the metadata store.

For a detailed look at what's changed, see below.

What's Changed

Full Changelog: 0.6.0...0.6.1

Contributors

stefannica, htahir1, and 4 other contributors
Loading

0.6.0

26 Jan 15:39
@htahir1 htahir1
0.6.0
ad530d6
Compare
Choose a tag to compare
0.6.0

ZenML 0.6.0 is out now. We've made some big changes under the hood, but our biggest public-facing addition is our new integration to support all your data logging needs: whylogs. Our core architecture was thoroughly reworked and is now in a much better place to support our ongoing development needs.

Smaller changes that you'll notice include extensive documentation additions, updates and fixes. For a detailed look at what's changed, see below.

📊 Whylogs logging

Whylogs is an open source library that analyzes your data and creates statistical summaries called whylogs profiles. Whylogs profiles can be visualized locally or uploaded to the WhyLabs platform where more comprehensive analysis can be carried out.

ZenML integrates seamlessly with Whylogs and WhyLabs. This example shows how easy it is to enhance steps in an existing ML pipeline with Whylogs profiling features. Changes to the user code are minimal while ZenML takes care of all aspects related to Whylogs session initialization, profile serialization, versioning and persistence and even uploading generated profiles to Whylabs.

Example of the visualizations you can make from Whylogs profiles

With our WhylogsVisualizer, as described in the associated example notes, you can visualize Whylogs profiles generated as part of a pipeline.

⛩ New Core Architecture

We implemented some fundamental changes to the core architecture to solve some of the issues we previously had and provide a more extensible design to support quicker implementations of different stack components and integrations. The main change was to refactor the Repository, Stack and StackComponent architectures. These changes had a pretty wide impact so involved changes in many files throughout the codebase, especially in the CLI which makes calls to all these pieces.

We've already seen how it helps us move faster in building integrations and we hope it helps making contributions as pain-free as possible!

🗒 Documentation and Example Updates

As the codebase and functionality of ZenML grows, we always want to make sure our documentation is clear, up-to-date and easy to use. We made a number of changes in this release that will improve your experience in this regard:

  • added a number of new explainers on key ZenML concepts and how to use them in your code, notably on how to create a custom materializer and how to fetch historic pipeline runs using the StepContext
  • fixed a number of typos and broken links
  • added versioning to our API documentation so you can choose to view the reference appropriate to the version that you're using. We now use mkdocs for this so you'll notice a slight visual refresh as well.
  • added new examples highlighting specific use cases and integrations:
    • how to create a custom materializer (example)
    • how to fetch historical pipeline runs (example)
    • how to use standard interfaces for common ML patterns (example)
    • whylogs logging (example)

➕ Other updates, additions and fixes

As with most releases, we made a number of small but significant fixes and additions. The most import of these were that you can now access the metadata store via the step context. This enables a number of new possible workflows and pipeline patterns and we're really excited to have this in the release.

We added in a markdown parser for the zenml example info … command, so now when you want to use our CLI to learn more about specific examples you will see beautifully parsed text and not markdown markup.

We improved a few of our error messages, too, like for when the return type of a step function doesn’t match the expected type, or if step is called twice. We hope this makes ZenML just that little bit easier to use.

New Contributors

[Full Changelog: https://github.com/zenml-io/zenml/compare/0.5.7...0.6.0]

Contributors

bhattbhuwan13 and wjayesh
Loading

0.5.7

13 Jan 14:00
@AlexejPenner AlexejPenner
0.5.7
e325f2a
Compare
Choose a tag to compare
0.5.7

0.5.7

ZenML 0.5.7 is here 💯 and it brings not one, but 🔥TWO🔥 brand new integrations 🚀! ZenML now support MLFlow for tracking pipelines as experiments and Evidently for detecting drift in your ML pipelines in production!

New Features

  • Introducing the MLFLow Tracking Integration, a first step towards our complete MLFlow Integration as described in the #115 poll. Full example found here.

Mlflow

evidently

Bugfixes

  • Prevent KFP install timeouts during stack up by @stefannica in #299
  • Prevent naming parameters same name as inputs/outputs to prevent kwargs-errors by @bcdurak in #300

What's Changed

New Contributors

Contributors

stefannica, htahir1, and 6 other contributors
Loading

0.5.6

23 Dec 17:07
@htahir1 htahir1
0.5.6
90e0413
Compare
Choose a tag to compare
0.5.6 
    )                    *      (     
 ( /(                  (  `     )\ )  
 )\())    (            )\))(   (()/(  
((_)\    ))\    (     ((_)()\   /(_)) 
 _((_)  /((_)   )\ )  (_()((_) (_))   
|_  /  (_))    _(_/(  |  \/  | | |    
 / /   / -_)  | ' \)) | |\/| | | |__  
/___|  \___|  |_||_|  |_|  |_| |____|

This release fixes some known bugs from previous releases and especially 0.5.5. Therefore, upgrading to 0.5.6 is a breaking change. You must do the following in order to proceed with this version:

cd zenml_enabled_repo
rm -rf .zen/

And then start again with ZenML init:

pip install --upgrade zenml
zenml init

New Features

  • Added zenml example run [EXAMPLE_RUN_NAME] feature: The ability to run an example with one command. In order to run this, do zenml example pull first and see all examples available by running zenml example list.
  • Added ability to specify a .dockerignore file before running pipelines on Kubeflow.
  • Kubeflow Orchestrator is now leaner and faster.
  • Added the describe command group to the CLI for groups stack, orchestrator, artifact-store, and metadata-store. E.g. zenml stack describe

Bug fixes and minor improvements

  • Adding StepContext to a branch now invalidates caching by default. Disable explicitly with enable_cache=True.
  • Docs updated to reflect minor changes in CLI commands.
  • CLI list commands now mentions active component. Try zenml stack list to check this out.
  • zenml version now has cooler art.

What's Changed

Full Changelog: 0.5.5...0.5.6

Contributors

htahir1, schustmi, and 4 other contributors
Loading