Merge pull request #26 from gizatechxyz/feature/integrate-ezkl

Add EZKL as a Framework

.github/workflows/onpush.yml

@@ -30,4 +30,4 @@ jobs:
           poetry run ruff giza
       - name: Testing
         run: |
-          poetry run pytest --cov=giza --cov-report term-missing --cov-fail-under=80
+          poetry run pytest --cov=giza --cov-report term-missing --cov-fail-under=65

docs/SUMMARY.md

@@ -4,22 +4,32 @@
 
 * [Giza CLI](README.md)
 * [Installation](welcome/Installation.md)
-
-## 📖 Examples
-
-* [Basic CLI Example](examples/basic.md)
-* [Full Transpilation](examples/full_transpilation.md)
-* [Create a Proof](examples/create_proof.md)
-* [Reset Your Password](examples/reset_password.md)
+* [Frameworks](welcome/frameworks.md)
 
 ## 📦 Resources
 
 * [Users](resources/users.md)
-* [Transpile](resources/transpile.md)
-* [Prove](resources/prove.md)
 * [Models](resources/models.md)
 * [Versions](resources/versions.md)
 
+## 📚 Frameworks
+
+* [Cairo](frameworks/cairo.md)
+  * [Transpile](frameworks/cairo/transpile.md)
+  * [Prove](frameworks/cairo/prove.md)
+
+* [EZKL](frameworks/ezkl.md)
+  * [Transpile](frameworks/ezkl/transpile.md)
+  * [Prove](frameworks/ezkl/prove.md)
+  * [Verify](frameworks/ezkl/verify.md)
+
+## 📖 Examples
+
+* [Basic Cairo CLI Example](examples/basic.md)
+* [Full Cairo Transpilation](examples/full_transpilation.md)
+* [Create a Cairo Proof](examples/create_proof.md)
+* [Reset Your Password](examples/reset_password.md)
+
 ## 🧱 Reference
 
 * [API Overview](reference/api/README.md)

docs/frameworks/cairo.md

@@ -0,0 +1,15 @@
+# Cairo
+
+Cairo is a programming language specifically designed for creating verifiable and efficient smart contracts and cryptographic proofs. Its core strength lies in its ability to create succinct and secure programs while ensuring formal verification of their correctness.
+
+In the realm of Giza, Cairo serves as the backbone for generating provable machine learning models. Leveraging Cairo as the backend provides a reliable infrastructure to transpile, prove and verify these models within the Giza platform.
+
+## Cairo Framework Features
+
+1. Transpilation Process:
+Dive into Giza's streamlined process to seamlessly convert ONNX machine learning models into Cairo code. This transformation ensures optimal compatibility and functionality within the Cairo ecosystem, leveraging the power of [✨Orion✨](https://github.com/gizatechxyz/orion). More on this in the [transpile documentation for cairo](./cairo/transpile.md).
+
+2. Creating Verifiable Proofs:
+Use the transpile model to generate the `casm.json` file to create a proof and validate the correctness and reliability of your transformed models. Discover how Giza harnesses Cairo's robust features to create evidence ensuring the credibility of your machine learning outputs. More on this in the [prove documentation for cairo](./cairo/prove.md).
+
+[Cairo official documentation](https://docs.cairo-lang.org/)

docs/resources/prove.md → docs/frameworks/cairo/prove.md

@@ -4,7 +4,7 @@ Time to create a Proof of our Cairo program!
 
 Once we have our `json` we can use the `prove` command to create our proof!
 
-Basically this command will create a proving job at Giza and once the job completes we will download the created proof.
+Basically, this command will create a proving job at Giza and once the job is completed we will download the created proof.
 
 ```console
 > giza prove orion_runner.casm.json --size M
@@ -15,4 +15,4 @@ Basically this command will create a proving job at Giza and once the job comple
 [giza][2023-08-04 11:37:43.007] Proof saved at: zk.proof
 ```
 
-As we can see the job takes some time, because of this we are actively waiting for the job to be completed, once it's done all information and the prove is retrieved.
+As we can see the job takes some time, and because of this we are actively waiting for the job to be completed, once it's done all information and the proof are retrieved.

docs/resources/transpile.md → docs/frameworks/cairo/transpile.md

@@ -47,16 +47,16 @@ There are three main methods for transpiling a model:
 
 ### **Method 1: Using the `giza transpile` command**
 
-- This is the simplest method and is recommended for most users. 
-- When you run this command, Giza handles everything for you. 
+- This is the simplest method and is recommended for most users.
+- When you run this command, Giza handles everything for you.
 - It first checks if a model with the specified name already exists. If not, it creates a new model and then transpiles it. 
 - The output of this process is saved in the `cairo_model/` folder by default, but you can specify a different output path using the `--output-path` option.
 - This is the strategy that we followed in the example before.
 
 ### **Method 2: Manually creating a model and then transpiling it**
 
-- This method gives you more control over the process. 
-- First, you create a model manually using the `giza models create` command. 
+- This method gives you more control over the process.
+- First, you create a model manually using the `giza models create` command.
 - After the model is created, you can transpile it using the `giza transpile --model-id ...` or `giza versions transpile --model-id` command.
 - This method is useful when you want to specify particular options or parameters during the model creation and transpilation process.
 

docs/frameworks/ezkl.md

@@ -0,0 +1,21 @@
+# EZKL
+
+`ezkl` is a library and command-line tool for doing inference for deep learning models and other computational graphs in a zk-snark (ZKML) using `halo2` as a backend. [Official documentation](https://docs.ezkl.xyz/).
+
+`ezkl` represents a pioneering framework in zero-knowledge machine learning (ZKML) trusted by Giza. It is designed with ease of use in mind and efficiency to make easy zero-knowledge inference.
+
+## EZKL Framework Features
+
+1. Transpilation Process
+
+The transpilation process refers to performing the `setup` for the model where an ONNX model and a set of input data are used to generate the `circuit settings`, `proving key` and `verification key` files that are used to perform the inference. In Giza we aim to provide an easy way to perform this step providing compute resources and a simple command line interface. More on this in the [transpile documentation for ezkl](./ezkl/transpile.md).
+
+1. Creating Verifiable Proofs
+
+Create a proof using the generated outputs of the setup process, we handle this step for you by loading the necessary files and performing the proof generation for you. More on this in the [prove documentation for ezkl](./ezkl/prove.md).
+
+1. Verifying Proofs
+
+Verify the proof generated by an `ezkl` version, here we manage the execution of the verification process and the compute as well for you. More on this in the [verify documentation for ezkl](./ezkl/verify.md)
+
+For detailed information check the [ezkl repository](https://github.com/zkonduit/ezkl)

docs/frameworks/ezkl/prove.md

@@ -0,0 +1,22 @@
+# Prove
+
+Time to create a Proof of our EZKL version!
+
+Once we have the input data that we want to use we can use the `prove` command to create our proof!
+
+This command will create a proving job at Giza and once the job is completed we will download the created proof.
+
+```console
+> giza prove --framework EZKL --model-id 1 --version-id 1 --size M input.json
+[giza][2023-12-04 19:41:55.236] Proving job created with name 'proof-ezkl-20231204-33e59441' and id -> 1 ✅
+[giza][2023-12-04 19:43:36.337] Proving job is successful ✅[giza][2023-12-04 19:43:36.447] Proof created with id -> 1 ✅
+[giza][2023-12-04 19:43:36.448] Proof metrics:
+{
+  "proving_time": 12.2238187789917
+}
+[giza][2023-12-04 19:43:37.046] Proof saved at: zk.proof
+```
+
+As we can see the job takes some time, and because of this we are actively waiting for the job to be completed, once it's done all information and the proof are retrieved.
+
+Now we can see that we have a proof successfully created! Now if we want we could verify it! Let's check the [verify](../frameworks/ezkl/verify.md) documentation to see how to do it!

docs/frameworks/ezkl/transpile.md

@@ -0,0 +1,113 @@
+# Transpile
+
+Transpilation is a crucial process in the deployment of Verifiable Machine Learning models and its powered by is powered by [ezkl](https://github.com/zkonduit/ezkl). It involves the usage of an ONNX model and input data to perform the `setup`. With this `setup` we can generate proofs that can be verified, ensuring the integrity and reliability of the model's predictions.
+
+The transpilation process begins by reading the model and input data from the specified path. The model and data are then sent for transpilation. By default, the output of this process is managed by Giza for later use.
+
+```console
+> giza transpile --framework EZKL --input-data input.json awesome_model.onnx
+[giza][2023-12-04 19:40:11.274] No model id provided, checking if model exists ✅ 
+[giza][2023-12-04 19:40:11.275] Model name is: awesome_model
+[giza][2023-12-04 19:40:11.424] Model Created with id -> 1! ✅ 
+[giza][2023-12-04 19:40:11.944] Sending model for setup ✅ 
+[giza][2023-12-04 19:40:13.046] Setup job created with name 'ezkl-job-20231204-32a4a1c1' and id -> 1 ✅
+[giza][2023-12-04 19:41:54.277] Setup job is successful ✅
+```
+
+The result of the transpilation process is the following:
+
+* Circuit settings
+* Proving key
+* Verification key
+
+## How do we transpile a model
+
+There are three main methods for transpiling a model:
+
+### **Method 1: Using the `giza transpile --framework EZKL` command**
+
+- This is the simplest method and is recommended for most users.
+- When you run this command, Giza handles everything for you.
+- It first checks if a model with the specified name already exists. If not, it creates a new model and then transpiles it.
+- The output of this process is managed by Giza.
+- This is the strategy that we followed in the example before.
+
+### **Method 2: Manually creating a model and then transpiling it**
+
+- This method gives you more control over the process.
+- First, you create a model manually using the `giza models create` command.
+- After the model is created, you can transpile it using the `giza transpile --framework EZKL --model-id ...` or `giza versions transpile --framework EZKL --model-id` command.
+- This method is useful when you want to specify particular options or parameters during the model creation and transpilation process.
+
+```console
+> giza models create --name awesome_model --description "A Model for testing different models"
+[giza][2023-09-13 14:04:59.532] Creating model ✅ 
+{
+  "id": 2,
+  "name": "awesome_model",
+  "description": "A Model for testing different models"
+}
+```
+
+```console
+> giza transpile --framework EZKL --model-id 2 --input-data input.json awesome_model.onnx
+[giza][2023-12-04 19:40:10.646] Model found with id -> 2! ✅
+[giza][2023-12-04 19:40:12.176] Sending model for Setup ✅ 
+[giza][2023-12-04 19:40:13.046] Setup job created with name 'ezkl-job-20231204-32a4a1c1' and id -> 13 ✅
+[giza][2023-12-04 19:41:54.277] Setup job is successful ✅
+```
+
+### **Method 3: Using a previous model**
+
+- If you have a previously created model, you can transpile it by indicating the model-id in the `giza transpile --framework EZKL --model-id ...` or `giza versions transpile --framework EZKL --model-id` command.
+- This method is useful when you want to create a new version of an existing model.
+- The output of the transpilation process is saved in the same location as the original model.
+
+```console
+# Using the previous model (id: 2) we can transpile a new model, which will create verion 2 of the model.
+> giza transpile --framework EZKL --model-id 2 --input-data input.json awesome_model.onnx
+[giza][2023-12-04 19:40:10.646] Model found with id -> 2! ✅
+[giza][2023-12-04 19:40:12.176] Sending model for Setup ✅ 
+[giza][2023-12-04 19:40:13.046] Setup job created with name 'ezkl-job-20231204-123fadc1' and id -> 14 ✅
+[giza][2023-12-04 19:41:54.277] Setup job is successful ✅
+```
+
+## What is happening with the models and versions
+
+In Giza, a model is essentially a container for versions. Each version represents a transpilation of a machine learning model at a specific point in time. This allows you to keep track of different versions of your model as it evolves and improves over time.
+
+To check the current models and versions that have been created, you can use the following steps:
+
+1. Use the `giza models list` command to list all the models that have been created.
+2. For each model, you can use the `giza versions list --model-id ...` command to list all the versions of that model.
+
+Remember, each version represents a specific transpilation of the model. So, if you have made changes to your machine learning model and transpiled it again, it will create a new version.
+
+This system of models and versions allows you to manage and keep track of the evolution of your machine learning models over time.
+
+For example, let's say you have created a model called `awesome_model` and transpiled it twice. This will create two versions of the model, version 1 and version 2. You can check the status of these versions using the `giza versions list --model-id ...` command.
+
+```console
+giza versions list --model-id 29
+[giza][2023-12-04 14:17:08.006] Listing versions for the model ✅ 
+[
+  {
+    "version": 1,
+    "size": 52735,
+    "status": "COMPLETED",
+    "message": "Setup Successful!",
+    "description": "Intial version",
+    "created_date": "2023-09-13T12:08:38.177605",
+    "last_update": "2023-09-13T12:08:43.986137"
+  },
+  {
+    "version": 2,
+    "size": 52735,
+    "status": "Setup",
+    "message": "Transpilation Successful!",
+    "description": "Intial version",
+    "created_date": "2023-12-04T12:11:30.165440",
+    "last_update": "2023-12-04T12:11:31.625834"
+  }
+]
+```

docs/frameworks/ezkl/verify.md

@@ -0,0 +1,15 @@
+# Verify
+
+We have created a Proof of our EZKL version, now it's time to verify it!
+
+As we already have the proof we can use the `verify` command to verify it, using the `--proof-id` option we can specify the proof that we want to verify.
+
+As we are using a managed `version` we need to specify `--model-id` and `--version-id` to indicate which version we want to verify, so we can retrieve the generated files in the `setup` process (`giza transpile`).
+
+```console
+> giza verify -f EZKL --model-id 1 --version-id 1 --proof-id 1 --size S
+[giza][2023-12-04 19:43:37.686] Verification job created with name 'verify-ezkl-20231204-32f44715' and id -> 1 ✅
+[giza][2023-12-04 19:45:18.683] Verification job is successful ✅
+```
+
+If the job is successful that means that the proof is valid! Otherwise, we will get an error message.

docs/reference/api/README.md

@@ -12,9 +12,13 @@
 - [`commands.prove`](./commands.prove.md#module-commandsprove)
 - [`commands.reset_password`](./commands.reset_password.md#module-commandsreset_password)
 - [`commands.users`](./commands.users.md#module-commandsusers)
+- [`commands.verify`](./commands.verify.md#module-commandsverify)
 - [`commands.version`](./commands.version.md#module-commandsversion)
 - [`commands.versions`](./commands.versions.md#module-commandsversions)
 - [`exceptions`](./exceptions.md#module-exceptions)
+- [`frameworks`](./frameworks.md#module-frameworks)
+- [`frameworks.cairo`](./frameworks.cairo.md#module-frameworkscairo)
+- [`frameworks.ezkl`](./frameworks.ezkl.md#module-frameworksezkl)
 - [`options`](./options.md#module-options)
 - [`utils`](./utils.md#module-utils)
 - [`utils.decorators`](./utils.decorators.md#module-utilsdecorators)
@@ -30,9 +34,12 @@
 - [`client.ProofsClient`](./client.md#class-proofsclient): Client to interact with `proofs` endpoint.
 - [`client.TranspileClient`](./client.md#class-transpileclient): Client to interact with `users` endpoint.
 - [`client.UsersClient`](./client.md#class-usersclient): Client to interact with `users` endpoint.
+- [`client.VersionJobsClient`](./client.md#class-versionjobsclient): Client to interact with `jobs` endpoint.
 - [`client.VersionsClient`](./client.md#class-versionsclient): Client to interact with `versions` endpoint.
 - [`exceptions.PasswordError`](./exceptions.md#class-passworderror)
 - [`echo.Echo`](./utils.echo.md#class-echo): Helper class to use when printin output of the CLI.
+- [`enums.Framework`](./utils.enums.md#class-framework)
+- [`enums.JobKind`](./utils.enums.md#class-jobkind)
 - [`enums.JobSize`](./utils.enums.md#class-jobsize)
 - [`enums.JobStatus`](./utils.enums.md#class-jobstatus)
 - [`enums.VersionStatus`](./utils.enums.md#class-versionstatus)
@@ -42,9 +49,10 @@
 - [`callbacks.debug_callback`](./callbacks.md#function-debug_callback): If a call adds the `--debug` flag debugging mode is activated for external requests and API Clients.
 - [`callbacks.version_callback`](./callbacks.md#function-version_callback): Prints the current version when `--version` flag is added to a call.
 - [`cli.entrypoint`](./cli.md#function-entrypoint)
+- [`models.create`](./commands.models.md#function-create): Command to create a model. Asks for the new model's information and validates the input,
 - [`models.get`](./commands.models.md#function-get): Command to create a user. Asks for the new users information and validates the input,
 - [`models.list`](./commands.models.md#function-list): Command to list all models.
-- [`prove.prove`](./commands.prove.md#function-prove): Command to prove as spceific cairo program, previously converted to CASM.
+- [`prove.prove`](./commands.prove.md#function-prove)
 - [`reset_password.handle_http_error`](./commands.reset_password.md#function-handle_http_error): Handle an HTTP error.
 - [`reset_password.prompt_for_input`](./commands.reset_password.md#function-prompt_for_input): Prompt the user for input.
 - [`reset_password.request_reset_password_token`](./commands.reset_password.md#function-request_reset_password_token): Request a password reset token for a given email.
@@ -53,11 +61,17 @@
 - [`users.login`](./commands.users.md#function-login): Logs the current user into Giza. Under the hood this will retrieve the token for the next requests.
 - [`users.me`](./commands.users.md#function-me): Retrieve information about the current user and print it as json to stdout.
 - [`users.resend_email`](./commands.users.md#function-resend_email): Command to resend verification email. Asks for the user's email and sends the request to the API
+- [`verify.verify`](./commands.verify.md#function-verify)
 - [`version.version_entrypoint`](./commands.version.md#function-version_entrypoint): Prints the current CLI version.
 - [`versions.download`](./commands.versions.md#function-download): Retrieve information about the current user and print it as json to stdout.
 - [`versions.get`](./commands.versions.md#function-get)
 - [`versions.list`](./commands.versions.md#function-list)
-- [`versions.transpile`](./commands.versions.md#function-transpile): This function is responsible for transpiling a model. The overall objective is to prepare a model for use by converting it into a different format (transpiling).
+- [`versions.transpile`](./commands.versions.md#function-transpile)
 - [`versions.update`](./commands.versions.md#function-update)
+- [`cairo.prove`](./frameworks.cairo.md#function-prove): Command to prove as spceific cairo program, previously converted to CASM.
+- [`cairo.transpile`](./frameworks.cairo.md#function-transpile): This function is responsible for transpiling a model. The overall objective is to prepare a model for use by converting it into a different format (transpiling).
+- [`ezkl.prove`](./frameworks.ezkl.md#function-prove)
+- [`ezkl.setup`](./frameworks.ezkl.md#function-setup): This function executes the setup of the model and creates the outputs, handled by Giza.
+- [`ezkl.verify`](./frameworks.ezkl.md#function-verify): Create a verification job.
 - [`utils.get_response_info`](./utils.md#function-get_response_info): Utility to retrieve information of the client response.
 - [`decorators.auth`](./utils.decorators.md#function-auth): Check that we have the token and it is not expired before executing

docs/reference/api/cli.md

@@ -13,7 +13,7 @@
 
 ---
 
-<a href="https://github.com/gizatechxyz/giza-cli/blob/main/giza/cli.py#L114"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="https://github.com/gizatechxyz/giza-cli/blob/main/giza/cli.py#L149"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
 ## <kbd>function</kbd> `entrypoint`