release v0.1.0 (#23)

* add changelog + security

* Deployment (#10)

* add changelog + security (#9)

* added dockerfile

* fix bug on build

* change the wkdir to /data

* change the wkdir to /

* fix typo

* back to /node

* push to dockerhub instead

* deployment done

---------

Co-authored-by: Kenny P <[email protected]>

* use internal network dns

* Deployment (#13)

* change the wkdir to /data

* change the wkdir to /

* fix typo

* back to /node

* deploy 2 reputer and workers for topic 1

* random

* API Adapter with README ORA-2066 (#12)

* API Adapter with README

* move API adapter readme inside dir

* Config file/env var + factory adapters + minor fixes

* updating readme to tell you to copy the env file and fill it out before running docker-compose (#14)

* readme changes (#15)

* chore: fix typos README.md (#17)

Hello, I found some typos in README.md

"intentend" to "intended"

"be use" to "be used"

And there were also missing space in "InferenceEndpointis required if InferenceEntrypoint is defined.". This canged for "InferenceEndpoint is required if InferenceEntrypoint is defined."

Hope this helps.

Signed-off-by: osrm <[email protected]>

* update dependencies, do not print conf (#18)

* Cleanups (#16)

* cleanups

* efficient deployment updates

* fix bug

---------

Co-authored-by: Kenny <[email protected]>

* fix issues with docker-compose initialize

* readme fix

* forecast should return nodeValue

* Update dependencies + reputer nonce fix (#19)

* fix deprecated docker compose command

* string interpolation

* uniform backoff (#21)

* uniform backoff

* README update

* Update and rename start.docker to init.docker (#22)

* Update and rename start.docker to init.docker

on devnet, when running the command:

```
chmod +x start.docker
./start.docker
```

 the worker node won't register because it is not funded. The wallet for the worker node is being created and the node is attempting to register to the network in one go. However, for the user this will never work. What they will have to do is
1. run the command
2. kill the terminal
3. fund their wallet
4. rerun the chmod command

The fix proposed is:

1. run init.docker to generate address and then you can get the address funded manually
2. run docker compose --build separately to start your services

Signed-off-by: kush-alloralabs <[email protected]>

* Update README.md

Signed-off-by: kush-alloralabs <[email protected]>

* Update README.md

Signed-off-by: kush-alloralabs <[email protected]>

---------

Signed-off-by: kush-alloralabs <[email protected]>

* fix the conflicting account issues

* add pull request template

* correct the wkdir for source github actions

---------

Signed-off-by: osrm <[email protected]>
Signed-off-by: kush-alloralabs <[email protected]>
Co-authored-by: Kenny <[email protected]>
Co-authored-by: Kenny P <[email protected]>
Co-authored-by: Diego C <[email protected]>
Co-authored-by: Madison Williams <[email protected]>
Co-authored-by: osrm <[email protected]>
Co-authored-by: kush-alloralabs <[email protected]>

.env.example

@@ -1,4 +1 @@
-ALLORA_ACCOUNT_NAME=secret
-ALLORA_ACCOUNT_MNEMONIC=secret
-ALLORA_ACCOUNT_PASSPHRASE=secret
-ALLORA_NODE_RPC=https://localhost:26657
+ALLORA_OFFCHAIN_NODE_CONFIG_JSON='{"wallet":{"addressKeyName":"test-offchain","addressRestoreMnemonic":"surge verify input...","alloraHomeDir":"","gas":"1000000","gasAdjustment":1,"nodeRpc":"http://localhost:26657","maxRetries":3,"delay":1,"submitTx":false},"worker":[{"topicId":1,"inferenceEntrypointName":"api-worker-reputer","loopSeconds":5,"parameters":{"InferenceEndpoint":"http://localhost:8000/inference/{Token}","Token":"ETH"}}]}'

.github/pull_request_template.md

@@ -0,0 +1,45 @@
+<!-- < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < ☺
+v                               ✰  Thanks for creating a PR! ✰
+v    Before smashing the submit button please review the checkboxes.
+v    If a checkbox is n/a - please still include it but + a little note why
+v    If your PR doesn't close an issue, that's OK!  Just remove the Closes: #XXX line!
+☺ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >  -->
+
+Closes: #XXX
+
+## What is the purpose of the change
+
+> Add a description of the overall background and high level changes that this PR introduces
+
+*(E.g.: This pull request improves documentation of area A by adding ....)*
+
+## Testing and Verifying
+
+*(Please pick one of the following options)*
+
+This change is a trivial rework / code cleanup without any test coverage.
+
+*(or)*
+
+This change is already covered by existing tests, such as *(please describe tests)*.
+
+*(or)*
+
+This change added tests and can be verified as follows:
+
+*(example:)*
+  - *Added unit test that validates ...*
+  - *Added integration tests for end-to-end deployment with ...*
+  - *Extended integration test for ...*
+  - *Manually verified the change by ...*
+
+## Documentation and Release Note
+
+  - [ ] Does this pull request introduce a new feature or user-facing behavior changes?
+
+
+Where is the change documented?
+  - [ ] Specification (`x/{module}/README.md`)
+  - [ ] Allora documentation site `docs.allora.network` source code at: `https://github.com/allora-network/docs`
+  - [ ] Code comments?
+  - [ ] N/A

.github/workflows/build_push_docker_hub.yml

@@ -0,0 +1,50 @@
+# This workflow will build and push a new container image to Docker hub based on the tag
+name: Build and Push docker image to Docker Hub
+
+on:
+  push:
+    tags:
+      - 'v*'
+      - dev-latest
+
+concurrency:
+  group: ${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  build-push:
+    name: Build and Push docker image
+    runs-on: ubuntu-latest
+
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+
+      - name: Login to Docker Hub
+        uses: docker/login-action@v1
+        with:
+          username: ${{ secrets.DOCKER_HUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_PASSWORD }}
+
+      - name: Build, tag, and push image to Docker Hub
+        id: build-push-image
+        env:
+          DOCKERHUB_USERNAME: alloranetwork
+          DOCKERHUB_REPOSITORY: ${{github.event.repository.name}}
+        run: |
+          GIT_TAG="$(echo $GITHUB_REF| sed 's#refs/tags/##')"
+
+          IMAGE_TAG="${GITHUB_SHA:0:8}"
+          EXTRA_IMAGE_TAGS=$GIT_TAG
+
+          if [[ ${GIT_TAG} == v* ]]; then
+            EXTRA_IMAGE_TAGS="${EXTRA_IMAGE_TAGS};latest"
+          fi
+
+          docker build --pull -t $DOCKERHUB_USERNAME/$DOCKERHUB_REPOSITORY:$IMAGE_TAG .
+          docker push $DOCKERHUB_USERNAME/$DOCKERHUB_REPOSITORY:$IMAGE_TAG
+
+          for tag in $(echo $EXTRA_IMAGE_TAGS| tr ";" "\n"); do
+            docker tag $DOCKERHUB_USERNAME/$DOCKERHUB_REPOSITORY:$IMAGE_TAG $DOCKERHUB_USERNAME/$DOCKERHUB_REPOSITORY:$tag
+            docker push $DOCKERHUB_USERNAME/$DOCKERHUB_REPOSITORY:$tag
+          done

.github/workflows/build_push_ecr.yml

@@ -1,5 +1,4 @@
-# This workflow will build and push a new container image to Amazon ECR,
-# and then will deploy a new task definition to Amazon ECS which will be run by Fargate when a release is created
+# This workflow will build and push a new container image to private AWS ECR repo based on the tag
 name: Build and Push docker image to ECR
 
 on:
@@ -35,11 +34,12 @@ jobs:
         id: login-ecr
         uses: aws-actions/amazon-ecr-login@v1
 
-      - name: offchain node build, tag, and push image to Amazon ECR
-        id: build-push-image-node
+      - name: Source Build, tag, and push image to Amazon ECR
+        id: build-push-image-source
+        working-directory: ./adapter/api/source
         env:
           ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
-          ECR_REPOSITORY: ${{github.event.repository.name}}  # Naming convention: ECR registry name == GITHUB repo name
+          ECR_REPOSITORY: ${{github.event.repository.name}}-source
         run: |
           #! Due to we trigger on push.tags GITHUB_REF - is the tag name
           GIT_TAG="$(echo $GITHUB_REF| sed 's#refs/tags/##')"
@@ -53,8 +53,7 @@ jobs:
           fi
 
           # Build a docker container and push it to ECR so that it can be deployed to ECS.
-          docker build --pull -f Dockerfile_b7s \
-              -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
+          docker build --pull -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
           docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
 
           # Build and PUSH additional tags

.gitignore

@@ -25,3 +25,5 @@ go.work.sum
 
 # Personal data
 .env
+.allorad
+config.json

CONTRIBUTING.md

@@ -0,0 +1,31 @@
+Any contribution that you make to this repository will
+be under the Apache 2 License, as dictated by that
+[license](http://www.apache.org/licenses/LICENSE-2.0.html):
+
+~~~
+5. Submission of Contributions. Unless You explicitly state otherwise,
+   any Contribution intentionally submitted for inclusion in the Work
+   by You to the Licensor shall be under the terms and conditions of
+   this License, without any additional terms or conditions.
+   Notwithstanding the above, nothing herein shall supersede or modify
+   the terms of any separate license agreement you may have executed
+   with Licensor regarding such Contributions.
+~~~
+
+Contributors must sign-off each commit by adding a `Signed-off-by: ...`
+line to commit messages to certify that they have the right to submit
+the code they are contributing to the project according to the
+[Developer Certificate of Origin (DCO)](https://developercertificate.org/).
+
+Contributors are encouraged to enable our pre-commit hooks to facilitate
+teh passage of CI/CD-related checks in our repositories. To enable them,
+please run the following command in the root of the repository:
+
+```bash
+chmod +x .hooks/pre-commit
+git config core.hooksPath .hooks/pre-commit 
+```
+
+This will enable the pre-commit hooks for the repository and ensure that
+all commits are checked for compliance with our formatter and other
+checks before they are accepted into the repository.

Dockerfile

@@ -1,9 +1,13 @@
 FROM golang:alpine
 
-WORKDIR /app
+WORKDIR /node
 
-COPY . /app
+COPY go.mod go.sum ./
+
+RUN go mod download
+
+COPY . .
 
 RUN go build -o allora_offchain_node
 
-CMD ["go", "run", "allora_offchain_node"]
+CMD ["./allora_offchain_node"]

README.md

@@ -2,6 +2,31 @@
 
 Allora off-chain nodes publish inferences, forecasts, and losses informed by a configurable ground truth to the Allora chain.
 
+## How to run with docker
+1. Clone the repository
+2. Make sure to remove any .env file so it doesn't clash with the automated environment variables
+3. Copy config.example.json and populate with your variables
+
+```shell
+cp config.example.json config.json
+```
+4. Run
+
+```shell
+chmod +x init.docker
+./init.docker 
+```
+
+from the root diectory. This will:
+   - Automatically create allora keys for you. You will have to request for some tokens from faucet to be able to register your worker and stake your reputer. You can find your address in ./data/env_file
+   - Automatically export the needed variables from the account created to be used by the offchain node and bundles it with the your provided config.json and then pass them to the node as environemnt variable
+
+5. Run `docker compose up --build`. This will:
+   - Run the both the offchain node and the source services, communicating through endpoints attached to the internal dns
+
+Please note that the environment variable will be created as bumdle of your config.json and allora account secrets, please make sure to remove every sectrets before commiting to remote git repository
+
+
 ## How to run without docker
 
 1. Clone the repository
@@ -27,10 +52,20 @@ cp .env.example .env
 9. Run the following commands:
 
 ```shell
-chmod +x run
-./run
+chmod +x start.local
+./start.local
 ```
 
+
+## How to configure
+
+There are several ways to configure the node. In order of preference, you can do any of these: 
+* Set the `ALLORA_OFFCHAIN_NODE_CONFIG_JSON` env var with a configuration as a JSON string.
+* Set the `ALLORA_OFFCHAIN_NODE_CONFIG_FILE_PATH` env var pointing to a file, which contains configuration as JSON. An example if provided in `config.example.json`.
+
+Each option completely overwrites the other options.
+
+
 This is the entrypoint for the application that simply builds and runs the Go program.
 
 It spins off a distinct processes per role worker, reputer per topic configered in `config.json`.
@@ -46,7 +81,7 @@ It spins off a distinct processes per role worker, reputer per topic configered
    1. Get and set latest_open_worker_nonce_from_chain from the chain
    2. If latest_open_worker_nonce_from_chain does not exist or nil then continue to next loop
       1. i.e. wait another config.loop_seconds
-   3. Retry request_retries times with backoff:
+   3. Retry request_retries times with uniform backoff:
       1. Invoke configured `inferenceEntrypoint`, `forecastEntrypoint` for topic and get results
          1. Else, break this inner retry loop
       2. Attempt to commit inference and forecast bundle to the chain
@@ -68,7 +103,7 @@ It spins off a distinct processes per role worker, reputer per topic configered
    1. Get and set latest_open_reputer_nonce_from_chain from the chain
    2. If latest_open_reputer_nonce_from_chain does not exist or nil then continue to next loop
       1. i.e. wait another config.loop_seconds
-   3. Retry request_retries times with backoff:
+   3. Retry request_retries times with uniform backoff:
       1. Invoke configured `truthEntrypoint, lossEntrypoint` for topic and get results
          1. Else, break this inner retry loop
       2. Attempt to commit loss bundle to the chain

adapter/README.md

@@ -0,0 +1,12 @@
+# Adapters
+
+This directory contains adapters for configuring different options in the node.
+
+## Adding an adapter
+
+To add an adapter: 
+* Add a directory that corresponds to the type eg API, Postgres, etc 
+* `cd` into the directory and add another directory that corresponds to the package name.
+* You can also add your source (eg API server, Postgres db, etc) into this directory
+* Create a main.go file inside the package implementing the interface `lib.AlloraAdapter`.
+* add a case in the switch in `adapter_factory.go`.

adapter/api/source/Dockerfile

@@ -0,0 +1,15 @@
+FROM python:3.9-slim
+
+RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /app
+
+COPY requirements.txt .
+
+RUN pip install --no-cache-dir -r requirements.txt
+
+COPY . .
+
+EXPOSE 8000
+
+CMD ["python", "main.py"]

adapter/api/source/README.md

@@ -0,0 +1,32 @@
+# Offchain API Source (Server)
+This is a small Python Flask API that can be run locally or within a Docker container. It's meant to demonstrate what your server will look like generating model inference and forecast to be called by the adapter
+
+### Prerequisites
+* Python 3.x
+* Flask
+* Docker (if you want to run the API in a Docker container)
+
+### Installing Dependencies
+Before running the API, you need to install the required dependencies.
+
+```bash
+pip install -r requirements.txt
+```
+
+### Running the API Locally
+To run the Flask API locally, use the following command:
+```bash
+python main.py
+```
+This will start the Flask development server on http://127.0.0.1:8000/
+
+### Running the API with Docker
+Alternatively, you can run the API using Docker.
+```bash
+docker build -t offchain-api-source .
+```
+Run the Docker container
+```bash
+docker run -p 8000:8000 offchain-api-source
+```
+This will start the Flask API server on http://localhost:8000/

adapter/api/source/main.py

@@ -0,0 +1,35 @@
+from flask import Flask, jsonify
+import random
+
+app = Flask(__name__)
+
+class NodeValue:
+    def __init__(self, worker, value):
+        self.worker = worker
+        self.value = value
+
+@app.route('/', methods=['GET'])
+def health():
+    return "Hello, World, I'm alive!"
+
+@app.route('/inference/<token>', methods=['GET'])
+def get_inference(token):
+    random_float = str(random.uniform(0.0, 100.0))
+    return random_float
+
+@app.route('/forecast', methods=['GET'])
+def get_forecast():
+    node_values = [
+        NodeValue("Worker1", str(random.uniform(0.0, 100.0))),
+        NodeValue("Worker2", str(random.uniform(0.0, 100.0))),
+        NodeValue("Worker3", str(random.uniform(0.0, 100.0))),
+    ]
+    return jsonify([nv.__dict__ for nv in node_values])
+
+@app.route('/truth/<token>/<blockheight>', methods=['GET'])
+def get_truth(token, blockheight):
+    random_float = str(random.uniform(0.0, 100.0))
+    return random_float
+
+if __name__ == '__main__':
+    app.run(debug=True, host='0.0.0.0', port=8000)

adapter/api/source/requirements.txt

@@ -0,0 +1 @@
+Flask