Deploying openyurt on top of kubernetes & knative

unit test with github workflows and test_configs for mock variables change template to yaml, add into subfolder configs add openyurt description to main readme Signed-off-by: Jason Chua <[email protected]>
vhive-serverless · Feb 5, 2024 · 850bdfe · 850bdfe
1 parent c57d8b6
commit 850bdfe
Show file tree

Hide file tree

Showing 49 changed files with 2,144 additions and 1,248 deletions.
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
@@ -7,7 +7,7 @@ on:
     - 'docs/**'
     - '**.md'
   pull_request:
-    branches: [ main, legacy-firecracker-v0.24.0-with-upf-support, openyurt ]
+    branches: [ main, legacy-firecracker-v0.24.0-with-upf-support ]
     paths-ignore:
     - 'docs/**'
     - '**.md'

diff --git a/.github/workflows/build_setup.yml b/.github/workflows/build_setup.yml
@@ -7,7 +7,7 @@ on:
       - 'docs/**'
       - '**.md'
     pull_request:
-      branches: [ main, legacy-firecracker-v0.24.0-with-upf-support, openyurt ]
+      branches: [ main, legacy-firecracker-v0.24.0-with-upf-support ]
       paths-ignore:
       - 'docs/**'
       - '**.md'

diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml
@@ -16,7 +16,7 @@ on:
     branches: [ main, legacy-firecracker-v0.24.0-with-upf-support ]
   pull_request:
     # The branches below must be a subset of the branches above
-    branches: [ main, legacy-firecracker-v0.24.0-with-upf-support, openyurt ]
+    branches: [ main, legacy-firecracker-v0.24.0-with-upf-support ]
   workflow_dispatch:
   schedule:
     - cron: '40 13 * * 2'

diff --git a/.github/workflows/firecracker_cri_tests.yml b/.github/workflows/firecracker_cri_tests.yml
@@ -8,7 +8,7 @@ on:
       - '**.md'
       - 'function-images/**'
   pull_request:
-    branches: [ main, legacy-firecracker-v0.24.0-with-upf-support, openyurt ]
+    branches: [ main, legacy-firecracker-v0.24.0-with-upf-support ]
     paths-ignore:
       - 'docs/**'
       - '**.md'

diff --git a/.github/workflows/gvisor_cri_tests.yml b/.github/workflows/gvisor_cri_tests.yml
@@ -8,7 +8,7 @@ on:
     - '**.md'
     - 'function-images/**'
   pull_request:
-    branches: [ main, legacy-firecracker-v0.24.0-with-upf-support, openyurt ]
+    branches: [ main, legacy-firecracker-v0.24.0-with-upf-support ]
     paths-ignore:
     - 'docs/**'
     - '**.md'

diff --git a/.github/workflows/integration_tests.yml b/.github/workflows/integration_tests.yml
@@ -8,7 +8,7 @@ on:
     - '**.md'
     - 'function-images/**'
   pull_request:
-    branches: [ main, legacy-firecracker-v0.24.0-with-upf-support, openyurt ]
+    branches: [ main, legacy-firecracker-v0.24.0-with-upf-support ]
     paths-ignore:
     - 'docs/**'
     - '**.md'

diff --git a/.github/workflows/linters.yml b/.github/workflows/linters.yml
@@ -3,7 +3,7 @@ on:
   push:
     branches: [ main, legacy-firecracker-v0.24.0-with-upf-support ]
   pull_request:
-    branches: [ main, legacy-firecracker-v0.24.0-with-upf-support, openyurt ]
+    branches: [ main, legacy-firecracker-v0.24.0-with-upf-support ]
 
 jobs:
   build:

diff --git a/.github/workflows/openyurt-unit-test.yml b/.github/workflows/openyurt-unit-test.yml
@@ -0,0 +1,41 @@
+name: Build and Test OpenYurt Deployer
+
+on:
+    push:
+      branches: [ main, legacy-firecracker-v0.24.0-with-upf-support ]
+      paths-ignore:
+      - 'docs/**'
+      - '**.md'
+    pull_request:
+      branches: [ main, legacy-firecracker-v0.24.0-with-upf-support ]
+      paths-ignore:
+      - 'docs/**'
+      - '**.md'
+    workflow_dispatch:
+
+env:
+    GOOS: linux
+    GO111MODULE: on
+
+jobs:
+  openyurt-unit-test:
+    runs-on: ubuntu-latest
+
+    steps:
+    - name: Set up Go 1.19
+      uses: actions/setup-go@v4
+      with:
+        go-version: '1.19'
+
+    - name: Check out the code
+      uses: actions/checkout@v4
+
+    - name: Build scripts
+      run: 
+        pushd scripts/openyurt-deployer && go build -o oy_deploy && popd
+
+    - name: Run Unit Test on OpenYurt helper function 
+      run: | 
+        cd scripts/openyurt-deployer
+        chmod +x unit_test_workflow.sh
+        ./unit_test_workflow.sh
diff --git a/.github/workflows/stargz_tests.yml b/.github/workflows/stargz_tests.yml
@@ -8,7 +8,7 @@ on:
     - '**.md'
     - 'function-images/**'
   pull_request:
-    branches: [ main, legacy-firecracker-v0.24.0-with-upf-support, openyurt ]
+    branches: [ main, legacy-firecracker-v0.24.0-with-upf-support ]
     paths-ignore:
     - 'docs/**'
     - '**.md'

diff --git a/.github/workflows/unit_tests.yml b/.github/workflows/unit_tests.yml
@@ -8,7 +8,7 @@ on:
     - '**.md'
     - 'function-images/**'
   pull_request:
-    branches: [ main, legacy-firecracker-v0.24.0-with-upf-support, openyurt ]
+    branches: [ main, legacy-firecracker-v0.24.0-with-upf-support ]
     paths-ignore:
     - 'docs/**'
     - '**.md'

diff --git a/README.md b/README.md
@@ -31,6 +31,7 @@ including functions autoscaling and cold-start delay optimization with several s
 
 vHive has added support for the state-of-the-art extension [eStargz](https://github.com/containerd/stargz-snapshotter) to container layers and lazy pull support for container images.
 
+vHive has added support for [`OpenYurt`](https://openyurt.io/), an open platform that extends upstream Kubernetes to the edge. It will assist in node management for cloud and edge nodes and can be deployed with [`openyurt-deployer`](scripts/openyurt-deployer/README.md).
 
 ## vHive architecture
 

diff --git a/scripts/openyurt-deployer/README.md b/scripts/openyurt-deployer/README.md
@@ -1,11 +1,26 @@
 # Quick set-up `OpenYurt`
 
-## 1. Introduction
+## 1.1 Introduction
 
 This program extends [`EasyOpenyurt`](https://github.com/flyinghorse0510/easy_openyurt) to automate the set up process of an `OpenYurt` cluster. 
 
 It support setting up a Kubernetes cluster using kubeadm and then deploy `OpenYurt` and Knative on it. It is compatible with vHive stock-only mode.
 
+## 1.2 About [`OpenYurt`](https://openyurt.io/docs/#:~:text=Furthermore%2C%20OpenYurt%20enhances%20node%20reliability,node%20heartbeats%20to%20the%20cloud.)
+
+### Key Features
+
+#### 1. Powerful edge autonomy capability
+`OpenYurt` addresses this issue by implementing a per-node proxy (`YurtHub`) along with local storage to cache the state of the cloud apiserver. Consequently, when a node loses its connection, the cached states remain accessible to Kubelet, `KubeProxy`, and any user Pods.
+
+#### 2. Cross `NodePool` network communication capability
+
+In an edge computing Kubernetes cluster, nodes are often distributed across various geographical regions. Consequently, when relying on a native Container Network Interface (CNI) solution, Pods within different `NodePools` may be unable to communicate using Pod IP, Service IP, or Node IP, particularly if each `NodePool` resides within its own isolated LAN. Raven offers a networking solution that enables `cross-NodePool` communication within an `OpenYurt` cluster.
+
+#### 3. `Multi-NodePool` Management
+
+In order to manage applications and traffic in multiple node pools conveniently, `YurtAppSet` and `YurtAppDaemon` are introduced for managing workloads in `multi-nodepool`, and service topology capability for routing traffic in node pool.
+
 ## 2. Brief overview
 
 **Prerequisite of nodes:**
@@ -19,8 +34,8 @@ It support setting up a Kubernetes cluster using kubeadm and then deploy `OpenYu
 | :----------: | :---: |
 | main.go | script entry point |
 | `conf.json` | json files that stores cluster's configuration |
-| node | executing commands on remote nodes through ssh |
-| configs | node runtime configurations |
+| node.go | node structure and related functions for Section 3.2 |
+| configs | yaml templates |
 
 **Description**
 
@@ -30,9 +45,7 @@ It support setting up a Kubernetes cluster using kubeadm and then deploy `OpenYu
 4. (Optional) Deploy Knative (vHive stock-only mode compatible)
 
 ## 3. Usage
-```bash
-./openyurt_deployer deploy # deploy openyurt on the cluster 
-```
+
 ```bash
 ./openyurt_deployer clean # clean the openyurt cluster and restore it to initial state 
 ```
@@ -56,24 +69,38 @@ It support setting up a Kubernetes cluster using kubeadm and then deploy `OpenYu
 
 ### 3.2 Run Script
 
+#### 3.2.1 To set up Kubernetes and Kubernetes cluster
 ```bash
 go build .
-./openyurt_deployer deploy
-```
-If it gives out error like: 
-```
-FATA[0001] Failed to connect to: username@host
-```
-Please execute:
-```
-eval `ssh-agent -s` && ssh-add ~/.ssh/<your private key>
-```
-For example:
-```
-eval `ssh-agent -s` && ssh-add ~/.ssh/id_rsa
+./openyurt_deployer k8s
+```
+---
+**Note:**
+> If it gives out error like: 
+> ```
+> FATA[0001] Failed to connect to: username@host
+> ```
+> Please execute:
+> ```
+> eval `ssh-agent -s` && ssh-add ~/.ssh/<your private key>
+> ```
+> For example:
+> ```
+> eval `ssh-agent -s` && ssh-add ~/.ssh/id_rsa
+> ```
+> And try again
+
+---
+
+#### 3.2.2 To set up Knative 
+```bash
+./openyurt_deployer knative
 ```
-And try again
 
+#### 3.2.2 To set up Open Yurt 
+```bash
+./openyurt_deployer openyurt
+```
 
 ## 4. Demo: Create `NodePool` And Deploy service on it
 **Referenced from [`OpenYurt`](https://openyurt.io/docs/user-manuals/workload/node-pool-management)*
@@ -82,31 +109,27 @@ The demo would deploy a helloworld function to cloud node pool or edge node pool
 
 Deploy the demo:
 ```
-./openyurt_deployer demo-c
+./openyurt_deployer demo-deploy-on-cloud
 ```
 or:
 ```
-./openyurt_deployer demo-e
+./openyurt_deployer demo-deploy-on-edge
 ```
-where `demo-c` would deploy the service to the cloud node pool and `demo-e` would deploy the service to the edge node pool.
+where `demo-deploy-on-cloud` would deploy the service to the cloud node pool and `demo-deploy-on-edge` would deploy the service to the edge node pool.
 
 The demo code will also show information about node pool after deployment.
-The name for `demo-c` would be `helloworld-cloud`, while the name for `demo-e` would be `helloworld-edge`
+The name for `demo-deploy-on-cloud` would be `helloworld-cloud`, while the name for `demo-deploy-on-edge` would be `helloworld-edge`
 It will also show the services' `URL` so you can try to invoke it on the master node.
 
 You can check the node pool information simply by:
 ```
 ./openyurt_deployer demo-print
 ```
-Or delete the services deployed on node pool by:
-```
-./openyurt_deployer demo-clear
-```
 
 The demo code will also show information about node pool after deployment.
 
 ### 4.1 Invoke the Services (Optional)
-You can try to invoke the services created by `demo-c` or `demo-e` on master node.
+You can try to invoke the services created by `demo-deploy-on-cloud` or `demo-deploy-on-edge` on master node.
 First, ssh to master node, following commands should all be executed on master node.
 ```
 ssh <master-user>@<master-ip>

diff --git a/scripts/openyurt-deployer/configs/benchmarkTemplate.yaml b/scripts/openyurt-deployer/configs/benchmarkTemplate.yaml
@@ -0,0 +1,15 @@
+apiVersion: serving.knative.dev/v1
+kind: Service
+metadata:
+  name: helloworld-python-isCloud
+  namespace: default
+spec:
+  template:
+    spec:
+      nodeSelector:
+        apps.openyurt.io/nodepool: poolName
+      containers:
+        - image: docker.io/vhiveease/hello-isCloud:latest
+          ports:
+            - name: h2c
+              containerPort: 50000
diff --git a/scripts/openyurt-deployer/configs/cloudNodePoolTemplate.yaml b/scripts/openyurt-deployer/configs/cloudNodePoolTemplate.yaml
@@ -0,0 +1,6 @@
+apiVersion: apps.openyurt.io/v1beta1
+kind: NodePool
+metadata:
+  name: poolName
+spec:
+  type: Cloud
diff --git a/scripts/openyurt-deployer/configs/configs.go b/scripts/openyurt-deployer/configs/configs.go
Original file line number	Diff line number	Diff line change
Expand Up		@@ -31,6 +31,7 @@ including functions autoscaling and cold-start delay optimization with several s

		vHive has added support for the state-of-the-art extension [eStargz](https://github.com/containerd/stargz-snapshotter) to container layers and lazy pull support for container images.

		vHive has added support for [`OpenYurt`](https://openyurt.io/), an open platform that extends upstream Kubernetes to the edge. It will assist in node management for cloud and edge nodes and can be deployed with [`openyurt-deployer`](scripts/openyurt-deployer/README.md).

		## vHive architecture

Expand Down