Skip to content

Commit

Permalink
Added download yq to fix the issue.
Browse files Browse the repository at this point in the history
Seperated demo for node pool management
Added overview and link to tutorial
Added comments to CHANGELOG

Signed-off-by: Jason Chua <[email protected]>
  • Loading branch information
jchua99 authored and ustiugov committed Feb 16, 2024
1 parent 3b1cd51 commit 712925b
Show file tree
Hide file tree
Showing 9 changed files with 125 additions and 84 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,4 @@
# Binaries
vhive
setup_tool
go.work.sum
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@

### Added

- Added support for [`OpenYurt`](https://openyurt.io/), an open platform that extends upstream Kubernetes to run on edge node pools. More details on how to the `Knative-atop-OpenYurt` mode are described [here](scripts/openyurt-deployer/README.md).


### Changed

- Removed the utils and examples from the vHive repo, moved to [vSwarm](https://github.com/vhive-serverless/vSwarm).
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ including functions autoscaling and cold-start delay optimization with several s

vHive has added support for the state-of-the-art extension [eStargz](https://github.com/containerd/stargz-snapshotter) to container layers and lazy pull support for container images.

vHive has added support for [`OpenYurt`](https://openyurt.io/), an open platform that extends upstream Kubernetes to the edge. It will assist in node management for cloud and edge nodes and can be deployed with [`openyurt-deployer`](scripts/openyurt-deployer/README.md).
vHive supports running Knative on top of [`OpenYurt`](https://openyurt.io/), an open platform that extends upstream Kubernetes to run on edge node pools. More details on how to the `Knative-atop-OpenYurt` mode are described [here](scripts/openyurt-deployer/README.md).

## vHive architecture

Expand Down
Binary file added docs/figures/knative_atop_openyurt.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions go.work.sum
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
golang.org/x/crypto v0.16.0 h1:mMMrFzRSCF0GvB7Ne27XVtVAaXLrPmgPC7/v0tkwHaY=
golang.org/x/crypto v0.16.0/go.mod h1:gCAAfMLgwOJRpTjQ2zCCt2OcSfYMTeZVSRtQlPC7Nq4=
golang.org/x/crypto v0.18.0 h1:PGVlW0xEltQnzFZ55hkuX5+KLyrMYhHld1YHO4AKcdc=
golang.org/x/crypto v0.18.0/go.mod h1:R0j02AL6hcrfOiy9T4ZYp/rcWeMxM3L6QYxlOuEG1mg=
golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/term v0.15.0/go.mod h1:BDl952bC7+uMoWR75FIrCDx79TPU9oHkTZ9yRbYOrX0=
golang.org/x/term v0.16.0/go.mod h1:yn7UURbUtPyrVJPGPq404EukNFxcm/foM+bV/bfcDsY=
24 changes: 18 additions & 6 deletions scripts/openyurt-deployer/README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,27 @@
# Quick set-up `OpenYurt`

## 1.1 Introduction
![Architecture Diagram](/docs/figures/knative_atop_openyurt.png)

Refer to [Serverless Functions atop of Cloud-Edge Deployment](https://docs.google.com/presentation/d/1QPwwvBhE1dZy430x_wDZPgsUYtlFNlteT9TPk3VbRsY/edit?usp=drive_link) tutorial to understand its application.

This program extends [`EasyOpenyurt`](https://github.com/flyinghorse0510/easy_openyurt) to automate the set up process of an `OpenYurt` cluster.
## 1.1 Introduction

It support setting up a Kubernetes cluster using kubeadm and then deploy `OpenYurt` and Knative on it. It is compatible with vHive stock-only mode.
`OpenYurt` extends Kubernetes from a single cluster to a federation of clusters, offers a seamless way to manage distributed nodes and applications efficiently. This guide outlines how to establish a Kubernetes cluster via kubeadm and then deploy `OpenYurt` and Knative on it, which is compatible with vHive stock-only mode.

## 1.2 About [`OpenYurt`](https://openyurt.io/docs/#:~:text=Furthermore%2C%20OpenYurt%20enhances%20node%20reliability,node%20heartbeats%20to%20the%20cloud.)

### Key Features

#### 1. Powerful edge autonomy capability
`OpenYurt` addresses this issue by implementing a per-node proxy (`YurtHub`) along with local storage to cache the state of the cloud apiserver. Consequently, when a node loses its connection, the cached states remain accessible to Kubelet, `KubeProxy`, and any user Pods.
`OpenYurt` is designed to ensure that applications running at the edge of the network remain operational, even in poor connectivity conditions. `OpenYurt` addresses this issue by implementing a per-node proxy (`YurtHub`) along with local storage to cache the state of the cloud apiserver. Consequently, when a node loses its connection, the cached states remain accessible to Kubelet, `KubeProxy`, and any user Pods.

#### 2. Cross `NodePool` network communication capability

In an edge computing Kubernetes cluster, nodes are often distributed across various geographical regions. Consequently, when relying on a native Container Network Interface (CNI) solution, Pods within different `NodePools` may be unable to communicate using Pod IP, Service IP, or Node IP, particularly if each `NodePool` resides within its own isolated LAN. Raven offers a networking solution that enables `cross-NodePool` communication within an `OpenYurt` cluster.
In an edge computing Kubernetes cluster, nodes are often distributed across various geographical regions. The edge nodes fell within same physical region are referred to as `Pools` in `OpenYurt`. Consequently, when relying on a native Container Network Interface (CNI) solution, Pods within different `NodePools` may be unable to communicate using Pod IP, Service IP, or Node IP, particularly if each `NodePool` resides within its own isolated LAN. `OpenYurt` introduces a gateway called `Raven`, which offers a networking solution that enables `cross-NodePool` communication within an `OpenYurt` cluster.

#### 3. `Multi-NodePool` Management

In order to manage applications and traffic in multiple node pools conveniently, `YurtAppSet` and `YurtAppDaemon` are introduced for managing workloads in `multi-nodepool`, and service topology capability for routing traffic in node pool.
In order to manage applications and traffic in multiple node pools conveniently, `YurtAppSet` and `YurtAppDaemon` are introduced for managing workloads in `multi-nodepool`, and service topology capability (geographical location, network latency, etc.) for routing traffic in node pool.

## 2. Brief overview

Expand Down Expand Up @@ -105,6 +107,16 @@ go build .
## 4. Demo: Create `NodePool` And Deploy service on it
**Referenced from [`OpenYurt`](https://openyurt.io/docs/user-manuals/workload/node-pool-management)*

### 4.1

This will set up the node pool management
```
./openyurt_deployer demo
```


### 4.2

The demo would deploy a helloworld function to cloud node pool or edge node pool

Deploy the demo:
Expand Down
6 changes: 4 additions & 2 deletions scripts/openyurt-deployer/configurations.go
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,7 @@ type SystemEnvironmentStruct struct {
PmuToolsRepoUrl string
ProtocVersion string
ProtocDownloadUrlTemplate string
YqDownloadUrl string
}

type YurtEnvironment struct {
Expand Down Expand Up @@ -126,8 +127,8 @@ type VHiveConfigStruct struct {
var Demo = DemoEnvironment{
CloudYamlFile: "cloud.yaml",
EdgeYamlFile: "edge.yaml",
CloudBenchYamlFile: "cloud-bench.yaml",
EdgeBenchYamlFile: "edge-bench.yaml",
CloudBenchYamlFile: "cloudBench.yaml",
EdgeBenchYamlFile: "edgeBench.yaml",
YurtAppSetYamlFile: "yurt.yaml",
CloudPoolName: "cloud",
EdgePoolName: "edge",
Expand Down Expand Up @@ -159,6 +160,7 @@ var System = SystemEnvironmentStruct{
CurrentDir: "",
UserHomeDir: "",
NodeHostName: "",
YqDownloadUrl: "https://github.com/mikefarah/yq/releases/latest/download/yq_linux_%s",
}

var VHive = VHiveConfigStruct{
Expand Down
33 changes: 29 additions & 4 deletions scripts/openyurt-deployer/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,9 @@ func main() {
deployNodes(*deployerConf)
case "knative":
deployKnative(*deployerConf)
case "demo-deploy-on-edge":
case "demo": // this is to add node management properties
buildDemo(*deployerConf)
case "demo-deploy-on-edge": // to add benchmark
demo(*deployerConf, false)
case "demo-deploy-on-cloud":
demo(*deployerConf, true)
Expand Down Expand Up @@ -237,16 +239,25 @@ func deployOpenYurt(deployerConfFile string) {
port := lines[1]
token := lines[2]

// 2.1 label your nodes
for _, worker := range workerNodes {
masterNode.YurtMasterExpand(&worker)
utils.InfoPrintf("Master has expanded to worker:%s\n", worker.Configs.System.NodeHostName)
}
utils.SuccessPrintf("Master has expaned to all nodes!\n")

// 2.2 Set up up Yurthub & 2.3 Configure Kubelet on Master
for _, worker := range workerNodes {
worker.YurtWorkerJoin(addr, port, token)
utils.InfoPrintf("worker %s joined yurt cluster!\n", worker.Configs.System.NodeHostName)
}
utils.SuccessPrintf("All nodes joined yurt cluster, start to expand\n")

// 2.4 Restart pods on edge
for _, worker := range workerNodes {
masterNode.YurtMasterExpand(&worker)
masterNode.YurtRestart(&worker)
utils.InfoPrintf("Master has expanded to worker:%s\n", worker.Configs.System.NodeHostName)
}
utils.SuccessPrintf("Master has expaned to all nodes!\n")

for _, node := range nodeList {
utils.InfoPrintf("node: %s\n", node.Name)
Expand All @@ -269,12 +280,26 @@ func demo(deployerConfFile string, isCloud bool) {
workerNodes := nodeList[1:]

// run demo, should only be executed after deployment
utils.SuccessPrintf("Start to init demo\n")
utils.SuccessPrintf("Start to build cloud and edge nodepools\n")
masterNode.Demo(isCloud)
utils.SuccessPrintf("Demo finished!\n")
masterNode.PrintDemoInfo(workerNodes, isCloud)
}

func buildDemo(deployerConfFile string) {

nodesInfo, err := readAndUnMarshall(deployerConfFile)
utils.CheckErrorWithMsg(err, "Failed to read and unmarshal deployer configuration JSON")
nodeList := initializeNodes(nodesInfo)
masterNode := nodeList[0]
workerNodes := nodeList[1:]

// run workload for node pool management
utils.SuccessPrintf("Run Build demo\n")
masterNode.BuildDemo(workerNodes)
utils.SuccessPrintf("Demo finished!\n")
}

func printDemo(deployerConfFile string) {

nodesInfo, err := readAndUnMarshall(deployerConfFile)
Expand Down
Loading

0 comments on commit 712925b

Please sign in to comment.