Skip to content

Latest commit

 

History

History
242 lines (178 loc) · 5.8 KB

HOWTO.md

File metadata and controls

242 lines (178 loc) · 5.8 KB
Raw

Permissioned Chain

This tutorial will walk you through setting up a Tezos based permissioned blockchain. Upon successful completion of this tutorial you will have a permissioned chain running with three nodes in a peer-to-peer network. This tutorial assumes the use of Minikube on MacOS.

Prerequisites

  • docker
  • minikube
  • python3
  • a ZeroTier network and api access token
  • jq

Installing prerequisites

This section varies depending on OS.

Mac w/ homebrew

Make sure homebrew is installed:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

Install prerequisites:

brew install python3
brew install jq
brew install minikube

Arch Linux

pacman -Syu && pacman -S python3 jq minikube kubectl kubectx linux

Ensure that the kernel used is compiled with the nfs module. The default arch linux kernel should suffice.

Enable and start the rpcbind and nfs-server systemd services:

sudo systemctl enable rpcbind && sudo systemctl start rpcbind
sudo systemctl enable nfs-server && sudo systemctl enable nfs-server

Setting up mkchain

Ensure minikube is running:

minikube start

Install the mkchain program:

python3 -m venv .venv-tezos-k8s
source .venv-tezos-k8s/bin/activate
pip install https://github.com/tqtezos/tezos-k8s/archive/master.zip

Create a ZeroTier network:

  • Go to https://my.zerotier.com
  • Login with google credentials or create a new account
  • Create a new API access token by clicking on the "Generate New Token" button. Save the generated access token. e.g. "yEflQt726fjXuSUyQ73WqXvAFoijXkLt"
  • Go to https://my.zerotier.com/network
  • Create a new network by clicking on the "Create a Network" button. Save the 16 character generated network id. e.g. "1c33c1ced02a5eee"

Set environment variables so we can access these values with later commands:

ZT_TOKEN=yEflQt726fjXuSUyQ73WqXvAFoijXkLt
ZT_NET=1c33c1ced02a5eee

Give your private chain a name:

CHAIN_NAME=my_chain

Set unbuffered IO for python:

PYTHONUNBUFFERED=x

Run the following command to create the chain yaml:

mkchain --create --baker --zerotier-network $ZT_NET \
--zerotier-token $ZT_TOKEN $CHAIN_NAME > output.yaml

Prepare your cluster storage for the node. On MacOS your storage directory needs to be exported to Minikube via NFS:

For mac:

sudo sh -c 'echo "/System/Volumes/Data/Users/ -alldirs -mapall=501:1000 $(minikube ip)" >> /etc/exports'
sudo nfsd restart

For Linux:

sudo sh -c 'echo "$HOME/.tq $(minikube ip)(rw,sync,no_subtree_check,all_squash,anonuid=$(id -u),anongid=$(id -g))" >> /etc/exports'
sudo exportfs -arv

If you're on Linux, you may also need to ensure that your minikube virtual machine has the appropriate NFS support:

minikube ssh
sudo -s
apt update && apt install nfs-common
echo -e "[ NFSMount_Global_Options ]\nDefaultvers=3" > /etc/nfsmount.conf

Feed the chain yaml file to kubernetes:

kubectl apply -f output.yaml

Your kubernetes cluster will now be running a series of jobs to perform the following tasks:

  • generate a node identity
  • generate a genesis block for your chain
  • activate the protocol
  • bake the first block
  • start the node and a baker to validate the chain

You can find your node in the tqtezos namespace using kubectl.

kubectl -n tqtezos get pods

You can view logs for your node using the following command:

kubectl -n tqtezos logs -l app=tezos-node -c tezos-node -f

Congratulations! You now have an operational Tezos based permissioned chain running one node.

Add Nodes

Additional nodes can be added to your network by sharing a yaml file generated by the mkchain invite command

create the invitation

IP=$(kubectl -n tqtezos exec  daemonsets/zerotier-bridge -- zerotier-cli get $ZT_NET ip)
mkchain --invite --bootstrap-peer $IP $CHAIN_NAME > join-$CHAIN_NAME.yaml

Share the resulting join-$CHAIN_NAME.yaml with your partner via a secure channel.

on the computer of the joining node

Member clusters will also need to have storage allocated for their node. Assuming Minikube on MacOS the following commands will establish an NFS share and a k8s PersistentVolume to satisfy the claims of the node.

sudo sh -c 'echo "/System/Volumes/Data/Users/ -alldirs -mapall=501:1000 $(minikube ip)" >> /etc/exports'
sudo nfsd restart

NODE_DIR=$HOME/.tq
mkdir -p $NODE_DIR
NFS_HOST=$(minikube ssh "ip route show default"|awk '{print $3}')
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolume
metadata:
  labels:
    storage-type: var-files
  name: tezos-var-volume
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 15Gi
  nfs:
    path: $NODE_DIR
    server: $NFS_HOST
  persistentVolumeReclaimPolicy: Retain
  storageClassName: manual
EOF

The member can now apply the shared yaml to their Minikube cluster:

kubectl apply -f join-$CHAIN_NAME.yaml

At this point additional nodes will be added in a star network topology. In order to get nodes communicating via p2p one of the nodes must be told about the VPN addresses used by the clients.

The following command can be used to force a peering connection over the VPN network.

curl -H "Authorization: bearer $ZT_TOKEN" \
https://my.zerotier.com/api/network/$ZT_NET/member|jq -r ".[].config.ipAssignments[0]" | while read i
do
    kubectl -n tqtezos exec deployment/tezos-node -- /usr/local/bin/tezos-admin-client connect address $i:30732
done

Congratulations! You now have a multi-node Tezos based permissioned chain.

Check that the nodes have matching heads by comparing their hashes:

kubectl -n tqtezos exec deployment/tezos-node -c tezos-node -- /usr/local/bin/tezos-client rpc get /chains/main/blocks/head/hash