diff --git a/CHECKLIST.md b/CHECKLIST.md new file mode 100644 index 0000000..8558336 --- /dev/null +++ b/CHECKLIST.md @@ -0,0 +1,80 @@ +Corda Kubernetes Depolyment checklist + +Tested with: + +- Corda Enterprise version 4.0 + +Pre-requisites: + + - Clone the repository to any local folder + - Installation requires the following tools: + - Docker (tested with Docker 19.03.5, API 1.40, newer versions should be fine) + - Kubectl (tested with kubectl v1.12.8, newer versions should be fine) + - Helm (requires Helm version 2.x, tested with Helm v2.14.3, newer v2.x versions should be fine) + - Azure CLI (tested with az cli 2.1.0, newer versions should be fine) + +Azure Cloud Setup: + + - Azure Setup + - Azure Account connected to a subscription with permissions to create resources + - Azure Kubernetes Service + - Azure Container Registry + - Azure Service Principals + - Azure Storage Account, create three new File shares for each of the following: + - node, named for example node--storage, where name would match the nodes x500 name to some degree and should match values.yaml files "fileShareName" parameter as well + - bridge, named for example bridge--storage, where name would match the nodes x500 name to some degree and should match values.yaml files "fileShareName" parameter as well + - float, named for example float--storage, where name would match the nodes x500 name to some degree and should match values.yaml files "fileShareName" parameter as well + - Public IP Addresses in the "KubernetesPlayground-NodePool" resource group + - Node, to enable RPC connections from GUI, named for example node--ip + - Float, to enable inbound connections from other nodes on the network, named for example float--ip + - Azure Config + - az login, make sure at this point that if you have many subscriptions, that the one you want to use has isDefault=true, if not use "az account list" and "az account set -s " to fix it + - az aks get-credentials --resource-group KubernetesPlayground --name KubernetesPlaygroundAKS # KubernetesPlayground is just an example name, use your own resource names + - kubectl create namespace # name could be for example firstname-lastname in lowercase, if you are deploying many instances in a test environment + - kubectl config set-context --current --namespace + +AWS Cloud setup: + + - Coming soon + +Configuration: + + - Deployment folder (repository folder) + - Binaries, jar files (the name of the jars should match the configuration) + - Docker-images/bin + - Corda Enterprise jar (eg. corda-ent-4.0.jar) + - Health-survey-tool jar (eg. corda-tools-healthsurvey-4.0.jar) + - Corda Firewall jar (eg. corda-firewall-4.0.jar) + - Pki-firewall/bin + - Optional on windows: Copy Key tool jar + dll to bin folder (pki-firewall/bin) + - Config + - docker_config.sh + - Define versions so that they match what is in the values.yaml file. + - Values.yaml + - Config containerRegistry section. + - Config storage section. + - Config fileShareName for node/bridge/float + - Config identityManagerAddress and networkmapAddress (without http:// prefix) + - You can use any network, but please note that if you want to use Testnet, we will have to skip the initial registration step and download the full Testnet node from the dashboard + - Config resourceName to reflect the x500 name of the node, please note to use lowercase letters and numbers only + - Config legalName to define the x500 name of the node + - Download network root truststore to ./helm/files/network with the name "networkRootTrustStore.jks" + - Configure matching truststorePassword to the truststore. + - Execution + - Run one-time-setup.sh once, which does the following: + - Creates and pushes Docker images to the container registry + - Generates certificates for the Corda Firewall TLS tunnel + - Performs initial registration of the node + - Copies the generates certificates for the next step, which is the deployment + - Deploy using deploy.sh or helm/helm_compile.sh, which does the following: + - Compiles the Helm charts from templates to Kubernetes resource definition files + - Applies the generated Kubernetes resources definition files to the Kubernetes cluster + - Three pods should be at status ‘Runningʼ for node, bridge and float after a while + - Please have a look at the logs files for the three pods to make sure they are running without errors (kubectl get pods + kubectl logs -f ) + - Run delete_all.sh to remove all resources from the Kubernetes cluster if you need to start fresh + +Useful commands: + + - Check deployment status with: kubectl get pods, expect to see 'Running' if the pods are working normally + - Check logs of running components with : kubectl logs -f + - Investigate Corda Node log by gaining a shell into the running pod with: (winpty on windows) kubectl exec -it bash, then cd to folder /opt/corda/workspace/logs and look at most recent node log file diff --git a/README.md b/README.md index f98fff5..d065595 100644 --- a/README.md +++ b/README.md @@ -31,17 +31,6 @@ It is strongly recommended you review all of the documentation there before sett --- -## BINARIES - -This deployment is targeting an Enterprise deployment, which should include a Corda Node, but also the Corda Firewall, which is an Enterprise only feature. - -In order to execute the following scripts correctly, you will have to have access to the Corda Enterprise binaries. -The files should be downloaded first and placed in the following folder: ``docker-images/bin`` - -Please see [docker-images/README.md](docker-images/README.md) for more information. - ---- - ## Azure cloud instructions Setting up the relevant cloud services is currently left to the reader, this may change in future versions of the scripts. @@ -91,7 +80,20 @@ A guide on setting up Public IP addresses in Azure: [Create, change, or delete a #### BINARIES -The scripts contained within require you to have the necessary binaries in place, as previously mentioned in this document. +This deployment is targeting an Enterprise deployment, which should include a Corda Node, but also the Corda Firewall, which is an Enterprise only feature. + +In order to execute the following scripts correctly, you will have to have access to the Corda Enterprise binaries. +The files should be downloaded first and placed in the following folder: ``docker-images/bin`` + +Please see [docker-images/README.md](docker-images/README.md) for more information. + +#### SETUP CHECKLIST + +Since there are a number of prerequisites that need to be met and then a certain order of running everything, a checklist has been collated that you may find useful. + +Please see [CHECKLIST.md](CHECKLIST.md) for more information. + +--- #### CONFIGURATION VALUES diff --git a/corda-pki-generator/copy_certs_to_helm_files.sh b/corda-pki-generator/copy_certs_to_helm_files.sh index c966e63..d12452c 100755 --- a/corda-pki-generator/copy_certs_to_helm_files.sh +++ b/corda-pki-generator/copy_certs_to_helm_files.sh @@ -56,14 +56,12 @@ ensureFileExistsAndCopy () { TO=$2 if [ -f "$FROM" ] then - if [ ! -f "$TO" ] + if [ -f "$TO" ] then + echo "Existing certificate already existed, but it is safe to replace, since this is just the Corda Firewall tunnel keys." + fi cp -f $FROM $TO else - echo "Existing certificate already existed, skipping copying as a safe-guard: $TO" - exit 1 - fi - else echo "File did not exist, probably an issue with certificate creation: $FROM" exit 1 fi diff --git a/docker-images/build_docker_images.sh b/docker-images/build_docker_images.sh index ec29aeb..f0c41df 100755 --- a/docker-images/build_docker_images.sh +++ b/docker-images/build_docker_images.sh @@ -59,6 +59,14 @@ then NO_CACHE=--no-cache fi +if [ ! -f "$DIR/bin/$CORDA_VERSION.jar" -o ! -f "$DIR/bin/corda-tools-health-survey-$HEALTH_CHECK_VERSION.jar" -o ! -f "$DIR/bin/$CORDA_FIREWALL_VERSION.jar" ]; then + echo "Missing binaries, check that you have the correct files with the correct names in the following folder $DIR/bin" + echo "$DIR/bin/$CORDA_VERSION.jar" + echo "$DIR/bin/$CORDA_FIREWALL_VERSION.jar" + echo "$DIR/bin/corda-tools-health-survey-$HEALTH_CHECK_VERSION.jar" + exit 1 +fi + cp $DIR/bin/$CORDA_VERSION.jar $DIR/$CORDA_IMAGE_PATH/corda.jar cp $DIR/bin/corda-tools-health-survey-$HEALTH_CHECK_VERSION.jar $DIR/$CORDA_IMAGE_PATH/corda-tools-health-survey.jar cd $DIR/$CORDA_IMAGE_PATH diff --git a/docker-images/docker_config.sh b/docker-images/docker_config.sh index 6cebb4a..89d30e4 100755 --- a/docker-images/docker_config.sh +++ b/docker-images/docker_config.sh @@ -1,8 +1,46 @@ #!/bin/sh +set -ux +DIR="." +function GetPathToCurrentlyExecutingScript () { + # Absolute path of this script, e.g. /opt/corda/node/foo.sh + ABS_PATH=$(readlink -f "$0") + if [ "$?" -ne "0" ]; then + echo "readlink issue workaround..." + # Unfortunate MacOs issue with readlink functionality, see https://github.com/corda/corda-kubernetes-deployment/issues/4 + TARGET_FILE=$0 + + cd `dirname $TARGET_FILE` + TARGET_FILE=`basename $TARGET_FILE` + local ITERATIONS=0 + + # Iterate down a (possible) chain of symlinks + while [ -L "$TARGET_FILE" ] + do + TARGET_FILE=`readlink $TARGET_FILE` + cd `dirname $TARGET_FILE` + TARGET_FILE=`basename $TARGET_FILE` + ((++ITERATIONS)) + if [ "$ITERATIONS" -gt 1000 ]; then + echo "symlink loop. Critical exit." + exit 1 + fi + done + + # Compute the canonicalized name by finding the physical path + # for the directory we're in and appending the target file. + PHYS_DIR=`pwd -P` + ABS_PATH=$PHYS_DIR/$TARGET_FILE + fi + + # Absolute path of the directory this script is in, thus /opt/corda/node/ + DIR=$(dirname "$ABS_PATH") +} +GetPathToCurrentlyExecutingScript set -eux DOCKER_REGISTRY="" +DOCKER_REGISTRY=$(grep -A 3 'containerRegistry:' $DIR/../helm/values.yaml | grep 'serverAddress: "' | cut -d '"' -f 2) VERSION="4.0" HEALTH_CHECK_VERSION="4.0" diff --git a/docker-images/push_docker_images.sh b/docker-images/push_docker_images.sh index bc0af1a..4d396b9 100755 --- a/docker-images/push_docker_images.sh +++ b/docker-images/push_docker_images.sh @@ -41,6 +41,11 @@ set -eux source $DIR/docker_config.sh +if [ "$DOCKER_REGISTRY" == "" ]; then + echo "You must specify a valid container registry in the values.yaml file" + exit 1 +fi + docker login $DOCKER_REGISTRY docker tag $CORDA_IMAGE_PATH:$CORDA_DOCKER_IMAGE_VERSION $DOCKER_REGISTRY/$CORDA_IMAGE_PATH:$CORDA_DOCKER_IMAGE_VERSION diff --git a/helm/helm_compile.sh b/helm/helm_compile.sh index 08d71b2..e747fcf 100755 --- a/helm/helm_compile.sh +++ b/helm/helm_compile.sh @@ -52,7 +52,13 @@ fi set -eux -TEMPLATE_NAMESPACE="cordatest" +TEMPLATE_NAMESPACE="" +TEMPLATE_NAMESPACE=$(grep -A 3 'config:' $DIR/values.yaml | grep 'namespace: "' | cut -d '"' -f 2) + +if [ "$TEMPLATE_NAMESPACE" == "" ]; then + echo "Kubernetes requires a namespace to deploy resources to, no namespace is defined in values.yaml, please define one." + exit 1 +fi helm template $DIR --name $TEMPLATE_NAMESPACE --namespace $TEMPLATE_NAMESPACE --output-dir $DIR/output mv $DIR/output/corda/templates/pre-install.sh.yml $DIR/output/corda/templates/pre-install.sh diff --git a/helm/initial_registration/initial_registration.sh b/helm/initial_registration/initial_registration.sh index f6ea91e..b63ac21 100755 --- a/helm/initial_registration/initial_registration.sh +++ b/helm/initial_registration/initial_registration.sh @@ -39,12 +39,11 @@ function GetPathToCurrentlyExecutingScript () { GetPathToCurrentlyExecutingScript set -eux -TEMPLATE_NAMESPACE="cordatest" OUTPUT_DIR=$DIR/output SCRIPT=$OUTPUT_DIR/corda/templates/initial_registration.sh NODE_CONF=$OUTPUT_DIR/corda/templates/node.conf -helm template $DIR -f $DIR/../values.yaml --name $TEMPLATE_NAMESPACE --namespace $TEMPLATE_NAMESPACE --output-dir $OUTPUT_DIR --set-file node_conf=$DIR/../files/node.conf +helm template $DIR -f $DIR/../values.yaml --output-dir $OUTPUT_DIR --set-file node_conf=$DIR/../files/node.conf mv $OUTPUT_DIR/corda/templates/initial_registration.sh.yml $SCRIPT mv $OUTPUT_DIR/corda/templates/node.conf.yml $NODE_CONF diff --git a/helm/templates/create-docker-secret.sh.yml b/helm/templates/create-docker-secret.sh.yml index 6928fce..9746f4b 100644 --- a/helm/templates/create-docker-secret.sh.yml +++ b/helm/templates/create-docker-secret.sh.yml @@ -1,6 +1,41 @@ #!/bin/sh -ABS_PATH=$(readlink -f "$0") -DIR=$(dirname "$ABS_PATH") +set -ux +DIR="." +function GetPathToCurrentlyExecutingScript () { + # Absolute path of this script, e.g. /opt/corda/node/foo.sh + ABS_PATH=$(readlink -f "$0") + if [ "$?" -ne "0" ]; then + echo "readlink issue workaround..." + # Unfortunate MacOs issue with readlink functionality, see https://github.com/corda/corda-kubernetes-deployment/issues/4 + TARGET_FILE=$0 + + cd `dirname $TARGET_FILE` + TARGET_FILE=`basename $TARGET_FILE` + local ITERATIONS=0 + + # Iterate down a (possible) chain of symlinks + while [ -L "$TARGET_FILE" ] + do + TARGET_FILE=`readlink $TARGET_FILE` + cd `dirname $TARGET_FILE` + TARGET_FILE=`basename $TARGET_FILE` + ((++ITERATIONS)) + if [ "$ITERATIONS" -gt 1000 ]; then + echo "symlink loop. Critical exit." + exit 1 + fi + done + + # Compute the canonicalized name by finding the physical path + # for the directory we're in and appending the target file. + PHYS_DIR=`pwd -P` + ABS_PATH=$PHYS_DIR/$TARGET_FILE + fi + + # Absolute path of the directory this script is in, thus /opt/corda/node/ + DIR=$(dirname "$ABS_PATH") +} +GetPathToCurrentlyExecutingScript set -eux kubectl create secret docker-registry --dry-run=true container-registry-secret \ diff --git a/helm/templates/pre-install.sh.yml b/helm/templates/pre-install.sh.yml index bcf0eb7..d3eb8ea 100644 --- a/helm/templates/pre-install.sh.yml +++ b/helm/templates/pre-install.sh.yml @@ -1,10 +1,43 @@ #!/bin/sh +set -ux +DIR="." +function GetPathToCurrentlyExecutingScript () { + # Absolute path of this script, e.g. /opt/corda/node/foo.sh + ABS_PATH=$(readlink -f "$0") + if [ "$?" -ne "0" ]; then + echo "readlink issue workaround..." + # Unfortunate MacOs issue with readlink functionality, see https://github.com/corda/corda-kubernetes-deployment/issues/4 + TARGET_FILE=$0 -set -eux + cd `dirname $TARGET_FILE` + TARGET_FILE=`basename $TARGET_FILE` + local ITERATIONS=0 + + # Iterate down a (possible) chain of symlinks + while [ -L "$TARGET_FILE" ] + do + TARGET_FILE=`readlink $TARGET_FILE` + cd `dirname $TARGET_FILE` + TARGET_FILE=`basename $TARGET_FILE` + ((++ITERATIONS)) + if [ "$ITERATIONS" -gt 1000 ]; then + echo "symlink loop. Critical exit." + exit 1 + fi + done -ABS_PATH=$(readlink -f "$0") -DIR=$(dirname "$ABS_PATH") + # Compute the canonicalized name by finding the physical path + # for the directory we're in and appending the target file. + PHYS_DIR=`pwd -P` + ABS_PATH=$PHYS_DIR/$TARGET_FILE + fi + # Absolute path of the directory this script is in, thus /opt/corda/node/ + DIR=$(dirname "$ABS_PATH") +} +GetPathToCurrentlyExecutingScript + +set -eux APP={{ .Values.config.resourceName }}-node POD=$(kubectl get pod -l app=$APP -o jsonpath="{.items[0].metadata.name}") CORDAPPS_PATH=$(readlink -f "$DIR/../../../files/cordapps") diff --git a/helm/values.yaml b/helm/values.yaml index a386bd2..20d4790 100644 --- a/helm/values.yaml +++ b/helm/values.yaml @@ -38,6 +38,8 @@ setup: # config defines the options that relate to the deployment but not specifically to the Corda Node config: + # namespace defines the Kubernetes namespace that all resources will be deployed to + namespace: "cordatest" # resourceName defines the Kubernetes resource Name that all resources will share (with some added extensions to differentiate the different resources) resourceName: "corda-node-1" # Does not need editing unless deploying multiple nodes in the same cluster which should use unique resource names # dockerImageNode defines the Docker Image we will be deploying for the Corda Node diff --git a/one-time-setup.sh b/one-time-setup.sh index f2061ed..27c8542 100755 --- a/one-time-setup.sh +++ b/one-time-setup.sh @@ -57,5 +57,15 @@ $DIR/docker-images/push_docker_images.sh checkStatus $? $DIR/corda-pki-generator/generate_firewall_pki.sh checkStatus $? -$DIR/helm/initial_registration/initial_registration.sh -checkStatus $? + +INITIAL_REGISTRATION="" +INITIAL_REGISTRATION=$(grep -A 3 'initialRegistration:' $DIR/helm/values.yaml | grep 'enabled: ' | cut -d ':' -f 2 | xargs) + +if [ "$INITIAL_REGISTRATION" == "true" ]; then + $DIR/helm/initial_registration/initial_registration.sh + checkStatus $? +else + echo "Skipping initial registration step. (disabled in values.yaml)" +fi + +echo "One time setup script complete."