Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix relay chain path #272

Merged
merged 1 commit into from
Jul 7, 2023
Merged

Fix relay chain path #272

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion charts/node/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ apiVersion: v2
name: node
description: A Helm chart to deploy Substrate/Polkadot nodes
type: application
version: 5.1.0
version: 5.1.1
maintainers:
- name: Parity
url: https://github.com/paritytech/helm-charts
Expand Down
2 changes: 1 addition & 1 deletion charts/node/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ This is intended behaviour. Make sure to run `git add -A` once again to stage ch

# Substrate/Polkadot node helm chart

![Version: 5.1.0](https://img.shields.io/badge/Version-5.1.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square)
![Version: 5.1.1](https://img.shields.io/badge/Version-5.1.1-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square)

## Maintainers

Expand Down
18 changes: 9 additions & 9 deletions charts/node/templates/statefulset.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -160,36 +160,36 @@ spec:
- -c
- |
set -eu -o pipefail {{ if .Values.initContainers.downloadChainSnapshot.debug }}-x{{ end }}
if [ -d "/chain-data/chains/${RELAY_CHAIN_PATH}/{{ $databasePath }}" ]; then
if [ -d "/relaychain-data/chains/${RELAY_CHAIN_PATH}/{{ $databasePath }}" ]; then
echo "Database directory already exists, skipping chain snapshot download"
else
trap 'echo -e "Snapshot restoration failed. Checkout the logs for errors.\nRemoving /chain-data/chains/${CHAIN_PATH}/{{ $databasePath }} ..."; rm -rf /chain-data/chains/${CHAIN_PATH}/{{ $databasePath }}' ERR
trap 'echo -e "Snapshot restoration failed. Checkout the logs for errors.\nRemoving /relaychain-data/chains/${RELAY_CHAIN_PATH}/{{ $databasePath }} ..."; rm -rf /relaychain-data/chains/${RELAY_CHAIN_PATH}/{{ $databasePath }}' ERR
PARALLEL_TRANFERS="$(($(nproc --all) * 5 < 50 ? $(nproc --all) * 5 : 50))" # MIN(vCPU_count * 5, 50)
echo "Downloading chain snapshot"
SNAPSHOT_URL="{{ .Values.node.collatorRelayChain.chainData.chainSnapshot.url }}"
mkdir -p /chain-data/chains/${RELAY_CHAIN_PATH}/{{ $databasePath }}/
mkdir -p /relaychain-data/chains/${RELAY_CHAIN_PATH}/{{ $databasePath }}/

if [ "${METHOD}" == "http-single-tar-lz4" ]; then
apk add lz4 --no-cache
rclone copyurl {{ .Values.initContainers.downloadChainSnapshot.cmdArgs }} --stdout --error-on-no-transfer ${SNAPSHOT_URL} | lz4 -c -d - | tar -x -C /chain-data/chains/${RELAY_CHAIN_PATH}/
chown -R {{ .Values.podSecurityContext.runAsUser }}:{{ .Values.podSecurityContext.fsGroup }} /chain-data/chains/${RELAY_CHAIN_PATH}/
rclone copyurl {{ .Values.initContainers.downloadChainSnapshot.cmdArgs }} --stdout --error-on-no-transfer ${SNAPSHOT_URL} | lz4 -c -d - | tar -x -C /relaychain-data/chains/${RELAY_CHAIN_PATH}/
chown -R {{ .Values.podSecurityContext.runAsUser }}:{{ .Values.podSecurityContext.fsGroup }} /relaychain-data/chains/${RELAY_CHAIN_PATH}/

elif [ "${METHOD}" == "http-single-tar" ]; then
rclone copyurl {{ .Values.initContainers.downloadChainSnapshot.cmdArgs }} --stdout --error-on-no-transfer ${SNAPSHOT_URL} | tar -x -C /chain-data/chains/${RELAY_CHAIN_PATH}/
rclone copyurl {{ .Values.initContainers.downloadChainSnapshot.cmdArgs }} --stdout --error-on-no-transfer ${SNAPSHOT_URL} | tar -x -C /relaychain-data/chains/${RELAY_CHAIN_PATH}/

elif [ "${METHOD}" == "gcs" ]; then
LATEST=$(rclone cat {{ .Values.initContainers.downloadChainSnapshot.cmdArgs }} --quiet :gcs:${SNAPSHOT_URL}/latest_version.meta.txt)
if [ -z "$LATEST" ]; then
echo "Failed to retrieve latest_version.meta.txt file. Will download everything from ${SNAPSHOT_URL} instead"
fi
rclone sync {{ .Values.initContainers.downloadChainSnapshot.cmdArgs }} --fast-list --transfers $PARALLEL_TRANFERS --progress --error-on-no-transfer :gcs:${SNAPSHOT_URL}/${LATEST} /chain-data/chains/${RELAY_CHAIN_PATH}/{{ $databasePath }}/
rclone sync {{ .Values.initContainers.downloadChainSnapshot.cmdArgs }} --fast-list --transfers $PARALLEL_TRANFERS --progress --error-on-no-transfer :gcs:${SNAPSHOT_URL}/${LATEST} /relaychain-data/chains/${RELAY_CHAIN_PATH}/{{ $databasePath }}/

elif [ "${METHOD}" == "s3" ]; then
LATEST=$(rclone cat {{ .Values.initContainers.downloadChainSnapshot.cmdArgs }} --quiet :s3:${SNAPSHOT_URL}/latest_version.meta.txt )
if [ -z "$LATEST" ]; then
echo "Failed to retrieve latest_version.meta.txt file. Will download everything from ${SNAPSHOT_URL} instead"
fi
rclone sync {{ .Values.initContainers.downloadChainSnapshot.cmdArgs }} --fast-list --transfers $PARALLEL_TRANFERS --progress --error-on-no-transfer :s3:${SNAPSHOT_URL}/${LATEST} /chain-data/chains/${RELAY_CHAIN_PATH}/{{ $databasePath }}/
rclone sync {{ .Values.initContainers.downloadChainSnapshot.cmdArgs }} --fast-list --transfers $PARALLEL_TRANFERS --progress --error-on-no-transfer :s3:${SNAPSHOT_URL}/${LATEST} /relaychain-data/chains/${RELAY_CHAIN_PATH}/{{ $databasePath }}/

elif [ "${METHOD}" == "http-filelist" ]; then
LATEST=$(rclone copyurl {{ .Values.initContainers.downloadChainSnapshot.cmdArgs }} --stdout ${SNAPSHOT_URL}/latest_version.meta.txt )
Expand All @@ -199,7 +199,7 @@ spec:
SNAPSHOT_URL="${SNAPSHOT_URL}/${LATEST}"
fi
rclone copyurl {{ .Values.initContainers.downloadChainSnapshot.cmdArgs }} --error-on-no-transfer ${SNAPSHOT_URL}/{{ .Values.node.collatorRelayChain.chainData.chainSnapshot.filelistName }} /tmp/filelist.txt
rclone copy {{ .Values.initContainers.downloadChainSnapshot.cmdArgs }} --progress --error-on-no-transfer --transfers $PARALLEL_TRANFERS --http-url ${SNAPSHOT_URL} --no-traverse --http-no-head --disable-http2 --files-from /tmp/filelist.txt :http: /chain-data/chains/${RELAY_CHAIN_PATH}/{{ $databasePath }}/
rclone copy {{ .Values.initContainers.downloadChainSnapshot.cmdArgs }} --progress --error-on-no-transfer --transfers $PARALLEL_TRANFERS --http-url ${SNAPSHOT_URL} --no-traverse --http-no-head --disable-http2 --files-from /tmp/filelist.txt :http: /relaychain-data/chains/${RELAY_CHAIN_PATH}/{{ $databasePath }}/
fi
fi
env:
Expand Down