-
-
Notifications
You must be signed in to change notification settings - Fork 8
Kubernetes in Cloud
kubectl create secret generic secrets \
--from-file=$HOME/.cloudvolume/secrets/google-secret.json \
--from-file=$HOME/.cloudvolume/secrets/aws-secret.json \
--from-file=$HOME/.cloudvolume/secrets/boss-secret.json
delete secret kubectl delete secret --all
get the pod id
kubectl get pods
watch the logs
watch kubectl logs pod-id
config dns servers
kubectl edit configmap kube-dns-autoscaler --namespace=kube-system
We can monitor the traffic operations per second in Kubernetes deployment monitor interface.
There is no bandwidth usage in Kubernetes, we can only monitor it in instance group
monitor interface.
- create:
kubectl apply -f deploy.yml
- check:
kubectl get deployments
- delete:
kubectl delete deployment inference
- scale:
kubectl scale --replicas=85 -f deploy.yml
The complete documentation.
easy to use web console
gcloud container clusters resize my-cluster --region us-central1 --node-pool gpu-pool-1 --num-nodes 1
gcloud container clusters get-credentials my-cluster
kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/container-engine-accelerators/stable/nvidia-driver-installer/cos/daemonset-preloaded.yaml
we use kops to create clusters in AWS. It is easy to use and manage. kops is the kubectl of cluster. AWS was officially supported and it can also work with other cloud vendors.
follow the setup here.
create-cluster.sh
#!/bin/sh
# use k8x.local in cluster name to enable gossip DNS setup
export NAME=chunkflow.k8s.local
export KOPS_STATE_STORE=s3://chunkflow-cluster-state-store
kops create cluster $NAME \
--zones us-east-1c \
--master-count 1 \
--master-zones us-east-1c \
--master-size m3.medium \
--master-volume-size 10 \
--node-count 1 \
--node-size p2.xlarge \
--node-volume-size 20 \
--cloud-labels "Team=MyTeam,Owner=MyName" \
--dry-run \
-o yaml > $NAME.yaml
run this script and it will generate yml file for kops
./create-cluster.sh
edite the yaml file and add maxPrice: "your_bidding_price"
under the worker section. The maxPrice
item will make kops allocate spot instances rather than default on-demand instances. This feature was not supported in command line interface yet. That's why we need the yaml file here.
kops create -f chunkflow.k8s.local.yaml
kops create secret --name chunkflow.k8s.local sshpublickey admin -i ~/.ssh/id_rsa.pub
kops update cluster chunkflow.k8s.local --yes
kubectl get nodes
delete cluster
kops delete -f chunkflow.k8s.local.yaml --yes
Note that the setup only support CUDA9.1. K80 GPUs needs cuda9 and T4 needs cuda10. This is required for both Google Cloud and AWS!
In default, kubectl will be set up after using kops. We'll lost the kubectl connection to AWS once we connect it to other cloud, such as Google Cloud. We can reconnect kubectl to AWS following the steps here.