Docker and Kubernetes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Docker and Kubernetes\n",
    "> - This notebook describes the process of using a production-style development workflow to build, test and deploy Docker applications with Kubernetes, a set of notes from https://www.udemy.com/docker-and-kubernetes-the-complete-guide/.\n",
    "- Author: Brant Yukan, b.rant@ucla.edu\n",
    "- Date: 9/13/19"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Goal: Use Docker to easily install and run software without troubleshooting setup  or dependencies.\n",
    "- e.g. docker run -it redis\n",
    "- An image is a file with all the dependencies and configurations required to run a program.  Containers are instances of an image."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### install on mac"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mWarning:\u001b[0m Cask 'docker' is already installed.\r\n",
      "\r\n",
      "To re-install docker, run:\r\n",
      "  \u001b[32mbrew cask reinstall docker\u001b[39m\r\n"
     ]
    }
   ],
   "source": [
    "!brew cask install docker"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Open Docker Desktop and sign in.  Here is my Docker Hub profile: https://hub.docker.com/u/byukan\n",
    "- For NLPIA, Docker instructions are in the readme https://github.com/totalgood/nlpia:\n",
    "- https://hub.docker.com/r/hoangchunghien/nlpia\n",
    "```console\n",
    "TODO: find a simple tutorial on docker workflow with jupyter notebook\n",
    "docker pull hoangchunghien/nlpia\n",
    "docker build -t nlpia .\n",
    "docker run -p 8888:8888 hoangchunghien/nlpia\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Client: Docker Engine - Community\r\n",
      " Version:           19.03.2\r\n",
      " API version:       1.40\r\n",
      " Go version:        go1.12.8\r\n",
      " Git commit:        6a30dfc\r\n",
      " Built:             Thu Aug 29 05:26:49 2019\r\n",
      " OS/Arch:           darwin/amd64\r\n",
      " Experimental:      false\r\n",
      "\r\n",
      "Server: Docker Engine - Community\r\n",
      " Engine:\r\n",
      "  Version:          19.03.2\r\n",
      "  API version:      1.40 (minimum version 1.12)\r\n",
      "  Go version:       go1.12.8\r\n",
      "  Git commit:       6a30dfc\r\n",
      "  Built:            Thu Aug 29 05:32:21 2019\r\n",
      "  OS/Arch:          linux/amd64\r\n",
      "  Experimental:     false\r\n",
      " containerd:\r\n",
      "  Version:          v1.2.6\r\n",
      "  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb\r\n",
      " runc:\r\n",
      "  Version:          1.0.0-rc8\r\n",
      "  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f\r\n",
      " docker-init:\r\n",
      "  Version:          0.18.0\r\n",
      "  GitCommit:        fec3683\r\n"
     ]
    }
   ],
   "source": [
    "!docker version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### using docker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r\n",
      "Hello from Docker!\r\n",
      "This message shows that your installation appears to be working correctly.\r\n",
      "\r\n",
      "To generate this message, Docker took the following steps:\r\n",
      " 1. The Docker client contacted the Docker daemon.\r\n",
      " 2. The Docker daemon pulled the \"hello-world\" image from the Docker Hub.\r\n",
      "    (amd64)\r\n",
      " 3. The Docker daemon created a new container from that image which runs the\r\n",
      "    executable that produces the output you are currently reading.\r\n",
      " 4. The Docker daemon streamed that output to the Docker client, which sent it\r\n",
      "    to your terminal.\r\n",
      "\r\n",
      "To try something more ambitious, you can run an Ubuntu container with:\r\n",
      " $ docker run -it ubuntu bash\r\n",
      "\r\n",
      "Share images, automate workflows, and more with a free Docker ID:\r\n",
      " https://hub.docker.com/\r\n",
      "\r\n",
      "For more examples and ideas, visit:\r\n",
      " https://docs.docker.com/get-started/\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "# start up new container using image with name hello-world\n",
    "# first time, daemon pulls image from docker hub, after that it's saved in image cache\n",
    "!docker run hello-world"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- A container is a set of processes that have a groupig of resources specifically assigned to it.\n",
    "- An image is a filesystem snapshot and startup command.\n",
    "- When you create an image, your Linux kernel creates a control gorup, isolates a portion of the hard drive, RAM, CPU, and network to a container.  The filesystem snapshot gets placed in that subsection of the hard drive.  The startup command uses namespacing and runs on the isolated set of resources."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### docker CLI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hi there\r\n"
     ]
    }
   ],
   "source": [
    "# default command overrride\n",
    "!docker run busybox echo hi there"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bin\r\n",
      "dev\r\n",
      "etc\r\n",
      "home\r\n",
      "proc\r\n",
      "root\r\n",
      "sys\r\n",
      "tmp\r\n",
      "usr\r\n",
      "var\r\n"
     ]
    }
   ],
   "source": [
    "# folders inside the container\n",
    "!docker run busybox ls"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "docker: Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused \"exec: \\\"echo\\\": executable file not found in $PATH\": unknown.\r\n",
      "\u001b[31mERRO\u001b[0m[0001] error waiting for container: context canceled \r\n"
     ]
    }
   ],
   "source": [
    "# ls and echo don't exist inside the hello-world filesystem.\n",
    "!docker run hello-world echo hi there"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root:x:0:0:root:/root:/bin/sh\r\n",
      "daemon:x:1:1:daemon:/usr/sbin:/bin/false\r\n",
      "bin:x:2:2:bin:/bin:/bin/false\r\n",
      "sys:x:3:3:sys:/dev:/bin/false\r\n",
      "sync:x:4:100:sync:/bin:/bin/sync\r\n",
      "mail:x:8:8:mail:/var/spool/mail:/bin/false\r\n",
      "www-data:x:33:33:www-data:/var/www:/bin/false\r\n",
      "operator:x:37:37:Operator:/var:/bin/false\r\n",
      "nobody:x:65534:65534:nobody:/home:/bin/false\r\n"
     ]
    }
   ],
   "source": [
    "# linux systems used to store passwords in here, but now it's in shadow which is hidden.\n",
    "!docker run busybox cat /etc/passwd\n",
    "# !docker run busybox cat /etc/shadow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES\r\n"
     ]
    }
   ],
   "source": [
    "# lists all running containers running on machine\n",
    "# !docker run busybox ping google.com\n",
    "!docker ps\n",
    "!docker ps --all  # all containers that we've ever started up on the machine"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- docker run creates and starts an image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "758d268415b3d06b98486971a18851856b2a7d51d8a69f55a12b67b43022fb02\r\n"
     ]
    }
   ],
   "source": [
    "!docker create hello-world\n",
    "# !docker create busybox echo hi there"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r\n",
      "Hello from Docker!\r\n",
      "This message shows that your installation appears to be working correctly.\r\n",
      "\r\n",
      "To generate this message, Docker took the following steps:\r\n",
      " 1. The Docker client contacted the Docker daemon.\r\n",
      " 2. The Docker daemon pulled the \"hello-world\" image from the Docker Hub.\r\n",
      "    (amd64)\r\n",
      " 3. The Docker daemon created a new container from that image which runs the\r\n",
      "    executable that produces the output you are currently reading.\r\n",
      " 4. The Docker daemon streamed that output to the Docker client, which sent it\r\n",
      "    to your terminal.\r\n",
      "\r\n",
      "To try something more ambitious, you can run an Ubuntu container with:\r\n",
      " $ docker run -it ubuntu bash\r\n",
      "\r\n",
      "Share images, automate workflows, and more with a free Docker ID:\r\n",
      " https://hub.docker.com/\r\n",
      "\r\n",
      "For more examples and ideas, visit:\r\n",
      " https://docs.docker.com/get-started/\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "# -a attaches to the the container, and prints out output\n",
    "!docker start -a 758d268415b3d06b98486971a18851856b2a7d51d8a69f55a12b67b43022fb02"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- start a container again that has exited"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                      PORTS               NAMES\r\n",
      "de35e291bbf4        hello-world         \"/hello\"            3 minutes ago       Exited (0) 2 minutes ago                        frosty_raman\r\n",
      "db6aa915f408        busybox             \"ping google.com\"   10 minutes ago      Exited (0) 10 minutes ago                       vigorous_tharp\r\n",
      "55d90a513220        busybox             \"cat /etc/passwd\"   18 hours ago        Exited (0) 18 hours ago                         cranky_dirac\r\n",
      "fd72095fd08d        busybox             \"cat /etc/shadow\"   18 hours ago        Exited (0) 18 hours ago                         brave_galileo\r\n",
      "e4e789077ee7        busybox             \"cat /etc/passwd\"   18 hours ago        Exited (0) 18 hours ago                         nice_solomon\r\n",
      "9ff64b236e53        hello-world         \"echo hi there\"     18 hours ago        Created                                         infallible_antonelli\r\n",
      "454cbaa57b0a        hello-world         \"ls\"                18 hours ago        Created                                         magical_burnell\r\n",
      "f920e4544300        busybox             \"ls\"                18 hours ago        Exited (0) 18 hours ago                         pedantic_northcutt\r\n",
      "688feb761a84        busybox             \"ls\"                18 hours ago        Exited (0) 18 hours ago                         condescending_agnesi\r\n",
      "68e42d6ce2ea        busybox             \"echo hi there\"     18 hours ago        Exited (0) 18 hours ago                         brave_hypatia\r\n",
      "4a3b9833d964        busybox             \"echo hi there\"     18 hours ago        Exited (0) 18 hours ago                         bold_hellman\r\n",
      "eb32d4ab2d1f        busybox             \"sh\"                18 hours ago        Exited (0) 18 hours ago                         nostalgic_wescoff\r\n",
      "07f1936944ec        hello-world         \"/hello\"            18 hours ago        Exited (0) 18 hours ago                         compassionate_northcutt\r\n",
      "4546544a3b64        hello-world         \"/hello\"            19 hours ago        Exited (0) 19 hours ago                         stupefied_lederberg\r\n",
      "f0963f1fa6a6        hello-world         \"/hello\"            19 hours ago        Exited (0) 19 hours ago                         zen_perlman\r\n"
     ]
    }
   ],
   "source": [
    "!docker ps --all"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hi there\r\n"
     ]
    }
   ],
   "source": [
    "!docker start -a 68e42d6ce2ea\n",
    "# can't replace default command of a container that's already been created"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "delete containers taking up disk space\n",
    "docker system prune"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "340349d7bb40cf1f97029babc21ab86328eff7d6bafea33c2684477661262a0c\r\n"
     ]
    }
   ],
   "source": [
    "!docker create busybox echo hi there"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "340349d7bb40cf1f97029babc21ab86328eff7d6bafea33c2684477661262a0c\r\n"
     ]
    }
   ],
   "source": [
    "!docker start 340349d7bb40cf1f97029babc21ab86328eff7d6bafea33c2684477661262a0c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hi there\r\n"
     ]
    }
   ],
   "source": [
    "# docker logs does not re-run.  It gets a record of all the logs that have been emitted from that container.\n",
    "!docker logs 340349d7bb40cf1f97029babc21ab86328eff7d6bafea33c2684477661262a0c"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### stop running container, ^C or stop/kill\n",
    "- stop sends a SIGTERM hardware signal message to stop at its own time; docker automatically kills after 10 seconds\n",
    "- kill issues a SIGKILL, shut down right now"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "d6904dc52d0d3c039401cfb9a63cdfec70dd8ab4af60f8995aff1091114e041d\r\n"
     ]
    }
   ],
   "source": [
    "!docker create busybox ping google.com"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "d6904dc52d0d3c039401cfb9a63cdfec70dd8ab4af60f8995aff1091114e041d\n",
      "PING google.com (172.217.6.78): 56 data bytes\n",
      "64 bytes from 172.217.6.78: seq=0 ttl=37 time=5.438 ms\n"
     ]
    }
   ],
   "source": [
    "!docker start d6904dc52d0d3c039401cfb9a63cdfec70dd8ab4af60f8995aff1091114e041d\n",
    "!docker logs d6904dc52d0d3c039401cfb9a63cdfec70dd8ab4af60f8995aff1091114e041d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES\r\n",
      "f7769ba90a0a        busybox             \"ping google.com\"   5 minutes ago       Up 3 minutes                            eloquent_boyd\r\n"
     ]
    }
   ],
   "source": [
    "!docker ps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "f7769ba90a0a\r\n"
     ]
    }
   ],
   "source": [
    "!docker stop f7769ba90a0a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES\r\n"
     ]
    }
   ],
   "source": [
    "!docker ps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Error response from daemon: Cannot kill container: f7769ba90a0a: Container f7769ba90a0a605cc53ed8313b6c2f79e401eeb796ed3e3984d32f5fcd64035b is not running\r\n"
     ]
    }
   ],
   "source": [
    "# !docker kill f7769ba90a0a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- redis is an in-memory data store commonly used with web applications"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In iTerm2, run:\n",
    "```console\n",
    "docker run redis\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/bin/sh: redis-cli: command not found\r\n"
     ]
    }
   ],
   "source": [
    "# you have to execute commands inside the container\n",
    "!redis-cli"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES\r\n",
      "b8219f9e6d4c        redis               \"docker-entrypoint.s…\"   8 minutes ago       Up 8 minutes        6379/tcp            amazing_northcutt\r\n"
     ]
    }
   ],
   "source": [
    "!docker ps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-it allows us to provide input to the container, -i means attach terminal to STDIN of the container's command ; -t lets the output show up pretty\n",
    "***\n",
    "**docker exec -it b8219f9e6d4c redis-cli**  \n",
    "**set myvalue 5**  \n",
    "**get myvalue**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### how to open up shell in context of your running container"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS               NAMES\r\n",
      "b8219f9e6d4c        redis               \"docker-entrypoint.s…\"   22 minutes ago      Up 22 minutes       6379/tcp            amazing_northcutt\r\n"
     ]
    }
   ],
   "source": [
    "!docker ps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get full terminal access to the container, get out with ^D\n",
    "sh is a command processor to open a shell.  Others are bash, powershell, zsh, sh.  Most containers have sh, some have bash.\n",
    "\n",
    "\n",
    "**docker exec -it b8219f9e6d4c sh**"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "docker run -it busybox sh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Container Isolation\n",
    "- containers don't share their file system\n",
    "- You have to specifically form a connection between two containers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating Docker Images\n",
    "- The process is to specify a base image, run commands to install additional programs, then provde a command to run on startup.\n",
    "- create a dockerfile, plaintext with lines of configuration about programs it contains and what it does when it starts up\n",
    "- client passes the file to a server, which builds a usable image\n",
    "    1. specify a base image\n",
    "    2. run commands to install additional programs\n",
    "    3. specify a command to run on container startup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# create an image that runs redis-server\n",
    "mkdir redis-image\n",
    "cd redis-image\n",
    "code ."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- make a file named Dockerfile:\n",
    "\n",
    "Dockerfile contents:\n",
    "- Use an existing docker image as a base\n",
    "- alpine comes with a preinstalled set of programs that are useful, such as apk package manager\n",
    "```Dockerfile\n",
    "FROM alpine\n",
    "```\n",
    "\n",
    "- Download and install a dependency\n",
    "- executes command while preparing custom image\n",
    "- apk is the Apache package manager.\n",
    "```Dockerfile\n",
    "RUN apk add --update redis\n",
    "```\n",
    "- Tell the image what to do when it starts as a container\n",
    "- what should be ran when image is used to start a new container\n",
    "```Dockerfile\n",
    "CMD [\"redis-server\"]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- build uses the Dockerfile to generate an image. '.' is the build context, which are the files and folders in our project\n",
    "- The build process creates a temporary intermediate container and takes a snapshot of its filesystem and its default command.\n",
    "- At each instrution step, we create a new container from the previous step, execute a command or change its file system, and take a snapshot of its filesystem and save it as an ouptut for next instruction along the chain.  The image from the last step is the final one.\n",
    "\n",
    "```console\n",
    "cd redis-image\n",
    "\n",
    "docker build .\n",
    "\n",
    "Successfully built 74873fee6d3c\n",
    "\n",
    "docker run 74873fee6d3c\n",
    "```\n",
    "... Ready to accept connections"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Docker build uses an image that's already cached in your machine if the command is the same and has been done before.\n",
    "- But the sequence of commands has to be the same.  Docker build uses the cache only up until any saved sequence or subsequence.\n",
    "- If you add a line to change the Dockerfile, try to add it at the bottom."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tagging an Image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sending build context to Docker daemon  2.048kB\n",
      "Step 1/4 : FROM alpine\n",
      " ---> 961769676411\n",
      "Step 2/4 : RUN apk add --update gcc\n",
      " ---> Running in 3118057c32d8\n",
      "fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz\n",
      "fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/x86_64/APKINDEX.tar.gz\n",
      "(1/10) Installing binutils (2.32-r0)\n",
      "(2/10) Installing gmp (6.1.2-r1)\n",
      "(3/10) Installing isl (0.18-r0)\n",
      "(4/10) Installing libgomp (8.3.0-r0)\n",
      "(5/10) Installing libatomic (8.3.0-r0)\n",
      "(6/10) Installing libgcc (8.3.0-r0)\n",
      "(7/10) Installing mpfr3 (3.1.5-r1)\n",
      "(8/10) Installing mpc1 (1.1.0-r0)\n",
      "(9/10) Installing libstdc++ (8.3.0-r0)\n",
      "(10/10) Installing gcc (8.3.0-r0)\n",
      "Executing busybox-1.30.1-r2.trigger\n",
      "OK: 92 MiB in 24 packages\n",
      "Removing intermediate container 3118057c32d8\n",
      " ---> d55abcf1f349\n",
      "Step 3/4 : RUN apk add --update redis\n",
      " ---> Running in 1c9e138529d2\n",
      "(1/1) Installing redis (5.0.5-r0)\n",
      "Executing redis-5.0.5-r0.pre-install\n",
      "Executing redis-5.0.5-r0.post-install\n",
      "Executing busybox-1.30.1-r2.trigger\n",
      "OK: 93 MiB in 25 packages\n",
      "Removing intermediate container 1c9e138529d2\n",
      " ---> 70a4bcee0f90\n",
      "Step 4/4 : CMD [\"redis-server\"]\n",
      " ---> Running in b679d80b31ac\n",
      "Removing intermediate container b679d80b31ac\n",
      " ---> b3e201d7b578\n",
      "Successfully built b3e201d7b578\n",
      "Successfully tagged byukan/redis:latest\n"
     ]
    }
   ],
   "source": [
    "!docker build -t byukan/redis:latest redis-image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1:C 23 Sep 2019 00:37:27.760 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo\n",
      "1:C 23 Sep 2019 00:37:27.760 # Redis version=5.0.5, bits=64, commit=47edd290, modified=0, pid=1, just started\n",
      "1:C 23 Sep 2019 00:37:27.760 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf\n",
      "1:M 23 Sep 2019 00:37:27.762 * Running mode=standalone, port=6379.\n",
      "1:M 23 Sep 2019 00:37:27.762 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.\n",
      "1:M 23 Sep 2019 00:37:27.762 # Server initialized\n",
      "1:M 23 Sep 2019 00:37:27.762 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.\n",
      "1:M 23 Sep 2019 00:37:27.762 * Ready to accept connections\n",
      "^C\n",
      "1:signal-handler (1569199550) Received SIGINT scheduling shutdown...\n",
      "1:M 23 Sep 2019 00:45:50.966 # User requested shutdown...\n",
      "1:M 23 Sep 2019 00:45:50.966 * Saving the final RDB snapshot before exiting.\n",
      "1:M 23 Sep 2019 00:45:50.974 * DB saved on disk\n",
      "1:M 23 Sep 2019 00:45:50.974 # Redis is now ready to exit, bye bye...\n"
     ]
    }
   ],
   "source": [
    "!docker run byukan/redis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Generate Image from Container\n",
    "- without Dockerfile\n",
    "- Generally, you'll want to use the Dockerfile because it's easier to rerun."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- docker run -it alpine sh\n",
    "> - apk add --update redis\n",
    "- get id of running container:\n",
    "- docker ps\n",
    "- docker commit -c 'CMD [\"redis-server\"]' 891302133f1d\n",
    "\n",
    "- Then, \n",
    "- docker run 811bcd5935af581\n",
    "- stats up a new container out of the image we just created"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create Node JS web app and wrap it in a Docker container\n",
    "- create Node JS web app\n",
    "- create a Dockerfile\n",
    "- Build image from dockerfile\n",
    "- run image as a container\n",
    "- connect to web app from a browser"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```console\n",
    "cd redis-image\n",
    "mkdir simpleweb\n",
    "cd simpleweb\n",
    "code .\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- install dependencies\n",
    "- npm install\n",
    "- start server, this command runs on container startup\n",
    "- npm start"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Add files inside a container\n",
    "- copying build files\n",
    "- first argument is copy from path on your machine realative to build context, meaning what is specified in the __docker build .__ command.\n",
    "- second argument is place to copy stuff to inside the container\n",
    "```Dockerfile\n",
    "COPY ./ ./\n",
    "```\n",
    "\n",
    "copy put files into the root directory of the container.  \n",
    "The build process we used is not best practice because if we have any files or folers that conflict with the default folder   system, then we'd overwrite files or folders inside the container.  \n",
    "ls shows files that were created by npm  \n",
    "- Copy into a nested directory instead.  e.g.:\n",
    "```Dockerfile\n",
    "WORKDIR /usr/app\n",
    "```\n",
    "- Any following command will be executed relative to this path.\n",
    "- Then, Copy won't copy into the root directory, it will copy into the workdir instead.\n",
    "\n",
    "\n",
    "```Dockerfile\n",
    "# Specify a base image\n",
    "FROM node:alpine\n",
    "\n",
    "WORKDIR /usr/app\n",
    "\n",
    "# Install some dependencies\n",
    "COPY ./ ./\n",
    "RUN npm install\n",
    "\n",
    "# Default command\n",
    "CMD [\"npm\", \"start\"]\n",
    "```\n",
    "\n",
    "\n",
    "- We dont't have to rerun all the steps of reinstalling dependencies after copy if you modify some files.\n",
    "\n",
    "```Dockerfile\n",
    "# Specify a base image\n",
    "FROM node:alpine\n",
    "\n",
    "WORKDIR /usr/app\n",
    "\n",
    "# Install some dependencies\n",
    "# With this build process, if you make changes to index.js, you won't have to reinstall all the dependencies again.\n",
    "# You won't invalidate the cache for parts taht are working.\n",
    "# Split the copy operation out into 2 different steps\n",
    "# npm install only cares about the package.json file\n",
    "COPY ./package.json ./\n",
    "RUN npm install \n",
    "COPY ./ ./\n",
    "\n",
    "# Default command\n",
    "CMD [\"npm\", \"start\"]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "in codechef-NLP/redis-image/simpleweb:\n",
    "```console\n",
    "docker build -t byukan/simpleweb .\n",
    "docker run byukan/simpleweb\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```console\n",
    "BYUKAN-M-H10C:simpleweb byukan$ docker run byukan/simpleweb\n",
    "\n",
    "> @ start /\n",
    "> node index.js\n",
    "\n",
    "Listening on port 8080\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Container Port Mapping"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- http://localhost:8080/ in browser\n",
    "- The container has its own isolated set of ports that can receive traffic, but by default no incomming traffic to your computer is going to be directed to your container.\n",
    "- Create a port mapping so that requests made to a port on your local network get forwarded to a port on the docker container.\n",
    "- There's no limitation by default of the container's ability to reach out.\n",
    "\n",
    "route incoming requests on this port on local host to : port inside container\n",
    "\n",
    "build if you haven't already\n",
    "```console\n",
    "docker build -t byukan/simpleweb .\n",
    "docker run -p 5000:8080 byukan/simpleweb\n",
    "\n",
    "```\n",
    "\n",
    "Then, go to http://localhost:5000/\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```console\n",
    "docker run -it byukan/simpleweb sh\n",
    "```  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```console\n",
    "docker ps\n",
    "docker exec -it  sh\n",
    "```\n",
    "- When you modify a file inside the container, you have to do additional fancy configuration.  You'd have to completely rebuild the container."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Web App that displays number of times someone has visited the server\n",
    "- need 2 separate components:\n",
    "- will need a Node App web server to respond to http requests and generate html to show inside browser\n",
    "- also needs a redis server (in-memory data store) to keep track of number of times the page has been visited.\n",
    "- If you put both components in 1 container, you'd run into trouble if you introduce more web application servers to respond to http requests.  Then you'd have disconnected instances.  \n",
    "- Instead have 1 single instance and if you want to scale, then create more instances of the Node server.  Each Docker container containing the node app connects to the container w/ the Redis server."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### App Server code"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir visits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "modify index.js      package.json files  Dockerfile\n",
    "- Connect containers with some form of networking.\n",
    "- To connect 2 separate containers, you can use Docker Compose, which is used to start up multiple Docker containers at the same time.\n",
    "- Docker compose automates some of the arguments we were passing to 'docker run'.\n",
    "```console\n",
    "docker build -t byukan/visits:latest .\n",
    "docker run redis\n",
    "docker run byukan/visits\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Docker Compose\n",
    "- compile all the docker cli commands into a docker-compose.yml file.  Then feed it into a docker-compose CLI.\n",
    "- We'll use docker-compose to create a resdis-ser ver using the 'redis' image, then a node-app container using the dockerfile that maps port 8081 to 8081.\n",
    "```yml\n",
    "# docker-compose.yml\n",
    "version: '3'\n",
    "services:\n",
    "  redis-server:\n",
    "    image: 'redis'\n",
    "  node-app:\n",
    "    build: .\n",
    "      - \"4001:8081\"\n",
    "```\n",
    "- By defining the two containers in the same docker-compose.yml, they will automatically get configured to talk to each other.\n",
    "\n",
    "- Instead of saying run, and instead of saying build and run:\n",
    "- -d launches a group of containers in the background\n",
    "```console\n",
    "docker-compose up\n",
    "docker-compose up --build\n",
    "docker-compose up -d\n",
    "```\n",
    "\n",
    "\n",
    "- close them using:\n",
    "```console\n",
    "docker-compose down\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Restart container when server crashes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```javascript\n",
    "const redis = require('redis');\n",
    "const process = require('process');\n",
    "\n",
    "// instance of express application\n",
    "const app = express();\n",
    "// set up connection to redis server\n",
    "// specify the host/url/address we're trying to connect to\n",
    "const client = redis.createClient({\n",
    "    host: 'redis-server',\n",
    "    post: 6379\n",
    "});\n",
    "client.set('visits', 0);\n",
    "\n",
    "// route handler for root route\n",
    "app.get('/', (req, res) => {\n",
    "    process.exit(0);\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Automatic container restart\n",
    "\n",
    "- When you run docker-compose up --build, the app crashes, and docker ps shows that the node-app container is no longer running, only the redis container.\n",
    "- Specify a restart policy:\n",
    "    - \"no\", quotes must be specifically put, because in a yml file, no gets interpreted as false, default\n",
    "    - always\n",
    "    - on-failure, only restart if the container stops with an error code\n",
    "    - unless-stopped, always restart unless we forcibly stop it\n",
    "```yml\n",
    "version: '3'\n",
    "services:\n",
    "  redis-server:\n",
    "    image: 'redis'\n",
    "  node-app:\n",
    "    restart: always\n",
    "    build: .\n",
    "    ports:\n",
    "      - \"4001:8081\"\n",
    "```\n",
    "\n",
    "- You'll see that the server crashes, then restarts:\n",
    "\n",
    "```console\n",
    "node-app_1      |\n",
    "node-app_1      | > @ start /app\n",
    "node-app_1      | > node index.js\n",
    "node-app_1      |\n",
    "node-app_1      | Listening on port 8081\n",
    "node-app_1      |\n",
    "node-app_1      | > @ start /app\n",
    "node-app_1      | > node index.js\n",
    "node-app_1      |\n",
    "node-app_1      | Listening on port 8081\n",
    "visits_node-app_1 exited with code 0\n",
    "node-app_1      |\n",
    "node-app_1      | > @ start /app\n",
    "node-app_1      | > node index.js\n",
    "node-app_1      |\n",
    "node-app_1      | Listening on port 8081\n",
    "visits_node-app_1 exited with code 0\n",
    "node-app_1      |\n",
    "node-app_1      | > @ start /app\n",
    "node-app_1      | > node index.js\n",
    "node-app_1      |\n",
    "node-app_1      | Listening on port 8081s\n",
    "```\n",
    "\n",
    "- docker-compose ps looks inside the directory for a docker-compose.yml as a reference to understand from which specific containers it's supposed to get the status\n",
    "```console\n",
    "docker-compose ps\n",
    "        Name                       Command               State     Ports\n",
    "--------------------------------------------------------------------------\n",
    "visits_node-app_1       docker-entrypoint.sh npm start   Exit 0\n",
    "visits_redis-server_1   docker-entrypoint.sh redis ...   Up       6379/tcp\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Development Workflow\n",
    "- develop in a docker container and then push it to an outside hosting service like AWS\n",
    "- Flow:  Development, Testing, Deployment, repeat\n",
    "> Github Repo\n",
    "- 2 branches:  \n",
    "feature  : development branch, add code, make changes  \n",
    "master:  very clean working copy of codebase, changes get automatically deployed to our hosting provider\n",
    "\n",
    "## DEV\n",
    "- pull all the latest code from feature branch (any non-master branch)\n",
    "- push changes to the feature branch\n",
    "- then make a pull request to take all the changes in feature and merge them to the master branch\n",
    "\n",
    "## TEST\n",
    "- once you create a pull request, we'll set up a workflow that automatically sets off a series of actions that govern how you manage your codebase\n",
    "    - the workflow we setup will push the service to Travis CI, a continuous integration provider, which pulls down your code and runs a set of tests that you write on your codebase.\n",
    "    \n",
    "## PROD\n",
    "    - Then Travis CI automatically takes your codebase and pushes it to AWS Hosting (AWS Elastic Beanstalk)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Project Generator\n",
    "- React frontend wrapped inside Docker container, test and deployed automatically\n",
    "\n",
    "download node if you don't have it\n",
    "```console\n",
    "node -v\n",
    "```\n",
    "npx Create React App Generation\n",
    "\n",
    "npx create-react-app client\n",
    "```console\n",
    "npx create-react-app frontend\n",
    "cd frontend\n",
    "npm start\n",
    "npm run test\n",
    "npm run build\n",
    "ls build/static/js\n",
    "npm run start\n",
    "```\n",
    "\n",
    "npm run start starts up a development server only  \n",
    "npm run test runs tests associated with the project  \n",
    "npm run build builds a production version of the application\n",
    "\n",
    "- eventually, you'll want to provide build/index.html and build/static/js/runtime-main.fca44c8b.js to AWS server\n",
    "\n",
    "\n",
    "- run docker file with custom file name\n",
    "```console\n",
    "docker build -f Dockerfile.dev .\n",
    "```\n",
    "\n",
    "- you can remove the node_modules folder inside of our working directory because there are 2 copies.\n",
    "\n",
    "```console\n",
    "docker run 90a08b3d20ac\n",
    "```\n",
    "\n",
    "- go to src/App.js\n",
    "```js\n",
    "Edit <code>src/App.js</code> and save to reload.\n",
    "```\n",
    "- When you create an image, it takes a snapshot.  If you want changes to be reflected inside the container, you either have to rebuild the image or do another step to make changes automatically propagate.\n",
    "\n",
    "# Docker Volumes\n",
    "- instead of doing a straight copy, adjust the docker run command\n",
    "- put in a reference in the container that points back to folders in the local machine.  It's a bit like setting up the port mappings.\n",
    "- put a bookmark on the node_modules folder\n",
    "- map the pwd into the /app folder\n",
    "- Note:  (pwd) won't work if there are spaces in the path.\n",
    "```console\n",
    "docker run -p 3000:3000 -v /app/node_modules -v $(pwd):/app 90a08b3d20ac\n",
    "```\n",
    "\n",
    "- Inside our current local directory, there's no node_modules folder, but we deleted it from local.  It's on the docker container, but it's mapping to nothing on local.  Therefore, we put that first -v switch to put a bookmark on the node_modules folder.\n",
    "- Refering to the first switch, when you do -v and then put the path of a folder inside the container, it means you want to set a placeholder for what's inside the container; don't map it to anything.\n",
    "\n",
    "\n",
    "#### Put the long command into Docker Compose\n",
    "```yml\n",
    "# docker-compose.yml\n",
    "version: '3'\n",
    "services:\n",
    "  web:\n",
    "    build: \n",
    "      # context specifies where you want all your files and folder to be pulled from\n",
    "      context: .\n",
    "      dockerfile: Dockerfile.dev\n",
    "    ports:\n",
    "      - \"3000:3000\"\n",
    "    volumes:\n",
    "      - /app/node_modules\n",
    "      - .:/app\n",
    "```\n",
    "\n",
    "```console\n",
    "docker-compose up\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# First, run test on development env then we'll move on to TraivsCI\n",
    "\n",
    "- docker run start: start dev server for development use only\n",
    "- runs tests associated with the project\n",
    "- builds a production version of the application\n",
    "```console\n",
    "docker build -f Dockerfile.dev .\n",
    "docker run -it eec49a31318a npm run test\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Live Updating \n",
    "- test doesn't have the volume mapping set up, so if you modify App.test.js and rerun the test, it would not reflect the changes.\n",
    "- You could map a volume as we did before, or you could do it another way:\n",
    "- Rather than makeing  2nd service in the docker-compose file, attach to existing container that's created:\n",
    "- Then, execute a commmand to start up our test suite inside there, which gives us access to a container that already has all that container mapping set up.\n",
    "- Reuse the existing container we have and start up our test suite inside there.\n",
    "```console\n",
    "docker-compose up\n",
    "docker ps\n",
    "docker exec -it 36f16628fed9 npm run test\n",
    "```\n",
    "- Now, you'll see that if you comment out one of the tests and save, the tests will rerun and recognize that the test file has been modified.\n",
    "- This method is not ideal because you have to run docker-compose and then get the container id and do docker exec, which is hard to remember."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a test running service inside docker-compose\n",
    "```yml\n",
    "# docker-compose.yml\n",
    "version: '3'\n",
    "services:\n",
    "  web:\n",
    "    build: \n",
    "      # context specifies where you want all your files and folder to be pulled from\n",
    "      context: .\n",
    "      dockerfile: Dockerfile.dev\n",
    "    ports:\n",
    "      - \"3000:3000\"\n",
    "    volumes:\n",
    "      - /app/node_modules\n",
    "      - .:/app\n",
    "  tests:\n",
    "    build:\n",
    "      context: .\n",
    "      dockerfile: Dockerfile.dev\n",
    "    volumes:\n",
    "      - /app/node_modules\n",
    "      - .:/app\n",
    "    command: [\"npm\", \"run\", \"test\"]\n",
    "```\n",
    "```console\n",
    "docker-compose up --build\n",
    "```\n",
    "\n",
    "- The downside to this approach is that you get all the output from test suite inside the logging interface of docker compose.  So, you can't input to interface with it.\n",
    "- Let's try attach directly to the container, then we can have custom input, like p, t q, w, to manipulate the test suite."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Test Container: npm run test  \n",
    "Web Container: npm run start  \n",
    "- Every process has its own stdin, stdout, and stderr.\n",
    "- Let's run a separate terminal to attatch to the Test Suite.\n",
    "- get the id of frontend_web\n",
    "```console\n",
    "docker ps\n",
    "docker attach 53807b4c36b1\n",
    "```\n",
    "- docker attach attaches to the stdin, stdout, and stderr of the primary process inside the container.\n",
    "- You can't supply input to manipulate the test suite.  Unfortunately, this is as good as it gets when using docker-compose.\n",
    "- It's not possible because of the different processes inside the container.  When you run npm run test inside hte test container, it's really running npm, which then runs another process (start.js) which runs the test suite.  However, when you use docker attach, you attach to the stdin of the primary process.  It's not the primary npm's job to be in charge of the interactive stdin where you interact with the test suite.\n",
    "```console\n",
    "BYUKAN-M-33RF:frontend byukan$ docker exec -it 24312db621cc sh\n",
    "/app # ps\n",
    "PID   USER     TIME  COMMAND\n",
    "    1 root      0:00 npm\n",
    "   17 root      0:00 node /app/node_modules/.bin/react-scripts start\n",
    "   24 root      0:08 node /app/node_modules/react-scripts/scripts/start.js\n",
    "   60 root      0:00 sh\n",
    "   66 root      0:00 ps\n",
    "```\n",
    "- There are other test suites that don't have interactive manipulation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build Production version\n",
    "- npm run build\n",
    "- need for Nginx: https://hub.docker.com/_/nginx\n",
    "- npm run build processes all the js files and creates a single folder on your hard drive\n",
    "- The production environment drops the dev server inside the web container of the dev environment.  Instead, it has a production server (nginx)\n",
    "- Create a separate dockerfile that has a nginx server which servers up the index.html and main.js\n",
    "- dependencies only need to execute npm run build\n",
    "\n",
    "## Multi-step build process: build phase and run phase\n",
    "- See the nginx image on docker hub to host simple static content.  We want node:alpine as well as nginx.  We'll have a build phase and a run phase.\n",
    "- create Dockerfile\n",
    "```\n",
    "docker build .\n",
    "docker run -p 8080:80 20028428c5d4\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Implement workflow: Github, Travis CI, AWS\n",
    "-  Start using https://github.com/byukan/docker-react"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Travis CI\n",
    "- watches for changes to github repo.  We set up Travis to test the code and deploy it to AWS.\n",
    "- https://travis-ci.org/byukan/docker-react\n",
    "- make the .travis.yml file\n",
    "\n",
    "```yml\n",
    "language: generic # resolves \"rakefile not found\" and the travis build failing\n",
    "sudo: required\n",
    "services:\n",
    "  - docker\n",
    "\n",
    "before_install:\n",
    "  # tag the image so you don't have to juggle the image id\n",
    "  - docker build -t byukan/docker-react -f Dockerfile.dev .\n",
    "\n",
    "script:\n",
    "  - docker run -e CI=true byukan/docker-react npm run test\n",
    "```\n",
    "\n",
    "output from the build appears in \n",
    "https://travis-ci.org/byukan/docker-react"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploy on AWS\n",
    "- Elastic Beanstalk: Run and Manage Web Apps\n",
    "\n",
    "Welcome to AWS Elastic Beanstalk\n",
    "\n",
    "With Elastic Beanstalk, you can deploy, monitor, and scale an application quickly and easily. Let us do the heavy lifting so you can focus on your business.\n",
    "\n",
    "To deploy your existing web application, create an application source bundle and then create a new application. If you're using Git and would prefer to use it with our command line tool, please see Getting Started with the EB CLI.\n",
    "\n",
    "To deploy a sample application, click Get started, choose a name, select a platform and click Create app.\n",
    "\n",
    "By launching the sample application, you allow AWS Elastic Beanstalk to administer AWS resources and necessary permissions on your behalf. Learn more\n",
    "\n",
    "- Create a New Application\n",
    "- name:  docker-react\n",
    "- Web Server Enviornment\n",
    "- Base configuration, under platform select Docker"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Reguest gets handled by Load Balancer.  The VM running docker monitors requests and automatically scales for multiple nodes.  It automatically scales everything up for us.  \n",
    "http://dockerreact-env.espy28hb3b.us-east-2.elasticbeanstalk.com/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Configure Travis CI to automatically deploy our application to AWS once tests have passed\n",
    "```yml\n",
    "# .travais.yml\n",
    "language: generic # resolves \"rakefile not found\" and the travis build failing\n",
    "sudo: required\n",
    "services:\n",
    "  - docker\n",
    "\n",
    "before_install:\n",
    "  # tag the image so you don't have to juggle the image id\n",
    "  - docker build -t byukan/docker-react -f Dockerfile.dev .\n",
    "\n",
    "script:\n",
    "  - docker run -e CI=true byukan/docker-react npm run test\n",
    "\n",
    "deploy:\n",
    "  provider: elasticbeanstalk\n",
    "  region: us-east-2\n",
    "  app: \"docker-react\"\n",
    "  env: \"DockerReact-env\"\n",
    "  # TRAVIS CI zips the contents of github and places in S3\n",
    "  bucket_name: \"elasticbeanstalk-us-east-2-033116074849\"\n",
    "  # default is the app name\n",
    "  bucket_path: \"docker-react\"\n",
    "  on:\n",
    "    branch: master\n",
    "  access_key_id:\n",
    "    secure: \"$AWS_ACCESS_KEY\"\n",
    "  secret_access_key:\n",
    "    secure: \"$AWS_SECRET_KEY\"\n",
    "```\n",
    "\n",
    "- add an IAM user, docker-react-travis-ci.  Give programatic access only.\n",
    "- Attach existing policies directly (permissions).  Add Full access to Elastic Beanstalk: AWSElasticBeanstalkFullAccess \n",
    "- Save the access key id and secret access key.\n",
    "- Go to https://travis-ci.org/byukan/docker-react/settings, and put the name AWS_ACCESS_KEY and the value access key ID under Enviornment Variables.  Do not display value in build log.\n",
    "- Push changes to .travis.yml to Github and Travis CI will run.\n",
    "- Next you have to do port mapping."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exposing ports through the Dockerfile \n",
    "This container needs to get a port mapped to port 80.  Elastic beanstalk looks for the expose and maps to that port directly, automatically.\n",
    "```Dockerfile\n",
    "EXPOSE 80\n",
    "```\n",
    "- Push to github, and see the Travis CI reruns the test, after which Elastic Beanstalk kicks off updating the enviornment and deploying the application.\n",
    "- Visit the url and you'll see the React App."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Work on a feature branch  \n",
    "git checkout -b feature\n",
    "git add .\n",
    "git commit -m \"changed app text\"\n",
    "git push origin feature\n",
    "\n",
    "Create a pull request and merge the pull request.  \n",
    "You'll see 2 sets of tests run by Travis CI on the page to merge pull request.\n",
    "After merging the pull request, Travis CI runs tests again, and when they pass, Elastic Beanstalk will redeploy.  You'll see the new changes refresh on the url in a few minutes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Environment Cleanup\n",
    "- go to Elastic Beanstalk dashboard, actions, delete application, then confirm the delete."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Disadvantages of single container deploymnet issues\n",
    "- no outside dependencies, like databases\n",
    "- our image was built multiple times, on Travis CI, and then on Elastic Beanstalk.  We were using the web server application to build our image.  We'd want the web server to focus on hosting, not on building the image.\n",
    "- How do we connec tto a database or use an api?  We'll build a multi-container application that makes use of multiple databases, tie up with docker-compose and integrate with Travis CI and Elastic Beanstalk."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Backend architecture\n",
    "- nginx web server routes to either react server for html file or js file or expres server (functions as our api) if submitting or retrieving numbers.\n",
    "- Express server connects to Redis (temporary or cahced values) and Postgres (Permanent stored set of data)\n",
    "- A worker (Node js process) handles the Redis server.\n",
    "\n",
    "Flow:\n",
    "        - user submits number\n",
    "        - react app makes ajax request to express server, which stores the number in postgres (permanent list of indicies that have been received) and stores all indicies and calculated values as key value pairs in Redis.  The worker watches Redis for new indicies, pulls each new indice, and calculates new values then puts it back into Redis."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Worker process setup\n",
    "- mkdir complex, which has worker, client, and server folders\n",
    "- cd worker\n",
    "- node index.js\n",
    "- check out the files in there.\n",
    "- The Postgres database gets created in server/index.js\n",
    "\n",
    "- cd server\n",
    "- node index.js"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# See the complex repo\n",
    "https://github.com/byukan/complex\n",
    "Instead of this:\n",
    "\n",
    "~~npm install -g create-react-app~~\n",
    "\n",
    "~~create-react-app client~~\n",
    "\n",
    "Just do this:\n",
    "\n",
    "npx create-react-app client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}