https://fenics.readthedocs.io/projects/containers/en/latest/jupyter.html
https://medium.com/the-code-review/top-10-docker-commands-you-cant-live-without-54fb6377f481
https://towardsdatascience.com/15-docker-commands-you-should-know-970ea5203421
get docker ip
sudo docker inspect <container-id> | grep '"IPAddress"' | head -n 1
run new docker container
docker run -d -p 8888:8888 jupyter/pyspark-notebook
check docker
docker ps -q
docker pa -a: see all the past dockers that is also exitted
check docker logs and link
docker logs <container-id>
docker copy to local
docker cp <docker-id>:/home/jovyan/<filename> "<path>"
remove all docker container not running
docker rm $(docker ps -a -q)
access docker container
docker exec -it <docker name> bash
remove all container that is stopped
docker container prune
https://medium.com/@suci/running-pyspark-on-jupyter-notebook-with-docker-602b18ac4494
Need to replace localhost with docker ip by getting docker-machine ip on docker terminal
overview of docker: https://www.dataquest.io/blog/docker-data-science/
cd to docker yml file directory
run docker compose
sudo docker-compose up
check docker ip on local host
sudo docker-compose ps
elk docker set-up : https://www.youtube.com/watch?v=PWUuyDrqvt0
the sample docker-compose.yml file in the documentation does not work, refer to this
but do not replace kibana.yml file as stated. The guy changed elasticsearch.hosts from https://localhost:9200 to https://odfe-node1:9200 which is causing main problems in kibana ui:all icons are clickable but only show the ES server stats
build docker image
sudo docker build -t {name}:latest .
run docker
sudo docker run -d -p 5010:5010 {name}
to mount a host directory to docker container, and not rebuild container anytime with code changes
docker run -d -p 5010:5010 --name credit-scoring --mount type=bind,source="$(pwd)",target=/app {name} apps.py
docker run -d -p 5010:5010 --name {container name} -v "$(pwd)":/app {image name}:latest apps.py
Dockerfile command difference
CMD enables change of arg at runtime
Docker issues
python have some encoding issue with docker need to add below in Dockerfile
RUN apt-get clean && apt-get -y update && apt-get install -y locales && locale-gen en_US.UTF-8
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
update and install is not always needed unless locale error throw up
If there's need to read other files in flask app.py, just name the path /app/filename (with condition of docker file dir named /app)
gunicorn and nginx with docker compose
note that in gunicorn+flask container, because the entrypoint is not python, "if name==main" section will not run
docker-compose build
docker-compose up