curvilinear bits 'n pieces

.github/pull_request_template.md

@@ -0,0 +1,8 @@
+# PR Checklist
+
+- [ ] I have read the `CONTRIBUTING.rst` document.
+- [ ] I have updated the docstrings accordingly.
+- [ ] I have updated `CHANGES.md`.
+- [ ] I have added tests which give complete coverage for my changes.
+- [ ] I have provided a usage example for my changes. 
+- [ ] All new and existing tests pass.

.github/workflows/docker_build_push.yml

@@ -0,0 +1,28 @@
+name: docker image build and push to dockerhub
+
+on:
+  push:
+    branches:
+      - 'jgiordani/curvi-bits'
+
+jobs:
+  docker:
+    runs-on: ubuntu-latest
+    steps:
+      -
+        name: Checkout
+        uses: actions/checkout@v2
+      -
+        name: Login to DockerHub
+        uses: docker/login-action@v1 
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+      -
+        name: Build and push
+        uses: docker/build-push-action@v2
+        with:
+          context: .
+          file: ./docs/development/docker/underworld2/Dockerfile
+          push: true
+          tags: underworldcode/underworld2:curvi-bits       

CHANGES.md

@@ -3,20 +3,29 @@ CHANGES: Underworld2
 
 Release 2.11.0 []
 -----------------
-Changes
+Changes:
 * Enabled user defined Gauss integration swarms for all systems. 
 * Update docker base images: switch to ubuntu(20.04), update petsc(3.1.4) & mpich(3.3.2), other tweaks. 
 * Cleaner Python compile time configuration. 
+* Add runtime check for solver availability (for example, 'mumps'). Also add test for check.
 
-New:
+Docker image changes:
+* Update base image.
+* Updated to PETSc 3.15.1
+* Updated to MPICH 3.4.2
+* Removed petsc4py.
+* Switched `/opt` to ugo+rwx to allow users to install Python packages.
 
+New:
+* Model for EBA convection, based on King et al. (2010) benchmarks. See `docs/test/14_Convection_EBA.ipynb`
 * Mesh/MeshVariable/Swarm/SwarmVariable objects now support loading and saving
   of units information, as well as additional attributes. For usage examples, 
   see `docs/test/mesh_aux.py` and `docs/test/swarm_aux.py`. 
 * Added `underworld.function.count()` method, which counts function calls.
 * Conda binaries available via underworldcode conda channel `conda install -c underworldcode underworld2`
 * Added `underworld.function.count()` method, which counts function calls. 
 * Added GADI install/run scripts @ ./docs/install_guides/nci_gadi/
+* Updated pull request related documentation and added template.
 
 Fixes:
 * Updates for SCons4.1.
@@ -25,6 +34,9 @@ Fixes:
   structures to accomodate. 
 * Tester uses `jupyter-nbconvert` which no longer defaults to Python. Update
   to explicitly select Python.
+* Switched h5 file save to mode "w" instead of "a" as append mode resulted
+  in data from previous datasets (with identical name) not being removed from 
+  file, and file sizes therefore growing unnecessarily.  
 
 Release 2.10.1 [2020-08-28]
 ---------------------------

CONTRIBUTING.rst

@@ -15,23 +15,24 @@ For Bug reports and Suggestions / Requests please submit an Issue on the Underwo
 
 Click here to submit an Issue https://github.com/underworldcode/underworld2/issues
 
-
-For Code / Documentation changes please submit a GitHub Pull Request (PR). This allows us to review and discuss the contributions before merging it into our ``development`` branch.
-As part of the PR please provide a documentation change summary (for our ``Changes.md``) and a test where applicable.
-
-For creating Pull Request (PR) we recommend following the workflow outlined https://guides.github.com/activities/forking/.
+For Code / Documentation changes please submit a GitHub Pull Request (PR). This allows us to review and discuss the contributions before merging it into our ``development`` branch. For creating Pull Request (PR) we recommend following the workflow outlined https://guides.github.com/activities/forking/.
 More specifically:
 
 1. Fork Underworld via GitHub and clone it to your machine.
 
-2. Add the original Underworld repository as `upstream` and branch your contribution off its development branch.
+  .. code-block::
+
+    git clone https://github.com/YOUR_GITHUB_ACCOUNT/underworld2
+
+2. Add the master Underworld repository as an additional remote source (named `uwmaster`) for your local repo and pull down its latest changesets. Checkout to the master/development repo state, and then create a new local branch which will contain your forthcoming changes.
 
   .. code-block::
 
-    git remote add upstream https://github.com/underworldcode/underworld2
-    git checkout upstream/development ; git checkout -b newFeature
+    git remote add uwmaster https://github.com/underworldcode/underworld2
+    git pull uwmaster
+    git checkout uwmaster/development
+    git checkout -b newFeature
 
-3. Make your changes! Remember to write comments, a test if applicable and follow the code style of the project (see    ``./docs/development/guidelines.md`` for details).
-   We ask that a short description be made in the commit message that is appropriate for our ``Changes.md`` summary. If the change is sizeable or adds news functionality we ask for an associated blog post which describes the contribution in its full glory. Your time to shine! Details on how to write a blog post are coming soon.
+3. Make your changes! Remember to write comments, a test if applicable and follow the code style of the project (see ``./docs/development/guidelines.md`` for details).
 
 4. Push your changes to your GitHub fork and then submit a PR to the ``development`` branch of Underworld via Github.

Dockerfile

@@ -1,4 +1,4 @@
-FROM underworldcode/underworld2:dev
+FROM underworldcode/underworld2:2.11.0b
 
 # Set the UW_MACHINE env variable for metrics
 ENV UW_MACHINE binder

LICENSE.md

@@ -16,9 +16,9 @@ Underworld has been in development since 2003. It has always been released under
 
 ### Copyright holders
 
-Copyright Australian National University, 2020
-Copyright Melbourne University,           2014-2020
-Copyright Monash University,              2003-2020
+Copyright Australian National University, 2020-2021
+Copyright Melbourne University,           2014-2021
+Copyright Monash University,              2003-2021
 Copyright VPAC,                           2003-2009
 
 ### References

README.md

@@ -29,18 +29,15 @@ In particular, the *Getting Started* section of the User Guide might be useful p
 Trying out Underworld2
 ----------------------
 
-You can try out the code immediately via a Jupyter Binder cloud instance. Be aware that it can take a little while for the site to fire up and that it will time-out after 30 minutes of inactivity and reset if you log back in. 
+You can try out the code immediately via a Jupyter Binder cloud instance. The Binder environment is identical to that obtained through running an Underworld Docker image locally.
+Note that it can take a while for the site to fire up and that it will time-out after 30 minutes of inactivity and reset if you log back in. 
 
 | | |
 |-|-|
-| [![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/underworldcode/underworld2/v2.10.0b) | v2.10.0b (Py3) |
-| [![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/underworldcode/underworld2/v2.9.2b) | v2.9.2b (Py3) |
-| [![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/underworldcode/underworld2/v2.8.2b) | v2.8.2b (Py3) |
-| [![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/underworldcode/underworld2/v2.7.1b) | v2.7.1b (Py2) |
-| [![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/underworldcode/underworld2/development) | dev (Py3) |
-
-
-Note that the Binder environment is identical to that obtained through running an Underworld Docker image locally.
+| [![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/underworldcode/underworld2/v2.11.0b) | v2.11.0b |
+| [![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/underworldcode/underworld2/v2.10.0b) | v2.10.0b |
+| [![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/underworldcode/underworld2/v2.9.2b) | v2.9.2b |
+| [![Binder](https://mybinder.org/badge.svg)](https://mybinder.org/v2/gh/underworldcode/underworld2/development) | dev |
 
 
 Getting Underworld2

conda/LICENSE.md

@@ -0,0 +1 @@
+../LICENSE.md

docs/development/docker/base/Dockerfile

@@ -1,5 +1,5 @@
 FROM ubuntu:20.04 as base_runtime
-MAINTAINER https://github.com/underworldcode/
+LABEL maintainer="https://github.com/underworldcode/"
 ENV LANG=C.UTF-8
 ENV PYVER=3.8
 # Setup some things in anticipation of virtualenvs
@@ -9,6 +9,19 @@ ENV PATH=${VIRTUAL_ENV}/bin:$PATH
 # The following ensures venv packages are available when using system python (such as from jupyter)
 ENV PYTHONPATH=${PYTHONPATH}:${VIRTUAL_ENV}/lib/python${PYVER}/site-packages
 
+# add joyvan user, volume mount and expose port 8888
+EXPOSE 8888
+ENV NB_USER jovyan
+ENV NB_WORK /home/$NB_USER
+RUN useradd -m -s /bin/bash -N $NB_USER -g users \
+&&  mkdir -p /$NB_WORK/workspace \
+&&  chown -R $NB_USER:users $NB_WORK
+VOLUME $NB_WORK/workspace
+
+# make virtualenv directory and set permissions 
+RUN mkdir ${VIRTUAL_ENV} \
+&&  chmod ugo+rwx ${VIRTUAL_ENV}
+
 # install runtime requirements
 RUN apt-get update -qq \
 &&  DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
@@ -62,21 +75,9 @@ RUN apt-get update \
 &&  rm -rf /var/lib/apt/lists/* 
 RUN PYTHONPATH= /usr/bin/pip3 install -r /opt/requirements.txt
 
-# add tini, joyvan user, volume mount and expose port 8888
-EXPOSE 8888
-ENV NB_USER jovyan
-ENV NB_WORK /home/$NB_USER
-ADD https://github.com/krallin/tini/releases/download/v0.18.0/tini /tini
-RUN ipcluster nbextension enable \
-&&  chmod +x /tini \
-&&  useradd -m -s /bin/bash -N $NB_USER -g users \
-&&  mkdir -p /$NB_WORK/workspace \
-&&  chown -R $NB_USER:users $NB_WORK
-VOLUME $NB_WORK/workspace
-
 # jovyan user, finalise jupyter env
 USER $NB_USER
-RUN ipython profile create --parallel --profile=mpi \
-&&  echo "c.IPClusterEngines.engine_launcher_class = 'MPIEngineSetLauncher'" >> $NB_WORK/.ipython/profile_mpi/ipcluster_config.py
+# RUN ipython profile create --parallel --profile=mpi \
+# &&  echo "c.IPClusterEngines.engine_launcher_class = 'MPIEngineSetLauncher'" >> $NB_WORK/.ipython/profile_mpi/ipcluster_config.py
 WORKDIR $NB_WORK
 CMD ["jupyter", "notebook", "--no-browser", "--ip='0.0.0.0'"]

docs/development/docker/lavavu/Dockerfile

@@ -1,5 +1,5 @@
 FROM ubuntu:20.04 as base_runtime
-MAINTAINER https://github.com/underworldcode/
+LABEL maintainer="https://github.com/underworldcode/"
 ENV LANG=C.UTF-8
 ENV PYVER=3.8
 # Setup some things in anticipation of virtualenvs
@@ -9,6 +9,19 @@ ENV PATH=${VIRTUAL_ENV}/bin:$PATH
 # The following ensures venv packages are available when using system python (such as from jupyter)
 ENV PYTHONPATH=${PYTHONPATH}:${VIRTUAL_ENV}/lib/python${PYVER}/site-packages
 
+# add joyvan user, volume mount and expose port 8888
+EXPOSE 8888
+ENV NB_USER jovyan
+ENV NB_WORK /home/$NB_USER
+RUN useradd -m -s /bin/bash -N $NB_USER -g users \
+&&  mkdir -p /$NB_WORK/workspace \
+&&  chown -R $NB_USER:users $NB_WORK
+VOLUME $NB_WORK/workspace
+
+# make virtualenv directory and set permissions 
+RUN mkdir ${VIRTUAL_ENV} \
+&&  chmod ugo+rwx ${VIRTUAL_ENV}
+
 # install runtime requirements
 RUN apt-get update -qq \
 &&  DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
@@ -45,8 +58,8 @@ RUN apt-get update -qq
 RUN DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
         build-essential \
         python3-setuptools \
-        libpython${PYVER}-dev \
-sudo apt-get install        libpng-dev \
+        libpython${PYVER}-dev \ 
+        libpng-dev \
         libjpeg-dev \
         libtiff-dev \
         mesa-utils \
@@ -63,6 +76,7 @@ sudo apt-get install        libpng-dev \
 
 # lavavu
 # create a virtualenv to put new python modules
+USER $NB_USER
 RUN /usr/bin/python3 -m virtualenv --system-site-packages --python=/usr/bin/python3 ${VIRTUAL_ENV}
 RUN LV_OSMESA=1 pip3 install --no-cache-dir --no-binary=lavavu lavavu
 
@@ -78,13 +92,6 @@ RUN apt-mark showmanual >/opt/installed.txt
 
 # Add user environment
 FROM minimal
-EXPOSE 8888
-ADD https://github.com/krallin/tini/releases/download/v0.18.0/tini /tini
-RUN chmod +x /tini
-ENV NB_USER jovyan
-RUN useradd -m -s /bin/bash -N jovyan
 USER $NB_USER
-ENV NB_WORK /home/$NB_USER
-VOLUME $NB_WORK/workspace
 WORKDIR $NB_WORK
 CMD ["jupyter", "notebook", "--no-browser", "--ip='0.0.0.0'"]

docs/development/docker/petsc/Dockerfile

@@ -1,5 +1,5 @@
 FROM ubuntu:20.04 as base_runtime
-MAINTAINER https://github.com/underworldcode/
+LABEL maintainer="https://github.com/underworldcode/"
 ENV LANG=C.UTF-8
 ENV PYVER=3.8
 # Setup some things in anticipation of virtualenvs
@@ -9,6 +9,19 @@ ENV PATH=${VIRTUAL_ENV}/bin:$PATH
 # The following ensures venv packages are available when using system python (such as from jupyter)
 ENV PYTHONPATH=${PYTHONPATH}:${VIRTUAL_ENV}/lib/python${PYVER}/site-packages
 
+# add joyvan user, volume mount and expose port 8888
+EXPOSE 8888
+ENV NB_USER jovyan
+ENV NB_WORK /home/$NB_USER
+RUN useradd -m -s /bin/bash -N $NB_USER -g users \
+&&  mkdir -p /$NB_WORK/workspace \
+&&  chown -R $NB_USER:users $NB_WORK
+VOLUME $NB_WORK/workspace
+
+# make virtualenv directory and set permissions 
+RUN mkdir ${VIRTUAL_ENV} \
+&&  chmod ugo+rwx ${VIRTUAL_ENV}
+
 # install runtime requirements
 RUN apt-get update -qq \
 &&  DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
@@ -43,23 +56,26 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
         git
 # build mpi
 WORKDIR /tmp/mpich-build
-ARG MPICH_VERSION="3.3.2"
+ARG MPICH_VERSION="3.4.2"
 RUN wget http://www.mpich.org/static/downloads/${MPICH_VERSION}/mpich-${MPICH_VERSION}.tar.gz 
 RUN  tar xvzf mpich-${MPICH_VERSION}.tar.gz 
 WORKDIR /tmp/mpich-build/mpich-${MPICH_VERSION}              
-ARG MPICH_CONFIGURE_OPTIONS="--prefix=/usr/local --enable-g=option=none --disable-debuginfo --enable-fast=O3,ndebug --without-timing --without-mpit-pvars"
+ARG MPICH_CONFIGURE_OPTIONS="--prefix=/usr/local --enable-g=option=none --disable-debuginfo --enable-fast=O3,ndebug --without-timing --without-mpit-pvars --with-device=ch3"
 ARG MPICH_MAKE_OPTIONS="-j8"
 RUN ./configure ${MPICH_CONFIGURE_OPTIONS} 
 RUN make ${MPICH_MAKE_OPTIONS}             
 RUN make install                           
 RUN ldconfig
+
 # create venv now for forthcoming python packages
+USER $NB_USER
 RUN /usr/bin/python3 -m virtualenv --system-site-packages --python=/usr/bin/python3 ${VIRTUAL_ENV}
-
 RUN pip3 install --no-cache-dir mpi4py
+
+USER root
 # build petsc
 WORKDIR /tmp/petsc-build
-ARG PETSC_VERSION="3.14.0"
+ARG PETSC_VERSION="3.15.1"
 RUN wget http://ftp.mcs.anl.gov/pub/petsc/release-snapshots/petsc-lite-${PETSC_VERSION}.tar.gz
 RUN tar zxf petsc-lite-${PETSC_VERSION}.tar.gz
 WORKDIR /tmp/petsc-build/petsc-${PETSC_VERSION}
@@ -85,8 +101,7 @@ RUN make PETSC_DIR=/tmp/petsc-build/petsc-${PETSC_VERSION} PETSC_ARCH=arch-linux
 RUN rm -fr /usr/local/share/petsc 
 # build petsc4py
 ENV PETSC_DIR=/usr/local
-ENV PETSC_ARCH=arch-linux-c-opt
-RUN pip3 install --no-cache-dir petsc4py
+USER $NB_USER
 RUN CC=h5pcc HDF5_MPI="ON" HDF5_DIR=${PETSC_DIR} pip3 install --no-cache-dir --no-binary=h5py git+https://github.com/h5py/h5py@master
 
 FROM base_runtime AS minimal
@@ -100,16 +115,6 @@ RUN apt-mark showmanual >/opt/installed.txt
 
 # Add user environment
 FROM minimal
-EXPOSE 8888
-RUN ipcluster nbextension enable
-ADD https://github.com/krallin/tini/releases/download/v0.18.0/tini /tini
-RUN chmod +x /tini
-ENV NB_USER jovyan
-RUN useradd -m -s /bin/bash -N jovyan
 USER $NB_USER
-ENV NB_WORK /home/$NB_USER
-RUN ipython profile create --parallel --profile=mpi && \
-    echo "c.IPClusterEngines.engine_launcher_class = 'MPIEngineSetLauncher'" >> $NB_WORK/.ipython/profile_mpi/ipcluster_config.py
-VOLUME $NB_WORK/workspace
 WORKDIR $NB_WORK
 CMD ["jupyter", "notebook", "--no-browser", "--ip='0.0.0.0'"]

docs/development/docker/stampede2/Dockerfile

@@ -1,5 +1,5 @@
 FROM ubuntu:bionic
-MAINTAINER https://github.com/underworldcode/
+LABEL maintainer="https://github.com/underworldcode/"
 
 RUN  mkdir /home1 /work /scratch /gpfs /corral-repl /corral-tacc /data
 

docs/development/docker/underworld2/Dockerfile

@@ -2,8 +2,8 @@
 # Typically:
 #    $  docker build -f docs/development/docker/underworld2/Dockerfile .
 
-FROM underworldcode/base@sha256:f194ce40ea602305141a6a95fad640c90dd7ae5fae7532cd164dc566ec465edb as base_runtime
-MAINTAINER https://github.com/underworldcode/
+FROM underworldcode/base@sha256:27198726bedbe747889d4db096d5e15bf0c7cec5c9dc07c483b1d02acdfff543 as base_runtime
+LABEL maintainer="https://github.com/underworldcode/"
 # install runtime requirements
 USER root
 ENV PYVER=3.8
@@ -22,6 +22,7 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get install -yq --no-install-recommends \
         libxml2-dev
 RUN PYTHONPATH= /usr/bin/pip3 install --no-cache-dir setuptools scons 
 # setup further virtualenv to avoid double copying back previous packages (h5py,mpi4py,etc)
+USER $NB_USER
 RUN /usr/bin/python3 -m virtualenv --system-site-packages --python=/usr/bin/python3 ${VIRTUAL_ENV}
 WORKDIR /tmp
 COPY --chown=jovyan:users . /tmp/underworld2
@@ -47,7 +48,7 @@ COPY --chown=jovyan:users ./docs/test       $NB_WORK/Underworld/test
 COPY --from=build_base --chown=jovyan:users /tmp/UWGeodynamics/docs/examples   $NB_WORK/UWGeodynamics/examples
 COPY --from=build_base --chown=jovyan:users /tmp/UWGeodynamics/docs/tutorials  $NB_WORK/UWGeodynamics/tutorials
 COPY --from=build_base --chown=jovyan:users /tmp/UWGeodynamics/docs/benchmarks $NB_WORK/UWGeodynamics/benchmarks
-RUN  chown jovyan:users /home/jovyan/workspace /home/jovyan/UWGeodynamics
+RUN chown jovyan:users /home/jovyan/workspace /home/jovyan/UWGeodynamics
 RUN jupyter serverextension enable --sys-prefix jupyter_server_proxy
 USER $NB_USER
 WORKDIR $NB_WORK