Merge remote-tracking branch 'upstream/master' into patch-1

* upstream/master: (75 commits)
  Implement idxmax and idxmin functions (pydata#3871)
  Update pre-commit-config.yaml (pydata#3911)
  Revert "Use `fixes` in PR template (pydata#3886)" (pydata#3912)
  update the docstring of diff (pydata#3909)
  Un-xfail test_dayofyear_after_cftime_range (pydata#3907)
  Limit repr of arrays containing long strings (pydata#3900)
  expose a few zarr backend functions as semi-public api (pydata#3897)
  Use drawstyle instead of linestyle in plot.step. (pydata#3274)
  Implementation of polyfit and polyval (pydata#3733)
  misplaced quote in whatsnew (pydata#3889)
  Rename ordered_dict_intersection -> compat_dict_intersection (pydata#3887)
  Control attrs of result in `merge()`, `concat()`, `combine_by_coords()` and `combine_nested()` (pydata#3877)
  xfail test_uamiv_format_write (pydata#3885)
  Use `fixes` in PR template (pydata#3886)
  Tweaks to "how_to_release" (pydata#3882)
  whatsnew section for 0.16.0
  Release v0.15.1
  whatsnew for 0.15.1 (pydata#3879)
  update panel documentation (pydata#3880)
  reword the whats-new entry for unit support (pydata#3878)
  ...

Author: dcherian

.github/ISSUE_TEMPLATE/bug_report.md

@@ -7,6 +7,9 @@ assignees: ''
 
 ---
 
+<!-- A short summary of the issue, if appropriate -->
+
+
 #### MCVE Code Sample
 <!-- In order for the maintainers to efficiently understand and prioritize issues, we ask you post a "Minimal, Complete and Verifiable Example" (MCVE): http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports -->
 
@@ -22,8 +25,11 @@ assignees: ''
 <!-- this should explain why the current behavior is a problem and why the expected output is a better solution -->
 
 
-#### Output of ``xr.show_versions()``
-<details>
-# Paste the output here xr.show_versions() here
+#### Versions
+
+<details><summary>Output of `xr.show_versions()`</summary>
+
+<!-- Paste the output here xr.show_versions() here -->
+
 
 </details>

.pre-commit-config.yaml

@@ -5,13 +5,14 @@ repos:
     rev: 4.3.21-2
     hooks:
       - id: isort
+        files: .+\.py$
   # https://github.com/python/black#version-control-integration
   - repo: https://github.com/python/black
     rev: stable
     hooks:
       - id: black
-  - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v2.2.3
+  - repo: https://gitlab.com/pycqa/flake8
+    rev: 3.7.9
     hooks:
       - id: flake8
   - repo: https://github.com/pre-commit/mirrors-mypy

HOW_TO_RELEASE.md

@@ -1,4 +1,4 @@
-How to issue an xarray release in 14 easy steps
+How to issue an xarray release in 16 easy steps
 
 Time required: about an hour.
 
@@ -20,32 +20,37 @@ Time required: about an hour.
       ```
       pytest
       ```
- 4. On the master branch, commit the release in git:
+ 4. Check that the ReadTheDocs build is passing.
+ 5. On the master branch, commit the release in git:
       ```
-      git commit -a -m 'Release v0.X.Y'
+      git commit -am 'Release v0.X.Y'
       ```
- 5. Tag the release:
+ 6. Tag the release:
       ```
       git tag -a v0.X.Y -m 'v0.X.Y'
       ```
- 6. Build source and binary wheels for pypi:
+ 7. Build source and binary wheels for pypi:
       ```
       git clean -xdf  # this deletes all uncommited changes!
       python setup.py bdist_wheel sdist
       ```
- 7. Use twine to register and upload the release on pypi. Be careful, you can't
+ 8. Use twine to check the package build:
+      ```
+      twine check dist/xarray-0.X.Y*
+      ```
+ 9. Use twine to register and upload the release on pypi. Be careful, you can't
     take this back!
       ```
       twine upload dist/xarray-0.X.Y*
       ```
     You will need to be listed as a package owner at
     https://pypi.python.org/pypi/xarray for this to work.
- 8. Push your changes to master:
+10. Push your changes to master:
       ```
       git push upstream master
       git push upstream --tags
       ```
- 9. Update the stable branch (used by ReadTheDocs) and switch back to master:
+11. Update the stable branch (used by ReadTheDocs) and switch back to master:
      ```
       git checkout stable
       git rebase master
@@ -55,20 +60,45 @@ Time required: about an hour.
     It's OK to force push to 'stable' if necessary. (We also update the stable 
     branch with `git cherrypick` for documentation only fixes that apply the 
     current released version.)
-10. Add a section for the next release (v.X.(Y+1)) to doc/whats-new.rst.
-11. Commit your changes and push to master again:
+12. Add a section for the next release (v.X.Y+1) to doc/whats-new.rst:
+     ```
+     .. _whats-new.0.X.Y+1:
+
+     v0.X.Y+1 (unreleased)
+     ---------------------
+
+     Breaking changes
+     ~~~~~~~~~~~~~~~~
+
+
+     New Features
+     ~~~~~~~~~~~~
+
+
+     Bug fixes
+     ~~~~~~~~~
+
+
+     Documentation
+     ~~~~~~~~~~~~~
+
+
+     Internal Changes
+     ~~~~~~~~~~~~~~~~
+     ```
+13. Commit your changes and push to master again:
       ```
-      git commit -a -m 'New whatsnew section'
+      git commit -am 'New whatsnew section'
       git push upstream master
       ```
     You're done pushing to master!
-12. Issue the release on GitHub. Click on "Draft a new release" at
+14. Issue the release on GitHub. Click on "Draft a new release" at
     https://github.com/pydata/xarray/releases. Type in the version number, but
     don't bother to describe it -- we maintain that on the docs instead.
-13. Update the docs. Login to https://readthedocs.org/projects/xray/versions/
+15. Update the docs. Login to https://readthedocs.org/projects/xray/versions/
     and switch your new release tag (at the bottom) from "Inactive" to "Active".
     It should now build automatically.
-14. Issue the release announcement! For bug fix releases, I usually only email
+16. Issue the release announcement! For bug fix releases, I usually only email
     [email protected]. For major/feature releases, I will email a broader
     list (no more than once every 3-6 months):
       - [email protected]
@@ -83,15 +113,17 @@ Time required: about an hour.
     ```
     git log "$(git tag --sort="v:refname" | sed -n 'x;$p').." --format="%aN" | sort -u
     ```
-    or by replacing `v0.X.Y` with the _previous_ release in:
+    or by substituting the _previous_ release in:
     ```
-    git log v0.X.Y.. --format="%aN" | sort -u
+    git log v0.X.Y-1.. --format="%aN" | sort -u
     ```
+    NB: copying this output into a Google Groups form can cause 
+    [issues](https://groups.google.com/forum/#!topic/xarray/hK158wAviPs) with line breaks, so take care
 
 Note on version numbering:
 
 We follow a rough approximation of semantic version. Only major releases (0.X.0)
-show include breaking changes. Minor releases (0.X.Y) are for bug fixes and
+should include breaking changes. Minor releases (0.X.Y) are for bug fixes and
 backwards compatible new features, but if a sufficient number of new features
 have arrived we will issue a major release even if there are no compatibility
 breaks.

azure-pipelines.yml

@@ -32,15 +32,16 @@ jobs:
   steps:
   - template: ci/azure/unit-tests.yml
 
-- job: MacOSX
-  strategy:
-    matrix:
-      py38:
-        conda_env: py38
-  pool:
-    vmImage: 'macOS-10.13'
-  steps:
-  - template: ci/azure/unit-tests.yml
+# excluded while waiting for https://github.com/conda-forge/libwebp-feedstock/issues/26
+# - job: MacOSX
+#   strategy:
+#     matrix:
+#       py38:
+#         conda_env: py38
+#   pool:
+#     vmImage: 'macOS-10.15'
+#   steps:
+#   - template: ci/azure/unit-tests.yml
 
 - job: Windows
   strategy:

ci/azure/install.yml

@@ -19,7 +19,6 @@ steps:
         --upgrade \
         matplotlib \
         numpy \
-        pandas \
         scipy
     python -m pip install \
         --no-deps \
@@ -29,7 +28,9 @@ steps:
         git+https://github.com/zarr-developers/zarr \
         git+https://github.com/Unidata/cftime \
         git+https://github.com/mapbox/rasterio \
-        git+https://github.com/pydata/bottleneck
+        git+https://github.com/hgrecco/pint \
+        git+https://github.com/pydata/bottleneck \
+        git+https://github.com/pandas-dev/pandas
   condition: eq(variables['UPSTREAM_DEV'], 'true')
   displayName: Install upstream dev dependencies
 

ci/requirements/doc.yml

@@ -6,21 +6,22 @@ dependencies:
   - python=3.8
   - bottleneck
   - cartopy
-  - cfgrib
-  - h5netcdf
+  - cfgrib>=0.9
+  - dask>=2.10
+  - h5netcdf>=0.7.4
   - ipykernel
   - ipython
-  - iris
+  - iris>=2.3
   - jupyter_client
   - nbsphinx
-  - netcdf4
+  - netcdf4>=1.5
   - numba
-  - numpy
+  - numpy>=1.17
   - numpydoc
-  - pandas
-  - rasterio
+  - pandas>=1.0
+  - rasterio>=1.1
   - seaborn
   - setuptools
-  - sphinx
-  - sphinx_rtd_theme
-  - zarr
+  - sphinx>=2.3
+  - sphinx_rtd_theme>=0.4
+  - zarr>=2.4

ci/requirements/py36-min-all-deps.yml

@@ -26,6 +26,7 @@ dependencies:
   - isort
   - lxml=4.4  # Optional dep of pydap
   - matplotlib=3.1
+  - msgpack-python=0.6  # remove once distributed is bumped. distributed GH3491
   - mypy=0.761  # Must match .pre-commit-config.yaml
   - nc-time-axis=1.2
   - netcdf4=1.4

ci/requirements/py36-min-nep18.yml

@@ -8,9 +8,10 @@ dependencies:
   - coveralls
   - dask=2.4
   - distributed=2.4
+  - msgpack-python=0.6  # remove once distributed is bumped. distributed GH3491
   - numpy=1.17
   - pandas=0.25
-  - pint=0.9  # Actually not enough as it doesn't implement __array_function__yet!
+  - pint=0.11
   - pip
   - pytest
   - pytest-cov

conftest.py

@@ -21,3 +21,14 @@ def pytest_runtest_setup(item):
         pytest.skip(
             "set --run-network-tests to run test requiring an " "internet connection"
         )
+
+
+@pytest.fixture(autouse=True)
+def add_standard_imports(doctest_namespace):
+    import numpy as np
+    import pandas as pd
+    import xarray as xr
+
+    doctest_namespace["np"] = np
+    doctest_namespace["pd"] = pd
+    doctest_namespace["xr"] = xr

doc/api-hidden.rst

@@ -379,7 +379,6 @@
    Variable.min
    Variable.no_conflicts
    Variable.notnull
-   Variable.pad_with_fill_value
    Variable.prod
    Variable.quantile
    Variable.rank
@@ -453,7 +452,6 @@
    IndexVariable.min
    IndexVariable.no_conflicts
    IndexVariable.notnull
-   IndexVariable.pad_with_fill_value
    IndexVariable.prod
    IndexVariable.quantile
    IndexVariable.rank

doc/api.rst

@@ -30,6 +30,7 @@ Top-level functions
    zeros_like
    ones_like
    dot
+   polyval
    map_blocks
    show_versions
    set_options
@@ -165,18 +166,22 @@ Computation
    Dataset.groupby_bins
    Dataset.rolling
    Dataset.rolling_exp
+   Dataset.weighted
    Dataset.coarsen
    Dataset.resample
    Dataset.diff
    Dataset.quantile
    Dataset.differentiate
    Dataset.integrate
+   Dataset.polyfit
 
 **Aggregation**:
 :py:attr:`~Dataset.all`
 :py:attr:`~Dataset.any`
 :py:attr:`~Dataset.argmax`
 :py:attr:`~Dataset.argmin`
+:py:attr:`~Dataset.idxmax`
+:py:attr:`~Dataset.idxmin`
 :py:attr:`~Dataset.max`
 :py:attr:`~Dataset.mean`
 :py:attr:`~Dataset.median`
@@ -220,6 +225,7 @@ Reshaping and reorganizing
    Dataset.to_stacked_array
    Dataset.shift
    Dataset.roll
+   Dataset.pad
    Dataset.sortby
    Dataset.broadcast_like
 
@@ -340,6 +346,7 @@ Computation
    DataArray.groupby_bins
    DataArray.rolling
    DataArray.rolling_exp
+   DataArray.weighted
    DataArray.coarsen
    DataArray.dt
    DataArray.resample
@@ -349,13 +356,16 @@ Computation
    DataArray.quantile
    DataArray.differentiate
    DataArray.integrate
+   DataArray.polyfit
    DataArray.str
 
 **Aggregation**:
 :py:attr:`~DataArray.all`
 :py:attr:`~DataArray.any`
 :py:attr:`~DataArray.argmax`
 :py:attr:`~DataArray.argmin`
+:py:attr:`~DataArray.idxmax`
+:py:attr:`~DataArray.idxmin`
 :py:attr:`~DataArray.max`
 :py:attr:`~DataArray.mean`
 :py:attr:`~DataArray.median`
@@ -399,6 +409,7 @@ Reshaping and reorganizing
    DataArray.to_unstacked_dataset
    DataArray.shift
    DataArray.roll
+   DataArray.pad
    DataArray.sortby
    DataArray.broadcast_like
 
@@ -577,6 +588,22 @@ Rolling objects
    core.rolling.DatasetRolling.reduce
    core.rolling_exp.RollingExp
 
+Weighted objects
+================
+
+.. autosummary::
+   :toctree: generated/
+
+   core.weighted.DataArrayWeighted
+   core.weighted.DataArrayWeighted.mean
+   core.weighted.DataArrayWeighted.sum
+   core.weighted.DataArrayWeighted.sum_of_weights
+   core.weighted.DatasetWeighted
+   core.weighted.DatasetWeighted.mean
+   core.weighted.DatasetWeighted.sum
+   core.weighted.DatasetWeighted.sum_of_weights
+
+
 Coarsen objects
 ===============