Fixes to documentation (#9)

- Fix minor bug in jamaly cmv
- Add demo for plant digitization by @williamhobbs 
- introduce sphinx documentation re: JOSS review commentary

.gitignore

@@ -131,8 +131,15 @@ dmypy.json
 ###### My Additions ######
 
 # Large data files
-/demos/data/
+/demos/data/*
+
+# but inlcude the sample plant geojson file
+!/demos/data/digitized_plant*.*
 
 # IDE Files
 .idea*
-*.egg
+*.egg
+
+# Sphinx Generated files
+/docs/sphinx/build/*
+/docs/sphinx/source/generated/*

.readthedocs.yml

@@ -0,0 +1,20 @@
+# .readthedocs.yaml
+# Read the Docs configuration file
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+build:
+  os: "ubuntu-22.04"
+  tools:
+    python: "3.10"
+
+python:
+   install:
+      - method: pip
+        path: .
+
+sphinx:
+  configuration: docs/sphinx/source/conf.py
+  fail_on_warning: true

Changelog.md

@@ -53,4 +53,9 @@ site itself.
     - `field_reassignment_demo` - Automated remapping of the field data, includes a Jupyter notebook.
     - `field_demo_full_process` - Overview of the full field processing workflow 
     - `field_demo_full_process_multithread` - Same as previous but including multithreading support for speed
-- Added tests of these new functions for more completeness.
+- Added tests of these new functions for more completeness.
+# Version 0.4.1
+- Bug fix for `cmv` so that `jamaly` r_corr is always positive. 
+- Demo for digitizing a plant equipment layout added by @williamhobbs
+- Improve comments in `cmv_demo`
+- Add sphinx documentation for entire library

README.md

@@ -1,9 +1,11 @@
 # SolarToolbox
-`solartoolbox` is a package containing tools for dealing with analysis of solar 
-energy data. Its specific focus is on signal processing approaches and 
-addressing variability from a spatiotemporal perspective. Tools here might be 
-useful for dealing with distributed data sets, or performing analyses that 
-rely on a spatially distributed set of measurements.
+Spatial analysis tools for solar energy research
+
+`solartoolbox` is a python package containing implementations of various spatial algorithms for solar energy data. While excellent open source solar energy packages already exist (e.g. [pvlib-python](https://github.com/pvlib/pvlib-python) and [pvanalytics](https://github.com/pvlib/pvanalytics)), the complexity of some high-level analyses found in the academic literature makes them as a poor fit for the scope of existing packages. This package fills that gap by implementing techniques that we hope can facilitate common spatial tasks for solar energy researchers and provide a platform for consistency and efficiency improvements in these calculations. 
+
+Two examples of what are believed to be the most generally useful techniques this package implements are:
+- Processing the cloud motion vector from a distributed sensor network. 
+- Verifying the locations of field components (e.g. combiners) within a distributed network of measurements
 
 # Installation
 The package can be most easily installed via [PyPi](https://pypi.org/project/solartoolbox/) 
@@ -12,6 +14,18 @@ with the following command:
 pip install solartoolbox
 ```
 
+# Getting Started
+A number of example codes are available in the demos folder. These are meant to demonstrate what are believed to be the most useful functions in the package and applications of the functions to real sample data included with the library.
+
+
+# Common Data Formatting
+
+The algorithm implementations in this package primarily involve spatially distributed analyses of solar energy data, necessitating processing of multiple simultaneous time series. Most codes will assume that data is provided using a common format, based upon ```pandas``` DataFrame objects. The demo codes use the following convention for the two most important data variables. 
+- ```pos``` or ```pos_utm``` - Data for the location of each sensor should be stored in a DataFrame with the sensor id as the index and the latitude and longitude (or UTM coordinates) as columns. 
+- ```ts_data``` - Sensor time series data should be stored in a DataFrame with the timestamp as the index and the sensor id as the columns. 
+The index of ```pos``` needs to match the columns of ```ts_data``` so that the correspondence between the locations and the time series can be maintained.
+
+
 # Structure of the Library
 The codes are organized into a few subpackages and several function libraries.
 The subpackages are meant to contain tools that are related to a specific
@@ -37,7 +51,7 @@ area that could use some expansion in the future.
 
 ```demos```  
 Data and demonstration codes (including as jupyter notebooks) that demonstrate 
-the functionality of the package. An explanation the included data is 
+the functionality of the package. An explanation for the included data is 
 warranted. 
 
 - Anonymized Plant Combiner Data  
@@ -107,56 +121,19 @@ of each hour of the day over a 30 day window)
 Functions for predicting the position of field components on the basis of cloud
 motion. 
 
-## Common format for H5 files used for Data Storage
-
-I've tried to format the multisite time series measurements in a way that's 
-conveinent for loading the files and working with the data. This came about 
-from my initial work analyzing the HOPE Campaign, which used 100 individual 
-point measurements of GHI scattered through a region near Jülich, Germany.
-
-All data is collected into a single H5 file containing multiple fields. I use
-```pandas``` and specifically ```pandas.read_hdf()``` for getting the data
-into python. 
-
-- ```latlon```: The latitude/longitude of the individual measurement sites
-- ```utm```: The UTM coordinates of the individual measurement sites
-- ```data```: Global Horizontal Irradiance
-- ```data_tilt```: Global Tilted Irradiance (if available)
-
-#### Location Data
-Data about the location of each individual site is stored in the H5 file. Two
-possible keys are used depending on the projection. Both are available when 
-possible. The key ```latlon``` represents the site in a latitude coordinate
-system. The key ```utm``` will contain the positions using UTM (or similar) 
-projection that attempts to place the layout into a rectilinear coordinates. 
-Upon use of ```pandas.read_hdf()``` the data will be brought into a DataFrame 
-object.
-
-- The index of the DataFrame is the site id. The HOPE datasets use an integer 
-for the id, while NRCAN uses a string. 
-- Columns are labelled ```lat``` and ```lon``` and contain the lat and lon in 
-degrees for each of the distributed sensors (or ```E```, ```N``` in the case of 
-```utm```).
-
-#### Irradiance Data
-Measurements consist of the individual sensor time series with a shared time 
-index. Upon use of ```pandas.read_hdf()``` the data will be brought into a 
-DataFrame object. Each individual sensor has its own column. 
-
-- Index of the DataFrame is the timestamp referenced to a timezone
-- Columns contain the time series for each individual sensor, and are keyed by
-the site id (HOPE - integer, NRCAN - string).
 
 # Contributing
-This project is happy to accept contributions and hear from users. The best way 
-to interact right now is to open an [issue](https://github.com/jranalli/solartoolbox/issues) 
-on GitHub. This is the best way to ask a question, make a suggestion about a 
-feature or describe a bug that you've encountered. 
+This is an open source project and appreciates participation, engagement and contribution from community members. Development on the library is active and the project seeks to provide a useful tool for the research community. The project is currently maintained by an individual researcher, and the process for contributions is not as formalized as it might be for larger projects.     
+
+If you've found a bug or have an idea for a new feature, please open an [issue](https://github.com/jranalli/solartoolbox/issues) 
+on GitHub. Questions can be asked in the GitHub [discussions](https://github.com/jranalli/solartoolbox/discussions).
+
+Code contributions are also welcome! Please follow the instructions in GitHub's [getting started guide](https://docs.github.com/en/get-started/start-your-journey/hello-world) to open a pull request. 
+
+Changes to the contribution guidelines and policies may be made in the future in response to growth of the project and community.
 
 # License
-This project is licensed under the BSD 3-Clause License - see the 
-[LICENSE](https://github.com/jranalli/solartoolbox/blob/main/LICENSE) file
-for full details. 
+This project is licensed under the BSD 3-Clause License - see the [LICENSE](https://github.com/jranalli/solartoolbox/blob/main/LICENSE) file for full details. 
 
 # History
 Solartoolbox began as a library of tools developed to support my own research 

docs/sphinx/Makefile

@@ -0,0 +1,20 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line, and also
+# from the environment for the first two.
+SPHINXOPTS    ?=
+SPHINXBUILD   ?= sphinx-build
+SOURCEDIR     = source
+BUILDDIR      = build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

docs/sphinx/make.bat

@@ -0,0 +1,35 @@
+@ECHO OFF
+
+pushd %~dp0
+
+REM Command file for Sphinx documentation
+
+if "%SPHINXBUILD%" == "" (
+	set SPHINXBUILD=sphinx-build
+)
+set SOURCEDIR=source
+set BUILDDIR=build
+
+%SPHINXBUILD% >NUL 2>NUL
+if errorlevel 9009 (
+	echo.
+	echo.The 'sphinx-build' command was not found. Make sure you have Sphinx
+	echo.installed, then set the SPHINXBUILD environment variable to point
+	echo.to the full path of the 'sphinx-build' executable. Alternatively you
+	echo.may add the Sphinx directory to PATH.
+	echo.
+	echo.If you don't have Sphinx installed, grab it from
+	echo.https://www.sphinx-doc.org/
+	exit /b 1
+)
+
+if "%1" == "" goto help
+
+%SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+goto end
+
+:help
+%SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% %O%
+
+:end
+popd

docs/sphinx/source/cmv.rst

@@ -0,0 +1,28 @@
+.. currentmodule:: solartoolbox
+
+Cloud Motion Vector Identification
+----------------------------------
+The `cmv` module contains tools for identifying the cloud motion vector (CMV) from a distributed network of measurement sensors. Two methods are implemented, that of Jamaly and Kleissl [1] and of Gagne et al. [2].
+
+The tool also contains a function for identifying the optimum subset of CMV vectors from among a long list of CMV pairs. This functions serves as part of the automated CMV identification workflow that serves the :mod:`solartoolbox.field` module.
+
+See examples found in the :ref:`cmv-examples` section. `cmv_demo` highlights identification of the CMV, while `automate_cmv_demo` shows how a group of useful CMVs can be downselected from a long time series of data [3].
+
+[1] M. Jamaly and J. Kleissl, "Robust cloud motion estimation by spatio-temporal correlation analysis of irradiance data," Solar Energy, vol. 159, pp. 306-317, Jan. 2018. https://www.sciencedirect.com/science/article/pii/S0038092X17309556
+
+[2] A. Gagne, N. Ninad, J. Adeyemo, D. Turcotte, and S. Wong, "Directional Solar Variability Analysis," in 2018 IEEE Electrical Power and Energy Conference (EPEC) (2018) pp. 1-6, iSSN: 2381-2842 https://www.researchgate.net/publication/330877949_Directional_Solar_Variability_Analysis
+
+[3] J. Ranalli and W.B. Hobbs, “Automating Methods for Validating PV Plant Equipment Labels,” 52nd IEEE PV Specialists Conference, 2024.
+
+.. automodule:: solartoolbox.cmv
+
+
+    .. rubric:: Functions
+
+    .. autosummary::
+        :toctree: generated/
+
+            compute_cmv
+            optimum_subset
+
+

docs/sphinx/source/conf.py

@@ -0,0 +1,51 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# For the full list of built-in configuration values, see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Project information -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+
+project = 'solartoolbox'
+copyright = '2024, Joe Ranalli'
+author = 'Joe Ranalli'
+
+release = '0.4'
+version = '0.4.1'
+
+import os
+import sys
+sys.path.insert(0, os.path.abspath(os.path.join('..', '..', '..', 'src')))
+
+# -- General configuration ---------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration
+
+extensions = ['sphinx.ext.duration',
+              'sphinx.ext.doctest',
+              'sphinx.ext.autodoc',
+              'sphinx.ext.autosummary',
+              'sphinx.ext.intersphinx',
+              'nbsphinx',
+              ]
+
+intersphinx_mapping = {
+    'python': ('https://docs.python.org/3/', None),
+    'sphinx': ('https://www.sphinx-doc.org/en/master/', None),
+}
+
+templates_path = ['_templates']
+
+
+
+# -- Options for HTML output -------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#options-for-html-output
+
+html_theme = 'sphinx_rtd_theme'
+# html_static_path = ['_static']
+
+# Control the display of class members
+autodoc_member_order = 'bysource'
+
+# -- Options for EPUB output
+epub_show_urls = 'footnote'
+