Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing the internal renaming of the repository to solarspatialtools #10

Closed
wants to merge 19 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
33a8406
Minor bugfix for r_corr output in Jamaly CMV
jranalli Apr 16, 2024
d80a1d3
added demo of how to digitize and anonymize a plant (#5)
williamhobbs May 15, 2024
c040287
Update readme and changelog
jranalli May 15, 2024
b101f94
Fix mistake for gitignore re: changed filenames in the digitization d…
jranalli May 15, 2024
53443e4
filter on asset_type == combiner
williamhobbs May 16, 2024
f37fd7f
Merge pull request #6 from williamhobbs/dev
jranalli May 16, 2024
945ffa9
Hopefully fix the github rendering of cmv_demo.ipynb, add comments to…
jranalli Jul 16, 2024
917472e
Hopefully fix failing CI tests. Appears to be related to a numpy vers…
jranalli Jul 16, 2024
9d6563e
Updates to the README particularly in response to the issues with the…
jranalli Jul 16, 2024
cd823aa
Add GitHub Discussions for questions
jranalli Jul 16, 2024
4ee010e
Very preliminary effort at starting sphinx docs
jranalli Jul 15, 2024
6b726bd
not sure if these files are needed
jranalli Jul 15, 2024
1ed40ae
Moving demos and progress on sphinx
jranalli Jul 17, 2024
8b0d6dd
Correct location of demo data files for updated documentation location
jranalli Jul 18, 2024
cbbb277
Draft of sphinx docs for the functions
jranalli Jul 18, 2024
2d08cb8
Update sphinx docs draft
jranalli Jul 18, 2024
1eddb59
Initialize rename of repository to solarspatialtools. Github and pip …
jranalli Jul 25, 2024
aaa026f
Fixing dependencies for readthedocs hopefully
jranalli Jul 25, 2024
4b1370a
Trying to fix ipython warnings on readthedocs builds
jranalli Jul 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -131,8 +131,15 @@ dmypy.json
###### My Additions ######

# Large data files
/demos/data/
/demos/data/*

# but inlcude the sample plant geojson file
!/demos/data/digitized_plant*.*

# IDE Files
.idea*
*.egg
*.egg

# Sphinx Generated files
/docs/sphinx/build/*
/docs/sphinx/source/generated/*
22 changes: 22 additions & 0 deletions .readthedocs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# .readthedocs.yaml
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

# Required
version: 2

build:
os: "ubuntu-22.04"
tools:
python: "3.10"

python:
install:
- method: pip
path: .
extra_requirements:
- docs

sphinx:
configuration: docs/sphinx/source/conf.py
fail_on_warning: true
8 changes: 7 additions & 1 deletion Changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,4 +53,10 @@ site itself.
- `field_reassignment_demo` - Automated remapping of the field data, includes a Jupyter notebook.
- `field_demo_full_process` - Overview of the full field processing workflow
- `field_demo_full_process_multithread` - Same as previous but including multithreading support for speed
- Added tests of these new functions for more completeness.
- Added tests of these new functions for more completeness.
# Version 0.4.1
- Bug fix for `cmv` so that `jamaly` r_corr is always positive.
- Demo for digitizing a plant equipment layout added by @williamhobbs
- Improve comments in `cmv_demo`
- Add sphinx documentation for entire library
- Name change to SolarSpatialTools
92 changes: 35 additions & 57 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,31 @@
# SolarToolbox
`solartoolbox` is a package containing tools for dealing with analysis of solar
energy data. Its specific focus is on signal processing approaches and
addressing variability from a spatiotemporal perspective. Tools here might be
useful for dealing with distributed data sets, or performing analyses that
rely on a spatially distributed set of measurements.
# SolarSpatialTools
Spatial analysis tools for solar energy research

`solarspatialtools` is a python package containing implementations of various spatial algorithms for solar energy data. While excellent open source solar energy packages already exist (e.g. [pvlib-python](https://github.com/pvlib/pvlib-python) and [pvanalytics](https://github.com/pvlib/pvanalytics)), the complexity of some high-level analyses found in the academic literature makes them as a poor fit for the scope of existing packages. This package fills that gap by implementing techniques that we hope can facilitate common spatial tasks for solar energy researchers and provide a platform for consistency and efficiency improvements in these calculations.

Two examples of what are believed to be the most generally useful techniques this package implements are:
- Processing the cloud motion vector from a distributed sensor network.
- Verifying the locations of field components (e.g. combiners) within a distributed network of measurements

# Installation
The package can be most easily installed via [PyPi](https://pypi.org/project/solartoolbox/)
with the following command:
```bash
pip install solartoolbox
pip install solarspatialtools
```

# Getting Started
A number of example codes are available in the demos folder. These are meant to demonstrate what are believed to be the most useful functions in the package and applications of the functions to real sample data included with the library.


# Common Data Formatting

The algorithm implementations in this package primarily involve spatially distributed analyses of solar energy data, necessitating processing of multiple simultaneous time series. Most codes will assume that data is provided using a common format, based upon ```pandas``` DataFrame objects. The demo codes use the following convention for the two most important data variables.
- ```pos``` or ```pos_utm``` - Data for the location of each sensor should be stored in a DataFrame with the sensor id as the index and the latitude and longitude (or UTM coordinates) as columns.
- ```ts_data``` - Sensor time series data should be stored in a DataFrame with the timestamp as the index and the sensor id as the columns.
The index of ```pos``` needs to match the columns of ```ts_data``` so that the correspondence between the locations and the time series can be maintained.


# Structure of the Library
The codes are organized into a few subpackages and several function libraries.
The subpackages are meant to contain tools that are related to a specific
Expand All @@ -37,7 +51,7 @@ area that could use some expansion in the future.

```demos```
Data and demonstration codes (including as jupyter notebooks) that demonstrate
the functionality of the package. An explanation the included data is
the functionality of the package. An explanation for the included data is
warranted.

- Anonymized Plant Combiner Data
Expand Down Expand Up @@ -74,7 +88,7 @@ warranted.
- See `dataio\hope_campaign.py` for details on the original dataset.
- See `signalproc_demo.py` for examples using this data.

## Function libraries in solartoolbox (root level)
## Function libraries in solarspatialtools (root level)

```cmv```
Functions for computing the cloud motion vector from a distributed irradiance
Expand Down Expand Up @@ -107,60 +121,24 @@ of each hour of the day over a 30 day window)
Functions for predicting the position of field components on the basis of cloud
motion.

## Common format for H5 files used for Data Storage

I've tried to format the multisite time series measurements in a way that's
conveinent for loading the files and working with the data. This came about
from my initial work analyzing the HOPE Campaign, which used 100 individual
point measurements of GHI scattered through a region near Jülich, Germany.

All data is collected into a single H5 file containing multiple fields. I use
```pandas``` and specifically ```pandas.read_hdf()``` for getting the data
into python.

- ```latlon```: The latitude/longitude of the individual measurement sites
- ```utm```: The UTM coordinates of the individual measurement sites
- ```data```: Global Horizontal Irradiance
- ```data_tilt```: Global Tilted Irradiance (if available)

#### Location Data
Data about the location of each individual site is stored in the H5 file. Two
possible keys are used depending on the projection. Both are available when
possible. The key ```latlon``` represents the site in a latitude coordinate
system. The key ```utm``` will contain the positions using UTM (or similar)
projection that attempts to place the layout into a rectilinear coordinates.
Upon use of ```pandas.read_hdf()``` the data will be brought into a DataFrame
object.

- The index of the DataFrame is the site id. The HOPE datasets use an integer
for the id, while NRCAN uses a string.
- Columns are labelled ```lat``` and ```lon``` and contain the lat and lon in
degrees for each of the distributed sensors (or ```E```, ```N``` in the case of
```utm```).

#### Irradiance Data
Measurements consist of the individual sensor time series with a shared time
index. Upon use of ```pandas.read_hdf()``` the data will be brought into a
DataFrame object. Each individual sensor has its own column.

- Index of the DataFrame is the timestamp referenced to a timezone
- Columns contain the time series for each individual sensor, and are keyed by
the site id (HOPE - integer, NRCAN - string).

# Contributing
This project is happy to accept contributions and hear from users. The best way
to interact right now is to open an [issue](https://github.com/jranalli/solartoolbox/issues)
on GitHub. This is the best way to ask a question, make a suggestion about a
feature or describe a bug that you've encountered.
This is an open source project and appreciates participation, engagement and contribution from community members. Development on the library is active and the project seeks to provide a useful tool for the research community. The project is currently maintained by an individual researcher, and the process for contributions is not as formalized as it might be for larger projects.

If you've found a bug or have an idea for a new feature, please open an [issue](https://github.com/jranalli/solartoolbox/issues)
on GitHub. Questions can be asked in the GitHub [discussions](https://github.com/jranalli/solartoolbox/discussions).

Code contributions are also welcome! Please follow the instructions in GitHub's [getting started guide](https://docs.github.com/en/get-started/start-your-journey/hello-world) to open a pull request.

Changes to the contribution guidelines and policies may be made in the future in response to growth of the project and community.

# License
This project is licensed under the BSD 3-Clause License - see the
[LICENSE](https://github.com/jranalli/solartoolbox/blob/main/LICENSE) file
for full details.
This project is licensed under the BSD 3-Clause License - see the [LICENSE](https://github.com/jranalli/solartoolbox/blob/main/LICENSE) file for full details.

# History
Solartoolbox began as a library of tools developed to support my own research
activities on solar energy. It was always shared publicly to encourage use by
SolarSpatialTools began as a library of tools developed to support my own
research activities on solar energy. Initial publication took place under the
name of `solartoolbox`. It was always shared publicly to encourage use by
interested parties and make a small contribution to open science and promoting
reproducibility in the field. As a result, most initial releases lacked full
documentation and the structure of the library underwent some significant
Expand Down
446 changes: 0 additions & 446 deletions demos/automate_cmv_demo.ipynb

This file was deleted.

Loading
Loading