Skip to content

Commit

Permalink
Merge pull request #12 from MDBAuth/Ahsanul-UpdateDocs
Browse files Browse the repository at this point in the history
Ahsanul update docs
  • Loading branch information
BenBMDBA authored Jan 12, 2023
2 parents 38be59a + eba394c commit 9d46d62
Show file tree
Hide file tree
Showing 7 changed files with 2,553 additions and 520 deletions.
551 changes: 551 additions & 0 deletions .ipynb_checkpoints/Gauge_getter_example-checkpoint.ipynb

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
cff-version: 0.4.5
cff-version: 0.4.6
message: "If you use this software, please cite it as below."
authors:
- family-names: Bradshaw
Expand All @@ -15,6 +15,6 @@ authors:
given-names: Ahsanul

title: "MDBA_Gauge_Getter"
version: 0.4.5
version: 0.4.6
doi: 10.5281/zenodo.7435693
date-released: 2022-12-14
2,466 changes: 1,957 additions & 509 deletions Gauge_getter_example.ipynb

Large diffs are not rendered by default.

47 changes: 41 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,25 +15,60 @@
## Description
MDBA Gauge Getter provides a unified and simple interface to collect surface water data from the following state water portals:


|State|Site|Source|
| --- | --- | --- |
| NSW | realtimedata.waternsw.com.au |CP|
| QLD | water-monitoring.information.qld.gov.au |PUBLISH|
| VIC | data.water.vic.gov.au |AT|
| SA | bom.gov.au/waterdata/ | BOM Water Data Online|

The tool is configured to abstract away the details specific to each state water portal and return a consistent structure.
By default it will return a daily mean of a flow in ML/day for a given gauge number, but level and other intervals and aggregations are available
The tool is configured to abstract away the details specific to each state water portal and return a consistent structure. By default it will return a daily mean of a flow in ML/day for a given gauge number, but storage level, storage volume, other intervals and aggregations are available.


Example of its use can be seen in Gauge_getter_example.ipynb notebook, contact [email protected] for more details.

## Installation
## Local Installation via git

- Clone the repo in your local folder: for example, inside `~./users/john.doe` git clone `https://github.com/MDBAuth/MDBA_Gauge_Getter.git` which will create a git tracked project repository inside `~./users/john.doe/mdba_gauge_getter`.
- Create a virtual environment: for example, `conda create --name gauge_getter_env`.
- Activate the virtual environment: `conda activate gauge_getter_env`.
- Go inside the project folder i.e., `cd ~./users/john.doe/mdba_gauge_getter`.
- Check `git status` and `git branch`.
- Install dependencies with `pip3 install -r requirements.txt`
- (Optionally) install dev dependencies with `pip3 install -r requirements-dev.txt`
- Run `python3 setup.py install` to install the module

## Quick Start

- Install via pip with the command: `pip install mdba-gauge-getter`
- After installation, import the package with the command: `import mdba_gauge_getter.gauge_getter as gg`
- Import datetime for converting your intervals into python datetime object: `import datetime as dt`

## Usage

There are several options to call Gauge Getter which are as follows:
- `gauge_numbers` denotes the gauge(s) for which the parameters such as flow, lake/storage level, storage volume etc. will be obtained. It takes a list of strings (gauge numbers) as input.
- `start_time_user` denotes the start time of the userdefined interval. It takes a datetime python object as input.
- `end_time_user` denotes the end time of the userdefined interval. It takes a datetime python object as input.
- `data_source` denotes which state the gauge(s) belong(s) to which API to fetch the data from. Please note that SA does not currently have an API to obtain the data from; hence, the data is fetched from the BOM API. Different *data_source* options are:
- <STATE> i.e., 'NSW', 'VIC', 'QLD', 'SA'/'BOM'
- 'BOM'
- `var` denotes the parameter to retreieve such as flow, lake/storage level, storage volume etc. It takes a string indicating the parameter type as input. Different string notation for different *var* options are:
- 'F' for flow (default).
- 'L' for water level used for flow calculation.
- 'LL'/'SL' for lake/storage level.
- 'SV' for storage volume. Please note that this is exclusively a BOM API parameter. Please specify the data source as 'BOM' if you would like to retrieve this parameter.
- `interval` indicates the duration the parameter data are collected for aggregation. Different *interval* options are:
- 'day'. Alternate options for BOM API call is: 'd'.
- 'hour'. Alternate options for BOM API call is: 'h'.
- 'month'. Alternate options for BOM API call is: 'm'.
- 'year'. Alternate options for BOM API call is: 'y'.
- `data_type` inidcates the aggregation method. Different *data_type* options are:
- 'mean' (default). Alternate options for BOM API call are: 'avg', 'average', 'av' and 'a'.
- 'min'. Alternate options for BOM API call is: 'minimum'. Only available when obtaining *daily* interval data.
- 'max'. Alternate options for BOM API call is: 'maximum'. Only available when obtaining *daily* interval data.

## Support
For issues relating to the script, a tutorial, or feedback please contact Ben Bradshaw [email protected]
For data issues please see the corresponding state water portals
For issues relating to the script, a tutorial, or feedback please contact Ben Bradshaw ([email protected]) or Ahsanul Habib ([email protected]).

For data issues please see the corresponding state water portals.
1 change: 0 additions & 1 deletion mdba_gauge_getter/gauge_getter.py
Original file line number Diff line number Diff line change
Expand Up @@ -377,7 +377,6 @@ def gauge_pull_bom(gauge_numbers: List[str], start_time_user: datetime.date, end
response = bm.request(bm.actions.GetObservation, gauge, prop, procedure, t_begin, t_end)
# response_json = bm.xml_to_json(response.text)
ts = bm.parse_get_data(response)
print(ts)
if ts.empty:
ts = pd.DataFrame(columns=["DATASOURCEID","SITEID", "SUBJECTID", "DATETIME", "VALUE", "QUALITYCODE"])
collect.append(ts)
Expand Down
2 changes: 1 addition & 1 deletion mdba_gauge_getter/version.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = '0.4.5'
__version__ = '0.4.6'
2 changes: 1 addition & 1 deletion tests/test_gauge_getter.py
Original file line number Diff line number Diff line change
Expand Up @@ -373,7 +373,7 @@ def dict_matches(d1: Dict[str, Any], d2: Dict[str, Any]):

def test_sort_gauges_by_state():
gauge_getter.gauge_data_uri = StringIO(MOCK_CSV)
print(gauge_getter.gauge_data_uri)
# print(gauge_getter.gauge_data_uri)
gauge_getter.init()
ret = gauge_getter.sort_gauges_by_state(['1', '2', '3', '4', '5', '6', '10'])
expect = {'NSW': ['1', '3'], 'QLD': ['2', '3', '4'], 'VIC': ['4', '5'], 'SA': ['6'], 'rest': ['10']}
Expand Down

0 comments on commit 9d46d62

Please sign in to comment.