Wastewater nutrient concentration (g/use)

.gitignore

@@ -70,7 +70,7 @@ docs/_build/
 target/
 
 # Jupyter Notebook
-.ipynb_checkpoints
+#.ipynb_checkpoints
 
 # pyenv
 .python-version

docs/config.md

@@ -30,7 +30,8 @@ The custom directory must contain `.toml` files structured as follows:
 /my/custom/config/ 
 └── Region/ 
     ├── diurnal_patterns.toml
-    ├── household_statistics.toml 
+    ├── household_statistics.toml
+    ├── ww_nutrients.toml
     └── end_uses/
         ├── BathroomTap.toml
         ├── Bathtub.toml
@@ -61,6 +62,9 @@ It also contains total and weekend activity. Each activity within a section is g
 
 The `household_statistics.toml` contains statistical data for modeling household compositions and their characteristics based on different household types. The file is organised into several sections, each representing a different type of household. Each section contains sub-sections for specific demographic and employment characteristics. This configuration allows for the simulation of realistic household compositions and their characteristics, which can be used to model water demand patterns based on different household types.
 
+### Wastewater Nutrient Statistics
+The `ww_nutrients.toml` contains statistical data for nutrient concentration of wastewater for each enduse and usage subtype. Statistics are given in g/use to allow for the study of water quality as water usage varies.
+
 ### End Uses
 
 The `end_uses` folder contain `.toml` files for each household appliance that is simulated. The files contains statistical data for modeling the usage patterns of the related appliance. These files collectively provide a comprehensive model of household water usage patterns. They are organised into several sections, each representing different aspects of the appliance usage. These sections include general information about the appliance, frequency of use, and subtypes of end-uses.

docs/discharge/water_quality.md

@@ -0,0 +1,33 @@
+# Water Quality
+
+## Overview
+
+`pysimdeum` treats water quality calculations as a post-processing step after simulation. Simulated wastewater profiles in the form of a `discharge` object are the assumed input for both nutrient and temperature calculations.
+
+!!! info 
+    Note that water quality calculations are applied outside the simulation stage and will not be generated when simulating discharge patterns.
+
+## Nutrient Concentrations
+
+### Methodology
+
+The nutrient concentration calculations in `pysimdeum` are performed using a series of steps that process simulated discharge profiles and enrich it with nutrient concentrations based on input statistics from a [wastewater nutrient config](schema.md). These calculations inclusde steps to manipulate data from `xarray.Dataset` and `xarray.DataArray` objects, converting them to `pandas.DataFrame` objects for easier manipulation and calculation.
+
+1. **Data extraction and enrichment**
+    - Discharge data is extracted from `discharge` object (`xarray.Dataset`) and converted to a `pandas.DataFrame`. This DataFrame is then enriched with metadata, including `usage` and `event_label` columns, based on the discharge event metadata provided in the dataset.
+
+1. **Nutrient sampling**
+    - Nutrient concentrations (g/use) are sampled from a truncated normal distribution. The mean values for the distribution are read from a TOML config file, which contains statistics for enduse and usage type level, i.e. difference nutrient concentrations for `urine` and `faeces` use of a `Wc` enduse.
+
+1. **Nutrient concentration** 
+    - The nutrient concentration for each discharge event is calculated by dividing the sampled nutrient value by the total flow for the discharge event, resulting in a concentration in grams per liter (g/L).
+
+1. **Temporal aggregation**
+    - The user can input the period of time aggregation that the output is required in. This can be in seconds, minutes, 15 minutes, 30 minutes, or hourly periods. Defaults to hourly.
+
+!!! info 
+    Note that the period of temporal aggregation selected will affect runtime. Only aggregate in seconds if necessary, as this will significantly increase runtime especially when simulating multiple houses.
+
+## Temperature
+
+tbc

docs/schema.md

@@ -47,6 +47,33 @@ To create a custom configuration:
 - Save the `.toml` file in the appropriate config directory.
 
 
+## Wastewater Nutrient Statistics
+
+This config file has a section for each enduse and their following subtypes. If the user adds new enduses or enduse subtypes, these need to be added to this config file with their respecitive nutrient concentrations. Values are provided in g/use.
+
+Each enduse subtype section contains:
+
+- `n` (float): Nitrogen grams per use
+- `p`(float): Phosphorus grams per use
+- `cod`(float): COD grams per use
+- `bod5`(float): BOD grams per use
+- `ss`(float): Suspended solids grams per use
+- `amm`(float): Ammonia grams per use
+
+### Sample File Structure
+
+```toml
+[enduse]
+    [enduse.subtype]
+    n = 0.49
+    p = 0.07
+    cod = 13.93
+    bod5 = 7.43
+    ss = 13.93
+    amm = 0.06
+```
+
+
 ## Household Statistics
 
 This config file has a section for each household type. Household types include:

mkdocs.yml

@@ -12,6 +12,7 @@ nav:
     - Overview: discharge/overview.md
     - Common methods: discharge/common_methods.md
     - Enduse specifics: discharge/enduse_specifics.md
+    - Water quality: discharge/water_quality.md
   - config.md
   - Config Schema: schema.md
   - Changelog: CHANGELOG.md

pysimdeum/core/end_use.py

@@ -16,6 +16,7 @@ class EndUse:
     name: str = "EndUse"  # ... name of the end-use
     cold_water_temp = 10
     hot_water_temp = 60
+    discharge_events = []
 
     def init_consumption(self, users: list=None, time_resolution: str='1s') -> pd.DataFrame:
         """Initialization of a pandas dataframe to store the  consumptions.
@@ -93,7 +94,7 @@ def fct_duration_intensity_temperature(self):
 @dataclass
 class Bathtub(EndUse):
     """Class for Bathtub end-use."""
-
+    #discharge_events: list = field(default_factory=list)
 
     def __post_init__(self):
         """Initialisation function of Bathtub end-use class.
@@ -104,6 +105,7 @@ def __post_init__(self):
         """
         self.name = "Bathtub"
         self.wastewater_type = "greywater"
+        #self.discharge_events = []
 
     def fct_frequency(self, age=None):
         """Random function computing the frequency of use for the Bathtub end-use class.
@@ -169,6 +171,13 @@ def calculate_discharge(self, discharge, end, duration, intensity, temperature_f
         high = discharge_intensity_stats['high']
         discharge_flow_rate = dist(low=low, high=high)
 
+        self.discharge_events.append({
+            'enduse': self.name,
+            'usage': self.name, # no bath subtypes
+            'start': start,
+            'end': int(start + (remaining_water / discharge_flow_rate)),
+        })
+
         while remaining_water > 0:
             discharge_duration = remaining_water / discharge_flow_rate
             end = int(start + discharge_duration)
@@ -212,10 +221,12 @@ def simulate(self, consumption, discharge=None, users=None, ind_enduse=None, pat
 
 @dataclass
 class BathroomTap(EndUse):
+    #discharge_events: list = field(default_factory=list)
 
     def __post_init__(self):
         self.name = "BathroomTap"
         self.wastewater_type = "greywater"
+        #self.discharge_events = []
 
     def fct_frequency(self):
 
@@ -260,6 +271,13 @@ def calculate_discharge(self, discharge, start, duration, intensity, temperature
 
         start = offset_simultaneous_discharge(discharge, start, j, ind_enduse, pattern_num)
 
+        self.discharge_events.append({
+            'enduse': self.name,
+            'usage': self.subtype, # subtypes are inherited from chooser(toml)
+            'start': start,
+            'end': int(start + (remaining_water / discharge_flow_rate)),
+        })
+
         while remaining_water > 0:
             discharge_duration = remaining_water / discharge_flow_rate
             end = int(start + discharge_duration)            
@@ -301,10 +319,12 @@ def simulate(self, consumption, discharge, users=None, ind_enduse=None, pattern_
 
 @dataclass
 class Dishwasher(EndUse):
+    #discharge_events: list = field(default_factory=list)
 
     def __post_init__(self):
         self.name = "Dishwasher"
         self.wastewater_type = "blackwater"
+        #self.discharge_events = []
 
     def fct_frequency(self, numusers=None):
 
@@ -331,6 +351,13 @@ def calculate_discharge(self, discharge, start, j, ind_enduse, pattern_num, day_
             else:
                 discharge[discharge_time, j, ind_enduse, pattern_num, 1] = discharge_pattern[time]
 
+        self.discharge_events.append({
+            'enduse': self.name,
+            'usage': self.name, # no subtypes currently
+            'start': start,
+            'end': int (start + len(self.fct_duration_pattern())),
+        })
+
         return discharge
 
     def simulate(self, consumption, discharge=None, users=None, ind_enduse=None, pattern_num=1, day_num=0, total_days=1, simulate_discharge=False, spillover=False):
@@ -381,10 +408,12 @@ def simulate(self, consumption, discharge=None, users=None, ind_enduse=None, pat
 
 @dataclass
 class KitchenTap(EndUse):
+    #discharge_events: list = field(default_factory=list)
 
     def __post_init__(self):
         self.name = "KitchenTap"
         self.wastewater_type = "blackwater"
+        #self.discharge_events = []
 
     def fct_frequency(self, numusers=None):
 
@@ -430,7 +459,7 @@ def fct_duration_intensity_temperature(self):
 
         return duration, intensity, temperature
 
-    def calculate_discharge(self, discharge, start, duration, intensity, temperature_fraction, j, ind_enduse, pattern_num):
+    def calculate_discharge(self, discharge, start, duration, intensity, temperature_fraction, j, ind_enduse, pattern_num, usage):
         remaining_water = intensity * duration
         start = int(start)
 
@@ -450,6 +479,13 @@ def calculate_discharge(self, discharge, start, duration, intensity, temperature
         # Check if the tap is turned off before the end of the duration, if so, update the start time
         start = offset_simultaneous_discharge(discharge, start, j, ind_enduse, pattern_num)
 
+        self.discharge_events.append({
+            'enduse': self.name,
+            'usage': usage, # subtypes are from chooser(toml)
+            'start': start,
+            'end': int(start + (remaining_water / discharge_flow_rate)),
+        })
+
         while remaining_water > 0:
             discharge_duration = remaining_water / discharge_flow_rate
             end = int(start + discharge_duration)
@@ -484,6 +520,9 @@ def simulate(self, consumption, discharge=None, users=None, ind_enduse=None, pat
         for i in range(freq):
 
             duration, intensity, temperature = self.fct_duration_intensity_temperature()
+
+            # assign usage type (based on subtype)
+            usage = self.subtype
 
             prob_joint = normalize(prob_user * prob_usage)  # ToDo: Check if joint probability can be computed outside of for loop for all functions
 
@@ -497,7 +536,7 @@ def simulate(self, consumption, discharge=None, users=None, ind_enduse=None, pat
             if simulate_discharge:
                 if discharge is None:
                     raise ValueError("Discharge array is None. It must be initialized before being passed to the simulate function.")
-                discharge = self.calculate_discharge(discharge, start, duration, intensity, temperature_fraction, j, ind_enduse, pattern_num)
+                discharge = self.calculate_discharge(discharge, start, duration, intensity, temperature_fraction, j, ind_enduse, pattern_num, usage)
 
         return consumption, (discharge if simulate_discharge else None)
 
@@ -570,10 +609,12 @@ def simulate(self, consumption, discharge=None, users=None, ind_enduse=None, pat
 
 @dataclass
 class Shower(EndUse):
+    #discharge_events: list = field(default_factory=list)
 
     def __post_init__(self):
         self.name = "Shower"
         self.wastewater_type = "greywater"
+        #self.discharge_events = []
 
     def fct_frequency(self, age=None):
 
@@ -617,6 +658,13 @@ def calculate_discharge(self, discharge, start, duration, intensity, temperature
 
         start = offset_simultaneous_discharge(discharge, start, j, ind_enduse, pattern_num)
 
+        self.discharge_events.append({
+            'enduse': "Shower",
+            'usage': "Shower", # subtypes are class inheritance names
+            'start': start,
+            'end': int(start + (remaining_water / discharge_flow_rate)),
+        })
+
         while remaining_water > 0:
             discharge_duration = remaining_water / discharge_flow_rate
             end = int(start + discharge_duration)
@@ -669,10 +717,12 @@ def __post_init__(self):
 
 
 class WashingMachine(EndUse):
+    #discharge_events: list = field(default_factory=list)
 
     def __post_init__(self):
         self.name = "WashingMachine"
         self.wastewater_type = "blackwater"
+        #self.discharge_events = []
 
     def fct_frequency(self, numusers=None):
 
@@ -700,6 +750,13 @@ def calculate_discharge(self, discharge, start, j, ind_enduse, pattern_num, day_
             else:
                 discharge[discharge_time, j, ind_enduse, pattern_num, 1] = discharge_pattern[time]
 
+        self.discharge_events.append({
+            'enduse': "WashingMachine",
+            'usage': "WashingMachine", # no subtypes currently
+            'start': start,
+            'end': int(start + len(self.fct_duration_pattern())),
+        })
+
         return discharge
 
     def simulate(self, consumption, discharge=None, users=None, ind_enduse=None, pattern_num=1, day_num=0, total_days=1, simulate_discharge=False, spillover=False):
@@ -752,10 +809,12 @@ def simulate(self, consumption, discharge=None, users=None, ind_enduse=None, pat
 
 @dataclass
 class Wc(EndUse):
+    #discharge_events: list = field(default_factory=list)
 
     def __post_init__(self):
         self.name = "Wc"
         self.wastewater_type = "blackwater"
+        self.discharge_events = []
 
 
     def fct_frequency(self, age=None, gender=None):
@@ -788,13 +847,20 @@ def fct_duration_intensity_temperature(self):
         return duration, intensity, temperature
 
 
-    def calculate_discharge(self, discharge, start, duration, intensity, temperature_fraction, j, ind_enduse, pattern_num):
+    def calculate_discharge(self, discharge, start, duration, intensity, temperature_fraction, j, ind_enduse, pattern_num, usage):
         incoming_water = intensity * duration
         end = int(start)
 
         # Sample a value from the discharge_intensity distribution
         discharge_flow_rate = self.statistics['discharge_intensity']
 
+        self.discharge_events.append({
+            'enduse': "Wc",
+            'usage': usage,
+            'start': int(end - (incoming_water / discharge_flow_rate)),
+            'end': end,
+        })
+
         while incoming_water > 0:
             discharge_duration = incoming_water / discharge_flow_rate
             start = int(end - discharge_duration)
@@ -819,6 +885,9 @@ def simulate(self, consumption, discharge=None, users=None, ind_enduse=None, pat
 
                 duration, intensity, temperature = self.fct_duration_intensity_temperature()
 
+                # assign usage type (urine or faeces)
+                usage = "urine" if np.random.random() * 100 < self.statistics['prob_urine'] else "faeces"
+
                 prob_joint = normalize(prob_user * prob_usage)
                 start, end = sample_start_time(prob_joint, day_num, duration, previous_events)
                 previous_events.append((start, end))
@@ -830,7 +899,7 @@ def simulate(self, consumption, discharge=None, users=None, ind_enduse=None, pat
                 if simulate_discharge:
                     if discharge is None:
                         raise ValueError("Discharge array is None. It must be initialized before being passed to the simulate function.")
-                    discharge = self.calculate_discharge(discharge, start, duration, intensity, temperature_fraction, j, ind_enduse, pattern_num)
+                    discharge = self.calculate_discharge(discharge, start, duration, intensity, temperature_fraction, j, ind_enduse, pattern_num, usage)
 
         return consumption, (discharge if simulate_discharge else None)
 

pysimdeum/core/house.py

@@ -303,6 +303,15 @@ def simulate(self, date=None, duration='1 day', num_patterns=1, simulate_dischar
         if simulate_discharge:
             self.consumption = xr.DataArray(data=consumption, coords=[time, users, enduse, patterns, flowtype], dims=['time', 'user', 'enduse', 'patterns', 'flowtypes'])
             self.discharge = xr.DataArray(data=discharge, coords=[time, users, enduse, patterns, dischargetype], dims=['time', 'user', 'enduse', 'patterns', 'dischargetypes'])
+
+            # discharge event metadata
+            discharge_events = []
+            for appliance in self.appliances:
+                if hasattr(appliance, 'discharge_events'):
+                    discharge_events.extend(appliance.discharge_events)
+
+            self.discharge = xr.Dataset({'discharge': self.discharge})
+            self.discharge['discharge_events'] = xr.DataArray(discharge_events)
         else:
             self.consumption = xr.DataArray(data=consumption, coords=[time, users, enduse, patterns, flowtype], dims=['time', 'user', 'enduse', 'patterns', 'flowtypes'])