merge Analysis into main (#14)

* Create analysis folder * remove spurious file * add interval functions * update interval function docs * Update README.md * update requirements.txt * fix pandas deprecation of .append * fix relative path ModuleNotFoundError * Merge analysis into Analysis (#11) * update README * add links to README.md * Update README.md * Update README.md * Update README.md * Update README.md Add Gabriele et al. reference link * Update README.md Add publication link, add vessel statement. * Update README.md * Update README.md * Adding Analysis --------- Co-authored-by: Logan <[email protected]> * Rename helperfunctios.py to helperfunctions.py * remove DRAFT Jupyter notebook * move functions to computation.py * parallelize parsing for Ais class * update models.Ais to handle 100 percent ADS-B * sort NVSPL DatetimeIndex to be monotonic * update virtual environment requirements, pyproj and geopandas * adjust deprecated pd.DataFrame.append() to pd.concat() * update deprecated pd.DataFrame.append() with pd.concat(), adjust fw header row * in .add_annotation() ensure each new linestring is in EPSG 4326 * models.py update concurrent import statement * Initial commit AudibleTransits super class * Update _DENA README.md * initial CLI version of run_audible_transits * Updated duplicate removal, streamlined aircraft info extraction, and fixed bug in takeoff/landing detection for GPS * Initial commit for geography-based audibility metrics generator * add function * fix typo in README.md * Added 'split_paused_tracks()' method, streamlined 'create_segments()', and set raster path from config file * Modified 'query_tracks' to include the flight_id, which contains the n-number (previously used a simple numeric id) * Adjusted 'load_tracks_from_database()' for onboard GPS to use Dini's query_tracks() instead of Sam's * Update README.md * Added garbage tracking and exporting capabilities * Modified query_tracks and query_adsb for functionality with run_audible_transits * facilitate relative import in generate_active_space.py * resolve AttirbuteError pandas.Int64Index * AttributeError debug follow-through * document path environment var, remove debug console messages * adjust decimal precision in log and console * add an altitude cap in ADSB data model * add an altitude cap in Early_ADSB data model + generate_active_space * rename variable altitudes to altitudes_ * add lower altitude bound on ADS-B record * adjust get_deployments for d{3} site codes * add temporary debug information * Added 'calculate_spatial_statistics' and first attempt at stereotypical track detection * Updated inline documentation, added 'export_garbage_summary', and modified 'simplify_active_space' to remove small interior rings * Fixed typo in the imports * Small updates to visualization, changed folder to save tracks (to V drive) * Added gain to file naming * Initial commit for variant of run_audible_transits that performs the same tasks on a circular active space of equal area * fixed bug in file name gain * Got rid of plt.pause when generating garbage pdf, sorted ADSB track points by point_dt in 'create_segments()' to ensure proper sampling interval calculations * fix IndexError when all points audible * replace list with gpd.GeoDataFrame * add geometry field * update ground truthing segmentation conditions * add time pad, 80 Hz guideline * remove diagnostic console prints * Reduce figure size to 250px README.md * add module descriptions, links, reorder README.md * add __all__ * tidy docstrings, add in-line docs generate_geographic_metrics.py * rephrase active-space task README.md * add geographic-metrics API overview, tidy README.md * v2.1.0 * revise docstrings, inline documentation * additional docstring and inline documentation * anticipate AIS functionality by raising NotImplementedError * improve spacing in `utils` docs README.md * refactor console printouts, pt.1 * refactor console printouts, pt.2 * refactor console printouts, pt.3 * refactor console printouts, pt.4 * .simplify_active_space() control flow adjust * refactor console printouts, pt.5 * refactor console printouts, pt.6 * refactor console printouts, pt.7 * refactor console printouts, pt.9 * refactor console printouts, pt.10 --------- Co-authored-by: Betchkal <[email protected]> Co-authored-by: Logan <[email protected]> Co-authored-by: Brotman-Krass <[email protected]> Co-authored-by: jacobbk <[email protected]>
dbetchkal · Dec 30, 2024 · b41b074 · b41b074
1 parent 707a559
commit b41b074
Show file tree

Hide file tree

Showing 21 changed files with 43,065 additions and 124 deletions.
diff --git a/README.md b/README.md
@@ -7,36 +7,81 @@ This repository is designed to estimate active spaces for motorized noise source
 
 ## Example
 
-Consider an example active space, below. It was computed using data from a long term acoustic monitoring site in Denali National Park, DENAUWBT Upper West Branch Toklat ([Withers 2012](https://irma.nps.gov/DataStore/Reference/Profile/2184396)), indicated by the black point in the center of the image. The bold black polygon delineates an active space estimate for flights at 3000 meters altitude. Points interior to the polygon are predicted to be audible, those exterior, inaudible. <br> 
+Consider an example active space, below. It was computed using data from a long term acoustic monitoring site in Denali National Park, DENAUWBT Upper West Branch Toklat ([Withers 2012](https://irma.nps.gov/DataStore/Reference/Profile/2184396)). The bold black polygon delineates an active space estimate for flights at 3000 meters altitude. Points interior to the polygon are predicted to be audible, those exterior, inaudible. <br> 
+
+Superposed over the polygon are colored flight track polylines. `NPS-ActiveSpace` includes an application that leverages the acoustic record to ground-truth audibility of co-variate vehicle tracks from GPS databases. Ground-truthing is used to "tune" an active space to the appropriate geographic extent via mathematical optimization.<br>
 
-Superposed over the polygon are flight track points colored by their corresponding audibility status. `NPS-ActiveSpace` includes an application that leverages the acoustic record to ground-truth audibility of co-variate vehicle tracks from GPS databases. Ground-truthing is used to "tune" an active space to the appropriate geographic extent via mathematical optimization.<br>
 <br>
-<img src="https://github.com/dbetchkal/NPS-ActiveSpace/blob/main/nps_active_space/img/NPS-ActiveSpace_example.png" alt="active space polygon example" width="300">
+<img src="https://github.com/dbetchkal/NPS-ActiveSpace/blob/main/nps_active_space/img/NPS-ActiveSpace_example.png" alt="active space polygon example" width="200">
 
 
 ## Packages
 
-This project is made up of four packages:
+This project is made up of four modules:
 
-[`utils`](https://github.com/dbetchkal/NPS-ActiveSpace/tree/main/nps_active_space#utils): diverse utilities - file I/O, geoprocessing computations, acoustic propagation modelling, and detection statistics
-    
-[`ground-truthing`](https://github.com/dbetchkal/NPS-ActiveSpace/blob/main/_DENA/README.md#ground-truthing): a `tkinter`-based ground-truthing application
+[`ground-truthing`](https://github.com/dbetchkal/NPS-ActiveSpace/blob/main/_DENA/README.md#ground-truthing): a `tkinter`-based interactive GUI app for the annotation of georeferenced sound events.
+
+[`active-space`](https://github.com/dbetchkal/NPS-ActiveSpace/blob/main/_DENA/README.md#generate-active-space): observer-based audibility modelling procedures that produce an optimized active space estimate through synthesis.
 
-[`active-space`](https://github.com/dbetchkal/NPS-ActiveSpace/blob/main/_DENA/README.md#generate-active-space): generate and optimize active space polygons
+[`audible-transits`](https://github.com/dbetchkal/NPS-ActiveSpace/tree/main/nps_active_space#audible-transits): geoprocess to construct the spatiotemporal intersections of a set of tracks with an active space.
 
-`analysis`: estimate acoustic metrics from the intersection of an active space polygon and vehicle tracks (*currently in development, see `Analysis` branch*)
 
+[`geographic-metrics`](https://github.com/dbetchkal/NPS-ActiveSpace/tree/main/nps_active_space#geographic-metrics): tabulation of transits into a variety of acoustic and spatial metrics 
+
+[`utils`](https://github.com/dbetchkal/NPS-ActiveSpace/tree/main/nps_active_space#utils): diverse utilities - file I/O, geoprocessing, acoustic propagation modelling, and detection statistics
+
 Also included are noise source [data](https://github.com/dbetchkal/NPS-ActiveSpace/tree/v2/nps_active_space/data) for tuning active space polygons.
 
-***For more specific information on each package, view their individual READMEs.***
 
 ## Order of Operations
 
-While each package can be used and run individually, the project was designed so that outputs of one package work seamlessly as the inputs for another. 
+While each package can be used and run individually, the project was designed so that outputs of one package work seamlessly as the inputs for another. Packages were designed to be run in the following order:
+
+`ground-truthing` $\rightarrow$ `active-space` $\rightarrow$ `audible-transits` $\rightarrow$ `geographic-metrics`
+
+## ground-truthing
+
+<img src="https://ars.els-cdn.com/content/image/1-s2.0-S0301479723019898-gr2.jpg" alt="The provided `NPS-ActiveSpace.ground_truthing` module `tkinter`-based app. Reproduced from Betchkal et al. 2023, Fig. 2. A view of the NPS-ActiveSpace ground-truthing application with a completed spectrogram annotation for an audible helicopter overflying HAVO009A. The upper map frame shows ADS-B data (brown points) in the xy-plane and the user-estimated spatial extent of audibility (cyan highlight). The lower spectrogram frame includes the noise event as contrasted against the natural residual ambience. It also provides the user a cue: the timestamp corresponding to the most proximal ADS-B point (vertical green line). Audible extent was then estimated by adjusting the temporal boundary (cyan slider)." width="700">
+
+The `ground-truthing` module provides a `tkinter`-based interactive GUI app for the annotation of georeferenced sound events. This module is the initial step of the process. Prerequesite to using this module is logging a simultaneous pair of datasets in the field: (1) a canonical Type-1 NPS acoustic record (`Nvspl`) and (2) a transportation dataset (`Adsb`, `Ais`, or generalized `Tracks`).
+
+The module is initialized in the Command Line Interface (CLI). Detailed [CLI documentation is available to initialize the app](https://github.com/dbetchkal/NPS-ActiveSpace/tree/Analysis/_DENA#ground-truthing) from a park-specific configuration file (see [`template.config`](https://github.com/dbetchkal/NPS-ActiveSpace/blob/Analysis/_DENA/config/template.config)).
+
+## active-space
+
+The `active-space` module is a CLI implementation of observer-based audibility modelling procedures. It produces an active space estimate through synthesis. This module exists primarially as a wrapper for the `FORTRAN`-based physics engine `Nord2000` as implemented in `NMSIM`. Previously-saved `ground-truthing.Annotations` files are required as an input. Diverse spatial and sound source inputs are also required to stage the `NMSIM` simulation (see [Ikelheimer and Plotkin 2005](https://github.com/dbetchkal/NMSIM-Python/blob/main/NMSIM/Manual/NMSim%20Manual.pdf)). 
+
+Detailed [CLI documentation is available to configure a synthesis](https://github.com/dbetchkal/NPS-ActiveSpace/tree/Analysis/_DENA#generate-active-space) of the optimal active space estimate for a park listener in a specific location.
+
+## audible-transits
+
+The `audible-transits` module is a CLI geoprocess to construct the spatiotemporal intersections of a set of tracks with an active space. As part of the construction errant `Tracks` are removed and tabulated. Output `Tracks` are imbued with the information necessary to produce an audiblity time series.
+
+Detailed [CLI documentation is available to initialize the construction](https://github.com/dbetchkal/NPS-ActiveSpace/tree/Analysis/_DENA#audible-transits).
+
+## geographic-metrics
+
+The `geographic-metrics` module estimates what we hear. To do this, it collapses the set of `audible-transits` into a binary audibility sequence in time. 
+Then, from attributes of these *noise events* (or dualistically, *noise-free intervals*) a variety of acoustical and spatial metrics may be computed.
+
+At present, no CLI interface exists for `geographic-metrics`. Instead it has been designed to be imported into a more flexible IDE.
+
+## utils
 
-Packages were designed to be run in the following order:
+The utilities module `utils` contains two sub-modules:
+1. `computation` for tasks related to:
+   >**geoprocessing** `.build_src_point_mesh`,`.climb_angle`,`.coords_to_utm`,`.create_overlapping_mesh`,`.interpolate_spline`,`.NMSIM_bbox_utm`,`.project_raster`<br><br>
+   >**audibility** `.audibility_to_interval`,`.ambience_from_nvspl`,`.ambience_from_raster`,`.contiguous_regions` <br><br>
+   >**detection statistics** `.calculate_duration_summary`,`.compute_fbeta` <br>
+2.  and `models` containing classes which parse various forms of input data:
+    > **Automatic Dependent Surveillance–Broadcast (ADS-B)** broacasts from aircraft `.Adsb`, `.EarlyAdsb`<br><br>
+    > **Automatic Identification System (AIS)** broadcasts from ships `.Ais`<br><br>
+    > human **spectrogram annotations** from the `NPS-ActiveSpace.ground_truthing` module as `.Annotations`<br><br>
+    > descriptions of canonical NPS Type-1 acoustic monitoring **Deployments** `.Microphone`<br><br>
+    > an **acoustic record** as 1/3rd-octave band spectral sound levels from a Deployment `.Nvspl` <br><br>
+    > generalized `.Tracks`<br>
 
-`ground-truthing` $\rightarrow$ `active-space` $\rightarrow$ `analysis`
+Most users should not need to use `utils` directly, but the data parsing classes may have use to other transportation geography projects.
 
 ---
 

diff --git a/_DENA/README.md b/_DENA/README.md
@@ -150,4 +150,30 @@ $ python -u -W ignore _DENA/scripts/generate_active_space_mesh.py -e production
 
 ```bash
 $ python -u -W ignore _DENA/scripts/generate_active_space_mesh.py -e production -n DENAFULL -s C:/Users/yourname/Desktop/DENA.shp --headings 0 180 --omni-source -12.5 --mesh-spacing 10 --mesh-size 20 -l 1524
-```
+```
+
+### Audible Transits
+
+This script is used to estimate the geographic intersection of a set of tracks with an active space. It simplifies and enriches the geographic data with information necessary to produce an audiblity time series. 
+
+| command-line arg       | description                                                                                                                                                |
+|------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `-e`, `--environment`  | **required.**<br/>The configuration environment to use. *Ex*: To use `production.config` pass `-e production`                                              |
+| `-u`, `--unit`         | **required.**<br/>The 4 letter NPS unit code. *Ex*: Denali = DENA                                                                                          |
+| `-s`, `--site`         | **required.**<br/>The 4 letter site code. *Ex*: Cathedral = CATH                                                                                           |
+| `-y`, `--year`         | **required.**<br/>The deployment year, YYYY. *Ex*: 2018                                                                                                    |
+| `-g`, `--gain` | **required.**<br/>The signed gain of the optimal active space fit, float. *Ex.*: -20.5 |
+| `-t0`, `--begintracks` | **required.**<br/>Date to begin parsing the position record, YYYY-MM-DD. *Ex.*: 2018-01-01 |
+| `-tf`, `--endtracks` | **required.**<br/>Date to stop parsing the position record, YYYY-MM-DD. *Ex.*:  2018-06-05 |
+| `-t`, `--track-source` | ***default Database -> {Database, ADSB, AIS}***<br/>Which track source to use. Paths and login credentials for all source types are stored in config files |
+| `-garb`, `--exportgarbage` | **default 0 -> {0, 1}***<br/>Whether to export garbage tracks (1) or not (0).  *Ex.*:  1 |
+
+Example executions:
+
+```bash
+$ python -u -W ignore _DENA/scripts/run_audible_transits.py -e production -u DENA -s FANG -y 2018 -g -1.5 -t0 2018-01-01 -tf 2024-08-20
+```
+
+```bash
+$ python -u -W ignore _DENA/scripts/run_audible_transits.py -e production -u DENA -s FANG -y 2018 -g -1.5 -t0 2018-01-01 -tf 2024-08-20 -t ADSB
+```
diff --git a/_DENA/requirements.txt b/_DENA/requirements.txt
@@ -1,3 +1,4 @@
+setuptools<=65.6.3
 affine==2.3.1
 attrs==22.1.0
 certifi==2022.9.14
@@ -13,7 +14,7 @@ fonttools==4.37.3
 future==0.18.2
 GDAL==3.4.3
 geographiclib==1.52
-geopandas==0.10.2
+geopandas==0.14.0
 geopy==2.2.0
 greenlet==1.1.3
 idna==3.4
@@ -24,14 +25,14 @@ kiwisolver==1.4.4
 matplotlib==3.5.2
 mercantile==1.2.1
 munch==2.5.0
-numpy==1.21.6
+numpy>=1.22
 packaging==21.3
 pandas
 Pillow==9.1.1
 psycopg2==2.9.3
 pygeos==0.13
 pyparsing==3.0.9
-pyproj==3.2.1
+pyproj==3.3.0
 python-dateutil==2.8.2
 pytz==2022.2.1
 rasterio==1.2.10
@@ -43,6 +44,7 @@ snuggs==1.4.7
 SQLAlchemy==1.4.39
 tqdm==4.64.0
 typing_extensions==4.3.0
+tzwhere==3.0.3
 urllib3==1.26.12
 xyzservices==2022.9.0
 zipp==3.8.1
diff --git a/_DENA/resource/helpers.py b/_DENA/resource/helpers.py
@@ -32,7 +32,7 @@ def get_deployment(unit: str, site: str, year: int, filename: str, elevation: bo
     unit : str
         Four letter park service unit code E.g. 'DENA'
     site : str
-        Deployment site character code. E.g. 'TRLA'
+        Deployment site character code. E.g. 'TRLA', '009'
     year : int
         Deployment year. YYYY
     filename : str
@@ -46,7 +46,13 @@ def get_deployment(unit: str, site: str, year: int, filename: str, elevation: bo
     mic : Microphone
         A Microphone object containing the mic deployment site metadata from the specific unit/site/year combination.
     """
+
+    print(unit, site, year)
     metadata = pd.read_csv(filename, delimiter='\t', encoding='ISO-8859-1')
+
+    # this rather cumbersome line assures that any sites styled as '009' or '099' are correctly formatted as strings
+    metadata.loc[metadata["code"].astype('str').str.len() <= 3, "code"] = metadata.loc[metadata["code"].astype('str').str.len() <= 3, "code"].apply(lambda s: str(s).zfill(3))
+
     site_meta = metadata.loc[(metadata['unit'] == unit) & (metadata['code'] == site) & (metadata['year'] == year)]
 
     # Microphone coordinates are stored in WGS84, epsg:4326
@@ -61,7 +67,8 @@ def get_deployment(unit: str, site: str, year: int, filename: str, elevation: bo
 
 
 def query_tracks(engine: 'Engine', start_date: str, end_date: str,
-                 mask: Optional[gpd.GeoDataFrame] = None) -> gpd.GeoDataFrame:
+                 mask: Optional[gpd.GeoDataFrame] = None, 
+                 mask_buffer_distance: Optional[int] = None) -> gpd.GeoDataFrame:
     """
     Query flight tracks from the FlightsDB for a specific date range and optional within a specific area.
 
@@ -87,12 +94,15 @@ def query_tracks(engine: 'Engine', start_date: str, end_date: str,
         if mask.crs.to_epsg() != 4326:  # If mask is not already in WGS84, project it.
             mask = mask.to_crs(epsg='4326')
         mask['dissolve_field'] = 1
+        if mask_buffer_distance:
+            ak_albers_mask = mask.to_crs(epsg=3338)
+            mask.geometry = ak_albers_mask.buffer(mask_buffer_distance).to_crs(epsg=4326)
         mask_wkt = mask.dissolve(by='dissolve_field').squeeze()['geometry'].wkt
         wheres.append(f"ST_Intersects(geom, ST_GeomFromText('{mask_wkt}', 4326))")
 
     query = f"""
         SELECT
-            f.id as flight_id,
+            f.flight_id as flight_id,
             fp.altitude_ft * 0.3048 as altitude_m,
             fp.ak_datetime,
             fp.geom, 
@@ -110,7 +120,9 @@ def query_tracks(engine: 'Engine', start_date: str, end_date: str,
 
 
 def query_adsb(adsb_path: str,  start_date: str, end_date: str,
-               mask: Optional[gpd.GeoDataFrame] = None) -> Union[Adsb, EarlyAdsb]:
+               mask: Optional[gpd.GeoDataFrame] = None,
+               mask_buffer_distance: Optional[int] = None,
+               exclude_early_ADSB: Optional[bool] = False) -> Union[Adsb, EarlyAdsb]:
     """
     Query flight tracks from ADSB files for a specific date range and optional within a specific area.
 
@@ -130,7 +142,8 @@ def query_adsb(adsb_path: str,  start_date: str, end_date: str,
     adsb : ADSB or EarlyADSB
         An ADSB or EarlyADSB object of flight track points.
     """
-    if int(start_date[:4]) <= 2019:  # ADSB file formats changed after 2019.
+
+    if (int(start_date[:4]) <= 2019) & (exclude_early_ADSB == False):  # ADSB file formats changed after 2019.
         adsb_files = glob.glob(os.path.join(adsb_path, "*.txt"))
         adsb = EarlyAdsb(adsb_files)
     else:
@@ -141,6 +154,11 @@ def query_adsb(adsb_path: str,  start_date: str, end_date: str,
     if mask is not None:
         if not mask.crs.to_epsg() == 4326:  # If mask is not already in WGS84, project it.
             mask = mask.to_crs(epsg='4326')
+        if mask_buffer_distance:
+            ak_albers_mask = mask.to_crs(epsg=3338)
+            mask.geometry = ak_albers_mask.buffer(mask_buffer_distance).to_crs(epsg=4326)
+        print(adsb.crs)
+        adsb.set_crs(epsg='4326', inplace=True)
         adsb = gpd.clip(adsb, mask)
 
     adsb = adsb.loc[~(adsb.geometry.is_empty)]

diff --git a/_DENA/scripts/generate_active_space.py b/_DENA/scripts/generate_active_space.py
@@ -10,9 +10,20 @@
 
 import geopandas as gpd
 import matplotlib.pyplot as plt
+import pandas as pd
 from shapely.geometry import Point
 from tqdm import tqdm
 
+# for some users relative imports are prohibitive
+# we simplify imports by adding three directories to the path environment variable
+import sys
+repo_dir = os.path.dirname(os.path.dirname(os.path.dirname(__file__)))
+config_dir = os.path.join(repo_dir, "_DENA")
+script_dir = os.path.join(repo_dir, "nps_active_space")
+sys.path.append(repo_dir)
+sys.path.append(config_dir)
+sys.path.append(script_dir)
+
 import iyore
 
 import _DENA.resource.config as cfg
@@ -75,7 +86,7 @@ def _run_active_space(outfile: str, omni_source: str, generator: ActiveSpaceGene
         if active_spaces is None:
             active_spaces = active_space
         else:
-            active_spaces = active_spaces.append(active_space, ignore_index=True)
+            active_spaces = pd.concat([active_spaces, active_space], ignore_index=True)
 
     # Combine the active spaces from each heading into a single active space and write it to a geojson file.
     dissolved_active_space = active_spaces.dissolve()
@@ -126,16 +137,19 @@ def _run_active_space(outfile: str, omni_source: str, generator: ActiveSpaceGene
     if len(annotation_files) == 0:
         logger.info(f"No track annotations found for {args.unit}{args.site}{args.year}. Exiting...")
         exit(-1)
-    annotations = Annotations()
+    annotations = []
     for file in tqdm(annotation_files, desc='Loading annotation files', unit='files', colour='white'):
-        annotations = annotations.append(Annotations(file, only_valid=True), ignore_index=True)
+        annotations.append(Annotations(file, only_valid=True))
+    annotations = pd.concat(annotations)
 
     # If the user does not pass an altitude, calculate the average altitude of all valid tracks. Extract the altitudes
     #  from each linestring to get the average height (in meters) of audible flight segments.
     if not args.altitude:
         logger.info("Calculating average altitude (in meters)...")
         annotations['z_vals'] = (annotations['geometry'].apply(lambda geom: mean([coords[-1] for coords in geom.coords])))
-        altitude_ = int(mean(annotations[annotations.audible == True].z_vals.tolist()))
+        altitudes_ = annotations[annotations.audible == True].z_vals
+        altitudes_ = altitudes_[(altitudes_ > 0)&(altitudes_ <= 10000)] # NOTE removing the negative values could be severe for some ADS-B loggers
+        altitude_ = int(mean(altitudes_.tolist()))
         logger.info(f"Average altitude is: {altitude_}m")
     else:
         altitude_ = args.altitude
@@ -205,7 +219,7 @@ def _run_active_space(outfile: str, omni_source: str, generator: ActiveSpaceGene
         fbeta, precision, recall, n_tot = compute_fbeta(valid_points, res, args.beta)
         precisions.append(precision)
         recalls.append(recall)
-        print(f"omni: {omni} --> fbeta: {fbeta} precision: {precision} recall: {recall}")
+        print(f"omni: {omni} --> fbeta: {fbeta:0.3f} precision: {precision:0.3f} recall: {recall:0.3f}")
         if fbeta > max_fbeta:
             max_fbeta = fbeta
             best_omni = omni