Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change ESSL colorisation approach #3021

Merged
merged 6 commits into from
Jan 20, 2025
Merged

Change ESSL colorisation approach #3021

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
982a31c
Change ESSL colorisation approach
gerritholl Dec 18, 2024
b3e3b31
name should be standard_name
gerritholl Jan 7, 2025
d86ee60
change standard_name to name in enhancement
gerritholl Jan 8, 2025
06abb9a
Merge branch 'main' into new-essl-colorize
gerritholl Jan 17, 2025
449a084
Remove unused outdir argument
gerritholl Jan 17, 2025
cc850ab
Add imagery tests for colorized low level moisture
gerritholl Jan 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 0 additions & 110 deletions satpy/enhancements/atmosphere.py
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions satpy/etc/composites/visir.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -626,8 +626,8 @@ composites:
is still under development and may be subject to change.
compositor: !!python/name:satpy.composites.RatioCompositor
prerequisites:
- wavelength: 0.86
- wavelength: 0.905
- wavelength: 0.86
standard_name: essl_colorized_low_level_moisture

day_essl_colorized_low_level_moisture:
Expand All @@ -638,7 +638,7 @@ composites:
day_night: day_only
prerequisites:
- name: essl_colorized_low_level_moisture
standard_name: day_essl_colorized_low_level_moisture
standard_name: image_ready

rocket_plume_day:
description: >
Expand Down
114 changes: 108 additions & 6 deletions satpy/etc/enhancements/generic.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1243,12 +1243,114 @@ enhancements:
essl_colorized_low_level_moisture:
name: essl_colorized_low_level_moisture
operations:
- name: essl_moisture
method: !!python/name:satpy.enhancements.atmosphere.essl_moisture

day_essl_colorized_low_level_moisture:
standard_name: day_essl_colorized_low_level_moisture
operations: []
- name: colorize
method: !!python/name:satpy.enhancements.colorize
kwargs:
palettes:
- min_value: 0.625
max_value: 0.91
values:
- 0.6250
- 0.6290
- 0.6331
- 0.6372
- 0.6414
- 0.6456
- 0.6499
- 0.6542
- 0.6586
- 0.6631
- 0.6676
- 0.6722
- 0.6768
- 0.6815
- 0.6863
- 0.6911
- 0.6960
- 0.7010
- 0.7061
- 0.7112
- 0.7164
- 0.7216
- 0.7270
- 0.7324
- 0.7380
- 0.7436
- 0.7492
- 0.7550
- 0.7609
- 0.7668
- 0.7729
- 0.7790
- 0.7853
- 0.7916
- 0.7980
- 0.8046
- 0.8113
- 0.8180
- 0.8249
- 0.8319
- 0.8390
- 0.8463
- 0.8537
- 0.8612
- 0.8688
- 0.8766
- 0.8845
- 0.8925
- 0.9007
- 0.9091
colors:
- [63, 0, 47]
- [58, 0, 50]
- [53, 0, 52]
- [48, 0, 54]
- [42, 0, 56]
- [37, 0, 58]
- [32, 0, 59]
- [27, 5, 60]
- [22, 11, 61]
- [16, 17, 62]
- [11, 23, 63]
- [6, 28, 63]
- [1, 34, 63]
- [0, 40, 63]
- [0, 46, 63]
- [0, 52, 62]
- [0, 58, 62]
- [0, 64, 61]
- [0, 70, 60]
- [0, 76, 58]
- [0, 82, 57]
- [0, 88, 55]
- [0, 94, 53]
- [0, 100, 51]
- [3, 106, 49]
- [17, 112, 46]
- [31, 118, 43]
- [44, 124, 40]
- [58, 130, 37]
- [72, 136, 35]
- [86, 141, 42]
- [100, 147, 50]
- [114, 153, 58]
- [128, 159, 66]
- [142, 165, 74]
- [156, 171, 81]
- [169, 177, 89]
- [183, 183, 97]
- [197, 189, 105]
- [211, 195, 113]
- [225, 201, 120]
- [239, 207, 128]
- [253, 213, 136]
- [255, 219, 144]
- [255, 225, 152]
- [255, 231, 160]
- [255, 237, 167]
- [255, 243, 175]
- [255, 249, 183]
- [255, 255, 191]

rocket_plume:
standard_name: rocket_plume
Expand Down
17 changes: 9 additions & 8 deletions satpy/tests/behave/features/image_comparison.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,15 @@ Feature: Image Comparison

Scenario Outline: Compare generated image with reference image
Given I have a <composite> reference image file from <satellite> resampled to <area>
When I generate a new <composite> image file from <satellite> with <reader> for <area> with clipping <clip>
When I generate a new <composite> image file from <satellite> case <case> with <reader> for <area> with clipping <clip>
Then the generated image should be the same as the reference image

Examples:
|satellite |composite | reader | area | clip |
|Meteosat-12 | cloudtop | fci_l1c_nc | sve | True |
|Meteosat-12 | night_microphysics | fci_l1c_nc | sve | True |
|GOES17 |airmass | abi_l1b | null | null |
|GOES16 |airmass | abi_l1b | null | null |
|GOES16 |ash | abi_l1b | null | null |
|GOES17 |ash | abi_l1b | null | null |
|satellite | case | composite | reader | area | clip |
|Meteosat-12 | scan_night | cloudtop | fci_l1c_nc | sve | True |
|Meteosat-12 | scan_night | night_microphysics | fci_l1c_nc | sve | True |
|Meteosat-12 | mali_day | essl_colorized_low_level_moisture | fci_l1c_nc | mali | False |
|GOES17 | americas_night | airmass | abi_l1b | null | null |
|GOES16 | americas_night | airmass | abi_l1b | null | null |
|GOES16 | americas_night | ash | abi_l1b | null | null |
|GOES17 | americas_night | ash | abi_l1b | null | null |
7 changes: 4 additions & 3 deletions satpy/tests/behave/features/steps/image_comparison.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,16 +63,17 @@
context.area = area


@when("I generate a new {composite} image file from {satellite} with {reader} for {area} with clipping {clip}")
def step_when_generate_image(context, composite, satellite, reader, area, clip):
@when("I generate a new {composite} image file from {satellite} case {case} "
"with {reader} for {area} with clipping {clip}")
def step_when_generate_image(context, composite, satellite, case, reader, area, clip):
"""Generate test images."""
os.environ["OMP_NUM_THREADS"] = os.environ["MKL_NUM_THREADS"] = "2"
os.environ["PYTROLL_CHUNK_SIZE"] = "1024"
warnings.simplefilter("ignore")
dask.config.set(scheduler="threads", num_workers=4)

# Get the list of satellite files to open
filenames = glob(f"{ext_data_path}/satellite_data/{satellite}/*.nc")
filenames = glob(f"{ext_data_path}/satellite_data/{satellite}/{case}/*.nc")

Check notice on line 76 in satpy/tests/behave/features/steps/image_comparison.py

View check run for this annotation

CodeScene Delta Analysis / CodeScene Cloud Delta Analysis (main)

ℹ Getting worse: Excess Number of Function Arguments 

step_when_generate_image increases from 6 to 7 arguments, threshold = 4. This function has too many arguments, indicating a lack of encapsulation. Avoid adding more arguments.

reader_kwargs = {}
if clip != "null":
Expand Down
61 changes: 0 additions & 61 deletions satpy/tests/enhancement_tests/test_atmosphere.py
View file
Open in desktop

This file was deleted.

15 changes: 7 additions & 8 deletions utils/create_reference.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright (c) 2024 Satpy developers
# Copyright (c) 2024-2025 Satpy developers
#
# This file is part of satpy.
#
Expand Down Expand Up @@ -42,7 +42,8 @@ def generate_images(props):
props (namespace): Object with attributes corresponding to command line
arguments as defined by :func:get_parser.
"""
filenames = (props.basedir / "satellite_data" / props.satellite).glob("*")
filenames = (props.basedir / "satellite_data" / props.satellite /
props.case).glob("*")

scn = Scene(reader=props.reader, filenames=filenames)

Expand Down Expand Up @@ -75,20 +76,18 @@ def get_parser():
"reader", action="store", type=str,
help="Reader name.")

parser.add_argument(
"case", help="case to generate", type=str)

parser.add_argument(
"-b", "--basedir", action="store", type=pathlib.Path,
default=pathlib.Path("."),
help="Base directory for reference data. "
"This must contain a subdirectories satellite_data and "
"reference_images. The directory satellite_data must contain "
"input data in a subdirectory for the satellite. Output images "
"input data in a subdirectory for the satellite and case. Output images "
"will be written to the subdirectory reference_images.")

parser.add_argument(
"-o", "--outdir", action="store", type=pathlib.Path,
default=pathlib.Path("."),
help="Directory where to write resulting images.")

parser.add_argument(
"-c", "--composites", nargs="+", help="composites to generate",
type=str, default=["ash", "airmass"])
Expand Down
Loading