Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove f000 from atmos rocoto tasks for replay cases #2778

Merged
merged 25 commits into from
Jul 29, 2024
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
9e6206c
Update linters.yaml
EricSinsky-NOAA Jun 3, 2024
4e13fbe
Revert changes in linters.yaml
EricSinsky-NOAA Jun 4, 2024
1eb0c26
Merge branch 'NOAA-EMC:develop' into develop
EricSinsky-NOAA Jun 7, 2024
720276b
Merge branch 'NOAA-EMC:develop' into develop
EricSinsky-NOAA Jun 12, 2024
219adb2
Merge branch 'NOAA-EMC:develop' into develop
EricSinsky-NOAA Jun 20, 2024
37e2a6f
Merge branch 'NOAA-EMC:develop' into develop
EricSinsky-NOAA Jun 24, 2024
b7ba980
Merge branch 'NOAA-EMC:develop' into develop
EricSinsky-NOAA Jun 25, 2024
a3efe66
Merge branch 'NOAA-EMC:develop' into develop
EricSinsky-NOAA Jun 26, 2024
12e1ec1
Merge branch 'NOAA-EMC:develop' into develop
EricSinsky-NOAA Jul 8, 2024
e114de5
Merge branch 'NOAA-EMC:develop' into develop
EricSinsky-NOAA Jul 16, 2024
e09f27d
Merge branch 'NOAA-EMC:develop' into develop
EricSinsky-NOAA Jul 18, 2024
3fdcdae
Use FHMIN_GFS for atmos component in tasks.py
EricSinsky-NOAA Jul 18, 2024
235ee26
Use FHMIN_GFS in FV3_predet
EricSinsky-NOAA Jul 18, 2024
61b14e9
Merge remote-tracking branch 'EMC/develop' into feature/issue2771
EricSinsky-NOAA Jul 23, 2024
b8fec2a
Revert "Use FHMIN_GFS in FV3_predet"
EricSinsky-NOAA Jul 23, 2024
c8b49e2
Use offset_start_hour as atmos fhmin
EricSinsky-NOAA Jul 23, 2024
6e132ad
Simplify variable definitions
EricSinsky-NOAA Jul 23, 2024
82dd2b6
Remove 0 from fhrs in gefs/gfs tasks for ocn/ice
EricSinsky-NOAA Jul 23, 2024
c7fe7e7
Remove f000 task for atmos_prod when replaying
EricSinsky-NOAA Jul 24, 2024
5d0a31c
Remove f000 task for atmos_ensstat when replaying
EricSinsky-NOAA Jul 24, 2024
9e004a5
Add Neil Barton's bugfix in exglobal_stage_ic
EricSinsky-NOAA Jul 24, 2024
1f6c580
Remove deep copy according to reviewer's comment
EricSinsky-NOAA Jul 24, 2024
fcac4e9
Merge remote-tracking branch 'EMC/develop' into feature/issue2771
EricSinsky-NOAA Jul 24, 2024
c3d3e9c
Merge remote-tracking branch 'EMC/develop' into feature/issue2771
EricSinsky-NOAA Jul 25, 2024
09b60c1
Merge branch 'develop' into feature/issue2771
aerorahul Jul 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions scripts/exglobal_stage_ic.sh
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ for MEMDIR in "${MEMDIR_ARRAY[@]}"; do

# Atmosphere Perturbation Files (usually used with replay ICS)
# Extra zero on MEMDIR ensure we have a number even if the string is empty
if (( 0${MEMDIR:3} > 0 )) && [[ "${REPLAY_ICS:-NO}" == "YES" ]]; then
if (( $((10#0${MEMDIR:3})) > 0 )) && [[ "${REPLAY_ICS:-NO}" == "YES" ]]; then
YMD=${PDY} HH=${cyc} declare_from_tmpl COM_ATMOS_ANALYSIS:COM_ATMOS_ANALYSIS_TMPL
[[ ! -d "${COM_ATMOS_ANALYSIS}" ]] && mkdir -p "${COM_ATMOS_ANALYSIS}"
src="${BASE_CPLIC}/${CPL_ATMIC:-}/${PDY}${cyc}/${MEMDIR}/atmos/${DTG_PREFIX}.fv3_perturbation.nc"
Expand Down Expand Up @@ -130,7 +130,7 @@ for MEMDIR in "${MEMDIR_ARRAY[@]}"; do

# Ocean Perturbation Files
# Extra zero on MEMDIR ensure we have a number even if the string is empty
if (( 0${MEMDIR:3} > 0 )) && [[ "${REPLAY_ICS:-NO}" == "YES" ]]; then
if (( $((10#0${MEMDIR:3})) > 0 )) && [[ "${REPLAY_ICS:-NO}" == "YES" ]]; then
YMD=${PDY} HH=${cyc} declare_from_tmpl COM_OCEAN_ANALYSIS:COM_OCEAN_ANALYSIS_TMPL
[[ ! -d "${COM_OCEAN_ANALYSIS}" ]] && mkdir -p "${COM_OCEAN_ANALYSIS}"
src="${BASE_CPLIC}/${CPL_OCNIC:-}/${PDY}${cyc}/${MEMDIR}/ocean/${DTG_PREFIX}.mom6_perturbation.nc"
Expand Down
18 changes: 18 additions & 0 deletions workflow/rocoto/gefs_tasks.py
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,17 @@ def _atmosoceaniceprod(self, component: str):
'maxtries': '&MAXTRIES;'}

fhrs = self._get_forecast_hours('gefs', self._configs[config], component)

# when replaying, atmos component does not have fhr 0, therefore remove 0 from fhrs
local_config = self._configs[config].deepcopy()
is_replay = local_config['REPLAY_ICS']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to make a deep copy here. We're not changing the configs, just reading them. We can use the original for that.

Suggested change
local_config = self._configs[config].deepcopy()
is_replay = local_config['REPLAY_ICS']
is_replay = self._configs[config]['REPLAY_ICS']

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Deep copy has been removed here and for atmos_ensstat.

if is_replay and component in ['atmos'] and 0 in fhrs:
fhrs.remove(0)

# ocean/ice components do not have fhr 0 as they are averaged output
if component in ['ocean', 'ice'] and 0 in fhrs:
fhrs.remove(0)

fhr_var_dict = {'fhr': ' '.join([f"{fhr:03d}" for fhr in fhrs])}

fhr_metatask_dict = {'task_name': f'{component}_prod_#member#',
Expand Down Expand Up @@ -314,6 +325,13 @@ def atmos_ensstat(self):
'maxtries': '&MAXTRIES;'}

fhrs = self._get_forecast_hours('gefs', self._configs['atmos_ensstat'])

# when replaying, atmos component does not have fhr 0, therefore remove 0 from fhrs
local_config = self._configs['atmos_ensstat'].deepcopy()
is_replay = local_config['REPLAY_ICS']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here

Suggested change
local_config = self._configs['atmos_ensstat'].deepcopy()
is_replay = local_config['REPLAY_ICS']
is_replay = self._configs['atmos_ensstat']['REPLAY_ICS']

if is_replay and 0 in fhrs:
fhrs.remove(0)

fhr_var_dict = {'fhr': ' '.join([f"{fhr:03d}" for fhr in fhrs])}

fhr_metatask_dict = {'task_name': f'atmos_ensstat',
Expand Down
5 changes: 5 additions & 0 deletions workflow/rocoto/gfs_tasks.py
Original file line number Diff line number Diff line change
Expand Up @@ -1123,6 +1123,11 @@ def _atmosoceaniceprod(self, component: str):
}

fhrs = self._get_forecast_hours(self.cdump, self._configs[config], component)

# ocean/ice components do not have fhr 0 as they are averaged output
if component in ['ocean', 'ice'] and 0 in fhrs:
fhrs.remove(0)

fhr_var_dict = {'fhr': ' '.join([f"{fhr:03d}" for fhr in fhrs])}
metatask_dict = {'task_name': f'{self.cdump}{component}_prod',
'task_dict': task_dict,
Expand Down
4 changes: 0 additions & 4 deletions workflow/rocoto/tasks.py
Original file line number Diff line number Diff line change
Expand Up @@ -158,10 +158,6 @@ def _get_forecast_hours(cdump, config, component='atmos') -> List[str]:
fhrs_hf = range(fhmin, fhmax_hf + fhout_hf, fhout_hf)
fhrs = list(fhrs_hf) + list(range(fhrs_hf[-1] + fhout, fhmax + fhout, fhout))

# ocean/ice components do not have fhr 0 as they are averaged output
if component in ['ocean', 'ice'] and 0 in fhrs:
fhrs.remove(0)

return fhrs

def get_resource(self, task_name):
Expand Down
Loading