model blown-up error

[met-sree]

Dear Sir,
I am trying to run the COAWST model with coupled ROMS and WRF. However, I am experiencing a blow-up within the initial hours. I have attached the output files and the namelist file at the link below.

1. I checked on Google and found that this issue could be due to NaN values in the initial forcing files. However, I have checked all the variables, and there are no NaN values.
2. For your reference, I have attached plots of some variables (ubar, vbar, top-layer u, v, temperature, and salinity) from the history file. Do you notice any anomalies?
3. Interestingly, when using the Red Key Word for boundary conditions, the model runs for up to 5 days before blowing up (I didn't attach files related to this).
   Please provide any solutions or suggestions to overcome this issue.

Thank you.
Sree

https://drive.google.com/drive/folders/14cTGTv0xPpp6sXrAP4gvww0DjttDWQSq?usp=sharing

[jcwarner-usgs]

In general, when ROMS blows up, you need to look at the ocean_rst.nc (or whatever you call it) file.
for your case, there was a large velocity that developed.
" REASON: MaxSpeed = 2.2E+01"
So look at the nc file, i use ncview. Look around to find where the value got large. It wrote the blow up time step into time record 2 (see below).
Was it at the open boundary? if so, then look at your BC's there.
Was it along the coastline? if so, then look at the bathy.
etc.
You will need to do some investigative digging.
Since this happens after only 99 steps, maybe try saving the output more frequently to see what is happening.

-j

"Timing for main: time 2023-06-07_00:43:30 on domain 1: 1.22791 elapsed seconds
Timing for main: time 2023-06-07_00:43:30 on domain 1: 1.22791 elapsed seconds
99 2023-06-07 00:49:30.00 4.568744E-03 1.819105E+04 1.819106E+04 1.124104E+16
(046,003,25) 8.791737E-03 8.467620E-03 8.439933E-01 2.151231E+01
DIAG speed trouble 21.5123142373066
Found Error: 01 Line: 355 Source: ROMS/Nonlinear/main3d.F
Found Error: 01 Line: 330 Source: ROMS/Drivers/nl_roms.h, ROMS_run

Blowing-up: Saving latest model state into RESTART file
REASON: MaxSpeed = 2.2E+01

WRT_RST_NF90 - wrote re-start fields (Index=1,2) in record = 2"

[met-sree]

Dear Sir,
I plotted the variables from the RST file at the second time step, and large values (marked with stars) are observed on the western side (at the estuary region), which is considered an open boundary (Varibale_plot).
For your reference, I also plotted the initial boundary conditions at intial time along the western boundary (west_plot.png).

Both images are attached in the below link.

What would be the best solution to resolve this issue?

Thank you,
Sree
https://drive.google.com/drive/folders/14cTGTv0xPpp6sXrAP4gvww0DjttDWQSq?usp=sharing

[jcwarner-usgs]

there is something strange going on along the western edge, near the north.
what do the boundary file values look like in there? what did zeta look like?
does WRF cover that full area?
run the model but save the his data more often.
you will need to keep digging. something is going on up there.

[met-sree]

Dear Sir,
As per your suggestions, I set the restart interval (NHIS) to a lower value (~60 sec). Yes, the WRF domain is larger than the ROMS domain.

Due to instability at the western boundary, particularly on the northern side near the Gulf region. To address this, I attempted two different setups:
1. Including the Gulf region (modifid folder)
2. Excluding the Gulf region (modified2 folder)
But, in both cases, the model still blew up.

I observed fluctuations/gradients in the zeta values at the open boundries (south and west) in both experiments (as RST file). Could this cause any issues? I used the MATLAB script provided on the COAWST site to create this file (like the attached figure).

For your reference, I have attached the output files, *.in files, figures, and initial and boundary (surface and tidal) files for both experiments in the link below.

Please review the files and provide your valuable suggestions.

Thank you,
Sree

https://drive.google.com/drive/folders/14cTGTv0xPpp6sXrAP4gvww0DjttDWQSq?usp=sharing