static route missing from config, not configured

[mksheera508]

Description

Static route is missing, neither present in configuration nor any error reported.

Few queries:

1. what is the use-case of copying running to condidate on vty_close ?
2. DataStore Lock is per session. Is there any protection when two sessions try to modify? or such a scenario is unexpected?

Version

stable/9.1

How to reproduce

Issue reproduced in two scenarios.

1. configure vxlan gpe tunnel between two nodes, configure static route via tunnel
   and after a wait of 10 secs, remove route and tunnel,
   Repeat this in a script, will hit issue.
   Static route will be missing from route table and from configuration as well.
2. configure a vxlan gpe tunnel.
   add static route with this tunnel as nexthop and then remove the tunnel.
   For next iteration, add tunnel, configure two static routes and the remove the tunnel
   Repeat this in a script, incrementing the no of static routes configured in every iteration.
   Ran this with 255 static routes. script fails in between itself, one of the static routes missed to configured.

In our configurations, we config using "sudo vtysh -c/-f" option, and after every command we give "sudo vtysh -w" for saving the config.

Expected behavior

Static route to be present in running config as well as route table.

Actual behavior

Static route missing from config and route table. No error reported as well.

Additional context

Able to debug this issue to some context. Below are observations

1. in issue case, its seen that it failed in mgmt_txn_prepare_config with reason " No changes found to be committed!"

Further debugging, we see below logs, while adding a route
While a config us running with session-id is 3688, another session with id 3689 is doing "mgmt_ds_replace_dst_with_src_ds: ERROR: Replacing candidate with running"
which is erasing the config candidate has and hence that particular static route config is missing.

2024/09/05 20:54:49 MGMTD: [R8JKD-3KS69] FE-ADAPTER: mgmt_fe_adapter_handle_msg: ERROR: Got SETCFG_REQ (1 Xpaths, Implicit:F) on DS:candidate for session-id 3688 from 'vty-mgmtd-75657'
2024/09/05 20:54:49 ZEBRA: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/09/05 20:54:49 ZEBRA: [G6NKK-8C6DV] end_config: VTY:0x555eff307950, pending SET-CFG: 0
2024/09/05 20:54:49 MGMTD: [R8JKD-3KS69] FE-ADAPTER: mgmt_fe_adapter_handle_msg: ERROR: Got SETCFG_REQ (6 Xpaths, Implicit:F) on DS:candidate for session-id 3688 from 'vty-mgmtd-75657'
2024/09/05 20:54:49 STATIC: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/09/05 20:54:49 STATIC: [G6NKK-8C6DV] end_config: VTY:0x55f56ad6b6d0, pending SET-CFG: 0
2024/09/05 20:54:49 MGMTD: [VTVCM-Y2NW3] Configuration Read in Took: 00:00:00
2024/09/05 20:54:49 MGMTD: [G6NKK-8C6DV] end_config: VTY:0x5605fe17b420, pending SET-CFG: 2
2024/09/05 20:54:49 MGMTD: [QG9ZP-YR9QM] FE-CLIENT: mgmt_fe_send_commitcfg_req: ERROR: Sending COMMIT_CONFIG_REQ message for Src-DS:candidate, Dst-DS:running session-id 3688
2024/09/05 20:54:49 MGMTD: [V7B7X-7519Z] FE-ADAPTER: mgmt_fe_session_cfg_txn_cleanup: ERROR: mgmt_fe_session_cfg_txn_cleanup: Replacing candidate with runn session-id: 3689 from vty-mgmtd-75657
2024/09/05 20:54:49 MGMTD: [Z3EBM-B55CT] mgmt_ds_replace_dst_with_src_ds: ERROR: Replacing candidate with running
2024/09/05 20:54:49 MGMTD: [Z3EBM-B55CT] mgmt_ds_replace_dst_with_src_ds: ERROR: Replacing running with candidate
2024/09/05 20:54:49 MGMTD: [M6PEZ-DNXXW] mgmt_txn_prepare_config: ERROR: txn-id: 210 session-id: 3688- No changes found to be committed!
2024/09/05 20:54:49 MGMTD: [V7B7X-7519Z] FE-ADAPTER: mgmt_fe_session_cfg_txn_cleanup: ERROR: mgmt_fe_session_cfg_txn_cleanup: Replacing candidate with runn session-id: 3688 from vty-mgmtd-75657
2024/09/05 20:54:49 MGMTD: [Z3EBM-B55CT] mgmt_ds_replace_dst_with_src_ds: ERROR: Replacing candidate with running
2024/09/05 20:54:49 MGMTD: [V7B7X-7519Z] FE-ADAPTER: mgmt_fe_session_cfg_txn_cleanup: ERROR: mgmt_fe_session_cfg_txn_cleanup: Replacing candidate with runn session-id: 3688 from vty-mgmtd-75657
2024/09/05 20:54:49 MGMTD: [Z3EBM-B55CT] mgmt_ds_replace_dst_with_src_ds: ERROR: Replacing candidate with running

Checklist

• I have searched the open issues for this bug.
• I have not included sensitive information in this report.