weights not being updated when using misc_dynamic

[georgen-data-axle]

Hi Everyone.

I have been trying to upgrade to ipvsadm v1.31 2019/12/24 (compiled with popt and IPVS v1.2.1). Long story short, I have a MISC_CHECK script which returns either a 2 if the checker fails or 3 upon success. In the end, I expect the weight to be either set to 0 or 1 without notifications of up/down. No matter what I return from the checker exit status, the weight is not being updated.

`
these are the global defaults I have set

global_defs
{
vrrp_version 3
enable_script_security
}

here's basically how I have the real_servers checks configured.

real_server xx.xx.xx.xx xxxxx {
weight 1
MISC_CHECK {
misc_path "/etc/keepalived/myscript myvip myrealhostname xx.xx.xx.xx xxxxx"
misc_dynamic
}
}
`

I believe this may be related to the new feature I have enabled, enable_script_security. It seems maybe the script has to run as a privileged user to adjust weights however I cannot find any documentation which refers to when escalated privileges are required.

TIA,
George.

[pqarmitage]

You need to check the keepalived log entries in the system logs. That will tell you if enable_script_security is causing you a problem.

[georgen-data-axle]

Thanks for your response pqarmitage. All of the scripts appear to be successful in the logs. I am convinced now it's definitely not related to security. I ended up changing the checker to be this,

#!/bin/bash
exit 5

and the weights updated to 3 (my expectation). However, when I change it to ...

#!/bin/bash
exit 2

the weights will absolutely not change. In this situation I expect the weight to become 0. It definitely works in the older version I had been using [RHEL7 distro, ipvsadm v1.27 2008/5/15 (compiled with popt and IPVS v1.2.1)]

In the past I did it this way so I wouldn't get blasted with and email from servers x services every time the keepalived service was restarted. Additionally, my checker script has a neat way to offline servers using the misc_dynamic without requiring elevated privileges. I would love for it to work, but if you have any suggestions, I am all ears.

TIA,
George

[georgen-data-axle]

Hi pqarmitage,

I believe this to be a bug, please let me know if you agree. If so I'll file a bug. Also please accept my apologies, not sure why I provided the ivpsadm version info. The following is the version it works as I expect it to...

Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2

Copyright(C) 2001-2017 Alexandre Cassen, <[email protected]>

Build options:  PIPE2 LIBNL3 RTA_ENCAP RTA_EXPIRES RTA_PREF RTA_VIA FRA_OIFNAME FRA_SUPPRESS_PREFIXLEN FRA_TUN_ID RTAX_CC_ALGO RTAX_QUICKACK LIBIPTC LIBIPSET_DYNAMIC LVS LIBIPVS_NETLINK VRRP VRRP_AUTH VRRP_VMAC SOCK_NONBLOCK SOCK_CLOEXEC FIB_ROUTING INET6_ADDR_GEN_MODE SNMP_V3_FOR_V2 SNMP SNMP_KEEPALIVED SNMP_CHECKER SNMP_RFC SNMP_RFCV2 SNMP_RFCV3 SO_MARK

This is the version I believe it's broken in...

Keepalived v2.1.5 (07/13,2020)

Copyright(C) 2001-2020 Alexandre Cassen, <[email protected]>

Built with kernel headers for Linux 4.18.0
Running on Linux 4.18.0-425.3.1.el8.x86_64 #1 SMP Fri Sep 30 11:45:06 EDT 2022

configure options: --build=x86_64-redhat-linux-gnu --host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin --sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --disable-libiptc --disable-ipset --enable-snmp --enable-snmp-rfc --enable-sha1 --with-init=systemd build_alias=x86_64-redhat-linux-gnu host_alias=x86_64-redhat-linux-gnu PKG_CONFIG_PATH=:/usr/lib64/pkgconfig:/usr/share/pkgconfig CFLAGS=-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection LDFLAGS=-Wl,-z,relro -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld

Config options:  LVS VRRP VRRP_AUTH OLD_CHKSUM_COMPAT FIB_ROUTING SNMP_V3_FOR_V2 SNMP_VRRP SNMP_CHECKER SNMP_RFCV2 SNMP_RFCV3

System options:  PIPE2 SIGNALFD INOTIFY_INIT1 VSYSLOG EPOLL_CREATE1 IPV4_DEVCONF IPV6_ADVANCED_API LIBNL3 RTA_ENCAP RTA_EXPIRES RTA_NEWDST RTA_PREF FRA_SUPPRESS_PREFIXLEN FRA_SUPPRESS_IFGROUP FRA_TUN_ID RTAX_CC_ALGO RTAX_QUICKACK RTEXT_FILTER_SKIP_STATS FRA_L3MDEV FRA_UID_RANGE RTAX_FASTOPEN_NO_COOKIE RTA_VIA FRA_OIFNAME FRA_PROTOCOL FRA_IP_PROTO FRA_SPORT_RANGE FRA_DPORT_RANGE RTA_TTL_PROPAGATE IFA_FLAGS IP_MULTICAST_ALL LWTUNNEL_ENCAP_MPLS LWTUNNEL_ENCAP_ILA NET_LINUX_IF_H_COLLISION LIBIPTC_LINUX_NET_IF_H_COLLISION LIBIPVS_NETLINK IPVS_DEST_ATTR_ADDR_FAMILY IPVS_SYNCD_ATTRIBUTES IPVS_64BIT_STATS VRRP_VMAC VRRP_IPVLAN IFLA_LINK_NETNSID CN_PROC SOCK_NONBLOCK SOCK_CLOEXEC O_PATH GLOB_BRACE INET6_ADDR_GEN_MODE VRF SO_MARK SCHED_RESET_ON_FORK

[pqarmitage]

Commit 291801d introduced this problem, and the first version this was in was v2.0.16. The problem was correct with commit e2b4d10 and the first version that was in was v2.2.0.

Now that you have identified that you are running v2.1.5 it is clear that the version you are running has the problem where an exit code of 2 does not work properly.

Your solution is to upgrade to a recent version of keepalived (e.g. v2.3.1) or to backport commit e2b4d10 to v2.1.5.

[georgen-data-axle]

Just to report back... we rolled our own rpm from v2.3.1 and it works as expected now. Thanks for your guidance pqarmitage.