[systemd/systemd-sonic-generator]: Systemd midplane network service o…

…f Smart Switch (sonic-net#18178)

Why I did it
Add systemd network service of Smart Switch for configuring the midplane network.

How I did it

Add related systemd services and networkd configuration
According to paltform.json, the systemd-sonic-generator will install required services.
The changes are based on the HLD: sonic-net/SONiC#1534 and sonic-net/SONiC#1586

How to verify it
Check Azp that no failure to the existing testcases
Check NPU scenario:
2.1 Start a KVM and add the NPU platform.json as following
echo '{"DPUS":{"dpu0":{}, "dpu1":{}}}' | sudo tee /usr/share/sonic/device/x86_64-kvm_x86_64-r0/platform.json
2.2 You should get two specific database containers for DPU

admin@vlab-01:~$ docker ps -a
CONTAINER ID   IMAGE                                COMMAND                  CREATED          STATUS         PORTS     NAMES
4455f7ab611c   docker-database:latest               "/usr/local/bin/dock…"   5 minutes ago    Up 5 minutes             databasedpu0
8e983b5beb27   docker-database:latest               "/usr/local/bin/dock…"   5 minutes ago    Up 5 minutes             databasedpu1
2.3 Check the expected service midplane-network-npu.service is enabled but the midplane-network-dpu.service is disabled

admin@vlab-01:~$ sudo systemctl list-unit-files '*midplane*'
UNIT FILE                    STATE           PRESET
midplane-network-dpu.service disabled        enabled
midplane-network-npu.service enabled-runtime  enabled
2.4 Check the bridge-midplane has been created.

admin@vlab-01:~$ sudo brctl show
bridge name     bridge id               STP enabled     interfaces
...
bridge-midplane         8000.76c5a51a18f6       no              dummy-midplane
2.5 If we create two mock dpu interfaces, they should be added into the bridge-midplane automatically

admin@vlab-01:~$ sudo ip link add dpu0 type veth peer dpu1
admin@vlab-01:~$ sudo brctl show
bridge name     bridge id               STP enabled     interfaces
...
bridge-midplane         8000.76c5a51a18f6       no              dpu0 dpu1 dummy-midplane
Check DPU scenario:
3.1 Start a KVM and add the DPU platform.json as following
echo '{"DPU":{}}' | sudo tee /usr/share/sonic/device/x86_64-kvm_x86_64-r0/platform.json
3.2 Check the expected service midplane-network-dpu.service is enabled but the midplane-network-npu.service is disabled

admin@vlab-01:~$ sudo systemctl list-unit-files '*midplane*'
UNIT FILE                    STATE           PRESET
midplane-network-dpu.service disabled        enabled
midplane-network-npu.service enabled-runtime enabled
3.3 The database service should be started after(waiting for) midplane-network-dpu.service with a 10mins delay

admin@vlab-01:~$ uptime
 16:08:21 up 29 min,  1 user,  load average: 0.01, 0.08, 0.12
admin@vlab-01:~$ docker ps -a
CONTAINER ID   IMAGE                                COMMAND                  CREATED          STATUS          PORTS     NAMES
...
ae5195283cd6   docker-database:latest               "/usr/local/bin/dock…"   35 minutes ago   Up 18 minutes             database
29min-18min≈10mins (Because in the KVM, we don't have an exact eth0-midplane interface and its DHCP provider)


Signed-off-by: Ze Gan <[email protected]>

files/build_templates/sonic_debian_extension.j2

@@ -47,6 +47,7 @@ DOCKER_CTL_SCRIPT="$DOCKER_CTL_DIR/docker.sh"
 FILESYSTEM_ROOT_USR="$FILESYSTEM_ROOT/usr"
 FILESYSTEM_ROOT_USR_LIB="$FILESYSTEM_ROOT/usr/lib/"
 FILESYSTEM_ROOT_USR_LIB_SYSTEMD_SYSTEM="$FILESYSTEM_ROOT_USR_LIB/systemd/system"
+FILESYSTEM_ROOT_USR_LIB_SYSTEMD_NETWORK="$FILESYSTEM_ROOT_USR_LIB/systemd/network"
 FILESYSTEM_ROOT_USR_SHARE="$FILESYSTEM_ROOT_USR/share"
 FILESYSTEM_ROOT_USR_SHARE_SONIC="$FILESYSTEM_ROOT_USR_SHARE/sonic"
 FILESYSTEM_ROOT_USR_SHARE_SONIC_SCRIPTS="$FILESYSTEM_ROOT_USR_SHARE_SONIC/scripts"
@@ -637,6 +638,34 @@ sudo cp $IMAGE_CONFIGS/config-chassisdb/config-chassisdb $FILESYSTEM_ROOT/usr/bi
 echo "config-chassisdb.service" | sudo tee -a $GENERATED_SERVICE_FILE
 sudo LANG=C chroot $FILESYSTEM_ROOT systemctl enable config-chassisdb.service
 
+# Copy midplane network service file for smart switch
+sudo cp $IMAGE_CONFIGS/midplane-network/bridge-midplane.netdev $FILESYSTEM_ROOT_USR_LIB_SYSTEMD_NETWORK/bridge-midplane.netdev
+sudo cp $IMAGE_CONFIGS/midplane-network/bridge-midplane.network $FILESYSTEM_ROOT_USR_LIB_SYSTEMD_NETWORK/bridge-midplane.network
+sudo cp $IMAGE_CONFIGS/midplane-network/dummy-midplane.netdev $FILESYSTEM_ROOT_USR_LIB_SYSTEMD_NETWORK/dummy-midplane.netdev
+sudo cp $IMAGE_CONFIGS/midplane-network/dummy-midplane.network $FILESYSTEM_ROOT_USR_LIB_SYSTEMD_NETWORK/dummy-midplane.network
+sudo cp $IMAGE_CONFIGS/midplane-network/midplane-network-npu.network $FILESYSTEM_ROOT_USR_LIB_SYSTEMD_NETWORK/midplane-network-npu.network
+sudo cp $IMAGE_CONFIGS/midplane-network/midplane-network-dpu.network $FILESYSTEM_ROOT_USR_LIB_SYSTEMD_NETWORK/midplane-network-dpu.network
+sudo cp $IMAGE_CONFIGS/midplane-network/midplane-network-npu.service $FILESYSTEM_ROOT_USR_LIB_SYSTEMD_SYSTEM/midplane-network-npu.service
+sudo cp $IMAGE_CONFIGS/midplane-network/midplane-network-dpu.service $FILESYSTEM_ROOT_USR_LIB_SYSTEMD_SYSTEM/midplane-network-dpu.service
+
+# Disable smart switch unit by default, these units will be controlled by systemd-sonic-generator
+sudo ln -s /dev/null $FILESYSTEM_ROOT/etc/systemd/network/bridge-midplane.netdev
+sudo ln -s /dev/null $FILESYSTEM_ROOT/etc/systemd/network/bridge-midplane.network
+sudo ln -s /dev/null $FILESYSTEM_ROOT/etc/systemd/network/dummy-midplane.netdev
+sudo ln -s /dev/null $FILESYSTEM_ROOT/etc/systemd/network/dummy-midplane.network
+sudo ln -s /dev/null $FILESYSTEM_ROOT/etc/systemd/network/midplane-network-npu.network
+sudo ln -s /dev/null $FILESYSTEM_ROOT/etc/systemd/network/midplane-network-dpu.network
+echo "midplane-network-npu.service" | sudo tee -a $GENERATED_SERVICE_FILE
+sudo LANG=C chroot $FILESYSTEM_ROOT systemctl disable midplane-network-npu.service
+echo "midplane-network-dpu.service" | sudo tee -a $GENERATED_SERVICE_FILE
+sudo LANG=C chroot $FILESYSTEM_ROOT systemctl disable midplane-network-dpu.service
+
+# According to the issue: https://github.com/systemd/systemd/issues/19106, To disable ManageForeignRoutingPolicyRules to avoid the ip rules being deleted by systemd-networkd
+sudo sed -i 's/#ManageForeignRoutingPolicyRules=yes/ManageForeignRoutingPolicyRules=no/g' $FILESYSTEM_ROOT/etc/systemd/networkd.conf
+
+sudo LANG=C chroot $FILESYSTEM_ROOT systemctl enable systemd-networkd
+sudo LANG=C chroot $FILESYSTEM_ROOT systemctl disable systemd-networkd-wait-online.service
+
 # Copy backend-acl script and service file
 sudo cp $IMAGE_CONFIGS/backend_acl/backend-acl.service $FILESYSTEM_ROOT_USR_LIB_SYSTEMD_SYSTEM/backend-acl.service
 sudo cp $IMAGE_CONFIGS/backend_acl/backend_acl.py $FILESYSTEM_ROOT/usr/bin/backend_acl.py

files/image_config/midplane-network/bridge-midplane.netdev

@@ -0,0 +1,5 @@
+# Bridge interface for midplane network
+
+[NetDev]
+Name=bridge-midplane
+Kind=bridge

files/image_config/midplane-network/bridge-midplane.network

@@ -0,0 +1,8 @@
+# Network configuration for bridge midplane
+
+[Match]
+Name=bridge-midplane
+
+[Network]
+Address=169.254.200.254/24
+LinkLocalAddressing=no

files/image_config/midplane-network/dummy-midplane.netdev

@@ -0,0 +1,5 @@
+# Dummy interface for midplane network
+
+[NetDev]
+Name=dummy-midplane
+Kind=dummy

files/image_config/midplane-network/dummy-midplane.network

@@ -0,0 +1,9 @@
+# The systemd-networkd requires that the bridge interface(bridge-midplane) has at least one member interface, otherwise the IP address will not be configured.
+# This dummy interface will be added into the bridge-midplane as a member to guarantee the IP address configuration correctly.
+# Refer: https://github.com/systemd/systemd/issues/9252#issuecomment-771540028
+
+[Match]
+Name=dummy-midplane
+
+[Network]
+Bridge=bridge-midplane

files/image_config/midplane-network/midplane-network-dpu.network

@@ -0,0 +1,5 @@
+[Match]
+Name=eth0-midplane
+
+[Network]
+DHCP=yes

files/image_config/midplane-network/midplane-network-dpu.service

@@ -0,0 +1,15 @@
+# Oneshot midplane network service
+
+[Unit]
+Description=Midplane network service
+Requires=systemd-networkd.service
+After=systemd-networkd.service
+Before=database.service
+
+[Service]
+Type=oneshot
+User=root
+ExecStart=/usr/lib/systemd/systemd-networkd-wait-online -i eth0-midplane --timeout=600
+
+[Install]
+WantedBy=multi-user.target

files/image_config/midplane-network/midplane-network-npu.network

@@ -0,0 +1,5 @@
+[Match]
+Name=dpu*
+
+[Network]
+Bridge=bridge-midplane

files/image_config/midplane-network/midplane-network-npu.service

@@ -0,0 +1,14 @@
+# Oneshot midplane network service
+
+[Unit]
+Description=Midplane network service
+Requires=systemd-networkd.service
+After=systemd-networkd.service
+
+[Service]
+Type=oneshot
+User=root
+ExecStart=/usr/lib/systemd/systemd-networkd-wait-online -i bridge-midplane
+
+[Install]
+WantedBy=multi-user.target

src/systemd-sonic-generator/Makefile

@@ -3,14 +3,14 @@ CFLAGS += -std=gnu99 -D_GNU_SOURCE
 
 CXX=g++
 CXXFLAGS += -std=c++11 -D_GNU_SOURCE
-LDFLAGS += -lpthread -lboost_filesystem -lboost_system -lgtest
+LDFLAGS += -lpthread -lboost_filesystem -lboost_system -lgtest -ljson-c
 
 BINARY = systemd-sonic-generator
 
 $(BINARY): systemd-sonic-generator.c
 	rm -f ./systemd-sonic-generator
 
-	$(CC) $(CFLAGS) -o $@ $^
+	$(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS)
 
 install: $(BINARY)
 	mkdir -p $(DESTDIR)