Skip to content

Commit

Permalink
Get multiport-eswitch mode from metalnet rundir
Browse files Browse the repository at this point in the history
  • Loading branch information
PlagueCZ committed Oct 14, 2024
1 parent 025d117 commit e193110
Show file tree
Hide file tree
Showing 3 changed files with 31 additions and 2 deletions.
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Build the manager binary
FROM --platform=$BUILDPLATFORM golang:1.23 as builder
FROM --platform=$BUILDPLATFORM golang:1.23 AS builder

ARG GOARCH=''

Expand Down
22 changes: 22 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,25 @@

## CRD usage
* [Usage](./usage/crd_usage.md)

## Multiport-eswitch mode
When running dpservice with Mellanox in multiport-eswitch mode, it is important to tell metalnet about it:
```
metalnet --multiport-eswitch
```
or
```
echo -n "eswitch" > /var/run/metalnet/mode
```
This changes the way metalnet generates identifiers for virtual function representors that are sent over to dpservice.

If pf1-proxy is also in use, it is important to mark it as used in the metalnet VF database:
```
mkdir -p /var/lib/metalnet/netfns/claims
echo -n "$pf1_proxy_vf_pci" > /var/lib/metalnet/netfns/claims/00000001-0000-4000-0000-000000000000
```
Where the `$pf1_proxy_vf_pci` is the PCI address of the VF representor for pf1-proxy. This should be the only VF using `mlx5_core` driver instead of the `vfio-pci` driver. One of many ways to retrieve such address is as follows:
```
pf1_proxy_vf_name=$(/opt/local/bin/dpdk-devbind.py -s | grep "mlx5Gen Virtual Function" | grep "drv=mlx5_core" | awk -F'if=' '{print $2}' | awk '{print $1}')
pf1_proxy_vf_pci=$(/opt/local/bin/dpdk-devbind.py -s | grep "mlx5Gen Virtual Function" | grep $pf1_proxy_vf_name | awk '{print $1}')
```
9 changes: 8 additions & 1 deletion main.go
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ func main() {
flag.BoolVar(&enableIPv6Support, "enable-ipv6", false, "Enable IPv6 support")
flag.IntVar(&publicVNI, "public-vni", 100, "Virtual network identifier used for public routing announcements.")
flag.IPVar(&routerAddress, "router-address", net.IP{}, "The address of the next router.")
flag.BoolVar(&multiportEswitchMode, "multiport-eswitch", false, "Enable multiport eswitch support")
flag.BoolVar(&multiportEswitchMode, "multiport-eswitch", false, "Enable multiport eswitch support (unless specified in metalnet-dir)")
flag.BoolVar(&enableLeaderElection, "leader-elect", false,
"Enable leader election for controller manager. "+
"Enabling this will ensure there is only one active controller manager.")
Expand All @@ -125,6 +125,13 @@ func main() {
log.SetLevel(log.DebugLevel)
}

// detect multiport-eswitch mode automatically (overrides command-line)
// Usage example: 'echo -n "eswitch" > /var/run/metalnet/mode'
modeFilePath := filepath.Join(metalnetDir, "mode")
content, err := os.ReadFile(modeFilePath)
multiportEswitchMode = err == nil && string(content) == "eswitch"
setupLog.Info(fmt.Sprintf("Multiport Eswitch mode set to: %v", multiportEswitchMode))

defaultRouterAddr.PublicVNI = uint32(publicVNI)
defaultRouterAddr.SetBySubsciption = false

Expand Down

0 comments on commit e193110

Please sign in to comment.