Merge pull request #3241 from Icarus9913/feat/wk/autopool-switch

subnet feature: support to turn on or off the feature of managing automatic-ippool
spidernet-io · Apr 8, 2024 · 68d6660 · 68d6660
2 parents b06dd65 + 5b55d69
commit 68d6660
Show file tree

Hide file tree

Showing 21 changed files with 231 additions and 80 deletions.
diff --git a/charts/spiderpool/README.md b/charts/spiderpool/README.md
@@ -127,19 +127,20 @@ helm install spiderpool spiderpool/spiderpool --wait --namespace kube-system \
 
 ### ipam parameters
 
-| Name                                        | Description                                                                                      | Value  |
-| ------------------------------------------- | ------------------------------------------------------------------------------------------------ | ------ |
-| `ipam.enableIPv4`                           | enable ipv4                                                                                      | `true` |
-| `ipam.enableIPv6`                           | enable ipv6                                                                                      | `true` |
-| `ipam.enableStatefulSet`                    | the network mode                                                                                 | `true` |
-| `ipam.enableKubevirtStaticIP`               | the feature to keep kubevirt vm pod static IP                                                    | `true` |
-| `ipam.enableSpiderSubnet`                   | SpiderSubnet feature gate.                                                                       | `true` |
-| `ipam.subnetDefaultFlexibleIPNumber`        | the default flexible IP number of SpiderSubnet feature auto-created IPPools                      | `1`    |
-| `ipam.gc.enabled`                           | enable retrieve IP in spiderippool CR                                                            | `true` |
-| `ipam.gc.gcAll.intervalInSecond`            | the gc all interval duration                                                                     | `600`  |
-| `ipam.gc.statelessPod.zombieOnReadyNode`    | enable reclaim IP for the stateless pod who is over deleting graceful period on a ready node     | `true` |
-| `ipam.gc.statelessPod.zombieOnNotReadyNode` | enable reclaim IP for the stateless pod who is over deleting graceful period on a not-ready node | `true` |
-| `ipam.gc.gcDeletingTimeOutPodDelay`         | the gc delay seconds after the pod times out of deleting graceful period                         | `0`    |
+| Name                                                  | Description                                                                                      | Value  |
+| ----------------------------------------------------- | ------------------------------------------------------------------------------------------------ | ------ |
+| `ipam.enableIPv4`                                     | enable ipv4                                                                                      | `true` |
+| `ipam.enableIPv6`                                     | enable ipv6                                                                                      | `true` |
+| `ipam.enableStatefulSet`                              | the network mode                                                                                 | `true` |
+| `ipam.enableKubevirtStaticIP`                         | the feature to keep kubevirt vm pod static IP                                                    | `true` |
+| `ipam.spidersubnet.enable`                            | SpiderSubnet feature.                                                                            | `true` |
+| `ipam.spidersubnet.autoPool.enable`                   | SpiderSubnet Auto IPPool feature.                                                                | `true` |
+| `ipam.spidersubnet.autoPool.defaultRedundantIPNumber` | the default redundant IP number of SpiderSubnet feature auto-created IPPools                     | `1`    |
+| `ipam.gc.enabled`                                     | enable retrieve IP in spiderippool CR                                                            | `true` |
+| `ipam.gc.gcAll.intervalInSecond`                      | the gc all interval duration                                                                     | `600`  |
+| `ipam.gc.statelessPod.zombieOnReadyNode`              | enable reclaim IP for the stateless pod who is over deleting graceful period on a ready node     | `true` |
+| `ipam.gc.statelessPod.zombieOnNotReadyNode`           | enable reclaim IP for the stateless pod who is over deleting graceful period on a not-ready node | `true` |
+| `ipam.gc.gcDeletingTimeOutPodDelay`                   | the gc delay seconds after the pod times out of deleting graceful period                         | `0`    |
 
 ### grafanaDashboard parameters
 

diff --git a/charts/spiderpool/templates/configmap.yaml b/charts/spiderpool/templates/configmap.yaml
@@ -19,9 +19,10 @@ data:
     enableIPv6: {{ .Values.ipam.enableIPv6 }}
     enableStatefulSet: {{ .Values.ipam.enableStatefulSet }}
     enableKubevirtStaticIP: {{ .Values.ipam.enableKubevirtStaticIP }}
-    enableSpiderSubnet: {{ .Values.ipam.enableSpiderSubnet }}
-    {{- if .Values.ipam.enableSpiderSubnet }}
-    clusterSubnetDefaultFlexibleIPNumber: {{ .Values.ipam.subnetDefaultFlexibleIPNumber }}
+    enableSpiderSubnet: {{ .Values.ipam.spidersubnet.enable }}
+    enableAutoPoolForApplication: {{ .Values.ipam.spidersubnet.autoPool.enable }}
+    {{- if and .Values.ipam.spidersubnet.enable .Values.ipam.spidersubnet.autoPool.enable }}
+    clusterSubnetDefaultFlexibleIPNumber: {{ .Values.ipam.spidersubnet.autoPool.defaultRedundantIPNumber }}
     {{- else}}
     clusterSubnetDefaultFlexibleIPNumber: 0
     {{- end }}

diff --git a/charts/spiderpool/templates/pod.yaml b/charts/spiderpool/templates/pod.yaml
@@ -112,7 +112,7 @@ spec:
       value: {{ toJson .Values.clusterDefaultPool.ipv4IPRanges | quote }}
     - name: SPIDERPOOL_INIT_DEFAULT_IPV4_IPPOOL_GATEWAY
       value: {{ .Values.clusterDefaultPool.ipv4Gateway | quote }}
-    {{- if .Values.ipam.enableSpiderSubnet }}
+    {{- if .Values.ipam.spidersubnet.enable }}
     - name: SPIDERPOOL_INIT_DEFAULT_IPV4_SUBNET_NAME
       value: {{ .Values.clusterDefaultPool.ipv4SubnetName | quote }}
     {{- end }}
@@ -126,7 +126,7 @@ spec:
       value: {{ toJson .Values.clusterDefaultPool.ipv6IPRanges | quote }}
     - name: SPIDERPOOL_INIT_DEFAULT_IPV6_IPPOOL_GATEWAY
       value: {{ .Values.clusterDefaultPool.ipv6Gateway | quote }}
-    {{- if .Values.ipam.enableSpiderSubnet }}
+    {{- if .Values.ipam.spidersubnet.enable }}
     - name: SPIDERPOOL_INIT_DEFAULT_IPV6_SUBNET_NAME
       value: {{ .Values.clusterDefaultPool.ipv6SubnetName | quote }}
     {{- end }}

diff --git a/charts/spiderpool/values.yaml b/charts/spiderpool/values.yaml
@@ -53,11 +53,16 @@ ipam:
   ## @param ipam.enableKubevirtStaticIP the feature to keep kubevirt vm pod static IP
   enableKubevirtStaticIP: true
 
-  ## @param ipam.enableSpiderSubnet SpiderSubnet feature gate.
-  enableSpiderSubnet: true
+  spidersubnet:
+    ## @param ipam.spidersubnet.enable SpiderSubnet feature.
+    enable: true
 
-  ## @param ipam.subnetDefaultFlexibleIPNumber the default flexible IP number of SpiderSubnet feature auto-created IPPools
-  subnetDefaultFlexibleIPNumber: 1
+    autoPool:
+      ## @param ipam.spidersubnet.autoPool.enable SpiderSubnet Auto IPPool feature.
+      enable: true
+
+      ## @param ipam.spidersubnet.autoPool.defaultRedundantIPNumber the default redundant IP number of SpiderSubnet feature auto-created IPPools
+      defaultRedundantIPNumber: 1
 
   gc:
     ## @param ipam.gc.enabled enable retrieve IP in spiderippool CR

diff --git a/cmd/spiderpool-agent/cmd/config.go b/cmd/spiderpool-agent/cmd/config.go
@@ -29,6 +29,7 @@ import (
 	"github.com/spidernet-io/spiderpool/pkg/reservedipmanager"
 	"github.com/spidernet-io/spiderpool/pkg/statefulsetmanager"
 	"github.com/spidernet-io/spiderpool/pkg/subnetmanager"
+	spiderpooltypes "github.com/spidernet-io/spiderpool/pkg/types"
 	"github.com/spidernet-io/spiderpool/pkg/workloadendpointmanager"
 )
 
@@ -97,13 +98,7 @@ type Config struct {
 	MultusClusterNetwork string
 
 	// configmap
-	IpamUnixSocketPath                string `yaml:"ipamUnixSocketPath"`
-	EnableIPv4                        bool   `yaml:"enableIPv4"`
-	EnableIPv6                        bool   `yaml:"enableIPv6"`
-	EnableStatefulSet                 bool   `yaml:"enableStatefulSet"`
-	EnableKubevirtStaticIP            bool   `yaml:"enableKubevirtStaticIP"`
-	EnableSpiderSubnet                bool   `yaml:"enableSpiderSubnet"`
-	ClusterSubnetDefaultFlexibleIPNum int    `yaml:"clusterSubnetDefaultFlexibleIPNumber"`
+	spiderpooltypes.SpiderpoolConfigmapConfig
 }
 
 type AgentContext struct {
@@ -194,7 +189,7 @@ func (ac *AgentContext) LoadConfigmap() error {
 		return fmt.Errorf("failed to read configmap file, error: %v", err)
 	}
 
-	err = yaml.Unmarshal(configmapBytes, &ac.Cfg)
+	err = yaml.Unmarshal(configmapBytes, &ac.Cfg.SpiderpoolConfigmapConfig)
 	if nil != err {
 		return fmt.Errorf("failed to parse configmap, error: %v", err)
 	}

diff --git a/cmd/spiderpool-agent/cmd/daemon.go b/cmd/spiderpool-agent/cmd/daemon.go
@@ -145,6 +145,7 @@ func DaemonMain() {
 		EnableIPv4:                           agentContext.Cfg.EnableIPv4,
 		EnableIPv6:                           agentContext.Cfg.EnableIPv6,
 		EnableSpiderSubnet:                   agentContext.Cfg.EnableSpiderSubnet,
+		EnableAutoPoolForApplication:         agentContext.Cfg.EnableAutoPoolForApplication,
 		EnableStatefulSet:                    agentContext.Cfg.EnableStatefulSet,
 		EnableKubevirtStaticIP:               agentContext.Cfg.EnableKubevirtStaticIP,
 		EnableReleaseConflictIPsForStateless: agentContext.Cfg.EnableReleaseConflictIPsForStateless,

diff --git a/cmd/spiderpool-controller/cmd/config.go b/cmd/spiderpool-controller/cmd/config.go
@@ -31,6 +31,7 @@ import (
 	"github.com/spidernet-io/spiderpool/pkg/reservedipmanager"
 	"github.com/spidernet-io/spiderpool/pkg/statefulsetmanager"
 	"github.com/spidernet-io/spiderpool/pkg/subnetmanager"
+	spiderpooltypes "github.com/spidernet-io/spiderpool/pkg/types"
 	"github.com/spidernet-io/spiderpool/pkg/workloadendpointmanager"
 )
 
@@ -153,20 +154,14 @@ type Config struct {
 	WorkQueueMaxRetries              int
 	WorkQueueRequeueDelayDuration    int
 
+	EnableCoordinator               bool
 	CoordinatorInformerResyncPeriod int
 
 	EnableMultusConfig               bool
 	MultusConfigInformerResyncPeriod int
 
-	EnableCoordinator bool
-
 	// configmap
-	EnableIPv4                        bool `yaml:"enableIPv4"`
-	EnableIPv6                        bool `yaml:"enableIPv6"`
-	EnableStatefulSet                 bool `yaml:"enableStatefulSet"`
-	EnableKubevirtStaticIP            bool `yaml:"enableKubevirtStaticIP"`
-	EnableSpiderSubnet                bool `yaml:"enableSpiderSubnet"`
-	ClusterSubnetDefaultFlexibleIPNum int  `yaml:"clusterSubnetDefaultFlexibleIPNumber"`
+	spiderpooltypes.SpiderpoolConfigmapConfig
 }
 
 type ControllerContext struct {
@@ -286,7 +281,7 @@ func (cc *ControllerContext) LoadConfigmap() error {
 		return fmt.Errorf("failed to read configmap file, error: %v", err)
 	}
 
-	err = yaml.Unmarshal(configmapBytes, &cc.Cfg)
+	err = yaml.Unmarshal(configmapBytes, &cc.Cfg.SpiderpoolConfigmapConfig)
 	if nil != err {
 		return fmt.Errorf("failed to parse configmap, error: %v", err)
 	}

diff --git a/cmd/spiderpool-controller/cmd/daemon.go b/cmd/spiderpool-controller/cmd/daemon.go
@@ -365,8 +365,8 @@ func initControllerServiceManagers(ctx context.Context) {
 			logger.Fatal(err.Error())
 		}
 
-		logger.Sugar().Debugf("Begin to initialize cluster Subnet default flexible IP number to %d", controllerContext.Cfg.ClusterSubnetDefaultFlexibleIPNum)
-		*applicationinformers.ClusterSubnetDefaultFlexibleIPNumber = controllerContext.Cfg.ClusterSubnetDefaultFlexibleIPNum
+		logger.Sugar().Debugf("Begin to initialize cluster Subnet AutoPool default redimdamt IP number to %d", controllerContext.Cfg.ClusterSubnetAutoPoolDefaultRedundantIPNumber)
+		*applicationinformers.ClusterSubnetAutoPoolDefaultRedundantIPNumber = controllerContext.Cfg.ClusterSubnetAutoPoolDefaultRedundantIPNumber
 	} else {
 		logger.Info("Feature SpiderSubnet is disabled")
 	}
@@ -489,6 +489,7 @@ func setupInformers(k8sClient *kubernetes.Clientset) {
 		ippoolmanager.IPPoolControllerConfig{
 			IPPoolControllerWorkers:       controllerContext.Cfg.IPPoolInformerWorkers,
 			EnableSpiderSubnet:            controllerContext.Cfg.EnableSpiderSubnet,
+			EnableAutoPoolForApplication:  controllerContext.Cfg.EnableAutoPoolForApplication,
 			LeaderRetryElectGap:           time.Duration(controllerContext.Cfg.LeaseRetryGap) * time.Second,
 			MaxWorkqueueLength:            controllerContext.Cfg.IPPoolInformerMaxWorkQueueLength,
 			WorkQueueRequeueDelayDuration: time.Duration(controllerContext.Cfg.WorkQueueRequeueDelayDuration) * time.Second,
@@ -517,27 +518,31 @@ func setupInformers(k8sClient *kubernetes.Clientset) {
 			logger.Fatal(err.Error())
 		}
 
-		logger.Info("Begin to set up auto-created IPPool controller")
-		subnetAppController, err := applicationcontroller.NewSubnetAppController(
-			controllerContext.CRDManager.GetClient(),
-			controllerContext.CRDManager.GetAPIReader(),
-			controllerContext.SubnetManager,
-			applicationcontroller.SubnetAppControllerConfig{
-				EnableIPv4:                    controllerContext.Cfg.EnableIPv4,
-				EnableIPv6:                    controllerContext.Cfg.EnableIPv6,
-				AppControllerWorkers:          controllerContext.Cfg.SubnetAppControllerWorkers,
-				MaxWorkqueueLength:            controllerContext.Cfg.SubnetInformerMaxWorkqueueLength,
-				WorkQueueMaxRetries:           controllerContext.Cfg.WorkQueueMaxRetries,
-				WorkQueueRequeueDelayDuration: time.Duration(controllerContext.Cfg.WorkQueueRequeueDelayDuration) * time.Second,
-				LeaderRetryElectGap:           time.Duration(controllerContext.Cfg.LeaseRetryGap) * time.Second,
-			})
-		if nil != err {
-			logger.Fatal(err.Error())
-		}
+		if controllerContext.Cfg.EnableAutoPoolForApplication {
+			logger.Info("Begin to set up auto-created IPPool controller")
+			subnetAppController, err := applicationcontroller.NewSubnetAppController(
+				controllerContext.CRDManager.GetClient(),
+				controllerContext.CRDManager.GetAPIReader(),
+				controllerContext.SubnetManager,
+				applicationcontroller.SubnetAppControllerConfig{
+					EnableIPv4:                    controllerContext.Cfg.EnableIPv4,
+					EnableIPv6:                    controllerContext.Cfg.EnableIPv6,
+					AppControllerWorkers:          controllerContext.Cfg.SubnetAppControllerWorkers,
+					MaxWorkqueueLength:            controllerContext.Cfg.SubnetInformerMaxWorkqueueLength,
+					WorkQueueMaxRetries:           controllerContext.Cfg.WorkQueueMaxRetries,
+					WorkQueueRequeueDelayDuration: time.Duration(controllerContext.Cfg.WorkQueueRequeueDelayDuration) * time.Second,
+					LeaderRetryElectGap:           time.Duration(controllerContext.Cfg.LeaseRetryGap) * time.Second,
+				})
+			if nil != err {
+				logger.Fatal(err.Error())
+			}
 
-		err = subnetAppController.SetupInformer(controllerContext.InnerCtx, controllerContext.ClientSet, controllerContext.Leader)
-		if nil != err {
-			logger.Fatal(err.Error())
+			err = subnetAppController.SetupInformer(controllerContext.InnerCtx, controllerContext.ClientSet, controllerContext.Leader)
+			if nil != err {
+				logger.Fatal(err.Error())
+			}
+		} else {
+			logger.Sugar().Info("SpiderSubnet AutoPool feature is off")
 		}
 	}
 

diff --git a/docs/usage/install/upgrade-zh_CN.md b/docs/usage/install/upgrade-zh_CN.md
@@ -51,13 +51,13 @@
 您可以通过 `--set` 在升级时去更新 Spiderpool 配置，可用的 values 参数，请查看 [values](https://github.com/spidernet-io/spiderpool/tree/main/charts/spiderpool/README.md) 说明文档。 以下示例展示了如何开启 Spiderpool 的 [SpiderSubnet 功能](../spider-subnet-zh_CN.md)
 
 ```bash
-helm upgrade spiderpool spiderpool/spiderpool -n kube-system --version [upgraded-version] --set ipam.enableSpiderSubnet=true
+helm upgrade spiderpool spiderpool/spiderpool -n kube-system --version [upgraded-version] --set ipam.spidersubnet.enable=true
 ```
 
 同时您也可以使用 `--reuse-values` 重用上一个 release 的值并合并来自命令行的任何覆盖。但仅当 Spiderpool chart 版本保持不变时，才可以安全地使用 `--reuse-values` 标志，例如，当使用 helm upgrade 来更改 Spiderpool 配置而不升级 Spiderpool 组件。 `--reuse-values` 使用，参考如下示例：
 
 ```bash
-helm upgrade spiderpool spiderpool/spiderpool -n kube-system --version [upgraded-version] --set ipam.enableSpiderSubnet=true --reuse-values
+helm upgrade spiderpool spiderpool/spiderpool -n kube-system --version [upgraded-version] --set ipam.spidersubnet.enable=true --reuse-values
 ```
 
 相反，如果 Spiderpool chart 版本发生了变化，您想重用现有安装中的值，请将旧值保存在值文件中，检查该文件中是否有任何重命名或弃用的值，然后将其传递给 helm upgrade 命令，您可以使用以下命令检索并保存现有安装中的值：

diff --git a/docs/usage/install/upgrade.md b/docs/usage/install/upgrade.md
@@ -51,13 +51,13 @@ It is recommended to always upgrade to the latest and maintained patch version o
 You can use `--set` to update the Spiderpool configuration when upgrading. For available values parameters, please see the [values](https://github.com/spidernet-io/spiderpool/tree/main/charts/spiderpool/README.md) documentation. The following example shows how to enable Spiderpool's [SpiderSubnet function](../spider-subnet.md)
 
 ```bash
-helm upgrade spiderpool spiderpool/spiderpool -n kube-system --version [upgraded-version] --set ipam.enableSpiderSubnet=true
+helm upgrade spiderpool spiderpool/spiderpool -n kube-system --version [upgraded-version] --set ipam.spidersubnet.enable=true
 ```
 
 You can also use `--reuse-values` to reuse the values from the previous release and merge any overrides from the command line. However, it is only safe to use the `--reuse-values` flag if the Spiderpool chart version remains unchanged, e.g. when using helm upgrade to change the Spiderpool configuration without upgrading the Spiderpool components. For `--reuse-values` usage, see the following example:
 
 ```bash
-helm upgrade spiderpool spiderpool/spiderpool -n kube-system --version [upgraded-version] --set ipam.enableSpiderSubnet=true --reuse-values
+helm upgrade spiderpool spiderpool/spiderpool -n kube-system --version [upgraded-version] --set ipam.spidersubnet.enable=true --reuse-values
 ```
 
 Conversely, if the Spiderpool chart version has changed and you want to reuse the values from the existing installation, save the old values in a values file, check that file for any renamed or deprecated values, and pass it to helm upgrade command, you can retrieve and save values from existing installations using.

diff --git a/docs/usage/spider-subnet-zh_CN.md b/docs/usage/spider-subnet-zh_CN.md
@@ -32,7 +32,7 @@ SpiderSubnet 功能还支持众多的控制器，如：ReplicaSet、Deployment
 
 ### 安装 Spiderpool
 
-可参考 [安装](./readme-zh_CN.md) 安装 Spiderpool. 其中，务必确保 helm 安装选项 `ipam.enableSpiderSubnet=true`
+可参考 [安装教程](./readme-zh_CN.md) 来安装 Spiderpool. 请务必确保 helm 安装选项 `--ipam.spidersubnet.enable=true --ipam.spidersubnet.autoPool.enable=true`. 其中，`ipam.spidersubnet.autoPool.enable` 提供 `自动创建 IPPool` 的能力。
 
 ### 安装 CNI 配置
 

diff --git a/docs/usage/spider-subnet.md b/docs/usage/spider-subnet.md
@@ -33,7 +33,7 @@ This feature does not support the bare Pod.
 
 ### Install Spiderpool
 
-Refer to [Installation](./readme.md) to install Spiderpool. And make sure that the helm installs the option `ipam.enableSpiderSubnet=true`.
+Refer to [Installation](./readme.md) to install Spiderpool. And make sure that the helm installs the option `--ipam.spidersubnet.enable=true --ipam.spidersubnet.autoPool.enable=true`. The `ipam.spidersubnet.autoPool.enable` provide the `Automatically create IPPool` ability.
 
 ### Install CNI