Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

coodirnator: optimize the detectiong timeout for ip conflict and gateway detection #4511

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -277,7 +277,7 @@ unittest-tests: check_test_label
@echo "run unittest-tests"
$(QUIET) $(ROOT_DIR)/tools/scripts/ginkgo.sh \
--cover --coverprofile=./coverage.out --covermode set \
--json-report unittestreport.json --label-filter $(E2E_GINKGO_UTLABELS) \
--json-report unittestreport.json \
-randomize-suites -randomize-all --keep-going --timeout=1h -p \
-vv -r $(ROOT_DIR)/pkg $(ROOT_DIR)/cmd
$(QUIET) go tool cover -html=./coverage.out -o coverage-all.html
Expand Down
6 changes: 3 additions & 3 deletions cmd/coordinator/cmd/cni_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -274,8 +274,8 @@ func validateRPFilterConfig(rpfilter *int32, coordinatorConfig int64) (*int32, e
func ValidateDelectOptions(config *DetectOptions) (*DetectOptions, error) {
if config == nil {
return &DetectOptions{
Interval: "1s",
TimeOut: "3s",
Interval: "10ms",
TimeOut: "100ms",
Retry: 3,
}, nil
}
Expand All @@ -289,7 +289,7 @@ func ValidateDelectOptions(config *DetectOptions) (*DetectOptions, error) {
}

if config.TimeOut == "" {
config.TimeOut = "100ms"
config.TimeOut = "500ms"
}

_, err := time.ParseDuration(config.Interval)
Expand Down
2 changes: 1 addition & 1 deletion docs/concepts/coordinator-zh_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ EOF
| podRPFilter | 设置 Pod 的 sysctl 参数 rp_filter | 整数型 | optional | 0 |
| hostRPFilter | (遗弃)设置节点 的 sysctl 参数 rp_filter | 整数型 | optional | 0 |
| txQueueLen | 设置 Pod 的网卡传输队列 | 整数型 | optional | 0 |
| detectOptions | 检测地址冲突和网关可达性的高级配置项: 包括重试次数(默认为 3 次), 探测间隔(默认为 10ms) 和 超时时间(默认为 100ms) | 对象类型 | optional ||
| detectOptions | 检测地址冲突和网关可达性的高级配置项: 包括发送探测报文次数(retries: 默认为 3 次), 和响应的超时时间(timeout: 默认为 100ms),还有发送报文的间隔(interval:默认为 10ms, 将会在未来版本中移除) | 对象类型 | optional ||
| logOptions | 日志配置,包括 logLevel(默认为 debug) 和 logFile(默认为 /var/log/spidernet/coordinator.log) | 对象类型 | optional | - |

> 如果您通过 `SpinderMultusConfig CR` 帮助创建 NetworkAttachmentDefinition CR,您可以在 `SpinderMultusConfig` 中配置 `coordinator` (所有字段)。参考: [SpinderMultusConfig](../reference/crd-spidermultusconfig.md)
Expand Down
2 changes: 1 addition & 1 deletion docs/concepts/coordinator.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Let's delve into how coordinator implements these features.
| podRPFilter | Set the rp_filter sysctl parameter on the pod, which is recommended to be set to 0 | int | optional | 0 |
| hostRPFilter | (deprecated)Set the rp_filter sysctl parameter on the node, which is recommended to be set to 0 | int | optional | 0 |
| txQueueLen | set txqueuelen(Transmit Queue Length) of the pod's interface | int | optional | 0 |
| detectOptions | The advanced configuration of detectGateway and detectIPConflict, including retry numbers(default is 3), interval(default is 10ms) and timeout(default is 100ms) | obejct | optional | nil |
| detectOptions | The advanced configuration of detectGateway and detectIPConflict, including the number of the send packets(retries: default is 3) and the response timeout(timeout: default is 100ms) and the packet sending interval(interval: default is 10ms, which will be removed in the future version). | obejct | optional | nil |
| logOptions | The configuration of logging, including logLevel(default is debug) and logFile(default is /var/log/spidernet/coordinator.log) | obejct | optional | nil |

> You can configure `coordinator` by specifying all the relevant fields in `SpinderMultusConfig` if a NetworkAttachmentDefinition CR is created via `SpinderMultusConfig CR`. For more information, please refer to [SpinderMultusConfig](../reference/crd-spidermultusconfig.md).
Expand Down
89 changes: 2 additions & 87 deletions pkg/coordinatormanager/coordinator_informer.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ import (
"fmt"
"net"
"reflect"
"regexp"
"sort"
"strings"
"time"
Expand Down Expand Up @@ -473,7 +472,7 @@ func (cc *CoordinatorController) updatePodAndServerCIDR(ctx context.Context, log

if err == nil {
logger.Sugar().Info("Trying to fetch the ClusterCIDR from kube-system/kubeadm-config")
k8sPodCIDR, k8sServiceCIDR, err = ExtractK8sCIDRFromKubeadmConfigMap(&cm)
k8sPodCIDR, k8sServiceCIDR, err = utils.ExtractK8sCIDRFromKubeadmConfigMap(&cm)
if err == nil {
// Success to get ClusterCIDR from kubeadm-config
logger.Sugar().Infof("Success get CIDR from kubeadm-config: PodCIDR=%v, ServiceCIDR=%v", k8sPodCIDR, k8sServiceCIDR)
Expand Down Expand Up @@ -507,7 +506,7 @@ func (cc *CoordinatorController) updatePodAndServerCIDR(ctx context.Context, log
return coordCopy
}

k8sPodCIDR, k8sServiceCIDR = ExtractK8sCIDRFromKCMPod(&podList.Items[0])
k8sPodCIDR, k8sServiceCIDR = utils.ExtractK8sCIDRFromKCMPod(&podList.Items[0])
logger.Sugar().Infof("kube-controller-manager k8sPodCIDR %v, k8sServiceCIDR %v", k8sPodCIDR, k8sServiceCIDR)
}

Expand Down Expand Up @@ -778,90 +777,6 @@ func (cc *CoordinatorController) updateServiceCIDR(logger *zap.Logger, coordCopy
return nil
}

func ExtractK8sCIDRFromKubeadmConfigMap(cm *corev1.ConfigMap) ([]string, []string, error) {
if cm == nil {
return nil, nil, fmt.Errorf("kubeadm configmap is unexpected to nil")
}
var podCIDR, serviceCIDR []string

clusterConfig, exists := cm.Data["ClusterConfiguration"]
if !exists {
return podCIDR, serviceCIDR, fmt.Errorf("unable to get kubeadm configmap ClusterConfiguration")
}

podReg := regexp.MustCompile(`podSubnet:\s*(\S+)`)
serviceReg := regexp.MustCompile(`serviceSubnet:\s*(\S+)`)

podSubnets := podReg.FindStringSubmatch(clusterConfig)
serviceSubnets := serviceReg.FindStringSubmatch(clusterConfig)

if len(podSubnets) > 1 {
for _, cidr := range strings.Split(podSubnets[1], ",") {
cidr = strings.TrimSpace(cidr)
_, _, err := net.ParseCIDR(cidr)
if err != nil {
continue
}
podCIDR = append(podCIDR, cidr)
}
}

if len(serviceSubnets) > 1 {
for _, cidr := range strings.Split(serviceSubnets[1], ",") {
cidr = strings.TrimSpace(cidr)
_, _, err := net.ParseCIDR(cidr)
if err != nil {
continue
}
serviceCIDR = append(serviceCIDR, cidr)
}
}

return podCIDR, serviceCIDR, nil
}

func ExtractK8sCIDRFromKCMPod(kcm *corev1.Pod) ([]string, []string) {
var podCIDR, serviceCIDR []string

podReg := regexp.MustCompile(`--cluster-cidr=(.*)`)
serviceReg := regexp.MustCompile(`--service-cluster-ip-range=(.*)`)

var podSubnets, serviceSubnets []string
for _, l := range kcm.Spec.Containers[0].Command {
if len(podSubnets) == 0 {
podSubnets = podReg.FindStringSubmatch(l)
}
if len(serviceSubnets) == 0 {
serviceSubnets = serviceReg.FindStringSubmatch(l)
}
if len(podSubnets) != 0 && len(serviceSubnets) != 0 {
break
}
}

if len(podSubnets) != 0 {
for _, cidr := range strings.Split(podSubnets[1], ",") {
_, _, err := net.ParseCIDR(cidr)
if err != nil {
continue
}
podCIDR = append(podCIDR, cidr)
}
}

if len(serviceSubnets) != 0 {
for _, cidr := range strings.Split(serviceSubnets[1], ",") {
_, _, err := net.ParseCIDR(cidr)
if err != nil {
continue
}
serviceCIDR = append(serviceCIDR, cidr)
}
}

return podCIDR, serviceCIDR
}

func fetchType(cniDir string) (string, error) {
defaultCniName, err := utils.GetDefaultCniName(cniDir)
if err != nil {
Expand Down
54 changes: 0 additions & 54 deletions pkg/coordinatormanager/coordinator_informer_test.go

This file was deleted.

61 changes: 0 additions & 61 deletions pkg/coordinatormanager/coordinatormanager_suite_test.go

This file was deleted.

11 changes: 11 additions & 0 deletions pkg/errgroup/errgroup.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
package errgroup

import (
"context"
"fmt"
"runtime"
"sync"
Expand Down Expand Up @@ -43,6 +44,16 @@ func (g *Group) done() {
g.wg.Done()
}

// WithContext returns a new Group and an associated Context derived from ctx.
//
// The derived Context is canceled the first time a function passed to Go
// returns a non-nil error or the first time Wait returns, whichever occurs
// first.
func WithContext(ctx context.Context) (*Group, context.Context) {
ctx, cancel := context.WithCancelCause(ctx)
return &Group{cancel: cancel}, ctx
}

// Wait blocks until all function calls from the Go method have returned, then
// returns the first non-nil error (if any) from them.
func (g *Group) Wait() error {
Expand Down
77 changes: 77 additions & 0 deletions pkg/ippoolmanager/config_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
// Copyright 2024 Authors of spidernet-io
// SPDX-License-Identifier: Apache-2.0
package ippoolmanager

import (
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)

var _ = Describe("IPPoolManagerConfig", Label("ippool_manager_test"), func() {
var config IPPoolManagerConfig

Describe("setDefaultsForIPPoolManagerConfig", func() {
Context("when MaxAllocatedIPs is nil", func() {
BeforeEach(func() {
config = IPPoolManagerConfig{
MaxAllocatedIPs: nil,
EnableKubevirtStaticIP: false,
}
})

It("should set MaxAllocatedIPs to default value", func() {
result := setDefaultsForIPPoolManagerConfig(config)
Expect(result.MaxAllocatedIPs).NotTo(BeNil())
Expect(*result.MaxAllocatedIPs).To(Equal(defaultMaxAllocatedIPs))
})
})

Context("when MaxAllocatedIPs is set", func() {
BeforeEach(func() {
maxIPs := 3000
config = IPPoolManagerConfig{
MaxAllocatedIPs: &maxIPs,
EnableKubevirtStaticIP: true,
}
})

It("should retain the provided MaxAllocatedIPs value", func() {
result := setDefaultsForIPPoolManagerConfig(config)
Expect(result.MaxAllocatedIPs).NotTo(BeNil())
Expect(*result.MaxAllocatedIPs).To(Equal(3000))
})
})

Context("when EnableKubevirtStaticIP is true", func() {
BeforeEach(func() {
config = IPPoolManagerConfig{
MaxAllocatedIPs: nil,
EnableKubevirtStaticIP: true,
}
})

It("should set MaxAllocatedIPs to default value", func() {
result := setDefaultsForIPPoolManagerConfig(config)
Expect(result.MaxAllocatedIPs).NotTo(BeNil())
Expect(*result.MaxAllocatedIPs).To(Equal(defaultMaxAllocatedIPs))
Expect(result.EnableKubevirtStaticIP).To(BeTrue())
})
})

Context("when EnableKubevirtStaticIP is false", func() {
BeforeEach(func() {
config = IPPoolManagerConfig{
MaxAllocatedIPs: nil,
EnableKubevirtStaticIP: false,
}
})

It("should set MaxAllocatedIPs to default value", func() {
result := setDefaultsForIPPoolManagerConfig(config)
Expect(result.MaxAllocatedIPs).NotTo(BeNil())
Expect(*result.MaxAllocatedIPs).To(Equal(defaultMaxAllocatedIPs))
Expect(result.EnableKubevirtStaticIP).To(BeFalse())
})
})
})
})
Loading
Loading