手把手教你把 SafeLine 落地到云原生环境 #289

jangrui · 2023-09-04T08:38:20Z

jangrui
Sep 4, 2023

反馈内容

手把手教你把 SafeLine 落地到云原生环境

核心一览

Sealos 一键构建 k8s 环境
部署 LongHorn 云原生分布式存储
部署 Ingress Nginx 控制器
部署 CertManager 控制器
部署 SafeLine
配置 WAF

SealOS 一键构建 k8s 环境

环境信息

IP	主机名	操作系统	功能作用	数据盘
192.168.20.253	/	/	lb
192.168.20.101	silkdo-1	Anolis 8.8	master+longhorn	/dev/vdc
192.168.20.102	silkdo-2	Anolis 8.8	master+longhorn	/dev/vdc
192.168.20.103	silkdo-3	Anolis 8.8	master+longhorn	/dev/vdc
192.168.20.104	silkdo-4	Anolis 8.8	worker
192.168.20.105	silkdo-5	Anolis 8.8	worker
192.168.20.106	silkdo-6	Anolis 8.8	worker
192.168.20.107	silkdo-7	Anolis 8.8	worker
192.168.20.108	silkdo-8	Anolis 8.8	worker
192.168.20.109	silkdo-8	Anolis 8.8	worker

安装 Sealos

sudo cat > /etc/yum.repos.d/labring.repo << EOF
[fury]
name=labring Yum Repo
baseurl=https://yum.fury.io/labring/
enabled=1
gpgcheck=0
EOF
sudo yum clean all
sudo yum install -y sealos

Sealos 命令说明

配置clusterfile

sealos gen \
  --masters 192.168.20.101,192.168.20.102,192.168.20.103 \
  --nodes 192.168.20.104,192.168.20.105,192.168.20.106,192.168.20.107,192.168.20.108,192.168.20.109 \
  --pk ~/.ssh/id_rsa \
  labring/kubernetes:v1.25.11 \
  labring/nerdctl:v1.2.1 \
  labring/helm:v3.12.0 \
  labring/cilium:v1.13.0
  > clusterfile

# 替换国内镜像源
sed -i '/^ImageRepository/ s|""|"registry.aliyuncs.com/google_containers"|' clusterfile

配置负载均衡

我这里的云平台提供 LB 服务，但我又不想让 SVC 使用 LoadBalancer，所以，我这里只配置了 80、443、6443 三个端口，其中 80、443 是给 Ingress Nginx 准备的，6443 是给 APIServer 做负载用的。

另外，如果你需要在集群外访问 APIServer，则需要把负载 IP 也添加到 CertSANs 列表中。

# CertSANs 添加 lb/vip
sed -i '/^  CertSANs/a\  - 192.168.20.253' clusterfile

自定义 CIDR

如上，我这里计划用 Cilium 作为 CNI 插件，看 clusterfile 配置可以知道 Sealos 部署的 k8s 集群，默认 PodSubnet 是 100.64.0.0/10，而 Cilium 对应的 clusterPoolIPv4PodCIDR 默认为 10.0.0.0/8，所以，我们需要自定义 cilium Operator 的 clusterPoolIPv4PodCIDR 也为 100.64.0.0/10。

cat > Kubefile <<EOF
FROM labring/cilium:v1.13.0

CMD ["cp opt/cilium /usr/bin/","cp opt/hubble /usr/bin/","cilium install --chart-directory charts/cilium --helm-set kubeProxyReplacement=strict,k8sServiceHost=apiserver.cluster.local,k8sServicePort=6443,ipam.operator.clusterPoolIPv4PodCIDR=100.64.0.0/10"]
EOF

sealos build -t labring/cilium:v1.13.0-amd64 --platform linux/amd64 -f Kubefile .

安装 Kubernetes 集群

# 部署 Kubernetes
sealos apply -f clusterfile

至此，我们姑且认为您已经按照上述步骤完成 k8s 集群的安装。

部署 LongHorn 云原生分布式存储

磁盘挂载

如上，我这里规划三个 master 节点作为存储节点，并且磁盘都是 /dev/vdc

DISK=vdc
for i in `seq 1 3`;do cat <<-EOF | ssh 192.168.20.10$i;done
    echo "=========="
    hostname
    parted -s /dev/${DISK} mklabel gpt
    parted -s /dev/${DISK} mkpart p ext4 0 100%
    mkfs.ext4 -F /dev/${DISK}1
    sed -i '/longhorn/d' /etc/fstab
    echo "/dev/${DISK}1 /var/lib/longhorn ext4 defaults 0 0" >> /etc/fstab
    mkdir /var/lib/longhorn
    mount -a
    df -h /var/lib/longhorn
EOF

安装依赖

安装 iSCSI

cat <<-'EOF' | kubectl apply -f -
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: longhorn-iscsi-installation
  # namespace: longhorn-system
  labels:
    app: longhorn-iscsi-installation
  annotations:
    command: &cmd OS=$(grep "ID_LIKE" /etc/os-release | cut -d '=' -f 2); if [[ "${OS}" == *"debian"* ]]; then sudo apt-get update -q -y && sudo apt-get install -q -y open-iscsi && sudo systemctl -q enable iscsid && sudo systemctl start iscsid; elif [[ "${OS}" == *"suse"* ]]; then sudo zypper --gpg-auto-import-keys -q refresh && sudo zypper --gpg-auto-import-keys -q install -y open-iscsi && sudo systemctl -q enable iscsid && sudo systemctl start iscsid; else sudo yum makecache -q -y && sudo yum --setopt=tsflags=noscripts install -q -y iscsi-initiator-utils && echo "InitiatorName=$(/sbin/iscsi-iname)" > /etc/iscsi/initiatorname.iscsi && sudo systemctl -q enable iscsid && sudo systemctl start iscsid; fi && if [ $? -eq 0 ]; then echo "iscsi install successfully"; else echo "iscsi install failed error code $?"; fi
spec:
  selector:
    matchLabels:
      app: longhorn-iscsi-installation
  template:
    metadata:
      labels:
        app: longhorn-iscsi-installation
    spec:
      tolerations:
      - key: node-role.kubernetes.io/control-plane
        operator: Exists
        effect: NoSchedule
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule
      hostNetwork: true
      hostPID: true
      initContainers:
      - name: iscsi-installation
        command:
          - nsenter
          - --mount=/proc/1/ns/mnt
          - --
          - bash
          - -c
          - *cmd
        image: hub.silkdo.com/library/alpine:3.12
        securityContext:
          privileged: true
      containers:
      - name: sleep
        image: registry.cn-beijing.aliyuncs.com/kubesphereio/pause:3.5
  updateStrategy:
    type: RollingUpdate
EOF

安装 NFSv4 客户端

cat <<-'EOF' | kubectl apply -f -
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: longhorn-nfs-installation
  # namespace: longhorn-system
  labels:
    app: longhorn-nfs-installation
  annotations:
    command: &cmd OS=$(grep "ID_LIKE" /etc/os-release | cut -d '=' -f 2); if [[ "${OS}" == *"debian"* ]]; then sudo apt-get update -q -y && sudo apt-get install -q -y nfs-common; elif [[ "${OS}" == *"suse"* ]]; then sudo zypper --gpg-auto-import-keys -q refresh && sudo zypper --gpg-auto-import-keys -q install -y nfs-client; else sudo yum makecache -q -y && sudo yum --setopt=tsflags=noscripts install -q -y nfs-utils; fi && if [ $? -eq 0 ]; then echo "nfs install successfully"; else echo "nfs install failed error code $?"; fi
spec:
  selector:
    matchLabels:
      app: longhorn-nfs-installation
  template:
    metadata:
      labels:
        app: longhorn-nfs-installation
    spec:
      tolerations:
      - key: node-role.kubernetes.io/control-plane
        operator: Exists
        effect: NoSchedule
      - key: node-role.kubernetes.io/master
        operator: Exists
        effect: NoSchedule
      hostNetwork: true
      hostPID: true
      initContainers:
      - name: nfs-installation
        command:
          - nsenter
          - --mount=/proc/1/ns/mnt
          - --
          - bash
          - -c
          - *cmd
        image: hub.silkdo.com/library/alpine:3.12
        securityContext:
          privileged: true
      containers:
      - name: sleep
        image: registry.cn-beijing.aliyuncs.com/kubesphereio/pause:3.5
  updateStrategy:
    type: RollingUpdate
EOF

添加标签

kubectl label nodes silkdo-1 node.longhorn.io/create-default-disk=true
kubectl label nodes silkdo-2 node.longhorn.io/create-default-disk=true
kubectl label nodes silkdo-3 node.longhorn.io/create-default-disk=true

在标记节点上创建默认磁盘

部署 LongHorn

helm repo add longhorn https://charts.longhorn.io && helm repo update longhorn


cat <<-'EOF' | helm -n longhorn-system upgrade -i longhorn longhorn/longhorn --version v1.4.3 --create-namespace -f - 
image:
  longhorn:
    engine:
      repository: uhub.service.ucloud.cn/silkdo/longhornio/longhorn-engine
    manager:
      repository: uhub.service.ucloud.cn/silkdo/longhornio/longhorn-manager
    ui:
      repository: uhub.service.ucloud.cn/silkdo/longhornio/longhorn-ui
    instanceManager:
      repository: uhub.service.ucloud.cn/silkdo/longhornio/longhorn-instance-manager
    shareManager:
      repository: uhub.service.ucloud.cn/silkdo/longhornio/longhorn-share-manager
    backingImageManager:
      repository: uhub.service.ucloud.cn/silkdo/longhornio/backing-image-manager
    supportBundleKit:
      repository: uhub.service.ucloud.cn/silkdo/longhornio/support-bundle-kit
  csi:
    attacher:
      repository: uhub.service.ucloud.cn/silkdo/longhornio/csi-attacher
    provisioner:
      repository: uhub.service.ucloud.cn/silkdo/longhornio/csi-provisioner
    nodeDriverRegistrar:
      repository: uhub.service.ucloud.cn/silkdo/longhornio/csi-node-driver-registrar
    resizer:
      repository: uhub.service.ucloud.cn/silkdo/longhornio/csi-resizer
    snapshotter:
      repository: uhub.service.ucloud.cn/silkdo/longhornio/csi-snapshotter
    livenessProbe:
      repository: uhub.service.ucloud.cn/silkdo/longhornio/livenessprobe

service:
  ui:
    type: NodePort
    nodePort: 30890

defaultSettings:
  allowRecurringJobWhileVolumeDetached: true
  createDefaultDiskLabeledNodes: true
  replicaAutoBalance: "best-effort"
  taintToleration: "node-role.kubernetes.io/control-plane:NoSchedule;node-role.kubernetes.io/master:NoSchedule"
  priorityClass: "high-priority"
  nodeDownPodDeletionPolicy: "delete-both-statefulset-and-deployment-pod"
  concurrentAutomaticEngineUpgradePerNodeLimit: "5"

longhornManager:
  tolerations:
  - key: node-role.kubernetes.io/control-plane
    operator: Exists
    effect: NoSchedule
  - key: node-role.kubernetes.io/master
    operator: Exists
    effect: NoSchedule
longhornDriver:
  tolerations:
  - key: node-role.kubernetes.io/control-plane
    operator: Exists
    effect: NoSchedule
  - key: node-role.kubernetes.io/master
    operator: Exists
    effect: NoSchedule

longhornUI:
  tolerations:
  - key: node-role.kubernetes.io/control-plane
    operator: Exists
    effect: NoSchedule
  - key: node-role.kubernetes.io/master
    operator: Exists
    effect: NoSchedule
  nodeSelector:
    node.longhorn.io/create-default-disk: "true"

longhornConversionWebhook:
  tolerations:
  - key: node-role.kubernetes.io/control-plane
    operator: Exists
    effect: NoSchedule
  - key: node-role.kubernetes.io/master
    operator: Exists
    effect: NoSchedule
  nodeSelector:
    node.longhorn.io/create-default-disk: "true"

longhornAdmissionWebhook:
  tolerations:
  - key: node-role.kubernetes.io/control-plane
    operator: Exists
    effect: NoSchedule
  - key: node-role.kubernetes.io/master
    operator: Exists
    effect: NoSchedule
  nodeSelector:
    node.longhorn.io/create-default-disk: "true"

longhornRecoveryBackend:
  tolerations:
  - key: node-role.kubernetes.io/control-plane
    operator: Exists
    effect: NoSchedule
  - key: node-role.kubernetes.io/master
    operator: Exists
    effect: NoSchedule
  nodeSelector:
    node.longhorn.io/create-default-disk: "true"

enablePSP: false
EOF

验证

cat <<-EOF | kubectl apply -f -
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: rwo
spec:
  storageClassName: longhorn
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

---
kind: Pod
apiVersion: v1
metadata:
  name: rwo
spec:
  containers:
  - name: busybox
    image: busybox
    command:
    - sleep
    - "3600"
    volumeMounts:
    - name: rwo
      mountPath: "/pv-data"
      readOnly: false
  volumes:
  - name: rwo
    persistentVolumeClaim:
      claimName: rwo

---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: rwx
spec:
  storageClassName: longhorn
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi

---
kind: Pod
apiVersion: v1
metadata:
  name: rwx
spec:
  containers:
  - name: busybox
    image: busybox
    command:
    - sleep
    - "3600"
    volumeMounts:
    - name: rwx
      mountPath: "/pv-data"
      readOnly: false
  volumes:
  - name: rwx
    persistentVolumeClaim:
      claimName: rwx
EOF

至此，我们姑且认为您已经按照上述步骤完成 LongHorn 的安装。

部署 Ingress Nginx 控制器

前面说过，我不打算使用 Loadbalancer，因为它的代价实在是太高了，但也不想让用户访问时多加一个端口，所以，准备使用 hostNetwork 参数暴露宿主机上的 80 和 443 端口，配合 pod 亲和性和反亲和性，将 Ingress Nginx 的 pod 固定在三个 master 节点上，再利用上述负载均衡让 Ingress 可以暴露在互联网。

VERSION=4.7.1

curl -L https://ghproxy.com/https://github.com/kubernetes/ingress-nginx/releases/download/helm-chart-${VERSION}/ingress-nginx-${VERSION}.tgz -o ~/.cache/helm/repository/ingress-nginx-${VERSION}.tgz

cat << EOF | helm -n ingress-nginx upgrade -i ingress-nginx ~/.cache/helm/repository/ingress-nginx-${VERSION}.tgz --create-namespace -f -
controller:
  image:
    registry: uhub.service.ucloud.cn/silkdo/registry.k8s.io
    digest: 
    digestChroot: 
  config:
  dnsPolicy: ClusterFirstWithHostNet
  reportNodeInternalIp: true
  watchIngressWithoutClass: true
  hostNetwork: true
  hostPort:
    enabled: false
    ports:
      http: 80
      https: 443
  ingressClassResource:
    default: true
  publishService:
    enabled: false
  kind: Deployment
  updateStrategy:
    rollingUpdate:
      maxSurge: 0
      maxUnavailable: 1
    type: RollingUpdate
  tolerations:
    - key: node-role.kubernetes.io/control-plane
      operator: Exists
      effect: NoSchedule
    - key: node-role.kubernetes.io/master
      operator: Exists
      effect: NoSchedule
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: node-role.kubernetes.io/control-plane
            operator: Exists
  replicaCount: 3
  service:
    enabled: true
  opentelemetry:
    enabled: true
    image: uhub.service.ucloud.cn/silkdo/registry.k8s.io/ingress-nginx/opentelemetry:v20230527
    containerSecurityContext:
      allowPrivilegeEscalation: false
  admissionWebhooks:
    enabled: true
    patch:
      enabled: true
      image:
        registry: uhub.service.ucloud.cn/silkdo/registry.k8s.io
        digest: 
      tolerations:
        - key: node-role.kubernetes.io/control-plane
          operator: Exists
          effect: NoSchedule
        - key: node-role.kubernetes.io/master
          operator: Exists
          effect: NoSchedule
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: node-role.kubernetes.io/control-plane
                operator: Exists
defaultBackend:
  enabled: true
  name: defaultbackend
  image:
    registry: uhub.service.ucloud.cn/silkdo/registry.k8s.io
    digest: 
  updateStrategy:
    rollingUpdate:
      maxSurge: 0
      maxUnavailable: 1
    type: RollingUpdate
  tolerations:
    - key: node-role.kubernetes.io/control-plane
      operator: Exists
      effect: NoSchedule
    - key: node-role.kubernetes.io/master
      operator: Exists
      effect: NoSchedule
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: node-role.kubernetes.io/control-plane
            operator: Exists
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app.kubernetes.io/component
            operator: In
            values:
            - default-backend
        topologyKey: kubernetes.io/hostname
  replicaCount: 3
EOF

部署 CertManager 控制器

单纯为了获取免费 SSL 证书

VERSION=1.12.2

curl -L https://charts.jetstack.io/charts/cert-manager-v${VERSION}.tgz -o ~/.cache/helm/repository/cert-manager-v${VERSION}.tgz

cat <<-'EOF' | helm -n cert-manager upgrade -i cert-manager ~/.cache/helm/repository/cert-manager-v${VERSION}.tgz --create-namespace -f -
installCRDs: true

strategy:
  type: RollingUpdate
  rollingUpdate:
    maxSurge: 0
    maxUnavailable: 1

image:
  repository: uhub.service.ucloud.cn/silkdo/quay.io/jetstack/cert-manager-controller

extraEnv:
  TZ: Asia/Shanghai
tolerations:
- key: node-role.kubernetes.io/control-plane
  operator: Exists
  effect: NoSchedule
- key: node-role.kubernetes.io/master
  operator: Exists
  effect: NoSchedule
affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: node-role.kubernetes.io/control-plane
          operator: Exists

prometheus:
  enabled: true
  servicemonitor:
    enabled: true
    endpointAdditionalProperties:
      relabelings:
      - replacement: base
        targetLabel: group

webhook:
  image:
    repository: uhub.service.ucloud.cn/silkdo/quay.io/jetstack/cert-manager-webhook

cainjector:
  image:
    repository: uhub.service.ucloud.cn/silkdo/quay.io/jetstack/cert-manager-cainjector

acmesolver:
  image:
    repository: uhub.service.ucloud.cn/silkdo/quay.io/jetstack/cert-manager-acmesolver

startupapicheck:
  image:
    repository: uhub.service.ucloud.cn/silkdo/quay.io/jetstack/cert-manager-ctl
EOF

部署 SafeLine

Helm 部署 SafeLine

helm repo add jangrui https://github.com/jangrui/SafeLine --force-update
helm -n safeline upgrade -i safeline jangrui/safeline --create-namespace

创建一个 HTTP01 类型的 ACME 发行者

cat <<-'EOF' | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: waf.silkdo.com
  namespace: safeline
spec:
  acme:
    email: [email protected]
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: waf.silkdo.com.tls
    solvers:
    - http01:
        ingress:
          class: nginx
EOF

创建 Ingress

cat <<-'EOF' | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: waf.silkdo.com
  namespace: safeline
  annotations:
    cert-manager.io/issuer: "waf.silkdo.com"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/service-upstreamtrue: "true"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
  ingressClassName: nginx
  rules:
  - host: waf.silkdo.com
    http:
      paths:
      - path: /
        pathType: ImplementationSpecific
        backend:
          service:
            name: safeline-mgt-api
            port: 
              number: 1443
  tls:
  - hosts:
    - waf.silkdo.com
    secretName: waf.silkdo.com.tls
EOF

此处省略 DNS 解析过程，默认认为您的域名已经做好 DNS 解析。

至此，您可以通过域名访问直接访问 SafeLine 的管理后台；另一种方法，也可以通过 NodePort 的形式去访问。

配置 WAF

既然 SafeLine 已经安装好了，那么把我们的域名交给 WAF 去防护肯定是首选了。

添加人机验证

这里单纯为了一眼看出下面配置的网站是经过 WAF 防护的。

配置防护站点

我们这里直接把刚才的 waf.silkdo.com 作为要防护站点。

刚才我们已经给 SafeLine 添加了一个 Ingress，但后端 SVC 用的是 safeline-mgt-api，而现在，我们需要 Ingress 也经过 WAF 防护，此时只需要稍作修改，把 Ingress 的后端 SVC 改成 safeline-tengine 即可。

cat <<-'EOF' | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: waf.silkdo.com
  namespace: safeline
  annotations:
    cert-manager.io/issuer: "waf.silkdo.com"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/service-upstreamtrue: "true"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
  ingressClassName: nginx
  rules:
  - host: waf.silkdo.com
    http:
      paths:
      - path: /
        pathType: ImplementationSpecific
        backend:
          service:
            name: safeline-tengine
            port: 
              number: 80
  tls:
  - hosts:
    - waf.silkdo.com
    secretName: waf.silkdo.com.tls
EOF

别忘了 SVC 端口也需要跟着变动。

现在，打开一个无痕窗口，访问你的域名，出现上图，说明你的 WAF 成功生效，至此，Safeline 落地到云原生环境完结。

Lorna0 · 2023-09-06T07:27:03Z

Lorna0
Sep 6, 2023
Maintainer

好内容放 issue 里容易被淹没，我们把仓库的 discussion 开了给师傅放到这边来。

0 replies

junlintianxiazhifulinzhongguo · 2023-12-29T02:17:55Z

junlintianxiazhifulinzhongguo
Dec 29, 2023

我参考这篇文章，部署safeline 已经成功，在配置防护站点后，访问报 502，请问教程是否还有没有记录到的配置

1 reply

jangrui Jan 3, 2024
Author

最新版的 chart 包是 3.16.1，我测试是没有问题的。

可以提供日志看看

charnet1019 · 2024-06-03T08:13:49Z

charnet1019
Jun 3, 2024

按这个配置代理waf自身是可以正常访问的，但代理其他命名空间下的web服务访问不了；
但通过将safeline-tengine设置为loadbalancer后再设置代理就可以访问了(我的ingress controller是通过loadbalancer配置的)

基本流程:
loadbalancer ---> safeline-tengine ---> backend service

但这样配置后等于没有和ingress配合使用，不知道你那边是否也是这种配置的？

1 reply

jangrui Jun 3, 2024
Author

其他 ns 下的 ing 就没有用了，需要全转到 safeline ns 下。

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/proxy-body-size: 4096m
    nginx.ingress.kubernetes.io/service-upstreamtrue: "true"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
  name: xxx.xxx.xxx
  namespace: safeline
spec:
  ingressClassName: nginx
  rules:
  - host: xxx.xxx.xxx
    http:
      paths:
      - backend:
          service:
            name: safeline-tengine   # 所有 ns 中的 web 服务都可以让走 safeline namespace 中的 tengine 服务
            port:
              number: 80
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - xxx.xxx.xxx
    secretName: sixxx.xxx.tls

配置其他 ns 下的上游服务器示例：https://harbor.harbor-system.svc.cluster.local:80

charnet1019 · 2024-06-04T08:39:57Z

charnet1019
Jun 4, 2024

看了下官方文档，ingress controller通过安装safeline插件可以实现代理其他ns下域名，但这种方式有些功能是有限制无法使用的，如人机验证、身份认证

基本流程:
client -> lb -> ingress controller(安装有safeline插件) -> safeline-detector -> backend service

注意事项:

# safeline.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: safeline
  namespace: ingress-nginx
data:
    host: "detector_host" # 雷池检测引擎的地址, 此处需要填写完整域名，否则会报错，如 safeline-detector.safeline.svc.cluster.local
    port: "8000"

1 reply

jangrui Jun 4, 2024
Author

我艹，你不说我都没注意到已经出 ingress-nginx 的插件了。
你可以试试全新安装集成方式

charnet1019 · 2024-06-05T00:34:04Z

charnet1019
Jun 5, 2024

我已经测试过了，是可以代理其他ns下的ingress

0 replies

boonhanchng · 2024-07-19T10:21:25Z

boonhanchng
Jul 19, 2024

根据教程部署完之后，反代可以从域名到safeline-tengine 但是人机验证封锁都无效，只有反代功能有生效

3 replies

jangrui Jul 20, 2024
Author

我这边好着呢。
可以提供一些详细信息，研究研究。

boonhanchng Jul 22, 2024

师傅您的镜像仓库的镜像是不是和 docker io 的不一样？
之前因为我在马来西亚拉取不到您的 values里面的仓库我转向 docker io
今天我尝试一下用values.yml里面的仓库拉取的，可以正常运行

jangrui Jul 23, 2024
Author

对，tengine 和 detector 这两个服务是定制镜像，也就是按照官文把 socket 方式改为 http，存放在个人仓库，其他镜像默认使用雷池官方华为镜像仓库 swr.cn-east-3.myhuaweicloud.com/chaitin-safeline

Jerry12228 · 2024-09-16T13:27:47Z

Jerry12228
Sep 16, 2024

emmm小白路过，请问我能否认为这个教程是自行构建雷池镜像的过程？（我正在尝试自行构建镜像）

1 reply

jangrui Sep 16, 2024
Author

不是

rex7036 · 2024-09-22T10:58:48Z

rex7036
Sep 22, 2024

大佬，6.9.1版本无法上传证书，是不是也是因为tengine 和 detector 这两个服务是的镜像没有修改成http的原因，我用的是官方的镜像

3 replies

rex7036 Sep 22, 2024

应该不是这个原因，改了你定制过的镜像之后还是有这个问题

jangrui Sep 23, 2024
Author

卸载后，保留数据库 pvc，把其余 pvc 删掉，重新部署一遍。

helm -n safeline un safeline
kubectl -n safeline get pvc -o custom-columns=NAME:.metadata.name | grep ^safeline | xargs -I {} kubectl -n safeline delete pvc {}

helm repo add jangrui https://github.com/jangrui/SafeLine --force-update
helm -n safeline upgrade -i safeline jangrui/safeline --create-namespace

如果遇到 safeline-tengine 缺少 error.log 而起不来，需要手动创建。例如：

mkdir safeline-safeline-logs-pvc-2ca226e1-0d23-4fb5-aa35-d4cde11b8001/nginx
kubectl -n safeline delete po -l component=tengine

jangrui Sep 27, 2024
Author

6.10.2+ 已修复

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

手把手教你把 SafeLine 落地到云原生环境 #289

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 8 comments 10 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

手把手教你把 SafeLine 落地到云原生环境 #289

反馈内容

手把手教你把 SafeLine 落地到云原生环境

核心一览

SealOS 一键构建 k8s 环境

环境信息

安装 Sealos

配置clusterfile

配置负载均衡

自定义 CIDR

安装 Kubernetes 集群

部署 LongHorn 云原生分布式存储

磁盘挂载

安装依赖

安装 iSCSI

安装 NFSv4 客户端

添加标签

部署 LongHorn

验证

部署 Ingress Nginx 控制器

部署 CertManager 控制器

部署 SafeLine

Helm 部署 SafeLine

创建一个 HTTP01 类型的 ACME 发行者

创建 Ingress

配置 WAF

添加人机验证

配置防护站点

Replies: 8 comments · 10 replies

Lorna0 Sep 6, 2023 Maintainer

jangrui Jan 3, 2024 Author

jangrui Jun 3, 2024 Author

jangrui Jun 4, 2024 Author

jangrui Jul 20, 2024 Author

jangrui Jul 23, 2024 Author

jangrui Sep 16, 2024 Author

jangrui Sep 23, 2024 Author

jangrui Sep 27, 2024 Author

Replies: 8 comments 10 replies

Lorna0
Sep 6, 2023
Maintainer

jangrui Jan 3, 2024
Author

jangrui Jun 3, 2024
Author

jangrui Jun 4, 2024
Author

jangrui Jul 20, 2024
Author

jangrui Jul 23, 2024
Author

jangrui Sep 16, 2024
Author

jangrui Sep 23, 2024
Author

jangrui Sep 27, 2024
Author