|
@@ -0,0 +1,539 @@
|
|
|
+# 本地环境部署 K8s Nginx ingress
|
|
|
+
|
|
|
+[TOC]
|
|
|
+
|
|
|
+## 开始之前
|
|
|
+
|
|
|
+国内可能会无法连接一些网站,需要准备好代理,然后运行
|
|
|
+
|
|
|
+```sh
|
|
|
+export http_proxy=192.168.11.61:11714
|
|
|
+export https_proxy=192.168.11.61:11714
|
|
|
+export no_proxy=.cn,.aliyun.com,.aliyuncs.com,.163.com,.baiduce.com,.qiniu.com,.daocloud.io,127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,100.64.0.0/10,169.254.0.0/16
|
|
|
+```
|
|
|
+
|
|
|
+请将`192.168.11.61:11714`修改成你的代理服务器地址和端口。
|
|
|
+
|
|
|
+## 1. 部署并曝露 Nginx Ingress 的 80 和 443 端口
|
|
|
+
|
|
|
+**云厂商方案**:使用 LoadBalancer 曝露 80 和 443 端口,并通过 DNS 配置将流量指向负载均衡器。
|
|
|
+
|
|
|
+**在裸机环境中**,有几种方式可以暴露 Nginx Ingress Controller 的 80 和 443 端口:
|
|
|
+
|
|
|
+
|
|
|
+1. [通过集群外的负载均衡软件](./expose/expose-via-external-lb.md)
|
|
|
+2. [通过 MetalLB 组件](./expose/expose-via-metallb.md)
|
|
|
+3. [通过 HostNetwork 模式,将 Nginx-Ingress-Controller 部署成 DaemonSet](./expose/expose-via-hostnetwork.md)
|
|
|
+4. [修改 Kubelet 将 NodePort 地址范围修改成 1-32767 端口](./expose/expose-via-kubelet-nodeport.md)
|
|
|
+
|
|
|
+可以根据具体需求选择一种方式。可以重点关注一下 MetalLB 组件。
|
|
|
+
|
|
|
+## 2. 部署 cert-manager
|
|
|
+
|
|
|
+**请将下文的 yourdomain.com 替换成真实的域名。**
|
|
|
+
|
|
|
+### 2.1 [安装 Cert-Manager](https://cert-manager.io/docs/installation/helm/)
|
|
|
+
|
|
|
+1. **添加 Helm 仓库并安装 cert-manager**:
|
|
|
+
|
|
|
+ ```sh
|
|
|
+ helm repo add jetstack https://charts.jetstack.io
|
|
|
+ helm repo update
|
|
|
+
|
|
|
+ # 安装最新版本:
|
|
|
+ helm install cert-manager jetstack/cert-manager --namespace cert-manager --create-namespace --set installCRDs=true
|
|
|
+ # 指定安装版本:
|
|
|
+ helm install cert-manager jetstack/cert-manager \
|
|
|
+ --namespace cert-manager \
|
|
|
+ --create-namespace \
|
|
|
+ --version v1.15.2 \
|
|
|
+ --set installCRDs=true
|
|
|
+ ```
|
|
|
+
|
|
|
+2. **验证 cert-manager 是否成功部署**:
|
|
|
+
|
|
|
+ ```sh
|
|
|
+ kubectl get pods --namespace cert-manager
|
|
|
+ ```
|
|
|
+
|
|
|
+### 2.1 [部署 cert-manager-alidns-webhook](https://github.com/DEVmachine-fr/cert-manager-alidns-webhook)
|
|
|
+
|
|
|
+本地环境通常无法通过公网访问,无法通过http的方式让CA验证域名所有权。需要通过DNS的方式让CA验证域名所有权。下面以阿里云DNS为例。[其他的DNS resolver](https://cert-manager.io/docs/configuration/acme/dns01/#webhook)。
|
|
|
+
|
|
|
+1. **添加 cert-manager-webhook-alidns Helm 仓库**:
|
|
|
+
|
|
|
+ ```sh
|
|
|
+ helm repo add cert-manager-alidns-webhook https://devmachine-fr.github.io/cert-manager-alidns-webhook
|
|
|
+ helm repo update
|
|
|
+ ```
|
|
|
+
|
|
|
+2. **安装 cert-manager-alidns-webhook**:
|
|
|
+
|
|
|
+ ```sh
|
|
|
+ helm install alidns-webhook cert-manager-alidns-webhook/alidns-webhook \
|
|
|
+ --namespace cert-manager \
|
|
|
+ --set groupName=acme.yourdomain.com
|
|
|
+ ```
|
|
|
+
|
|
|
+ **配置说明**:
|
|
|
+
|
|
|
+ - `groupName` 应该是唯一的,建议使用你的域名或类似标识。
|
|
|
+
|
|
|
+3. **创建阿里云AKSK secret:**
|
|
|
+
|
|
|
+ 在阿里云控制台创建一个RAM用户,并赋予DNSFullAccess 策略。在这个用户下创建access key。然后在k8s 创建secret:
|
|
|
+
|
|
|
+ ```sh
|
|
|
+ kubectl -n cert-manager create secret generic alidns-secrets --from-literal="access-token=yourtoken" --from-literal="secret-key=yoursecretkey"
|
|
|
+ ```
|
|
|
+
|
|
|
+ 确保将 `yourtoken`和`yoursecretkey` 修改成从阿里云控制台获得的 AK/SK。
|
|
|
+
|
|
|
+## 3. 配置 ClusterIssuer 和 Issuer
|
|
|
+
|
|
|
+**请将下文的 yourdomain.com 替换成真实的域名。**
|
|
|
+
|
|
|
+### 步骤 1:创建 Let's Encrypt ClusterIssuer
|
|
|
+
|
|
|
+1. **创建 ClusterIssuer 文件:**
|
|
|
+
|
|
|
+ ```yaml
|
|
|
+ apiVersion: cert-manager.io/v1
|
|
|
+ kind: ClusterIssuer
|
|
|
+ metadata:
|
|
|
+ name: letsencrypt-staging
|
|
|
+ spec:
|
|
|
+ acme:
|
|
|
+ server: https://acme-staging-v02.api.letsencrypt.org/directory
|
|
|
+ email: your-email@example.com
|
|
|
+ privateKeySecretRef:
|
|
|
+ name: letsencrypt-staging
|
|
|
+ solvers:
|
|
|
+ - dns01:
|
|
|
+ webhook:
|
|
|
+ groupName: acme.yourdomain.com
|
|
|
+ solverName: alidns-solver
|
|
|
+ config:
|
|
|
+ regionId: cn-beijing
|
|
|
+ accessKeySecretRef:
|
|
|
+ name: alidns-secret
|
|
|
+ key: access-key-id
|
|
|
+ secretKeySecretRef:
|
|
|
+ name: alidns-secret
|
|
|
+ key: access-key-secret
|
|
|
+ selector:
|
|
|
+ dnsZones:
|
|
|
+ - yourdomain.com
|
|
|
+ ```
|
|
|
+
|
|
|
+ ```yaml
|
|
|
+ apiVersion: cert-manager.io/v1
|
|
|
+ kind: ClusterIssuer
|
|
|
+ metadata:
|
|
|
+ name: letsencrypt-prod
|
|
|
+ spec:
|
|
|
+ acme:
|
|
|
+ server: https://acme-v02.api.letsencrypt.org/directory
|
|
|
+ email: your-email@example.com
|
|
|
+ privateKeySecretRef:
|
|
|
+ name: letsencrypt-prod
|
|
|
+ solvers:
|
|
|
+ - dns01:
|
|
|
+ webhook:
|
|
|
+ groupName: acme.yourdomain.com
|
|
|
+ solverName: alidns-solver
|
|
|
+ config:
|
|
|
+ regionId: cn-beijing
|
|
|
+ accessKeySecretRef:
|
|
|
+ name: alidns-secret
|
|
|
+ key: access-key-id
|
|
|
+ secretKeySecretRef:
|
|
|
+ name: alidns-secret
|
|
|
+ key: access-key-secret
|
|
|
+ selector:
|
|
|
+ dnsZones:
|
|
|
+ - yourdomain.com
|
|
|
+
|
|
|
+ ```
|
|
|
+
|
|
|
+2. **应用配置**:
|
|
|
+
|
|
|
+ ```sh
|
|
|
+ kubectl apply -f letsencrypt-staging.yaml
|
|
|
+ kubectl apply -f letsencrypt-prod.yaml
|
|
|
+ ```
|
|
|
+
|
|
|
+**配置说明**:
|
|
|
+
|
|
|
+- `groupName` 需要和之前helm指定的一致。
|
|
|
+- `email` 需要修改为你的邮箱。
|
|
|
+
|
|
|
+### 步骤 2:创建 Let's Encrypt Issuer
|
|
|
+
|
|
|
+1. **创建 Issuer 文件:**
|
|
|
+
|
|
|
+ ```yaml
|
|
|
+ apiVersion: cert-manager.io/v1
|
|
|
+ kind: Issuer
|
|
|
+ metadata:
|
|
|
+ name: letsencrypt-staging-issuer
|
|
|
+ namespace: your-namespace
|
|
|
+ spec:
|
|
|
+ acme:
|
|
|
+ server: https://acme-staging-v02.api.letsencrypt.org/directory
|
|
|
+ email: your-email@example.com
|
|
|
+ privateKeySecretRef:
|
|
|
+ name: letsencrypt-staging
|
|
|
+ solvers:
|
|
|
+ - dns01:
|
|
|
+ webhook:
|
|
|
+ groupName: acme.yourdomain.com
|
|
|
+ solverName: alidns-solver
|
|
|
+ config:
|
|
|
+ regionId: cn-beijing
|
|
|
+ accessKeySecretRef:
|
|
|
+ name: alidns-secret
|
|
|
+ key: access-key-id
|
|
|
+ secretKeySecretRef:
|
|
|
+ name: alidns-secret
|
|
|
+ key: access-key-secret
|
|
|
+ selector:
|
|
|
+ dnsZones:
|
|
|
+ - yourdomain.com
|
|
|
+ ```
|
|
|
+
|
|
|
+ ```yaml
|
|
|
+ apiVersion: cert-manager.io/v1
|
|
|
+ kind: Issuer
|
|
|
+ metadata:
|
|
|
+ name: letsencrypt-prod-issuer
|
|
|
+ namespace: your-namespace
|
|
|
+ spec:
|
|
|
+ acme:
|
|
|
+ server: https://acme-v02.api.letsencrypt.org/directory
|
|
|
+ email: your-email@example.com
|
|
|
+ privateKeySecretRef:
|
|
|
+ name: letsencrypt-prod
|
|
|
+ solvers:
|
|
|
+ - dns01:
|
|
|
+ webhook:
|
|
|
+ groupName: acme.yourdomain.com
|
|
|
+ solverName: alidns-solver
|
|
|
+ config:
|
|
|
+ regionId: cn-beijing
|
|
|
+ accessKeySecretRef:
|
|
|
+ name: alidns-secret
|
|
|
+ key: access-key-id
|
|
|
+ secretKeySecretRef:
|
|
|
+ name: alidns-secret
|
|
|
+ key: access-key-secret
|
|
|
+ selector:
|
|
|
+ dnsZones:
|
|
|
+ - yourdomain.com
|
|
|
+ ```
|
|
|
+
|
|
|
+2. **应用配置**:
|
|
|
+
|
|
|
+ ```sh
|
|
|
+ kubectl apply -f letsencrypt-staging-issuer.yaml
|
|
|
+ kubectl apply -f letsencrypt-prod-issuer.yaml
|
|
|
+ ```
|
|
|
+
|
|
|
+**配置说明**:
|
|
|
+
|
|
|
+- `groupName` 需要和之前helm指定的一致。
|
|
|
+- `email` 需要修改为你的邮箱。
|
|
|
+- `namespace` 需要和你的应用的 namespace 一致,否则无法找到 Issuer, 无法颁发证书。而之前的ClusterIssuer是Cluster级别的,不需要指定namespace。
|
|
|
+
|
|
|
+## 4. 部署测试应用并请求证书
|
|
|
+
|
|
|
+### 步骤 1:部署测试应用
|
|
|
+
|
|
|
+1. **创建测试应用的 Deployment 和 Service**:
|
|
|
+
|
|
|
+ ```yaml
|
|
|
+ apiVersion: apps/v1
|
|
|
+ kind: Deployment
|
|
|
+ metadata:
|
|
|
+ name: test-app
|
|
|
+ namespace: your-namespace
|
|
|
+ spec:
|
|
|
+ replicas: 1
|
|
|
+ selector:
|
|
|
+ matchLabels:
|
|
|
+ app: test-app
|
|
|
+ template:
|
|
|
+ metadata:
|
|
|
+ labels:
|
|
|
+ app: test-app
|
|
|
+ spec:
|
|
|
+ containers:
|
|
|
+ - name: test-app
|
|
|
+ image: nginx:alpine
|
|
|
+ ports:
|
|
|
+ - containerPort: 80
|
|
|
+ ```
|
|
|
+
|
|
|
+ ```yaml
|
|
|
+ apiVersion: v1
|
|
|
+ kind: Service
|
|
|
+ metadata:
|
|
|
+ name: test-app
|
|
|
+ namespace: your-namespace
|
|
|
+ spec:
|
|
|
+ selector:
|
|
|
+ app: test-app
|
|
|
+ ports:
|
|
|
+ - port: 80
|
|
|
+ targetPort: 80
|
|
|
+ ```
|
|
|
+
|
|
|
+2. **应用配置**:
|
|
|
+
|
|
|
+ ```yaml
|
|
|
+ kubectl apply -f test-app-deployment.yaml
|
|
|
+ kubectl apply -f test-app-service.yaml
|
|
|
+ ```
|
|
|
+
|
|
|
+### 步骤 2:创建 Ingress 资源并请求证书
|
|
|
+
|
|
|
+1. **创建 Ingress 资源**:
|
|
|
+
|
|
|
+ 使用 Let's Encrypt Staging 请求证书:
|
|
|
+
|
|
|
+ ```yaml
|
|
|
+ apiVersion: networking.k8s.io/v1
|
|
|
+ kind: Ingress
|
|
|
+ metadata:
|
|
|
+ name: test-app-ingress
|
|
|
+ namespace: your-namespace
|
|
|
+ annotations:
|
|
|
+ cert-manager.io/cluster-issuer: "letsencrypt-staging"
|
|
|
+ spec:
|
|
|
+ rules:
|
|
|
+ - host: ingress-test.yourdomain.com
|
|
|
+ http:
|
|
|
+ paths:
|
|
|
+ - path: /
|
|
|
+ pathType: Prefix
|
|
|
+ backend:
|
|
|
+ service:
|
|
|
+ name: test-app
|
|
|
+ port:
|
|
|
+ number: 80
|
|
|
+ tls:
|
|
|
+ - hosts:
|
|
|
+ - ingress-test.yourdomain.com
|
|
|
+ secretName: test-app-tls
|
|
|
+ ```
|
|
|
+
|
|
|
+2. **应用配置**:
|
|
|
+
|
|
|
+ ```sh
|
|
|
+ kubectl apply -f test-app-ingress.yaml
|
|
|
+ ```
|
|
|
+
|
|
|
+3. **验证证书颁发情况**:
|
|
|
+
|
|
|
+ 查看 Ingress 的证书是否成功颁发:
|
|
|
+
|
|
|
+ ```sh
|
|
|
+ kubectl describe certificate test-app-tls -n your-namespace
|
|
|
+ ```
|
|
|
+
|
|
|
+4. **测试从浏览器访问**:
|
|
|
+
|
|
|
+ 先在阿里云DNS控制台创建DNS记录。然后在本地浏览器访问 https://ingress-test.yourdomain.com
|
|
|
+
|
|
|
+### 步骤 3:切换到 Let's Encrypt Production
|
|
|
+
|
|
|
+1. **更新 Ingress 资源**:
|
|
|
+
|
|
|
+ 将 `cert-manager.io/cluster-issuer` 注解从 `letsencrypt-staging` 更改为 `letsencrypt-prod`:
|
|
|
+
|
|
|
+ ```yaml
|
|
|
+ apiVersion: networking.k8s.io/v1
|
|
|
+ kind: Ingress
|
|
|
+ metadata:
|
|
|
+ name: test-app-ingress
|
|
|
+ namespace: your-namespace
|
|
|
+ annotations:
|
|
|
+ cert-manager.io/cluster-issuer: "letsencrypt-prod"
|
|
|
+ spec:
|
|
|
+ rules:
|
|
|
+ - host: ingress-test.yourdomain.com
|
|
|
+ http:
|
|
|
+ paths:
|
|
|
+ - path: /
|
|
|
+ pathType: Prefix
|
|
|
+ backend:
|
|
|
+ service:
|
|
|
+ name: test-app
|
|
|
+ port:
|
|
|
+ number: 80
|
|
|
+ tls:
|
|
|
+ - hosts:
|
|
|
+ - ingress-test.yourdomain.com
|
|
|
+ secretName: test-app-tls
|
|
|
+ ```
|
|
|
+
|
|
|
+2. **应用更新**:
|
|
|
+
|
|
|
+ ```sh
|
|
|
+ kubectl apply -f test-app-ingress.yaml
|
|
|
+ ```
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+### 步骤4. 使用 ECC 证书并配置密钥大小和轮换策略
|
|
|
+
|
|
|
+**步骤 1:指定颁发 ECC 证书**
|
|
|
+
|
|
|
+1. **修改 Ingress 或 Certificate 资源的注解**:
|
|
|
+
|
|
|
+ [注解文档](https://cert-manager.io/docs/usage/ingress/#supported-annotations)
|
|
|
+
|
|
|
+ 添加以下注解来指定 ECC 证书并设置密钥大小和轮换策略:
|
|
|
+
|
|
|
+ ```yaml
|
|
|
+ apiVersion: networking.k8s.io/v1
|
|
|
+ kind: Ingress
|
|
|
+ metadata:
|
|
|
+ name: test-app-ingress
|
|
|
+ namespace: your-namespace
|
|
|
+ annotations:
|
|
|
+ cert-manager.io/cluster-issuer: "letsencrypt-prod"
|
|
|
+ cert-manager.io/private-key-algorithm: "ECDSA"
|
|
|
+ cert-manager.io/private-key-size: "256"
|
|
|
+ cert-manager.io/private-key-rotation-policy: "Always"
|
|
|
+ spec:
|
|
|
+ rules:
|
|
|
+ - host: ingress-test.yourdomain.com
|
|
|
+ http:
|
|
|
+ paths:
|
|
|
+ - path: /
|
|
|
+ pathType: Prefix
|
|
|
+ backend:
|
|
|
+ service:
|
|
|
+ name: test-app
|
|
|
+ port:
|
|
|
+ number: 80
|
|
|
+ tls:
|
|
|
+ - hosts:
|
|
|
+ - ingress-test.yourdomain.com
|
|
|
+ secretName: test-app-tls
|
|
|
+ ```
|
|
|
+
|
|
|
+**配置说明**:
|
|
|
+
|
|
|
+- `private-key-size: "256"`:指定 ECC 密钥的大小为 256 位。常用的 ECC 密钥大小有 256、384 和 521 位,256 位通常足够提供高强度的安全性。
|
|
|
+- `private-key-rotation-policy: "Always"`:启用私钥轮换策略,即每次证书更新时都生成新的私钥。这种策略有助于提高安全性,因为它减少了私钥泄露或被攻击的风险。
|
|
|
+
|
|
|
+1. **应用更新**:
|
|
|
+
|
|
|
+ ```sh
|
|
|
+ kubectl apply -f test-app-ingress.yaml
|
|
|
+ ```
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+## 5. 总结
|
|
|
+
|
|
|
+- **部署并曝露 Nginx Ingress 的 80 和 443 端口**
|
|
|
+ - **云厂商方案**:使用 LoadBalancer 曝露 80 和 443 端口,并通过 DNS 配置将流量指向负载均衡器。
|
|
|
+
|
|
|
+ - **裸机环境方案:**
|
|
|
+ - **负载均衡软件**:使用 Nginx 或 HAProxy 将流量转发到 Ingress Controller。
|
|
|
+ - **MetalLB**:使用 MetalLB 组件在裸机环境中实现 LoadBalancer。
|
|
|
+ - **HostNetwork**:将 Nginx Ingress Controller 作为 DaemonSet 部署,并直接监听 80 和 443 端口。
|
|
|
+ - **修改 Kubelet NodePort 范围**:扩展 NodePort 范围以允许 80 和 443 端口。
|
|
|
+
|
|
|
+- **Cert manager 和 Ingress 配置 SSL 证书**:使用 cert-manager 自动管理和应用 SSL
|
|
|
+ - **Cert-Manager 和 Alidns Webhook**:通过 Helm 安装 Cert-Manager 和 cert-manager-alidns-webhook。
|
|
|
+ - **ClusterIssuer 和 Issuer**:配置 Let's Encrypt 的 Staging 和 Production Issuer。
|
|
|
+ - **测试应用部署**:使用 Let's Encrypt Staging 验证证书颁发流程,成功后切换到 Production。
|
|
|
+ - **ECC 证书配置**:通过注解指定 ECC 证书,并设置密钥大小和轮换策略以增强安全性。
|
|
|
+
|
|
|
+
|
|
|
+
|
|
|
+## 6. 清理资源
|
|
|
+
|
|
|
+### 1. 清理测试应用和证书
|
|
|
+
|
|
|
+**删除测试应用的 Deployment 和 Service**:
|
|
|
+
|
|
|
+```sh
|
|
|
+kubectl delete deployment test-app -n your-namespace
|
|
|
+kubectl delete service test-app -n your-namespace
|
|
|
+```
|
|
|
+
|
|
|
+**删除 Ingress 资源和证书**:
|
|
|
+
|
|
|
+```sh
|
|
|
+kubectl delete ingress test-app-ingress -n your-namespace
|
|
|
+kubectl delete certificate test-app-tls -n your-namespace
|
|
|
+```
|
|
|
+
|
|
|
+**删除 Issuer 和 ClusterIssuer**:
|
|
|
+
|
|
|
+```sh
|
|
|
+kubectl delete issuer letsencrypt-staging-issuer -n your-namespace
|
|
|
+kubectl delete issuer letsencrypt-prod-issuer -n your-namespace
|
|
|
+kubectl delete clusterissuer letsencrypt-staging
|
|
|
+kubectl delete clusterissuer letsencrypt-prod
|
|
|
+```
|
|
|
+
|
|
|
+### 2. 清理 Cert-Manager 和 Alidns Webhook
|
|
|
+
|
|
|
+**删除 Cert-Manager 和 Alidns Webhook**:
|
|
|
+
|
|
|
+```sh
|
|
|
+helm uninstall cert-manager -n cert-manager
|
|
|
+helm uninstall cert-manager-webhook-alidns -n cert-manager
|
|
|
+kubectl delete namespace cert-manager
|
|
|
+```
|
|
|
+
|
|
|
+### 3. 清理 Nginx Ingress Controller
|
|
|
+
|
|
|
+**删除 Nginx Ingress Controller**:
|
|
|
+
|
|
|
+```sh
|
|
|
+helm uninstall ingress-nginx -n ingress-nginx
|
|
|
+kubectl delete namespace ingress-nginx
|
|
|
+```
|
|
|
+
|
|
|
+### 4. 清理暴露 80 和 443 端口的资源
|
|
|
+
|
|
|
+**清理负载均衡软件(如 NGINX 或 HAProxy 配置文件)**:
|
|
|
+
|
|
|
+如果你使用了集群外的负载均衡软件如 NGINX 或 HAProxy,请移除你为暴露 80 和 443 端口而添加的配置。具体步骤请参考[之前的文档](./expose/expose-via-external-lb.md)。
|
|
|
+
|
|
|
+**删除 MetalLB 资源(如果使用了 MetalLB 方式)**:
|
|
|
+
|
|
|
+```
|
|
|
+kubectl delete -f metallb-addresspool.yaml
|
|
|
+kubectl delete -f metallb-l2advertisement.yaml
|
|
|
+kubectl delete -f metallb-native.yaml
|
|
|
+kubectl delete namespace metallb-system
|
|
|
+```
|
|
|
+
|
|
|
+**对于 HostNetwork**
|
|
|
+
|
|
|
+之前删除 Nginx Ingress Controller就已经完成清理。
|
|
|
+
|
|
|
+**恢复 Kubelet 配置(如果修改过 NodePort 范围)**:
|
|
|
+
|
|
|
+如果你修改过 Kubelet 的 `--nodeport-addresses` 范围,请恢复默认设置。具体步骤请参考[之前的文档](./expose/expose-via-kubelet-nodeport.md)。
|
|
|
+
|
|
|
+如果你修改过 `/var/lib/kubelet/config.yaml` 文件,可以更新这一行,或者删除这一行:
|
|
|
+
|
|
|
+```yaml
|
|
|
+serviceNodePortRange: 30000-32767
|
|
|
+```
|
|
|
+
|
|
|
+## 最终清理总结
|
|
|
+
|
|
|
+- **删除应用和证书**:删除所有测试应用、证书、Ingress 和相关资源。
|
|
|
+- **清理 Cert-Manager 和 Alidns Webhook**:删除 Cert-Manager 和 Alidns Webhook 以及它们的命名空间。
|
|
|
+- **删除 Nginx Ingress Controller**:卸载 Nginx Ingress Controller 并删除相关的命名空间。
|
|
|
+- **清理暴露端口的配置**:移除所有暴露 80 和 443 端口的配置,包括 MetaLB 和集群外的负载均衡软件配置,恢复 Kubelet 的 NodePort 配置。
|