

misterli's Blog.

misterli's Blog.

使用ssl-exporter监控ssl证书过期时间



使用ssl-exporter监控ssl证书过期时间

prometheus ssl-exporter

字数统计: 3.9k阅读时长: 22 min

 2021/04/01   Share

• 
• 
• 
• 
• 

虽然blackbox-exporter也可以检测证书过期时间，但是最近发现它偶尔采集到的部分证书的过期时间不太准确，于是用ssl-exporter去监控证书过期时间。

部署

apiVersion: v1
kind: Service
metadata:
  labels:
    name: ssl-exporter
  name: ssl-exporter
  namespace: monitoring
spec:
  ports:
    - name: ssl-exporter
      protocol: TCP
      port: 9219
      targetPort: 9219
  selector:
    app: ssl-exporter
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ssl-exporter
  namespace: monitoring
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ssl-exporter
  template:
    metadata:
      name: ssl-exporter
      labels:
        app: ssl-exporter
    spec:
      initContainers:
        # Install kube ca cert as a root CA
        - name: ca
          image: misterli/alpine:3.7-china
          command:
            - sh
            - -c
            - |
              set -e
              apk add --update ca-certificates
              cp /var/run/secrets/kubernetes.io/serviceaccount/ca.crt /usr/local/share/ca-certificates/kube-ca.crt
              update-ca-certificates
              cp /etc/ssl/certs/* /ssl-certs
          volumeMounts:
            - name: ssl-certs
              mountPath: /ssl-certs
      containers:
        - name: ssl-exporter
          image: ribbybibby/ssl-exporter:v2.2.0
          ports:
            - name: tcp
              containerPort: 9219
          volumeMounts:
            - name: ssl-certs
              mountPath: /etc/ssl/certs
      volumes:
        - name: ssl-certs
          emptyDir: {}

注意

1、这里的initcontainer 用的修改过的alpine 镜像，替换镜像源为国内源，如果用官方镜像可能执行apk add --update ca-certificates时下载过慢导致超时失败，网络好的同学也可以直接使用官方镜像，这里用的修改过的镜像的Dockerfile如下：

FROM alpine:3.7
RUN echo -e http://mirrors.ustc.edu.cn/alpine/v3.7/main/ >> /etc/apk/repositories

2、ssl-exporter的镜像这里使用的最新的v2.2.0，网上很多文章中用的0.6.0不建议使用，因为镜像版本0.6.0 ，提供的指标和作者最新的dashboard 不一致。

prometheus抓取

这里我的prometheus是使用prometheus-operator安装的，有两种方式去抓取监控指标。probe方式适合于使用较新的prometheus-operator安装的prometheus，老版本prometheus-operator安装的prometheus 不支持probe 可以使用additional。

使用probe

创建文件ssl-exporter-probe.yaml，内容如下

kind: Probe
apiVersion: monitoring.coreos.com/v1
metadata:
  name: ssl-exporter-test
  namespace: monitoring
spec:
  interval: 60s
  module: https
  prober:
    url: ssl-exporter.monitoring.svc.cluster.local:9219
  targets:
    staticConfig:
      static:
      - https://www.lishuai.fun:443
      - https://grafana.lishuai.fun:443
      - https://prometheus.lishuai.fun:443
      - https://minio.lishuai.fun:443
      - https://seafile.lishuai.fun:443
      - https://dashboard.lishuai.fun:443

执行命令kubectl apply -f ssl-exporter-probe.yaml，然后我们去prometheus ui上刷新即可看到新的target

使用additional

创建一个文件名为prometheus-additional.yaml，内容如下：

- job_name: ssl-exporter
  metrics_path: /probe
  static_configs:
  - targets:
    - kubernetes.default.svc:443
    - www.lishuai.fun:443
    - harbor.lishuai.fun:443
  relabel_configs:
  - source_labels: [__address__]
    target_label: __param_target
  - source_labels: [__param_target]
    target_label: instance
  - target_label: __address__
    replacement: ssl-exporter.monitoring:9219

注意：这里要监控的地址要加上https协议使用的端口号

使用上述文件创建一个secret，这里为了每次修改后创建方便使用如下脚本：

#!/bin/bash
kubectl delete secrets -n monitoring additional-configs
sleep 1
kubectl create secret generic additional-configs --from-file=prometheus-additional.yaml -n monitoring

修改prometheus-prometheus.yaml添加如下

.....
additionalScrapeConfigs:
   name: additional-config 
   key: prometheus-additional.yaml 
......

然后重新apply prometheus-prometheus.yaml

kubectl apply -f prometheus-prometheus.yaml

稍等片刻我们就可以在prometheus ui上看到target

告警

规则如下,定义了证书检测失败以及证书有效时间不足七天进行报警

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: ssl-exporter-rule
  namespace: monitoring
  labels:
    prometheus: k8s
    role: alert-rules
spec:
  groups:
  - name: "ssl证书过期时间检查"
    rules:
    - alert: "ssl证书过期警告"
      expr: (ssl_cert_not_after-time())/3600/24 <7
      for: 3h
      labels:
        severity: critical
      annotations:
        description: '证书{{ $labels.instance }}还有{{ printf "%.1f" $value }}天就过期了,请尽快更新证书'
        summary: "ssl证书过期警告"
    - alert: "ssl证书检查异常"
      expr: ssl_probe_success == 0
      for: 5m
      labels:
        severity: critical
      annotations:
        summary: "ssl证书检查异常"
        description: "证书 {{ $labels.instance }} 识别失败，请检查"

grafana展示

适用于prometheus-operator安装的prometheus的dashboard如下

{
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": "-- Grafana --",
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "type": "dashboard"
      }
    ]
  },
  "description": "Shows certitificate expiration times, as well as failed ssl connects",
  "editable": true,
  "gnetId": 11279,
  "graphTooltip": 0,
  "id": 2,
  "iteration": 1583741464883,
  "links": [],
  "panels": [
    {
      "cacheTimeout": null,
      "colorBackground": false,
      "colorPostfix": false,
      "colorPrefix": false,
      "colorValue": false,
      "colors": ["#299c46", "rgba(237, 129, 40, 0.89)", "#d44a3a"],
      "datasource": "prometheus",
      "decimals": 0,
      "description": "",
      "format": "none",
      "gauge": {
        "maxValue": 100,
        "minValue": 0,
        "show": false,
        "thresholdLabels": false,
        "thresholdMarkers": true
      },
      "gridPos": {
        "h": 7,
        "w": 4,
        "x": 0,
        "y": 0
      },
      "id": 9,
      "interval": null,
      "links": [],
      "mappingType": 1,
      "mappingTypes": [
        {
          "name": "value to text",
          "value": 1
        },
        {
          "name": "range to text",
          "value": 2
        }
      ],
      "maxDataPoints": 100,
      "nullPointMode": "connected",
      "nullText": null,
      "options": {},
      "postfix": "",
      "postfixFontSize": "50%",
      "prefix": "",
      "prefixFontSize": "50%",
      "rangeMaps": [
        {
          "from": "null",
          "text": "N/A",
          "to": "null"
        }
      ],
      "sparkline": {
        "fillColor": "rgba(31, 118, 189, 0.18)",
        "full": false,
        "lineColor": "rgb(31, 120, 193)",
        "show": false,
        "ymax": null,
        "ymin": null
      },
      "tableColumn": "",
      "targets": [
        {
          "expr": "count(max(ssl_cert_not_after{instance=~\"$instance\",job=~\"$job\"}) by (issuer_cn,serial_no))",
          "refId": "A"
        }
      ],
      "thresholds": "",
      "timeFrom": null,
      "timeShift": null,
      "title": "Total Unique Certificates",
      "transparent": true,
      "type": "singlestat",
      "valueFontSize": "80%",
      "valueMaps": [
        {
          "op": "=",
          "text": "N/A",
          "value": "null"
        }
      ],
      "valueName": "current"
    },
    {
      "cacheTimeout": null,
      "colorBackground": false,
      "colorPostfix": false,
      "colorPrefix": false,
      "colorValue": false,
      "colors": ["#299c46", "rgba(237, 129, 40, 0.89)", "#d44a3a"],
      "datasource": "prometheus",
      "decimals": 0,
      "description": "",
      "format": "none",
      "gauge": {
        "maxValue": 100,
        "minValue": 0,
        "show": false,
        "thresholdLabels": false,
        "thresholdMarkers": true
      },
      "gridPos": {
        "h": 7,
        "w": 4,
        "x": 4,
        "y": 0
      },
      "id": 10,
      "interval": null,
      "links": [],
      "mappingType": 1,
      "mappingTypes": [
        {
          "name": "value to text",
          "value": 1
        },
        {
          "name": "range to text",
          "value": 2
        }
      ],
      "maxDataPoints": 100,
      "nullPointMode": "connected",
      "nullText": null,
      "options": {},
      "postfix": "",
      "postfixFontSize": "50%",
      "prefix": "",
      "prefixFontSize": "50%",
      "rangeMaps": [
        {
          "from": "null",
          "text": "N/A",
          "to": "null"
        }
      ],
      "sparkline": {
        "fillColor": "rgba(31, 118, 189, 0.18)",
        "full": false,
        "lineColor": "rgb(31, 120, 193)",
        "show": false,
        "ymax": null,
        "ymin": null
      },
      "tableColumn": "",
      "targets": [
        {
          "expr": "count(max(ssl_cert_not_after{instance=~\"$instance\",job=~\"$job\"}) by (instance))",
          "refId": "A"
        }
      ],
      "thresholds": "",
      "timeFrom": null,
      "timeShift": null,
      "title": "Total Probe Targets",
      "transparent": true,
      "type": "singlestat",
      "valueFontSize": "80%",
      "valueMaps": [
        {
          "op": "=",
          "text": "N/A",
          "value": "null"
        }
      ],
      "valueName": "current"
    },
    {
      "aliasColors": {},
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": "prometheus",
      "fill": 0,
      "fillGradient": 0,
      "gridPos": {
        "h": 8,
        "w": 14,
        "x": 9,
        "y": 0
      },
      "hiddenSeries": false,
      "id": 6,
      "interval": "",
      "legend": {
        "alignAsTable": false,
        "avg": false,
        "current": false,
        "hideEmpty": false,
        "hideZero": false,
        "max": false,
        "min": false,
        "rightSide": false,
        "show": true,
        "total": false,
        "values": false
      },
      "lines": false,
      "linewidth": 0,
      "nullPointMode": "null",
      "options": {
        "dataLinks": [
          {
            "targetBlank": true,
            "title": "Open URL",
            "url": "https://${__field.labels.instance}"
          }
        ]
      },
      "percentage": false,
      "pointradius": 2,
      "points": true,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": true,
      "steppedLine": false,
      "targets": [
        {
          "expr": "(up{job=~\"$job\", instance=~\"$instance\"} == 0 or ssl_probe_success{job=~\"$job\", instance=~\"$instance\"} == 0)^0",
          "format": "time_series",
          "instant": false,
          "legendFormat": "{{instance}}",
          "refId": "A"
        }
      ],
      "thresholds": [],
      "timeFrom": null,
      "timeRegions": [],
      "timeShift": null,
      "title": "Failed SSL Connects History",
      "tooltip": {
        "shared": true,
        "sort": 1,
        "value_type": "individual"
      },
      "transparent": true,
      "type": "graph",
      "xaxis": {
        "buckets": null,
        "mode": "time",
        "name": null,
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "decimals": 0,
          "format": "short",
          "label": "",
          "logBase": 1,
          "max": null,
          "min": "0",
          "show": true
        },
        {
          "decimals": 0,
          "format": "short",
          "label": "",
          "logBase": 1,
          "max": null,
          "min": "0",
          "show": false
        }
      ],
      "yaxis": {
        "align": false,
        "alignLevel": null
      }
    },
    {
      "columns": [],
      "datasource": "prometheus",
      "description": "Possible reasons:\n- site is down\n- server is down\n- certificate has expired\n- certificate's CA is not trusted by the exporter\n- other connection errors\n- other certificate errors",
      "fontSize": "100%",
      "gridPos": {
        "h": 5,
        "w": 24,
        "x": 0,
        "y": 8
      },
      "id": 4,
      "links": [],
      "maxPerRow": 2,
      "options": {},
      "pageSize": 10,
      "repeat": "job",
      "repeatDirection": "h",
      "scopedVars": {
        "job": {
          "selected": false,
          "text": "ssl",
          "value": "ssl"
        }
      },
      "scroll": true,
      "showHeader": true,
      "sort": {
        "col": 2,
        "desc": false
      },
      "styles": [
        {
          "alias": "Time",
          "align": "auto",
          "dateFormat": "YYYY-MM-DD HH:mm:ss",
          "pattern": "Time",
          "type": "hidden"
        },
        {
          "alias": "",
          "align": "auto",
          "colorMode": null,
          "colors": [
            "rgba(245, 54, 54, 0.9)",
            "rgba(237, 129, 40, 0.89)",
            "rgba(50, 172, 45, 0.97)"
          ],
          "decimals": 2,
          "pattern": "job",
          "thresholds": [],
          "type": "hidden",
          "unit": "short"
        },
        {
          "alias": "",
          "align": "auto",
          "colorMode": null,
          "colors": [
            "rgba(245, 54, 54, 0.9)",
            "rgba(237, 129, 40, 0.89)",
            "rgba(50, 172, 45, 0.97)"
          ],
          "dateFormat": "YYYY-MM-DD HH:mm:ss",
          "decimals": 2,
          "mappingType": 1,
          "pattern": "__name__",
          "thresholds": [],
          "type": "hidden",
          "unit": "short"
        },
        {
          "alias": "SSL Failed",
          "align": "auto",
          "colorMode": "row",
          "colors": [
            "rgba(245, 54, 54, 0.9)",
            "rgba(237, 129, 40, 0.89)",
            "rgba(50, 172, 45, 0.97)"
          ],
          "dateFormat": "YYYY-MM-DD HH:mm:ss",
          "decimals": 0,
          "mappingType": 1,
          "pattern": "Value",
          "thresholds": ["1"],
          "type": "number",
          "unit": "short"
        },
        {
          "alias": "",
          "align": "auto",
          "colorMode": "row",
          "colors": [
            "#F2495C",
            "rgba(237, 129, 40, 0.89)",
            "rgba(50, 172, 45, 0.97)"
          ],
          "dateFormat": "YYYY-MM-DD HH:mm:ss",
          "decimals": 2,
          "link": true,
          "linkTargetBlank": true,
          "linkTooltip": "${__cell}",
          "linkUrl": "https://${__cell:raw}",
          "mappingType": 1,
          "pattern": "instance",
          "preserveFormat": false,
          "rangeMaps": [],
          "sanitize": true,
          "thresholds": [""],
          "type": "string",
          "unit": "short",
          "valueMaps": []
        }
      ],
      "targets": [
        {
          "expr": "ssl_probe_success{instance=~\"$instance\",job=~\"$job\"} == 0",
          "format": "table",
          "instant": true,
          "intervalFactor": 1,
          "legendFormat": "",
          "refId": "A"
        }
      ],
      "timeFrom": null,
      "timeShift": null,
      "title": "Failed SSL Connects - $job",
      "transform": "table",
      "transparent": true,
      "type": "table"
    },
    {
      "cacheTimeout": null,
      "columns": [],
      "datasource": "prometheus",
      "description": "",
      "fontSize": "100%",
      "gridPos": {
        "h": 25,
        "w": 24,
        "x": 0,
        "y": 18
      },
      "id": 2,
      "interval": "",
      "links": [],
      "options": {},
      "pageSize": null,
      "pluginVersion": "6.1.6",
      "scroll": true,
      "showHeader": true,
      "sort": {
        "col": 8,
        "desc": false
      },
      "styles": [
        {
          "alias": "Expires In",
          "align": "auto",
          "colorMode": "cell",
          "colors": [
            "rgba(245, 54, 54, 0.9)",
            "rgba(237, 129, 40, 0.89)",
            "rgba(50, 172, 45, 0.97)"
          ],
          "decimals": 1,
          "link": false,
          "pattern": "Value",
          "thresholds": ["3", "7"],
          "type": "number",
          "unit": "d"
        },
        {
          "alias": "",
          "align": "auto",
          "colorMode": null,
          "colors": [
            "rgba(245, 54, 54, 0.9)",
            "rgba(237, 129, 40, 0.89)",
            "rgba(50, 172, 45, 0.97)"
          ],
          "dateFormat": "YYYY-MM-DD HH:mm:ss",
          "decimals": 2,
          "mappingType": 1,
          "pattern": "Time",
          "thresholds": [],
          "type": "hidden",
          "unit": "short"
        },
        {
          "alias": "",
          "align": "auto",
          "colorMode": null,
          "colors": [
            "rgba(245, 54, 54, 0.9)",
            "rgba(237, 129, 40, 0.89)",
            "rgba(50, 172, 45, 0.97)"
          ],
          "dateFormat": "YYYY-MM-DD HH:mm:ss",
          "decimals": 2,
          "mappingType": 1,
          "pattern": "job",
          "thresholds": [],
          "type": "hidden",
          "unit": "short"
        },
        {
          "alias": "",
          "align": "auto",
          "colorMode": null,
          "colors": [
            "rgba(245, 54, 54, 0.9)",
            "rgba(237, 129, 40, 0.89)",
            "rgba(50, 172, 45, 0.97)"
          ],
          "dateFormat": "YYYY-MM-DD HH:mm:ss",
          "decimals": 2,
          "link": true,
          "linkTargetBlank": true,
          "linkTooltip": "${__cell_6}",
          "linkUrl": "https://${__cell:raw}/",
          "mappingType": 1,
          "pattern": "instance",
          "sanitize": false,
          "thresholds": [],
          "type": "string",
          "unit": "short"
        },
        {
          "alias": "",
          "align": "auto",
          "colorMode": null,
          "colors": [
            "rgba(245, 54, 54, 0.9)",
            "rgba(237, 129, 40, 0.89)",
            "rgba(50, 172, 45, 0.97)"
          ],
          "dateFormat": "YYYY-MM-DD HH:mm:ss",
          "decimals": 2,
          "mappingType": 1,
          "pattern": "dnsnames",
          "thresholds": [],
          "type": "hidden",
          "unit": "short"
        },
        {
          "alias": "",
          "align": "auto",
          "colorMode": null,
          "colors": [
            "rgba(245, 54, 54, 0.9)",
            "rgba(237, 129, 40, 0.89)",
            "rgba(50, 172, 45, 0.97)"
          ],
          "dateFormat": "YYYY-MM-DD HH:mm:ss",
          "decimals": 2,
          "mappingType": 1,
          "pattern": "ou",
          "thresholds": [],
          "type": "hidden",
          "unit": "short"
        }
      ],
      "targets": [
        {
          "expr": "((ssl_cert_not_after{instance=~\"$instance\",job=~\"$job\"} - time())/24/60/60)",
          "format": "table",
          "instant": true,
          "interval": "",
          "intervalFactor": 1,
          "legendFormat": "",
          "refId": "A"
        }
      ],
      "timeFrom": null,
      "timeShift": null,
      "title": "SSL Certificates",
      "transform": "table",
      "type": "table"
    }
  ],
  "refresh": "5m",
  "schemaVersion": 22,
  "style": "dark",
  "tags": ["ssl", "tls"],
  "templating": {
    "list": [
      {
        "allValue": null,
        "current": {
          "text": "All",
          "value": ["$__all"]
        },
        "datasource": "prometheus",
        "definition": "label_values(ssl_probe_success, job)",
        "hide": 0,
        "includeAll": true,
        "label": "Job",
        "multi": true,
        "name": "job",
        "options": [],
        "query": "label_values(ssl_probe_success, job)",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "sort": 5,
        "tagValuesQuery": "",
        "tags": [],
        "tagsQuery": "",
        "type": "query",
        "useTags": false
      },
      {
        "allValue": null,
        "current": {
          "text": "All",
          "value": ["$__all"]
        },
        "datasource": "prometheus",
        "definition": "label_values({job=~\"$job\"}, instance)",
        "hide": 0,
        "includeAll": true,
        "label": "Instance",
        "multi": true,
        "name": "instance",
        "options": [],
        "query": "label_values({job=~\"$job\"}, instance)",
        "refresh": 1,
        "regex": "",
        "skipUrlSync": false,
        "sort": 1,
        "tagValuesQuery": "",
        "tags": [],
        "tagsQuery": "",
        "type": "query",
        "useTags": false
      }
    ]
  },
  "time": {
    "from": "now-1h",
    "to": "now"
  },
  "timepicker": {
    "refresh_intervals": ["30s", "1m", "5m", "15m", "30m", "1h", "2h", "1d"],
    "time_options": ["5m", "15m", "1h", "6h", "12h", "24h", "2d", "7d", "30d"]
  },
  "timezone": "browser",
  "title": "SSL/TLS Exporter",
  "uid": "HyKQlVGWk",
  "version": 1
}

如果是别的方式安装的请替换"datasource": "xxxx"为相应的prometheus的名字

参考

https://github.com/ribbybibby/ssl_exporter#configuration

https://github.com/ribbybibby/ssl_exporter/issues/14

原文作者：Mister Li

原文链接：https://www.lishuai.fun/2021/04/01/ssl-exporter/

发表日期：April 1st 2021, 9:55:30 am

更新日期：May 3rd 2022, 11:25:55 pm

版权声明：本文采用知识共享署名-非商业性使用 4.0 国际许可协议进行许可

• Next Post
  prometheus使用missing-container-metrics监控pod oomkill
• Previous Post
  prometheus使用PrometheusAlert进行聚合报警

Powered by Hexotheme Archer

©豫ICP备19035521号

PV: :)

CATALOG

1. 1. 部署
2. 2. prometheus抓取
   1. 2.1. 使用probe
   2. 2.2. 使用additional
3. 3. 告警
4. 4. grafana展示
5. 5. 参考

• Archive
• Tag
• Cate

Total : 46



2023

• 05/05 记一次kuboard故障—Etcd空间爆满
• 05/05 K8s环境清理节点上旧的镜像
• 05/05 docker/buildkit 构建问题
• 05/05 分析redis 大key
• 05/05 Umami 安装并升级到v2.x

2022

• 06/15 SSO之使用Pomerium保护您的网站
• 05/09 使用sloop 监控kubernetes event
• 05/02 阿里云ACK集群 Ingress-nginx配置日志输出request body和request headers信息
• 04/06 K8s RBAC工具之akcess

2021

• 12/31 使用s3(minio)为kubernetes提供pv存储
• 09/28 Kubernetes 的 API 流量查看器---mizu
• 09/22 记一次longhorn 组件重启导致pv无法正常挂载
• 09/09 在Gitlab中规范提交的commit message的格式
• 09/06 监控minio的多种姿势
• 09/01 在k8s上部署支持多租户的minio
• 08/25 k8s 开启临时容器进行debug
• 08/12 史上最全之K8s使用nfs作为存储卷的五种方式
• 08/10 三个小工具带你进一步了解K8S集群内RBAC
• 08/09 使用Permission manager动态为k8s集群创建用户及kubeconfig
• 08/05 使用RBAC Manager 简化Kubernetes 中的授权
• 08/02 使用gitlab账号登陆Kubesphere
• 08/01 k8s安装cert-manager及签发泛域名证书
• 07/16 kubesphere 3.1.1 接入自定义 Prometheus
• 07/13 阿里云slb https代理harbor 总结
• 07/06 prometheus使用missing-container-metrics监控pod oomkill
• 04/01 使用ssl-exporter监控ssl证书过期时间
• 01/27 prometheus使用PrometheusAlert进行聚合报警
• 01/22 chaos-mesh 混沌工程（下）
• 01/21 Chaos-mesh 混沌工程(上）
• 01/06 扫描k8s集群中漏洞
• 01/06 利用k8s审计日志生成RBAC规则
• 01/05 同步secret和config到指定namespace

2020

• 12/08 kubevela之appfile详解（下）
• 12/08 kubevela之appfile详解（上）
• 12/03 kubevela小试牛刀（1）
• 12/02 使用Rancher的rke安装k8s
• 11/27 Prometheus监控redis及redis-cluster
• 11/18 Argo CD使用ding talk通知同步状态
• 11/17 k8s集群中pod镜像版本检查
• 11/16 阿里云rds 备份恢复到本地
• 11/06 记录一次kyverno重启解决过程
• 11/05 使用harbor代理缓存docker hub
• 09/23 一次cert-manager无法续签
• 09/07 使用traefik为服务配置sso
• 09/02 k8s之纵向扩缩容vpa
• 09/01 使用hugo搭建博客

 kubernetes  rbac  k8s  vulnerability  ingress-nginx  RBAC  argocd  devops  gitlab  harbor  slb  hugo  pv  csi  s3  kubesphere  prometheus  sso  kubevela  k8s准入控制  kyverno  longhorn  oomkill  event  configmap  secret  sync  ingress  cert-manager  chaos-mesh  minio  vpa  动态扩缩容  k8s kubevela  oamm  报警  ssl-exporter  traefik  混沌工程  rke k8s  pvc  nfs  prometheus redis  umami  etcd  kuboard  docker  ci  buildkit  redis



缺失模块，请参考主题文档进行安装配置：https://github.com/fi3ework/hexo-theme-archer#%E5%AE%89%E8%A3%85%E4%B8%BB%E9%A2%98



