k8s集群中pod镜像版本检查

k8s

字数统计: 1.7k阅读时长: 9 min

 2020/11/17 

概述

version-checker用于观察k8s集群中运行的镜像的当前版本以及上游可用的最新版本。这些检查作为Prometheus指标公开，可以在gara fana上查看。

该工具目前处于实验阶段。

version-checker支持以下仓库：

ACR
Docker Hub
ECR
GCR (inc gcr facades such as k8s.gcr.io)
Quay
自托管（符合Docker V2 API的registry，例如harbor,artifactory等）。可以一次配置多个自托管registry。

这些镜像仓库支持身份验证。

项目地址：https://github.com/jetstack/version-checker.git

安装

可以使用官方提供的yaml文件或者helm文件安装，均在项目下的deploy目录下

yaml安装

1	$ kubectl apply -k ./deploy/yaml

helm文件安装

1 2	$ cd ./deploy/charts/version-checker && kubectl create namespace version-checker $ helm install version-checker . -n version-checker

这里我使用yaml安装，安装后如下

[root@master-01 version-check]# kubectl get pod,svc -n version-checker  -o wide
NAME                                  READY   STATUS    RESTARTS   AGE   IP              NODE      NOMINATED NODE   READINESS GATES
pod/version-checker-8cfbf9f69-4htlv   1/1     Running   0          35m   100.67.79.188   node-01   <none>           <none>

NAME                      TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE   SELECTOR
service/version-checker   ClusterIP   10.104.135.133   <none>        8080/TCP   81m   app=version-checker

使用

Version-checker 暴露prometheus指标的地址为 http://podip:8080/metrics

[root@master-01 version-check]# curl http://100.67.79.188:8080/metrics
# HELP version_checker_is_latest_version Where the container in use is using the latest upstream registry version
# TYPE version_checker_is_latest_version gauge
version_checker_is_latest_version{container="version-checker",current_version="v0.2.1",image="quay.io/jetstack/version-checker",latest_version="v0.2.1",namespace="version-checker",pod="version-checker-8cfbf9f69-4htlv"}

version-checker 默认只会检查包含注解 enable.version-checker.io/my-container-name: "true" 的pod。所以上面只显示了 version-checker-8cfbf9f69-4htlv这个pod，可以通过启动时添加选项 -a, –test-all-containers 来检查所有pod

---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: version-checker
  name: version-checker
  namespace: version-checker
spec:
  replicas: 1
  selector:
    matchLabels:
      app: version-checker
  template:
    metadata:
      labels:
        app: version-checker
      annotations:
        prometheus.io/path: /metrics
        prometheus.io/port: "8080"
        prometheus.io/scrape: "true"
        enable.version-checker.io/version-checker: "true"
    spec:
      serviceAccountName: version-checker
      containers:
      - image: quay.io/jetstack/version-checker:v0.2.1
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
          name: web
        name: version-checker
        command: ["version-checker","-a"]

我们此时在访问metrics 如下

[root@master-01 version-check]# curl 100.67.79.156:8080/metrics
# HELP version_checker_is_latest_version Where the container in use is using the latest upstream registry version
# TYPE version_checker_is_latest_version gauge
version_checker_is_latest_version{container="alertmanager",current_version="v0.21.0",image="quay.io/prometheus/alertmanager",latest_version="v0.21.0",namespace="monitoring",pod="alertmanager-main-0"} 1
version_checker_is_latest_version{container="alertmanager-proxy",current_version="6.0.1",image="keycloak/keycloak-gatekeeper",latest_version="7.0.0",namespace="dex",pod="alertmanager-proxy-c4c8c5754-vh2tz"} 0
version_checker_is_latest_version{container="alertmanager-proxy",current_version="6.0.1",image="keycloak/keycloak-gatekeeper",latest_version="7.0.0",namespace="dex",pod="kuboard-proxy-765848d698-mdsdj"} 0
version_checker_is_latest_version{container="alertmanager-proxy",current_version="6.0.1",image="keycloak/keycloak-gatekeeper",latest_version="7.0.0",namespace="dex",pod="longhorn-proxy-f7cbfb745-8kvct"} 0
version_checker_is_latest_version{container="alertmanager-proxy",current_version="6.0.1",image="keycloak/keycloak-gatekeeper",latest_version="7.0.0",namespace="dex",pod="traefik-proxy-5c744bb4dc-t5t4t"} 0
version_checker_is_latest_version{container="argocd-application-controller",current_version="v1.7.5",image="argoproj/argocd",latest_version="v1.7.8",namespace="argocd",pod="argocd-application-controller-6c5dfc6fc5-spvvq"} 0
version_checker_is_latest_version{container="argocd-repo-server",current_version="v1.7.5",image="argoproj/argocd",latest_version="v1.7.8",namespace="argocd",pod="argocd-repo-server-67f8db4f7c-7524w"} 0
version_checker_is_latest_version{container="argocd-server",current_version="v1.7.5",image="argoproj/argocd",latest_version="v1.7.8",namespace="argocd",pod="argocd-server-646767ff75-2gld6"} 0
version_checker_is_latest_version{container="blackbox",current_version="v0.16.0",image="prom/blackbox-exporter",latest_version="v0.18.0",namespace="monitoring",pod="blackbox-549bfdd9dc-g679w"} 0
version_checker_is_latest_version{container="calico-node",current_version="v3.8.2",image="calico/node",latest_version="9512289",namespace="kube-system",pod="calico-node-jz8pj"} 0
version_checker_is_latest_version{container="calico-node",current_version="v3.8.2",image="calico/node",latest_version="9512289",namespace="kube-system",pod="calico-node-p8nv4"} 0
version_checker_is_latest_version{container="chartmuseum",current_version="v2.1.1",image="goharbor/chartmuseum-photon",latest_version="v2.1.1",namespace="harbor",pod="harbor-harbor-chartmuseum-78d9dcf76-9wrnt"} 1
version_checker_is_latest_version{container="check-ecs-price",current_version="v0.4",image="misterli/checkecsprice",latest_version="v0.4",namespace="default",pod="check-ecs-price-5b74cbf8dc-ghzf7"} 1
version_checker_is_latest_version{container="kubernetes-dashboard-proxy",current_version="6.0.1",image="keycloak/keycloak-gatekeeper",latest_version="7.0.0",namespace="kubernetes-dashboard",pod="kubernetes-dashboard-proxy-5bcf658b56-k5p97"} 0
version_checker_is_latest_version{container="kuboard",current_version="v2.0.6-beta.1",image="eipwork/kuboard",latest_version="v2.0.5",namespace="kube-system",pod="kuboard-5b5b5859f7-87flj"} 1
version_checker_is_latest_version{container="loki",current_version="2.0.0@sha256:77e138f81a8e253f1d0ea5d2dc329a02212ecab3247c87f85f1f2182a0160ccd",image="grafana/loki",latest_version="2.0.0@sha256:91b0a08eb482c677304a3ab09e3e71eb10a9e78b05309cc178b07be83c0b238e",namespace="monitoring",pod="loki-0"} 1
version_checker_is_latest_version{container="longhorn-csi-plugin",current_version="v1.0.2",image="longhornio/longhorn-manager",latest_version="v1.0.2",namespace="longhorn-system",pod="longhorn-csi-plugin-2gfls"} 1
version_checker_is_latest_version{container="longhorn-driver-deployer",current_version="v1.0.2",image="longhornio/longhorn-manager",latest_version="v1.0.2",namespace="longhorn-system",pod="longhorn-driver-deployer-7d957dcd9-pc74k"} 1
version_checker_is_latest_version{container="longhorn-manager",current_version="v1.0.2",image="longhornio/longhorn-manager",latest_version="v1.0.2",namespace="longhorn-system",pod="longhorn-manager-9qdcs"} 1
version_checker_is_latest_version{container="longhorn-ui",current_version="v1.0.2",image="longhornio/longhorn-ui",latest_version="v1.0.2",namespace="longhorn-system",pod="longhorn-ui-65d76ddf9b-f9vkw"} 1
.......略

version-checker还支持如下注释：

pin-major.version-checker.io/my-container: 4。将主版本固定为4（v4.0.0）。
pin-minor.version-checker.io/my-container: 3。将次要版本固定为3（v0.3.0）。
pin-patch.version-checker.io/my-container: 23。将补丁程序的版本固定为23（v0.0.23）。
use-metadata.version-checker.io/my-container: "true"。将允许搜索包含字符串第一部分之后的信息的镜像标签。例如，这可以是预发布或生成元数据（v1.2.4-alpha.0，v1.2.3-debian-r3）。
use-sha.version-checker.io/my-container: “true”`。将检查可用的最新SHA标签。如果未设置镜像标签或“latest”镜像标签，则将其静默设置为true。不能与任何其他选项一起使用。
match-regex.version-checker.io/my-container: ^v\d+\.\d+\.\d+-debian-。仅用于与匹配正则表达式集的镜像标签进行比较。例如，上面的注释将仅检查格式为的镜像标签v1.3.4-debian-r30。 use-metadata.version-checker.io设置时此项时不需要。设置此选项后，除URL覆盖以外的所有其他选项都将被忽略。
override-url.version-checker.io/my-container: docker.io/bitnami/etcd。用于更改URL，以查找最新图像版本所在的位置。在此示例中，my-container将把当前版本与docker.io/bitnami/etcd仓库中的镜像版本进行比较。

监控展示

我们还可以用prometheus和gtafana 进行监控展示

注意：通过helm安装会自动创建ServiceMonitor，这里我们需要手动创建ServiceMonitor

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: version-checker
  labels:
    release: prometheus-operator
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app: version-checker
  endpoints:
  - port: web
    path: /metrics
  jobLabel: version-checker
  namespaceSelector:
    matchNames:
    - version-checker