misterli's Blog.

kubevela之appfile详解（下）

k8s kubevela oamm
字数统计: 2.5k阅读时长: 12 min
2020/12/08

上一篇我们介绍了appfile中服务的几种类型以及相关参数，这一篇我们介绍一些appfile的traits。在traits中可以配置一些运维策略，包括routes（ingress ), auto-scaling , rollout, metrics.

routes

Route 用于配置对服务的外部访问。

可用属性列表

1
2
3
4
5
6
7
8
9
10
11
name: my-app-name

services:
  my-service-name:
    ...
    route:
      domain: example.com
      issuer: tls
      rules:
        - path: /testapp
          rewriteTarget: /

参数说明：

名称 类型 描述 笔记
domain string 指定此应用程序的URL [默认为（空）]
issuer string 指定是否使用cert-manager创建ssl证书。 [默认为不使用tls]
rules [[] RouteRules](见下表） [可选的]

RouteRules

名称 类型 描述 笔记
path string [默认为（空）]
RewriteTarget string [默认为（空）]

示例：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
name: testapp

services:
  test-web-service:
    # this image will be used in both build and deploy steps
    image: misterli/testapp:v1


      # Uncomment the following to push to local kind cluster
      # push:
      #   local: kind

    type: webservice 

    cmd: ["node", "server.js"]
    port: 8080
    cpu: "0.1"
    env:
      - name: FOO
        value: bar
    scaler:
      replicas: 1

    route:
      domain: kubevela.lishuai.fun
      rules:
        - path: /testapp
          rewriteTarget: /

我们部署后查看可以发现vela会自动为我们创建deployment,service,ingress对象

1
2
3
4
5
6
7
8
9
10
11
12
13
14
[root@master-01 testapp]# kubectl get pod,svc,ingress
NAME                                   READY   STATUS              RESTARTS   AGE
pod/check-ecs-price-7cdc97b997-j9w9q   1/1     Running             0          11d
pod/test-web-service-7f49665d-7n47c    0/1     ContainerCreating   0          5s
pod/test-web-service-7f49665d-w8shg    1/1     Terminating         0          99s

NAME                                       TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/kubernetes                         ClusterIP   10.96.0.1       <none>        443/TCP    12d
service/test-web-service-route-cfddb8945   ClusterIP   10.100.196.92   <none>        8080/TCP   5s
service/traefik-forward-auth               ClusterIP   10.96.227.183   <none>        4181/TCP   12d

NAME                                                    CLASS    HOSTS                  ADDRESS   PORTS   AGE
ingress.extensions/test-web-service-route-cfddb8945-0   <none>   kubevela.lishuai.fun             80      4s

我们可以访问我们ingress中定义的hosts

image-20201207152211130

autoscale

Autoscale 用于根据资源利用率指标和cron自动扩展工作负载

完整的配置列表

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
name: testapp

services:
  express-server:
    ...
  autoscale:
    min: 1
    max: 4
    cron:
      startAt:  "14:00"
      duration: "2h"
      days:     "Monday, Thursday"
      replicas: 2
      timezone: "America/Los_Angeles"
    cpuPercent: 10

参数

名称 类型 描述 笔记
min 整型 工作负载的最小副本 需要
Max 整型 工作负载的最大副本 需要
cpuPercent 整型 指定CPU利用率的值，例如80，表示80％
Cron {Cron} 仅用于appfile，不适用于Cli

cron参数

名称 类型 描述 笔记
startAt 串 开始缩放的时间，例如 08:00
duration 串 缩放将持续多长时间
days 串 几个工作日或周末，例如“星期一，星期二”
replicas 整型 要缩放到的目标副本
Timezone 串 时区，例如“ America / Los_Angeles”

示例：

利用cron进行扩缩放

我们这里定义周一和周四的15:40副本扩大到两个持续三分钟

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
name: testapp

services:
  test-web-service:
    # this image will be used in both build and deploy steps
    image: misterli/testapp:v1
    type: webservice 

    cmd: ["node", "server.js"]
    port: 8080
    cpu: "0.1"
    scaler:
      replicas: 1
    autoscale:
      min: 1
      max: 4
      cron:
        startAt:  "10:55"
        duration: "10m"
        days:     "Tuesday, Thursday"
        replicas: 2
        timezone: "Asia/Shanghai"

结果如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
[root@master-01 testapp]# date 
2020年 12月 08日 星期二 10:50:43 CST
[root@master-01 testapp]# kubectl get pod -w 
NAME                               READY   STATUS    RESTARTS   AGE
check-ecs-price-7cdc97b997-j9w9q   1/1     Running   0          11d
test-web-service-b898fdc47-4klq8   1/1     Running   0          67s
test-web-service-b898fdc47-pp5zs   0/1     Pending   0          0s #到达指定时间开始扩容
test-web-service-b898fdc47-pp5zs   0/1     Pending   0          0s
test-web-service-b898fdc47-pp5zs   0/1     ContainerCreating   0          0s
test-web-service-b898fdc47-pp5zs   0/1     ContainerCreating   0          1s
test-web-service-b898fdc47-pp5zs   1/1     Running             0          2s
test-web-service-b898fdc47-pp5zs   1/1     Terminating         0          15m
test-web-service-b898fdc47-pp5zs   0/1     Terminating         0          15m  #设置的扩容持续10m，此时开始缩容
test-web-service-b898fdc47-pp5zs   0/1     Terminating         0          15m
test-web-service-b898fdc47-pp5zs   0/1     Terminating         0          15m

[root@master-01 testapp]# vela status testapp
About:

  Name:         testapp                                
  Namespace:    default                                
  Created at:   2020-12-08 10:49:40.961289108 +0800 CST
  Updated at:   2020-12-08 10:49:40.961289248 +0800 CST

Services:
                                                                                                                                                                - Name: test-web-service
    Type: webservice
    HEALTHY Ready:1/1 
    Traits:                                                                                                                                                         - ✅ scaler: replicas=1                                                                                                                                        - ✅ autoscale: type: cron    replicas(min/max/current): 1/4/1
    Last Deployment:
      Created at: 2020-12-08 10:49:40 +0800 CST
      Updated at: 2020-12-08T10:49:40+08:00

利用cpu使用率进行扩缩容

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
name: testapp

services:
  test-web-service:
    # this image will be used in both build and deploy steps
    image: misterli/testapp:v1

    type: webservice 

    cmd: ["node", "server.js"]
    port: 8080
    cpu: "0.1"
    scaler:
      replicas: 1

    route:
      domain: kubevela.lishuai.fun
      rules:
        - path: /testapp
          rewriteTarget: /
    autoscale:
      min: 1
      max: 4
      cpuPercent: 10

我们用ab去压测service的地址，以便cpu快速达到阈值

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
[root@master-01 testapp]# kubectl get svc
NAME                               TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
kubernetes                         ClusterIP   10.96.0.1        <none>        443/TCP    12d
oam-express-server                 ClusterIP   10.109.203.178   <none>        4848/TCP   2m41s
test-web-service-route-cfddb8945   ClusterIP   10.102.14.30     <none>        8080/TCP   4s
traefik-forward-auth               ClusterIP   10.96.227.183    <none>        4181/TCP   12d
[root@master-01 testapp]# ab -n 10000 -c 200 http://10.102.14.30:8080/
.....
##pod的缩放如下
[root@master-01 ~]# kubectl get pod -w
NAME                               READY   STATUS    RESTARTS   AGE
check-ecs-price-7cdc97b997-j9w9q   1/1     Running   0          11d
test-web-service-b898fdc47-ppq4d   1/1     Running   0          49s
test-web-service-b898fdc47-5s6dx   0/1     Pending   0          0s
test-web-service-b898fdc47-nk8zm   0/1     Pending   0          0s
test-web-service-b898fdc47-rl2tg   0/1     Pending   0          0s
test-web-service-b898fdc47-5s6dx   0/1     Pending   0          0s
test-web-service-b898fdc47-rl2tg   0/1     Pending   0          0s
test-web-service-b898fdc47-nk8zm   0/1     Pending   0          0s
test-web-service-b898fdc47-5s6dx   0/1     ContainerCreating   0          0s
test-web-service-b898fdc47-rl2tg   0/1     ContainerCreating   0          1s
test-web-service-b898fdc47-nk8zm   0/1     ContainerCreating   0          1s
test-web-service-b898fdc47-rl2tg   0/1     ContainerCreating   0          1s
test-web-service-b898fdc47-nk8zm   0/1     ContainerCreating   0          2s
test-web-service-b898fdc47-5s6dx   0/1     ContainerCreating   0          2s
test-web-service-b898fdc47-nk8zm   1/1     Running             0          5s
test-web-service-b898fdc47-5s6dx   1/1     Running             0          5s
test-web-service-b898fdc47-rl2tg   1/1     Running             0          7s

不再压测，稍等几分钟之后，pod数量又重新缩放回1个

1
2
3
4
[root@master-01 servicemonitor]# kubectl get pod 
NAME                               READY   STATUS    RESTARTS   AGE
check-ecs-price-7cdc97b997-j9w9q   1/1     Running   0          11d
test-web-service-b898fdc47-ppq4d   1/1     Running   0          37m

rollout

Rollout 用于为您的应用程序配置金丝雀部署策略。

金丝雀部署可以参考：https://martinfowler.com/bliki/CanaryRelease.html

注意：目前当Rollout和Autoscle特性附加到同一服务时，它们两个将在部署期间争用实例数。因此，这是设计Rollout使然.replicas在推出期间将接管副本控制（由字段指定）。后续官方可能会在Appfile中引入一个单独的部分来定义发布阶段的配置，例如Rollout

完整的配置列表

1
2
3
4
5
6
7
8
servcies:
  express-server:
    ...

    rollout:
      replicas: 2
      stepWeight: 50
      interval: "10s"

参数选项：

名称 类型 描述 笔记
replicas 串 每个版本的服务实例的副本数 [默认为2]
stepWeight 串 金丝雀增量步长百分比（0-100） [默认为50]
interval 串 每个滚动更新步骤的等待间隔

rollout是如何工作的

Rollouttrait遵循Canary策略实施渐进式发布过程以推广您的应用程序。

简单的来讲，Rollout控制器将为您的应用程序创建一个金丝雀版本，然后逐渐将流量转移到金丝雀版本，同时测量关键性能指标（例如HTTP请求成功率）。

alt

在上图中，每一个10s，5%流量将转移到canary，直到canary流量到达50%。同时，canary的实例数将自动扩展到`replicas: 2.

根据流量转移期间的分析结果，如果分析失败，则金丝雀将被中止。如果没有问题，则服务版本将从v1完全升级到v2，流量将完全转移回到服务版本v2。升级完成后，金丝雀实例将被删除。

alt

示例：

这里金丝雀部署每次增加20%的副本，这里就是每次增加1个新版本pod，并且为了方便把interval设置的较短，实际使用应该自行调整。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
name: testapp

services:
  test-web-service:
    # this image will be used in both build and deploy steps
    image: oamdev/testapp:rolling01

    type: webservice 

    port: 80
    cpu: "0.1"
    rollout:
      replicas: 5
      stepWeight: 20
      interval: "5s"

    route:
      domain: kubevela.lishuai.fun

查看一下此时的状态,我们可以看到 HEALTHY Ready:5/5 ,说明pod都处于ready 状态

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
[root@master-01 testapp]# vela status testapp
About:

  Name:         testapp                                
  Namespace:    default                                
  Created at:   2020-12-07 16:52:41.408877074 +0800 CST
  Updated at:   2020-12-07 16:52:41.408877175 +0800 CST

Services:
                                                                                                                                                                - Name: test-web-service
    Type: webservice
    HEALTHY Ready:5/5 
    Traits:                                                                                                                                                         - ✅ rollout: replicas=5
                stepWeight=20
                interval=5s                                                                                                                                        - ✅ route: Checking timeout: IP/Hostname of test-web-service-route-6f6dbdd85b-auto-created ingress is generating
    Last Deployment:
      Created at: 2020-12-07 16:52:41 +0800 CST
      Updated at: 2020-12-07T16:52:41+08:00

访问http://kubevela.lishuai.fun如下

1
2
[root@master-01 testapp]# curl kubevela.lishuai.fun
Hello World -- Rolling 01

此时我们修改vela.yaml文件把镜像更新为oamdev/testapp:rolling02，并执行vela up

我们循环访问会发现随着时间的推移，慢慢的流量都切换到了新版本。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
[root@master-01 ~]# while true ; do curl http://kubevela.lishuai.fun&&sleep 1s;done  
Hello World -- Rolling 01
Hello World -- This is rolling 02
Hello World -- Rolling 01
Hello World -- Rolling 01
Hello World -- Rolling 01
Hello World -- Rolling 01
Hello World -- Rolling 01
Hello World -- This is rolling 02
Hello World -- Rolling 01
Hello World -- Rolling 01
Hello World -- Rolling 01
Hello World -- Rolling 01
....
Hello World -- This is rolling 02
Hello World -- Rolling 01
Hello World -- Rolling 01
Hello World -- Rolling 01
Hello World -- Rolling 01
Hello World -- This is rolling 02
Hello World -- This is rolling 02
Hello World -- Rolling 01
Hello World -- Rolling 01
Hello World -- This is rolling 02
Hello World -- This is rolling 02
Hello World -- This is rolling 02
....
Hello World -- Rolling 01
Hello World -- Rolling 01
Hello World -- This is rolling 02
Hello World -- This is rolling 02
Hello World -- This is rolling 02
...
Hello World -- This is rolling 02
Hello World -- This is rolling 02
Hello World -- This is rolling 02
Hello World -- This is rolling 02
Hello World -- This is rolling 02

metrics

Metrics 用于为您的服务配置监视指标。

完整的配置列表：

1
2
3
4
5
6
7
8
9
10
11
name: my-app-name

services:
  my-service-name:
    ...
    metrics:
      format: "prometheus"
      port: 8080
      path: "/metrics"
      scheme:  "http"
      enabled: true

参数

名称 类型 描述 笔记
path string 服务的指标路径 [默认为/ metrics]
format string 指标的格式，默认为prometheus [默认为prometheus]
scheme string [默认为http]
enabled bool [默认为true]
port int32 指标的端口，默认情况下会自动发现 [默认为0]，> = 1024和<= 65535
selector map [string]string Pod的标签选择器，默认情况下会自动发现 [可选的]

示例:

1
2
3
4
5
6
7
8
9
10
11
12
13
name: metricapp
services:
 metricapp:
   type: webservice
   image: christianhxc/gorandom:1.0
   port: 8080

   metrics:
     enabled: true
     format: prometheus
     path: /metrics
     port: 8080
     scheme: http

执行vela up后会创建一个service 和service monitor 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
[root@master-01 testapp]# kubectl get svc 
NAME                   TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
kubernetes             ClusterIP   10.96.0.1       <none>        443/TCP    13d
oam-metricapp          ClusterIP   10.104.9.200    <none>        4848/TCP   6s  #新创建的
traefik-forward-auth   ClusterIP   10.96.227.183   <none>        4181/TCP   12d
[root@master-01 update-yaml]# kubectl -n monitoring  get servicemonitors.monitoring.coreos.com 
NAME                          AGE
alertmanager                  48m
argocd-metrics                48m
argocd-repo-server-metrics    48m
argocd-server-metrics         48m
coredns                       48m
gitlab-ci                     48m
gitlab-runner                 48m
grafana                       48m
kube-apiserver                48m
kube-controller-manager       48m
kube-scheduler                48m
kube-state-metrics            48m
kubelet                       48m
metricapp-metrics-d545b7d8d   3m28s   #新创建的
minio                         48m
node-exporter                 48m
prometheus                    48m
prometheus-adapter            48m
prometheus-operator           48m
traefik                       48m

稍等片刻我们就可以在prometheus上看到这个target

image-20201207173600211

原文作者：Mister Li

原文链接：https://www.lishuai.fun/2020/12/08/kubevela-create-appfile-2/

发表日期：December 8th 2020, 11:21:49 am

更新日期：May 3rd 2022, 11:25:55 pm

版权声明：本文采用知识共享署名-非商业性使用 4.0 国际许可协议进行许可

CATALOG
  1. 1. routes
  2. 2. autoscale
  3. 3. rollout
  4. 4. metrics
Total : 46
2023
2022
2021
2020
kubernetes rbac k8s vulnerability ingress-nginx RBAC argocd devops gitlab harbor slb hugo pv csi s3 kubesphere prometheus sso kubevela k8s准入控制 kyverno longhorn oomkill event configmap secret sync ingress cert-manager chaos-mesh minio vpa 动态扩缩容 k8s kubevela oamm 报警 ssl-exporter traefik 混沌工程 rke k8s pvc nfs prometheus redis umami etcd kuboard docker ci buildkit redis