聶永的博客

          記錄工作/學(xué)習(xí)的點(diǎn)點(diǎn)滴滴。

          K8S Service 多種類型 Types 測(cè)試手記

          前言

          這里基于whoami示范服務(wù),部署3個(gè)實(shí)例,分別一一驗(yàn)證各種類型的K8S Service服務(wù)范疇。

          大致逐一從下面列表逐一驗(yàn)證每種類型的Service訪問(wèn)方式:

          • Service Name
          • 域名解析結(jié)果等
          • CLUSTER-IP
          • EXTERNAL-IP

          一些設(shè)定如下:

          • 測(cè)試環(huán)境K8S版本號(hào)為v1.27.3
          • K8S集群Node節(jié)點(diǎn)IP地址段范圍:10.0.1.0/24
          • K8S集群自動(dòng)生成Pod網(wǎng)段為10.43.0.0/24
          • 本書(shū)所列代碼皆可拷貝直接粘貼到終端界面直接運(yùn)行

          首先,部署whoami服務(wù)

          先部署包含3個(gè)實(shí)例的whoami

          # cat << 'EOF' | kubectl apply -f -
          apiVersion: apps/v1
          kind: Deployment
          metadata:
            name: whoami
            labels:
              app: whoami
          spec:
            replicas: 3
            selector:
              matchLabels:
                app: whoami
            template:
              metadata:
                labels:
                  app: whoami
              spec:
                containers:
                - name: whoami
                  image: containous/whoami
                  ports:
                  - containerPort: 80
                    name: web
          EOF
          

          查看一下:

          # kubectl get all
          NAME                                                      READY   STATUS      RESTARTS         AGE
          pod/whoami-767d459f67-qffqw                               1/1     Running     0                23m
          pod/whoami-767d459f67-xdv9p                               1/1     Running     0                23m
          pod/whoami-767d459f67-gwpgx                               1/1     Running     0                23m
          
          NAME                                                  READY   UP-TO-DATE   AVAILABLE   AGE
          deployment.apps/whoami                                3/3     3            3           23m
          
          NAME                                                            DESIRED   CURRENT   READY   AGE
          replicaset.apps/whoami-767d459f67                               3         3         3       23m
          

          其次,安裝busybox進(jìn)行調(diào)試

          安裝一個(gè)包含有curl的busybox方便后續(xù)調(diào)試:

          kubectl run busybox-curl --image=yauritux/busybox-curl --command -- sleep 3600
          

          另起一個(gè)終端,輸入下面命令進(jìn)入:

          kubectl exec -ti busybox-curl -n default -- sh
          

          環(huán)境準(zhǔn)備好之后,下面逐一測(cè)試各種類型:

          默認(rèn)Cluster IP模式

          K8S默認(rèn)Service為Cluster IP模式,面向內(nèi)部Pod以及通過(guò)Ingress對(duì)外提供服務(wù)。

          下面一張圖很清晰解釋清楚了PortTargetPort適用情景,Port為Service對(duì)外輸出的端口,TargetPort為服務(wù)后端Pod的端口,兩者之間有一個(gè)轉(zhuǎn)換:port -> targetPort -> containerPort

          創(chuàng)建一個(gè)Service:

          cat << 'EOF' | kubectl apply -f -
          apiVersion: v1
          kind: Service
          metadata:
            labels:
              name: whoami-clusterip
            name: whoami-clusterip
          spec:
            ports:
            - port: 80
              targetPort: 80
              protocol: TCP
            selector:
              app: whoami
          EOF
          

          部署后可以查看一下:

          NAME                          TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
          service/whoami-clusterip      ClusterIP      10.43.247.74    <none>        80/TCP         57s
          

          下面就需要逐一測(cè)試了。

          域名形式:

          # curl whoami-clusterip
          Hostname: whoami-767d459f67-gwpgx
          IP: 127.0.0.1
          IP: 10.42.8.35
          RemoteAddr: 10.42.9.32:35968
          GET / HTTP/1.1
          Host: whoami-clusterip
          User-Agent: curl/7.81.0
          Accept: */*
          

          Cluster IP形式:

          # curl 10.43.247.74
          Hostname: whoami-767d459f67-qffqw
          IP: 127.0.0.1
          IP: 10.42.3.73
          RemoteAddr: 10.42.9.32:42398
          GET / HTTP/1.1
          Host: 10.43.247.74
          User-Agent: curl/7.81.0
          Accept: */*
          

          域名解析,只解析到Cluster IP上:

          # nslookup whoami-clusterip
          Server:		10.43.0.10
          Address:	10.43.0.10:53
          
          Name:	whoami-clusterip.default.svc.cluster.local
          Address: 10.43.247.74
          

          External IP模式

          原理同Cluster IP模式,為指定服務(wù)綁定一個(gè)額外的一個(gè)IP地址。當(dāng)終端訪問(wèn)該IP地址,將流量一樣轉(zhuǎn)發(fā)到Service。

          當(dāng)訪問(wèn)external IP,其端口轉(zhuǎn)換過(guò)程:port -> targetPort -> containerPort

          與默認(rèn)Service相比,端口轉(zhuǎn)換流程沒(méi)有增加,但好處對(duì)外暴露了一個(gè)可訪問(wèn)的IP地址,不過(guò)可能需要在交換機(jī)/路由器層面提供動(dòng)靜態(tài)路由支持。

          cat << 'EOF' | kubectl apply -f -
          apiVersion: v1
          kind: Service
          metadata:
            labels:
              name: whoami-externalip
            name: whoami-externalip
          spec:
            ports:
            - port: 80
              targetPort: 80
              protocol: TCP
            selector:
              app: whoami
            externalIPs:
            - 10.10.10.10
          EOF
          

          服務(wù)顯示如下,綁定了指定的擴(kuò)展IP地址10.10.10.10

          # NAME                          TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
          service/whoami-externalip     ClusterIP      10.43.192.118   10.10.10.10   80/TCP         57s
          

          kube-proxy 將在每一個(gè)Node節(jié)點(diǎn)為10.10.10.10上建立一個(gè)轉(zhuǎn)發(fā)規(guī)則,該IP地址的80端口將直接轉(zhuǎn)發(fā)到對(duì)應(yīng)的后端三個(gè)whoami Pod 上。

          -A KUBE-SERVICES -d 10.10.10.10/32 -p tcp -m comment --comment "default/whoami-externalip external IP" -m tcp --dport 80 -j KUBE-EXT-QN5HIEVYUPDP6UNK
          
          ......
          -A KUBE-EXT-QN5HIEVYUPDP6UNK -j KUBE-SVC-QN5HIEVYUPDP6UNK
          ......
          
          -A KUBE-SVC-QN5HIEVYUPDP6UNK ! -s 10.42.0.0/16 -d 10.43.192.118/32 -p tcp -m comment --comment "default/whoami-externalip cluster IP" -m tcp --dport 80 -j KUBE-MARK-MASQ
          -A KUBE-SVC-QN5HIEVYUPDP6UNK -m comment --comment "default/whoami-externalip -> 10.42.2.79:80" -m statistic --mode random --probability 0.33333333349 -j KUBE-SEP-JSAT6D2KFCSF4YLF
          -A KUBE-SVC-QN5HIEVYUPDP6UNK -m comment --comment "default/whoami-externalip -> 10.42.3.77:80" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-2R66UI3G2AY2IMNM
          -A KUBE-SVC-QN5HIEVYUPDP6UNK -m comment --comment "default/whoami-externalip -> 10.42.8.42:80" -j KUBE-SEP-ZHHIL2SAN2G37GCM
          

          訪問(wèn)域名:

          # curl whoami-externalip
          Hostname: whoami-767d459f67-gwpgx
          IP: 127.0.0.1
          IP: 10.42.8.35
          RemoteAddr: 10.42.9.32:46746
          GET / HTTP/1.1
          Host: whoami-externalip
          User-Agent: curl/7.81.0
          Accept: */*
          

          訪問(wèn)ClusterIP形式:

          # curl 10.43.192.118
          Hostname: whoami-767d459f67-qffqw
          IP: 127.0.0.1
          IP: 10.42.3.73
          RemoteAddr: 10.42.9.32:47516
          GET / HTTP/1.1
          Host: 10.43.192.118
          User-Agent: curl/7.81.0
          Accept: */*
          

          訪問(wèn)暴露的External IP:

          # curl 10.10.10.10
          Hostname: whoami-767d459f67-gwpgx
          IP: 127.0.0.1
          IP: 10.42.8.35
          RemoteAddr: 10.42.9.0:38477
          GET / HTTP/1.1
          Host: 10.10.10.10
          User-Agent: curl/7.81.0
          Accept: */*
          

          域名解析結(jié)果只解析到其對(duì)應(yīng)的Cluster IP:

          # nslookup whoami-externalip
          Server:		10.43.0.10
          Address:	10.43.0.10:53
          
          Name:	whoami-externalip.default.svc.cluster.local
          Address: 10.43.192.118
          

          NodePort 模式

          Cluster IP相比,多了一個(gè)nodePort,這個(gè)NodePort會(huì)在K8S所有Node節(jié)點(diǎn)上都會(huì)開(kāi)放。

          這里有一個(gè)端口轉(zhuǎn)換過(guò)程:nodePort -> port -> targetPort -> containerPort,多了一層數(shù)據(jù)轉(zhuǎn)換過(guò)程。

          服務(wù)定義如下:

          cat << 'EOF' | kubectl apply -f -
          apiVersion: v1
          kind: Service
          metadata:
            labels:
              name: whoami-nodeport
            name: whoami-nodeport
          spec:
            type: NodePort
            ports:
            - port: 80
              targetPort: 80
              nodePort: 30080
              protocol: TCP
            selector:
              app: whoami
          EOF
          

          查看一下服務(wù)分配地址:

          NAME                          TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
          service/whoami-nodeport       NodePort       10.43.215.233   <none>        80:30080/TCP   57s
          

          訪問(wèn)域名:

          # curl whoami-nodeport
          Hostname: whoami-767d459f67-xdv9p
          IP: 127.0.0.1
          IP: 10.42.2.75
          RemoteAddr: 10.42.9.32:36878
          GET / HTTP/1.1
          Host: whoami-nodeport
          User-Agent: curl/7.81.0
          Accept: */*
          

          測(cè)試 CLUSTER IP :

          # curl 10.43.215.233
          Hostname: whoami-767d459f67-qffqw
          IP: 127.0.0.1
          IP: 10.42.3.73
          RemoteAddr: 10.42.9.32:40552
          GET / HTTP/1.1
          Host: 10.43.215.233
          User-Agent: curl/7.81.0
          Accept: */*
          

          因?yàn)槭窃诿恳粋€(gè)K8S Node節(jié)點(diǎn)上都會(huì)開(kāi)放一個(gè)30080端口,因此可以這樣訪問(wèn) {Node IP}:{nodePort},如下Node IP地址為10.0.1.11

          # curl 10.0.1.11:30080
          Hostname: whoami-767d459f67-qffqw
          IP: 127.0.0.1
          IP: 10.42.3.73
          RemoteAddr: 10.42.1.0:1880
          GET / HTTP/1.1
          Host: 10.0.1.11:30080
          User-Agent: curl/7.81.0
          Accept: */*
          

          域名還是只解析到對(duì)應(yīng)Cluster IP:

          # nslookup whoami-nodeport
          Server:		10.43.0.10
          Address:	10.43.0.10:53
          
          Name:	whoami-nodeport.default.svc.cluster.local
          Address: 10.43.215.233
          

          LoadBalancer 模式

          LoadBalancer模式,會(huì)強(qiáng)制K8S Service自動(dòng)開(kāi)啟nodePort

          這里有一張圖,詳細(xì)解析數(shù)據(jù)流向。

          服務(wù)數(shù)據(jù)端口轉(zhuǎn)換過(guò)程:port -> nodePort -> port -> targetPort -> containerPort

          • 與默認(rèn)Cluster IP相比,多了兩層數(shù)據(jù)轉(zhuǎn)換過(guò)程
          • nodePort相比,對(duì)了一層數(shù)據(jù)轉(zhuǎn)換過(guò)程
          • externalIP相比,在小流量場(chǎng)景下就沒(méi)有什么優(yōu)勢(shì)了

          具體服務(wù)定義:

          cat << 'EOF' | kubectl apply -f -
          apiVersion: v1
          kind: Service
          metadata:
            labels:
              name: whoami-clusterip-none
            name: whoami-clusterip-none
          spec:
            clusterIP: None
            ports:
            - port: 80
              targetPort: 80
              protocol: TCP
            selector:
              app: whoami
          EOF
          

          查看一下部署結(jié)果:

          NAME                          TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
          service/whoami-loadbalancer   LoadBalancer   10.43.63.92     <pending>     80:30906/TCP   57s
          

          服務(wù)域名形式:

          # curl whoami-loadbalancer
          Hostname: whoami-767d459f67-qffqw
          IP: 127.0.0.1
          IP: 10.42.3.73
          RemoteAddr: 10.42.9.32:57844
          GET / HTTP/1.1
          Host: whoami-loadbalancer
          User-Agent: curl/7.81.0
          Accept: */*
          

          測(cè)試 CLUSTER-IP

          # curl 10.43.63.92
          Hostname: whoami-767d459f67-xdv9p
          IP: 127.0.0.1
          IP: 10.42.2.75
          RemoteAddr: 10.42.9.32:42400
          GET / HTTP/1.1
          Host: 10.43.63.92
          User-Agent: curl/7.81.0
          Accept: */*
          

          域名解析到Cluster IP:

          #  nslookup whoami-loadbalancer
          Server:		10.43.0.10
          Address:	10.43.0.10:53
          
          Name:	whoami-loadbalancer.default.svc.cluster.local
          Address: 10.43.63.92
          

          安裝LoadBalancer

          此時(shí)whoami-loadbalancer服務(wù)對(duì)應(yīng)的EXTERNAL-IP<pending>,我們需要安裝一個(gè)負(fù)載均衡器,可以選擇MetalLB作為負(fù)載均衡器。

          # kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.13.11/config/manifests/metallb-native.yaml
          

          稍后分配可用的LoadBalaner可分配的地址池:

          cat << 'EOF' | kubectl apply -f -
          apiVersion: metallb.io/v1beta1
          kind: IPAddressPool
          metadata:
            name: default-pool
            namespace: metallb-system
          spec:
            addresses:
            - 10.0.1.100-10.0.1.200
          ---
          apiVersion: metallb.io/v1beta1
          kind: L2Advertisement
          metadata:
            name: default
            namespace: metallb-system
          spec:
            ipAddressPools:
            - default-pool
          EOF
          

          等安裝完成之后,可以看到服務(wù)whoami-loadbalancer分配的IP地址為 10.0.1.101

          NAME                          TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
          ......
          service/whoami-loadbalancer LoadBalancer   10.43.63.92     10.0.1.101         80:30906/TCP   27h
          ......
          

          測(cè)試負(fù)載均衡IP地址

          測(cè)試一下:

          # curl 10.0.1.101
          Hostname: whoami-767d459f67-xdv9p
          IP: 127.0.0.1
          IP: 10.42.2.78
          RemoteAddr: 10.42.8.0:33658
          GET / HTTP/1.1
          Host: 10.0.1.101
          User-Agent: curl/7.79.1
          Accept: */*
          

          我們看到該服務(wù)分配的端口為80:30906/TCP30906為K8S為該服務(wù)自動(dòng)生成的NodePort類型端口。

          可以找任一K8S Node節(jié)點(diǎn)IP地址測(cè)試一下:

          # curl 10.0.1.12:30906
          Hostname: whoami-767d459f67-qffqw
          IP: 127.0.0.1
          IP: 10.42.3.77
          RemoteAddr: 10.42.2.0:9717
          GET / HTTP/1.1
          Host: 10.0.1.12:30906
          User-Agent: curl/7.81.0
          Accept: */*
          

          分析一下路由表,可以分析到該負(fù)載均衡的External_IP:80的打流量到NodePort:30906上,然后走Service對(duì)應(yīng){Pod:80}流量分發(fā)邏輯。

          -A KUBE-NODEPORTS -p tcp -m comment --comment "default/whoami-loadbalancer" -m tcp --dport 30906 -j KUBE-EXT-NBTYBEEXACZI7DPC
          
          ......
          
          -A KUBE-SERVICES -d 10.0.1.101/32 -p tcp -m comment --comment "default/whoami-loadbalancer loadbalancer IP" -m tcp --dport 80 -j KUBE-EXT-NBTYBEEXACZI7DPC
          
          ......
          
          -A KUBE-EXT-NBTYBEEXACZI7DPC -m comment --comment "masquerade traffic for default/whoami-loadbalancer external destinations" -j KUBE-MARK-MASQ
          -A KUBE-EXT-NBTYBEEXACZI7DPC -j KUBE-SVC-NBTYBEEXACZI7DPC
          
          ......
          
          -A KUBE-SVC-NBTYBEEXACZI7DPC ! -s 10.42.0.0/16 -d 10.43.63.92/32 -p tcp -m comment --comment "default/whoami-loadbalancer cluster IP" -m tcp --dport 80 -j KUBE-MARK-MASQ
          -A KUBE-SVC-NBTYBEEXACZI7DPC -m comment --comment "default/whoami-loadbalancer -> 10.42.2.79:80" -m statistic --mode random --probability 0.33333333349 -j KUBE-SEP-E3K3SUYNFWT2VICE
          -A KUBE-SVC-NBTYBEEXACZI7DPC -m comment --comment "default/whoami-loadbalancer -> 10.42.3.77:80" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-HG5MYVVID7GJOZA7
          -A KUBE-SVC-NBTYBEEXACZI7DPC -m comment --comment "default/whoami-loadbalancer -> 10.42.8.42:80" -j KUBE-SEP-GFJH72YCBKBFB6OG
          

          Headless 無(wú)頭模式

          一般應(yīng)用在有狀態(tài)的服務(wù),或需要終端調(diào)用者自己實(shí)現(xiàn)負(fù)載均衡,等一些特定場(chǎng)景。

          通過(guò)調(diào)用者從端口角度分析,數(shù)據(jù)轉(zhuǎn)換流程:targetPort -> containerPort

          在意服務(wù)性能的場(chǎng)景,不妨試試無(wú)頭模式。


          服務(wù)定義:

          cat << 'EOF' | kubectl apply -f -
          apiVersion: v1
          kind: Service
          metadata:
            labels:
              name: whoami-clusterip-none
            name: whoami-clusterip-none
          spec:
            clusterIP: None
            ports:
            - port: 80
              targetPort: 80
              protocol: TCP
            selector:
              app: whoami
          EOF
          

          查看服務(wù)部署情況:

          NAME                          TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
          service/whoami-clusterip-none   ClusterIP      None            <none>                     80/TCP         9h
          

          通過(guò)service域名訪問(wèn),K8S會(huì)自動(dòng)根據(jù)服務(wù)域名whoami-clusterip-none進(jìn)行pick后端對(duì)應(yīng)Pod IP地址。

          # curl whoami-clusterip-none
          Hostname: whoami-767d459f67-xdv9p
          IP: 127.0.0.1
          IP: 10.42.2.75
          RemoteAddr: 10.42.9.32:34998
          GET / HTTP/1.1
          Host: whoami-clusterip-none
          User-Agent: curl/7.81.0
          Accept: */*
          

          查詢DNS會(huì)把所有節(jié)點(diǎn)都列出來(lái)。

          # nslookup whoami-clusterip-none
          Server:		10.43.0.10
          Address:	10.43.0.10:53
          
          Name:	whoami-clusterip-none.default.svc.cluster.local
          Address: 10.42.3.73
          Name:	whoami-clusterip-none.default.svc.cluster.local
          Address: 10.42.2.75
          Name:	whoami-clusterip-none.default.svc.cluster.local
          Address: 10.42.8.35
          

          External Name模式

          用于引進(jìn)帶域名的外部服務(wù),這里引入內(nèi)部服務(wù)作為測(cè)試。

          多了一層域名解析過(guò)程,端口轉(zhuǎn)換流程依賴于所引入服務(wù)的服務(wù)設(shè)定。

          服務(wù)定義:

          cat << 'EOF' | kubectl apply -f -
          apiVersion: v1
          kind: Service
          metadata:
            labels:
              name: whoami-externalname
            name: whoami-externalname
          spec:
            type: ExternalName
            externalName: whoami-clusterip.default.svc.cluster.local
          EOF
          

          這里外聯(lián)的是whoami-clusterip服務(wù)的完整訪問(wèn)域名。

          查看服務(wù)部署情況:

          NAME                          TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
          service/whoami-externalname     ExternalName   <none>          whoami-clusterip.default   <none>         9h
          

          根據(jù)域名訪問(wèn)測(cè)試:

          # curl whoami-externalname
          Hostname: whoami-767d459f67-qffqw
          IP: 127.0.0.1
          IP: 10.42.3.77
          RemoteAddr: 10.42.9.35:36756
          GET / HTTP/1.1
          Host: whoami-externalname
          User-Agent: curl/7.81.0
          Accept: */*
          

          DNS解析結(jié)果:

          # nslookup whoami-externalname
          Server:		10.43.0.10
          Address:	10.43.0.10:53
          
          whoami-externalname.default.svc.cluster.local	canonical name = whoami-clusterip.default.svc.cluster.local
          Name:	whoami-clusterip.default.svc.cluster.local
          Address: 10.43.247.74
          

          小結(jié)

          簡(jiǎn)要分析了各種類型Service定義、服務(wù)引用場(chǎng)景以及測(cè)試流程等,整理清楚了,也方便在具體業(yè)務(wù)場(chǎng)景中進(jìn)行抉擇選擇具體服務(wù)類型。

          posted on 2023-09-13 10:13 nieyong 閱讀(228) 評(píng)論(0)  編輯  收藏 所屬分類: 容器

          公告

          所有文章皆為原創(chuàng),若轉(zhuǎn)載請(qǐng)標(biāo)明出處,謝謝~

          新浪微博,歡迎關(guān)注:

          導(dǎo)航

          <2023年9月>
          272829303112
          3456789
          10111213141516
          17181920212223
          24252627282930
          1234567

          統(tǒng)計(jì)

          常用鏈接

          留言簿(58)

          隨筆分類(130)

          隨筆檔案(151)

          個(gè)人收藏

          最新隨筆

          搜索

          最新評(píng)論

          閱讀排行榜

          評(píng)論排行榜

          主站蜘蛛池模板: 台中县| 沧州市| 建平县| 梁河县| 潞城市| 三原县| 河池市| 南川市| 连云港市| 葫芦岛市| 房产| 盖州市| 资源县| 西贡区| 柳林县| 沅陵县| 水城县| 海淀区| 石楼县| 即墨市| 宿州市| 广饶县| 义乌市| 石台县| 武川县| 榆社县| 绥滨县| 武陟县| 辽阳市| 哈尔滨市| 桐乡市| 阿克苏市| 普兰县| 邹平县| 泽州县| 济宁市| 汉阴县| 巢湖市| 小金县| 阿坝| 呼玛县|