æ¥è‡ªåQ?nbsp;http://tonybai.com/2016/02/26/deploy-a-private-docker-registry/
安装部çÖvä¸€ä¸ªç§æœ‰çš„Docker Registry是引入ã€å¦ä¹ 和使用 Docker ˜q™é—¨æŠ€æœ¯çš„å¿…ç»ä¹‹èµ\之一。尤其是当Docker被所在组¾l‡æŽ¥å—,更多人ã€é¡¹ç›®å’Œäº§å“开始接触和使用Dockeræ—Óž¼Œå˜å‚¨å’Œåˆ†å‘自制的Docker image便æˆäº†åˆšéœ€ã€‚Docker Registry一如既往的ç‘ô承了“Dockerå‘多”的特点,为椘q™é‡Œž®†è‡ªå·±æå»?#8221;å„ç±»”Registry˜q‡ç¨‹ä¸æ‰§è¡Œçš„æ¥éª¤ã€é‡åˆ°çš„问题记录下æ¥åQŒäؓ己备忘,ä¸ÞZ»–å‚考ã€?/p>
Dockeråœ?015òq´æŽ¨å‡ÞZº† distribution ™å¹ç›®åQŒå³Docker Registry 2。相比于 old registry åQŒRegistry 2使用Go实现åQŒåœ¨å®‰å…¨æ€§ã€æ€§èƒ½æ–šw¢å‡æœ‰å¤§å¹…改进。Registry设计了全新的Rest APIåQŒåƈ且在imageå˜å‚¨æ ¼å¼½{‰æ–¹é¢ä¸å†å…¼å®¹äºŽold Registry。去òq?月䆾åQŒdocker官方hub使用Registriy 2.1替代了原先的old Registryã€‚å¦‚æžœä½ è¦ä¸ŽRegistry2交互åQŒä½ çš„Docker版本臛_°‘è¦æ˜¯Docker 1.6ã€?/p>
Docker的开å‘者也一直在致力于改善Registry安装和ä‹É用的体验åQŒé€šè¿‡æä¾› 官方Registry Imageä»¥åŠ Docker Compose工具 ½{‰æ¥½Ž€åŒ–Registry的酾|®ã€‚丘q‡åœ¨æœ¬æ–‡ä¸ï¼Œæˆ‘ä»¬åªæ˜¯åˆ©ç”¨Docker以åŠRegistry的官方Imageæ¥éƒ¨¾|²RegistryåQŒè¿™æ äh›´ä¾¿äºŽå…¨é¢äº†è§£Registry的部¾|²é…¾|®ç»†èŠ‚ã€?/p>
Registry2在镜åƒå˜å‚¨æ–¹é¢ä¸ä»…æ”¯æŒæœ¬åœ°ç›˜åQŒè¿˜æ”¯æŒè¯¸å¤šä¸ÀLµ½W¬ä¸‰æ–¹å˜å‚¨æ–¹æ¡ˆã€‚通过分布å¼å˜å‚¨ç³»¾lŸä½ ˜q˜å¯ä»¥å®žçŽîC¸€ä¸ªåˆ†å¸ƒå¼Docker RegistryæœåŠ¡ã€‚è¿™é‡Œä»…ä»¥æœ¬åœ°ç›˜ä»¥åŠsingle node registry2ä¸ÞZ¾‹ã€?/p>
˜q™é‡Œ˜q˜æ˜¯å¤ç”¨ä»¥å¾€æ–‡ç« ä¸çš„Docker环境åQ?/p>
Docker Registry Server: 10.10.105.71 Ubuntu 14.04 3.16.0-57-genericåQ›docker 1.9.1 其他两个工作ServeråQ? 10.10.105.72 Ubuntu 14.04 3.19.0-25-generic; docker 1.9.1 10.10.126.101 Ubuntu 12.04 3.16.7-013607-generic; docker 1.9.1
本次Registryä½¿ç”¨å½“å‰æœ€æ–°stable版本:Registry 2.3.0。由于镜åƒé‡‡ç”¨æœ¬åœ°ç£ç›˜å˜å‚¨ï¼Œroot分区较å°åQŒéœ€è¦æ˜ ž®„ä‹É用其他volumeã€?/p>
本以为Docker Registryçš„æå»ºæ˜¯ä½•å…¶½Ž€å•çš„åQŒç”šè‡³ç®€å•到通过一行命令就å¯ä»¥å®Œæˆçš„。比如我们在Registry Server上执行:
在~/dockerregistry下,执行åQ? $sudo docker run -d -p 5000:5000 -v `pwd`/data:/var/lib/registry --restart=always --name registry registry:2 Unable to find image 'registry:2' locally 2: Pulling from library/registry f32095d4ba8a: Pull complete 9b607719a62a: Pull complete 973de4038269: Pull complete 2867140211c1: Pull complete 8da16446f5ca: Pull complete fd8c38b8b68d: Pull complete 136640b01f02: Pull complete e039ba1c0008: Pull complete c457c689c328: Pull complete Digest: sha256:339d702cf9a4b0aa665269cc36255ee7ce424412d56bee9ad8a247afe8c49ef1 Status: Downloaded newer image for registry:2 e9088ef901cb00546c59f89defa4625230f4b36b0a44b3713f38ab3d2a5a2b44 $ docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE registry 2 c457c689c328 9 days ago 165.7 MB $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES e9088ef901cb registry:2 "/bin/registry /etc/d" About a minute ago Up About a minute 0.0.0.0:5000->5000/tcp registry
Registry containerå·²ç»è·‘è“væ¥äº†åQŒå…¶å¯åŠ¨æ—¥å¿—å¯ä»¥é€šè¿‡åQšdocker logs registry查看ã€?/p>
我们åœ?1本地¾l™busybox:latest打一个tagåQŒåƈž®è¯•ž®†æ–°tag下的image push到Registryä¸åŽ»åQ?/p>
$ docker tag busybox:latest 10.10.105.71:5000/tonybai/busybox:latest $ docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE registry 2 c457c689c328 9 days ago 165.7 MB busybox latest 65e4158d9625 9 days ago 1.114 MB 10.10.105.71:5000/tonybai/busybox latest 65e4158d9625 9 days ago 1.114 MB ... ...
push到Registryä¸ï¼š
$ docker push 10.10.105.71:5000/tonybai/busybox The push refers to a repository [10.10.105.71:5000/tonybai/busybox] (len: 1) unable to ping registry endpoint https://10.10.105.71:5000/v0/ v2 ping attempt failed with error: Get https://10.10.105.71:5000/v2/: Tunnel or SSL Forbidden v1 ping attempt failed with error: Get https://10.10.105.71:5000/v1/_ping: Tunnel or SSL Forbidden
å‡ºé”™äº†ï¼½Ž€å•分æžäº†ä¸€ä¸‹ï¼Œå¯èƒ½æ˜?1上docker daemoné…ç½®ä¸åŠ äº†http代ç†çš„ç¼˜æ•…ï¼Œå¯ÆD‡´æ— 法ping通registry endpoint。于是在/etc/default/docker䏿³¨é‡ŠæŽ‰export http_proxy=”xxx”的设¾|®ï¼Œòq‰™‡å¯docker daemonã€?/p>
冿¬¡ž®è¯•pushåQ?/p>
$ docker push 10.10.105.71:5000/tonybai/busybox The push refers to a repository [10.10.105.71:5000/tonybai/busybox] (len: 1) unable to ping registry endpoint https://10.10.105.71:5000/v0/ v2 ping attempt failed with error: Get https://10.10.105.71:5000/v2/: tls: oversized record received with length 20527 v1 ping attempt failed with error: Get https://10.10.105.71:5000/v1/_ping: tls: oversized record received with length 20527
虽然˜q˜æ˜¯å¤ÞpÓ|åQŒä½†é”™è¯¯ä¿¡æ¯å·²æœ‰æ‰€ä¸åŒäº†ã€‚è¿™‹Æ¡çœ‹æ¥è¿žæŽ¥æ˜¯å¯ä»¥å»ºç«‹çš„,但client端通过https讉K—®server端,ä¼ég¹Žæƒ³tls通信åQŒä½†˜q™ä¸€˜q‡ç¨‹òq¶æœªå®Œæˆã€?/p>
在其他机器上ž®è¯•push image到registry也é‡åˆîCº†åŒæ ·çš„错误输出,如下åQ?/p>
10.10.105.72: $ docker push 10.10.105.71:5000/tonybai/ubuntu The push refers to a repository [10.10.105.71:5000/tonybai/ubuntu] (len: 1) unable to ping registry endpoint https://10.10.105.71:5000/v0/ v2 ping attempt failed with error: Get https://10.10.105.71:5000/v2/: tls: oversized record received with length 20527 v1 ping attempt failed with error: Get https://10.10.105.71:5000/v1/_ping: tls: oversized record received with length 20527
ä»Žé”™è¯¯ä¿¡æ¯æ¥çœ‹ï¼Œclient与Registry交互åQŒé»˜è®¤å°†é‡‡ç”¨https讉K—®åQŒä½†æˆ‘们在install Registryæ—¶åÆˆæœªé…¾|®æŒ‡å®šä“Q何tls相关的keyå’Œcrtæ–‡äšgåQŒhttps讉K—®å®šç„¶å¤ÞpÓ|ã€‚è¦æƒ›_¼„清这个问题,åªèƒ½æŸ¥çœ‹ Registry Manual ã€?/p>
Registry的文档还是相对详ž®½çš„。在文档ä¸ï¼Œæˆ‘们扑ֈ°äº?nbsp;Insecure Registry åQŒå³æŽ¥æ”¶plain http讉K—®çš„Registry的酾|®å’Œä½¿ç”¨æ–ÒŽ(gu¨©)³•åQŒè™½ç„¶è¿™ä¸æ˜¯å®˜æ–¹æŽ¨èçš„ã€?/p>
实际上对于我们内部网¾lœè€Œè¨€åQŒInsecure Registry基本能满‘³éœ€æ±‚,部çÖv˜q‡ç¨‹ä¹Ÿé¿å…了secure registry的那些ç¹çæ¥éª¤ï¼Œæ¯”如制作和部¾|²è¯ä¹¦ç‰ã€?/p>
ä¸ÞZº†æå¾ä¸€ä¸ªInsecure RegistryåQŒæˆ‘们需è¦å…ˆæ¸…ç†ä¸€ä¸‹ä¸Šé¢å·²¾lå¯åŠ¨çš„Registry容器ã€?/p>
$ docker stop registry registry $ docker rm registry registry
修改Registry server上的Docker daemon的酾|®ï¼Œä¸ºDOCKER_OPTSå¢žåŠ –insecure-registryåQ?/p>
DOCKER_OPTS="--insecure-registry 10.10.105.71:5000 ....
é‡å¯Docker DaemonåQŒå¯åЍRegistry容器åQ?/p>
$ sudo service docker restart docker stop/waiting docker start/running, process 6712 $ sudo docker run -d -p 5000:5000 -v `pwd`/data:/var/lib/registry --restart=always --name registry registry:2 5966e92fce9c34705050e19368d19574e021a272ede1575385ef35ecf5cea019
ž®è¯•冿¬¡Push image:
$ docker push 10.10.105.71:5000/tonybai/busybox The push refers to a repository [10.10.105.71:5000/tonybai/busybox] (len: 1) 65e4158d9625: Pushed 5506dda26018: Pushed latest: digest: sha256:800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892 size: 2739
˜q™å›žpush okåQ?/p>
我们ž®†æœ¬åœ°çš„tagåšuntag处ç†åQŒå†ä»ŽRegistry pull相关imageåQ?/p>
$ docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE registry 2 c457c689c328 9 days ago 165.7 MB 10.10.105.71:5000/tonybai/busybox latest 65e4158d9625 9 days ago 1.114 MB busybox latest 65e4158d9625 9 days ago 1.114 MB ubuntu 14.04 6cc0fc2a5ee3 5 weeks ago 187.9 MB $ docker rmi 10.10.105.71:5000/tonybai/busybox Untagged: 10.10.105.71:5000/tonybai/busybox:latest $ docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE registry 2 c457c689c328 9 days ago 165.7 MB busybox latest 65e4158d9625 9 days ago 1.114 MB ubuntu 14.04 6cc0fc2a5ee3 5 weeks ago 187.9 MB $ docker pull 10.10.105.71:5000/tonybai/busybox Using default tag: latest latest: Pulling from tonybai/busybox Digest: sha256:800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892 Status: Downloaded newer image for 10.10.105.71:5000/tonybai/busybox:latest $ docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE registry 2 c457c689c328 9 days ago 165.7 MB 10.10.105.71:5000/tonybai/busybox latest 65e4158d9625 9 days ago 1.114 MB busybox latest 65e4158d9625 9 days ago 1.114 MB ubuntu 14.04 6cc0fc2a5ee3 5 weeks ago 187.9 MB
å¯ä»¥çœ‹åˆ°åQšPull˜q‡ç¨‹ä¹Ÿå¾ˆ™åºåˆ©ã€?/p>
在Private Registry2䏿Ÿ¥çœ‹æˆ–‹‚€ç´¢Repository或imagesåQ?nbsp;ž®†ä¸èƒ½ç”¨docker search åQ?/p>
$ docker search 10.10.105.71:5000/tonybai/busybox/ Error response from daemon: Unexpected status code 404
但通过v2版本的APIåQŒæˆ‘们å¯ä»¥å®žçŽ°ç›¸åŒç›®çš„:
$curl http://10.10.105.71:5000/v2/_catalog {"repositories":["tonybai/busybox"]} $ curl http://10.10.105.71:5000/v2/tonybai/busybox/tags/list {"name":"tonybai/busybox","tags":["latest"]}
在其他主æœÞZ¸ŠåQŒæˆ‘们å°è¯•pull busyboxåQ?/p>
10.10.105.72: $docker pull 10.10.105.71:5000/tonybai/busybox Using default tag: latest Error response from daemon: unable to ping registry endpoint https://10.10.105.71:5000/v0/ v2 ping attempt failed with error: Get https://10.10.105.71:5000/v2/: tls: oversized record received with length 20527 v1 ping attempt failed with error: Get https://10.10.105.71:5000/v1/_ping: tls: oversized record received with length 20527
我们å‘çŽ°ä¾æ—§ä¸èƒ½pullå’ŒpushåQ在Registry手册ä¸è®²åˆŽÍ¼Œå¦‚果采用insecure registry的模å¼ï¼Œé‚£ä¹ˆæ‰€æœ‰ä¸ŽRegistry交互的主æœÞZ¸Šçš„Docker Daemon都è¦é…ç½®åQ?#8211;insecure-registry选项ã€?/p>
我们按照上é¢çš„é…¾|®æ–¹æ³•,修改105.72上的/etc/default/dockeråQŒé‡å¯Docker daemonåQŒå†æ‰§è¡Œpull/pushž®×ƒ¼šå¾—到æ£ç¡®çš„结果:
$ sudo vi /etc/default/docker $ sudo service docker restart docker stop/waiting docker start/running, process 10614 $ docker pull 10.10.105.71:5000/tonybai/busybox Using default tag: latest latest: Pulling from tonybai/busybox 5506dda26018: Pull complete 65e4158d9625: Pull complete Digest: sha256:800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892 Status: Downloaded newer image for 10.10.105.71:5000/tonybai/busybox:latest $ docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE ubuntu 14.04 36248ae4a9ac 8 days ago 187.9 MB 10.10.105.71:5000/tonybai/ubuntu 14.04 36248ae4a9ac 8 days ago 187.9 MB 10.10.105.71:5000/tonybai/busybox latest 65e4158d9625 9 days ago 1.114 MB $ docker push 10.10.105.71:5000/tonybai/ubuntu The push refers to a repository [10.10.105.71:5000/tonybai/ubuntu] (len: 1) 36248ae4a9ac: Pushed 8ea5373bf5a6: Pushed 2e0188208e83: Pushed e3c70beaa378: Pushed 14.04: digest: sha256:72e56686cb9fb38438f0fd68fecf02ef592ce2ef7069bbf97802d959d568c5cc size: 6781
Docker官方是推èä½ é‡‡ç”¨Secure Registry的工作模å¼çš„åQŒå³transport采用tls。这æ ähˆ‘们就需è¦äØ“Registryé…ç½®tls所需的keyå’Œcrtæ–‡äšg了ã€?/p>
我们首先清ç†ä¸€ä¸‹çŽ¯å¢ƒï¼Œž®†ä¸Šé¢çš„Insecure RegistryåœæŽ‰òq¶rm掉;ž®†å„åîC¸»æœÞZ¸ŠDocker Daemonçš„DOCKER_OPTSé…ç½®ä¸çš„–insecure-registryåŽÀLމåQŒåƈé‡å¯Docker Daemonã€?/p>
å¦‚æžœä½ æ‹¥æœ‰ä¸€ä¸ªåŸŸå,域å下主机æä¾›RegistryæœåŠ¡åQŒåÆˆä¸”ä½ æ‹¥æœ‰æŸçŸ¥åCA½{„¡Övçš„è¯ä¹¦æ–‡ä»Óž¼Œé‚£ä¹ˆä½ å¯ä»¥å¾ç«‹è“v一个Secure Registry。丘q‡æˆ‘˜q™é‡Œæ²¡æœ‰çްæˆçš„è¯ä¹¦ï¼Œåªèƒ½ä½¿ç”¨è‡ªç¾¾|²çš„è¯ä¹¦ã€‚ä¸¥æ ¼æ¥è®ÔŒ¼Œä½¿ç”¨è‡ªç¾¾|²çš„è¯ä¹¦åœ¨Docker官方çœég¸ä¾æ—§å±žäºŽInsecureåQŒä¸˜q‡è¿™é‡Œåªæ˜¯å€ŸåŠ©è‡ªç¾¾|²çš„è¯ä¹¦æ¥è¯´æ˜Žä¸€ä¸‹Secure Registry的部¾|²æ¥éª¤çŞ了ã€?/p>
å¦‚æžœä½ æœ‰çŸ¥åCA½{„¡Övçš„è¯ä¹¦ï¼Œé‚£ä¹ˆ˜q™æ¥å¯ç›´æŽ¥å¿½ç•¥ã€?/p>
$ openssl req -newkey rsa:2048 -nodes -sha256 -keyout certs/domain.key -x509 -days 365 -out certs/domain.crt Generating a 2048 bit RSA private key ..............+++ ............................................+++ writing new private key to 'certs/domain.key' ----- You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [AU]:CN State or Province Name (full name) [Some-State]:Liaoning Locality Name (eg, city) []:shenyang Organization Name (eg, company) [Internet Widgits Pty Ltd]:foo Organizational Unit Name (eg, section) []:bar Common Name (e.g. server FQDN or YOUR name) []:mydockerhub.com Email Address []:bigwhite.cn@gmail.com
å¯åЍ另è¯ä¹¦çš„RegistryåQ?/p>
$ docker run -d -p 5000:5000 --restart=always --name registry \ -v `pwd`/data:/var/lib/registry \ -v `pwd`/certs:/certs \ -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \ -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \ registry:2 35e8ce77dd455f2bd50854e4581cd52be8a137f4aaea717239b6d676c5ea5777
ç”׃ºŽè¯ä¹¦çš„CN是mydockerhub.comåQŒæˆ‘们需è¦ä¿®æ”¹ä¸€ä¸?etc/hostsæ–‡äšg:
10.10.105.71 mydockerhub.com
釿–°ä¸ºbusybox制作一个tag:
$docker tag busybox:latest mydockerhub.com:5000/tonybai/busybox:latest
Push到Registry:
$ docker push mydockerhub.com:5000/tonybai/busybox The push refers to a repository [mydockerhub.com:5000/tonybai/busybox] (len: 1) unable to ping registry endpoint https://mydockerhub.com:5000/v0/ v2 ping attempt failed with error: Get https://mydockerhub.com:5000/v2/: x509: certificate signed by unknown authority v1 ping attempt failed with error: Get https://mydockerhub.com:5000/v1/_ping: x509: certificate signed by unknown authority
pushå¤ÞpÓ|了ï¼ä»Žé”™è¯¯æ—¥å¿—æ¥çœ‹ï¼Œdocker clientè®¤äØ“serverä¼ è¾“˜q‡æ¥çš„è¯ä¹¦çš„½{„¡Övæ–ÒŽ(gu¨©)˜¯ä¸€ä¸ªunknown authorityåQˆæœªçŸ¥çš„CAåQ‰ï¼Œå› æ¤éªŒè¯å¤ÞpÓ|。我们需è¦è®©docker client安装我们的CAè¯ä¹¦åQ?/p>
$ sudo mkdir -p /etc/docker/certs.d/mydockerhub.com:5000 $ sudo cp certs/domain.crt /etc/docker/certs.d/mydockerhub.com:5000/ca.crt $ sudo service docker restart //安装è¯ä¹¦åŽï¼Œé‡å¯Docker Daemon
冿‰§è¡ŒPushåQŒæˆ‘们看åˆîCº†æˆåŠŸçš„è¾“å‡ºæ—¥å¿—ã€‚ç”±äºŽdata目录下之å‰å·²¾l被push了tonybai/busybox repositoryåQŒå› æ¤æ½C?#8220;å·²å˜åœ?#8221;åQ?/p>
$docker push mydockerhub.com:5000/tonybai/busybox The push refers to a repository [mydockerhub.com:5000/tonybai/busybox] (len: 1) 65e4158d9625: Image already exists 5506dda26018: Image already exists latest: digest: sha256:800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892 size: 2739
我们æ¢å…¶ä»–机器试试访问这个secure registryã€‚æ ¹æ®ä¹‹å‰çš„è¦æ±‚åQŒæˆ‘们照猫画虎的修改一下hostsæ–‡äšgåQŒå®‰è£…ca.certåQŒåŽ»é™?#8211;insecure-registry选项åQŒåƈé‡å¯Docker daemon。之åŽå°è¯•从registry pull imageåQ?/p>
$ docker pull mydockerhub.com:5000/tonybai/busybox Using default tag: latest latest: Pulling from tonybai/busybox Digest: sha256:800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892 Status: Downloaded newer image for mydockerhub.com:5000/tonybai/busybox:latest $ docker images REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE 10.10.105.71:5000/tonybai/ubuntu 14.04 36248ae4a9ac 9 days ago 187.9 MB ubuntu 14.04 36248ae4a9ac 9 days ago 187.9 MB 10.10.105.71:5000/tonybai/busybox latest 65e4158d9625 9 days ago 1.114 MB mydockerhub.com:5000/tonybai/busybox latest 65e4158d9625 9 days ago 1.114 MB
˜q™æ ·æ¥çœ‹åQŒå¦‚æžœä‹É用自½{„¡Övçš„è¯ä¹¦ï¼Œé‚£ä¹ˆæ‰€æœ‰è¦ä¸ŽRegistry交互的Dockerä¸ÀLœºéƒ½éœ€è¦å®‰è£…mydockerhub.comçš„ca.crt(domain.crt)ã€‚ä½†å¦‚æžœä½ ä‹É用知åCAåQŒè¿™ä¸€æ¥ä¹Ÿž®±å¯ä»¥å¿½ç•¥ã€?/p>
Registryæä¾›äº†ä¸€¿U基¼‹€çš„é‰´æƒæ–¹å¼ã€‚æˆ‘ä»¬é€šè¿‡ä¸‹é¢æ¥éª¤å›_¯ä¸ºRegistryåŠ ä¸ŠåŸºç¡€é‰´æƒåQ?/p>
在Register server上,为Registryå¢žåŠ foo用户åQŒå¯†ç foo123åQšï¼ˆä¹‹å‰éœ€è¦åœæŽ‰å·²æœ‰çš„RegistryåQŒåÆˆåˆ é™¤ä¹‹ï¼‰
//生æˆé‰´æƒå¯†ç æ–‡äšg $ mkdir auth $ docker run --entrypoint htpasswd registry:2 -Bbn foo foo123 > auth/htpasswd $ ls auth htpasswd //å¯åЍ另鉴æƒåŠŸèƒ½çš„RegistryåQ? $ docker run -d -p 5000:5000 --restart=always --name registry \ -v `pwd`/auth:/auth \ -e "REGISTRY_AUTH=htpasswd" \ -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \ -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \ -v `pwd`/data:/var/lib/registry \ -v `pwd`/certs:/certs \ -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \ -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \ registry:2 199ad0b3591fb9613b21b1c96f017267f3c39661a7025d30df636c6805e7ab50
åœ?05.72上,我们ž®è¯•push image到RegistryåQ?/p>
$ docker push mydockerhub.com:5000/tonybai/busybox The push refers to a repository [mydockerhub.com:5000/tonybai/busybox] (len: 1) 65e4158d9625: Image push failed Head https://mydockerhub.com:5000/v2/tonybai/busybox/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4: no basic auth credentials
é”™è¯¯ä¿¡æ¯æç¤ºåQšé‰´æƒå¤±è´¥ã€?/p>
�2上执行docker login:
$docker login mydockerhub.com:5000 Username: foo Password: Email: bigwhite.cn@gmail.com WARNING: login credentials saved in /home/baiming/.docker/config.json Login Succeeded
loginæˆåŠŸåŽï¼Œå†è¡ŒPushåQ?/p>
$ docker push mydockerhub.com:5000/tonybai/busybox The push refers to a repository [mydockerhub.com:5000/tonybai/busybox] (len: 1) 65e4158d9625: Image already exists 5506dda26018: Image already exists latest: digest: sha256:800f2d4558acd67f52262fbe170c9fc2e67efaa6f230a74b41b555e6fcca2892 size: 2739
Push okåQ?/p>
å‰é¢æåˆ°˜q‡ï¼Œé€šè¿‡V2版Rest APIå¯ä»¥æŸ¥è¯¢Repositoryå’ŒimagesåQ?/p>
$ curl --cacert domain.crt --basic --user foo:foo123 https://mydockerhub.com:5000/v2/_catalog {"repositories":["tonybai/busybox","tonybai/ubuntu"]}
但如果è¦åˆ 除Registryä¸çš„Repository或æŸä¸ªtagçš„ImageåQŒç›®å‰v2˜q˜ä¸æ”¯æŒåQŒåŽŸå› è§ Registryçš„roadmapä¸çš„说明 ã€?/p>
ä¸è¿‡å¦‚æžœä½ çš„Registryçš„å˜å‚¨å¼•擎ä‹É用的是本地盘åQŒå€’是有一些第三方脚本å¯ä¾›ä½¿ç”¨åQŒæ¯”如:delete-docker-registry-image ã€?/p>
Registry2å‘布ä¸åˆ°1òqß_¼Œç›®å‰˜q˜æœ‰è®¸å¤šé—®é¢˜å¾…解冻I¼Œž®±æ¯”如delete imageçš„é—®é¢˜ï¼Œç›æ€¿¡åœ?.4以åŠåŽç®‹ç‰ˆæœ¬˜q™äº›é—®é¢˜ä¼šè¢«é€ä¸ªè§£å†³æŽ‰æˆ–能找åˆîC¸€ä¸ªç›¸å¯¹ç†æƒ³çš„æ–ÒŽ(gu¨©)¡ˆã€?/p>
上一节我们阘qîCº†Kubernetes的系¾lŸæž¶æž„,让大家对Kubernetesæœ‰ä¸€å®šçš„åˆæ¥äº†è§£åQŒä½†æ˜¯å°±å¦‚何使用KubernetesåQ?也许大家˜q˜ä¸çŸ¥å¦‚何下手。本文作者将带领大家如何在本地部¾|ŒÓ€é…¾|®Kubernetes集群¾|‘络环境以åŠé€šè¿‡å®žä¾‹æ¼”示跨机器æœåŠ¡é—´çš„é€šä¿¡åQŒä¸»è¦åŒ…括如下内容:
æœåŠ¡å™¨ä¿¡æ?
| Role | Hostname | IP Address | |:---------:|:----------:|:----------: | |APIServer |kubernetes |192.168.230.3| |Minion | minion1 |192.168.230.4| |Minion | minion2 |192.168.230.5|
在详¾l†ä»‹¾l部¾|²Kubernetes集群å‰ï¼Œå…ˆç»™å¤§å®¶å±•示下集¾Ÿ¤çš„逻辑架构。从下图å¯çŸ¥åQŒæ•´ä¸ªç³»¾lŸåˆ†ä¸ÞZ¸¤éƒ¨åˆ†åQŒç¬¬ä¸€éƒ¨åˆ†æ˜¯Kubernetes APIServeråQŒæ˜¯æ•´ä¸ª¾pÈ»Ÿçš„æ ¸å¿ƒï¼Œæ‰¿æ‹…é›†ç¾¤ä¸æ‰€æœ‰å®¹å™¨çš„½Ž¡ç†å·¥ä½œåQ›ç¬¬äºŒéƒ¨åˆ†æ˜¯minionåQŒè¿è¡ŒContainer DaemonåQŒæ˜¯æ‰€æœ‰å®¹å™¨æ –æ¯ä¹‹åœŽÍ¼ŒåŒæ—¶åœ¨minion上è¿è¡ŒOpen vSwitch½E‹åºåQŒé€šè¿‡GRE Tunnelè´Ÿè´£minion之间Pod的网¾lœé€šä¿¡å·¥ä½œã€?/p>
ä¸ÞZº†è§£å†³è·¨minion之间Pod的通信问题åQŒæˆ‘们在æ¯ä¸ªminion上安装Open vSwtichåQŒåƈ使用GRE或者VxLAN使得跨机器之间Pod能相互通信åQŒæœ¬æ–‡ä‹É用GREåQŒè€ŒVxLAN通常用在需è¦éš”¼›Èš„大规模网¾lœä¸ã€‚对于Open vSwitch的具体安装æ¥éª¤ï¼Œå¯å‚考这½‹?a style="text-decoration: none; color: #286ab2; outline: none !important; margin: 0px; border: 0px; padding: 0px;">åšå®¢åQŒæˆ‘们在˜q™é‡Œž®×ƒ¸å†è¯¦¾l†ä»‹¾l安装æ¥éª¤äº†ã€‚安装完Open vSwitchåŽï¼ŒæŽ¥ä¸‹æ¥ä¾¿å»ºç«‹minion1å’Œminion2之间的隧é“。首先在minion1å’Œminion2上å¾ç«‹OVS Bridge,
[root@minion1 ~]# ovs-vsctl add-br obr0
接下æ¥å¾ç«‹greåQŒåƈž®†æ–°å»ºçš„gre0æ·ÕdŠ åˆ°obr0åQŒåœ¨minion1上执行如下命令,
[root@minion1 ~]# ovs-vsctl add-port obr0 gre0 -- set Interface gre0 type=gre options:remote_ip=192.168.230.5
在minion2上执�
[root@minion2 ~]# ovs-vsctl add-port obr0 gre0 -- set Interface gre0 type=gre options:remote_ip=192.168.230.4
è‡Ïx¤åQŒminion1å’Œminion2之间的隧é“å·²¾lå¾ç«‹ã€‚ç„¶åŽæˆ‘们在minion1å’Œminion2上创建Linux¾|‘æ¡¥kbr0替代Docker默认的docker0åQˆæˆ‘们å‡è®¾minion1å’Œminion2都已安装DockeråQ‰ï¼Œè®„¡½®minion1çš„kbr0的地å€ä¸?72.17.1.1/24åQ?minion2çš„kbr0的地å€ä¸?72.17.2.1/24åQŒåƈæ·ÕdŠ obr0为kbr0的接å£ï¼Œä»¥ä¸‹å‘½ä×o在minion1å’Œminion2上执行ã€?/p>
[root@minion1 ~]# brctl addbr kbr0 //创å¾linux bridge [root@minion1 ~]# brctl addif kbr0 obr0 //æ·ÕdŠ obr0为kbr0的接å?[root@minion1 ~]# ip link set dev docker0 down //讄¡½®docker0为down状æ€?[root@minion1 ~]# ip link del dev docker0 //åˆ é™¤docker0
ä¸ÞZº†ä½¿æ–°å»ºçš„kbr0在毋ơ系¾lŸé‡å¯åŽä»È„¶æœ‰æ•ˆåQŒæˆ‘们在/etc/sysconfig/network-scripts/目录下新建minion1çš„ifcfg-kbr0如下åQ?/p>
DEVICE=kbr0 ONBOOT=yes BOOTPROTO=static IPADDR=172.17.1.1 NETMASK=255.255.255.0 GATEWAY=172.17.1.0 USERCTL=no TYPE=Bridge IPV6INIT=no
åŒæ ·åœ¨minion2上新建ifcfg-kbr0åQŒåªéœ€ä¿®æ”¹ipaddrä¸?72.17.2.1å’Œgatewayä¸?72.17.2.0å›_¯åQŒç„¶åŽæ‰§è¡Œsystemctl restart networké‡å¯¾pÈ»Ÿ¾|‘络æœåŠ¡åQŒä½ 能在minion1å’Œminion2上å‘现kbr0都设¾|®äº†ç›¸åº”çš„IP地å€ã€‚äØ“äº†éªŒè¯æˆ‘ä»¬åˆ›å»ºçš„éš§é“æ˜¯å¦èƒ½é€šä¿¡åQŒæˆ‘们在minion1å’Œminion2上相互pingå¯ÒŽ(gu¨©)–¹kbr0çš„IP地å€åQŒä»Žä¸‹é¢çš„结果å‘现是ä¸é€šçš„åQŒç»æŸ¥æ‰¾˜q™æ˜¯å› äØ“åœ¨minion1å’Œminion2上缺ž®‘访é—?72.17.1.1å’?72.17.2.1çš„èµ\ç”±ï¼Œå› æ¤æˆ‘ä»¬éœ€è¦æ·»åŠ èµ\ç”׃¿è¯å½¼æ¤ä¹‹é—´èƒ½é€šä¿¡ã€?/p>
[root@minion1 network-scripts]# ping 172.17.2.1 PING 172.17.2.1 (172.17.2.1) 56(84) bytes of data. ^C --- 172.17.2.1 ping statistics --- 2 packets transmitted, 0 received, 100% packet loss, time 1000ms [root@minion2 ~]# ping 172.17.1.1 PING 172.17.1.1 (172.17.1.1) 56(84) bytes of data. ^C --- 172.17.1.1 ping statistics --- 2 packets transmitted, 0 received, 100% packet loss, time 1000ms
ç”׃ºŽé€šè¿‡ip route addæ·ÕdŠ çš„èµ\ç”׃¼šåœ¨ä¸‹‹Æ¡ç³»¾lŸé‡å¯åŽå¤±æ•ˆåQŒäØ“æ¤æˆ‘们在/etc/sysconfig/network-scripts目录下新å»ÞZ¸€ä¸ªæ–‡ä»¶route-eth0å˜å‚¨è·¯ç”±åQŒè¿™é‡Œéœ€è¦æ³¨æ„的是route-eth0å’Œifcfg-eth0的黑体部分必™åÖM¿æŒä¸€è‡ß_¼Œå¦åˆ™ä¸èƒ½å·¥ä½œåQŒè¿™æ äh·»åŠ çš„è·¯ç”±åœ¨ä¸‹‹Æ¡é‡å¯åŽä¸ä¼šå¤±æ•ˆã€‚äØ“äº†ä¿è¯ä¸¤å°minionçš„kbr0能相互通信åQŒæˆ‘们在minion1çš„route-eth0é‡Œæ·»åŠ èµ\ç”?72.17.2.0/24 via 192.168.230.5 dev eno16777736åQŒeno16777736是minion1的网å¡ï¼ŒåŒæ ·åœ¨minion2çš„route-eth0é‡Œæ·»åŠ èµ\ç”?72.17.1.0/24 via 192.168.230.4 dev eno16777736。é‡å¯ç½‘¾lœæœåŠ¡åŽå†æ¬¡éªŒè¯åQŒå½¼æ¤kbr0的地å€å¯ä»¥ping通,如:
[root@minion2 network-scripts]# ping 172.17.1.1 PING 172.17.1.1 (172.17.1.1) 56(84) bytes of data. 64 bytes from 172.17.1.1: icmp_seq=1 ttl=64 time=2.49 ms 64 bytes from 172.17.1.1: icmp_seq=2 ttl=64 time=0.512 ms ^C --- 172.17.1.1 ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1002ms rtt min/avg/max/mdev = 0.512/1.505/2.498/0.993 ms
到现在我们已¾lå¾ç«‹äº†ä¸¤minion之间的隧é“,而且能棼‹®çš„å·¥ä½œã€‚ä¸‹é¢æˆ‘们将介ç»å¦‚何安装Kubernetes APIServeråŠkubeletã€proxy½{‰æœåŠ¡ã€?/p>
在安装APIServer之å‰åQŒæˆ‘们先下è²KubernetesåŠEtcdåQŒåšä¸€äº›å‡†å¤‡å·¥ä½œã€‚在kubernetes上的具体æ“作如下åQ?/p>
[root@kubernetes ~]# mkdir /tmp/kubernetes [root@kubernetes ~]# cd /tmp/kubernetes/ [root@kubernetes kubernetes]# wget https://github.com/GoogleCloudPlatform/kubernetes/releases/download/v0.5.2/kubernetes.tar.gz [root@kubernetes kubernetes]# wget https://github.com/coreos/etcd/releases/download/v0.4.6/etcd-v0.4.6-linux-amd64.tar.gz
ç„¶åŽè§£åދ䏋è²çš„kuberneteså’Œetcd包,òq¶åœ¨kubernetesã€minion1ã€minion2上创建目å½?opt/kubernetes/binåQ?/p>
[root@kubernetes kubernetes]# mkdir -p /opt/kubernetes/bin [root@kubernetes kubernetes]# tar xf kubernetes.tar.gz [root@kubernetes kubernetes]# tar xf etcd-v0.4.6-linux-amd64.tar.gz [root@kubernetes kubernetes]# cd ~/kubernetes/server [root@kubernetes server]# tar xf kubernetes-server-linux-amd64.tar.gz [root@kubernetes kubernetes]# /tmp/kubernetes/kubernetes/server/kubernetes/server/bin
å¤åˆ¶kube-apiserveråQŒkube-controller-manageråQŒkube-scheduleråQŒkubecfg到kubernetesçš?opt/kubernetes/bin目录下,而kubeletåQŒkube-proxy则å¤åˆ¶åˆ°minion1å’Œminion2çš?opt/kubernetes/binåQŒåƈ¼‹®ä¿éƒ½æ˜¯å¯æ‰§è¡Œçš„ã€?/p>
[root@kubernetes amd64]# cp kube-apiserver kube-controller-manager kubecfg kube-scheduler /opt/kubernetes/bin [root@kubernetes amd64]# scp kube-proxy kubelet root@192.168.230.4:/opt/kubernetes/bin [root@kubernetes amd64]# scp kube-proxy kubelet root@192.168.230.5:/opt/kubernetes/bin
ä¸ÞZº†½Ž€å•我们åªéƒ¨çÖv一å°etcdæœåŠ¡å™¨ï¼Œå¦‚æžœéœ€è¦éƒ¨¾|²etcd的集¾Ÿ¤ï¼Œè¯·å‚è€?a style="text-decoration: none; color: #286ab2; outline: none !important; margin: 0px; border: 0px; padding: 0px;">官方文档åQŒåœ¨æœ¬æ–‡ä¸å°†å…¶è·ŸKubernetes APIServer部çÖvåŒä¸€å°æœºå™¨ä¸ŠåQŒè€Œä¸”ž®†etcd攄¡½®åœ?opt/kubernetes/bin下,etcdctlè·ŸectdåŒä¸€ç›®å½•ã€?/p>
[root@kubernetes kubernetes]# cd /tmp/kubernetes/etcd-v0.4.6-linux-amd64 [root@kubernetes etcd-v0.4.6-linux-amd64]# cp etcd etcdctl /opt/kubernetes/bin
需注æ„的是kuberneteså’Œminionä¸?opt/kubernetes/bin目录下的文äšg都必™åÀL˜¯å¯æ‰§è¡Œçš„。到目å‰åQŒæˆ‘们准备工作已¾lå·®ä¸å¤šåQŒçŽ°åœ¨å¼€å§‹ç»™apiserveråQŒcontroller-manageråQŒscheduleråQŒetcdé…ç½®unitæ–‡äšg。首先我们用如下脚本etcd.shé…ç½®etcdçš„unitæ–‡äšgåQ?/p>
#!/bin/sh ETCD_PEER_ADDR=192.168.230.3:7001 ETCD_ADDR=192.168.230.3:4001 ETCD_DATA_DIR=/var/lib/etcd ETCD_NAME=kubernetes ! test -d $ETCD_DATA_DIR && mkdir -p $ETCD_DATA_DIR cat <<EOF >/usr/lib/systemd/system/etcd.service [Unit] Description=Etcd Server [Service] ExecStart=/opt/kubernetes/bin/etcd \\ -peer-addr=$ETCD_PEER_ADDR \\ -addr=$ETCD_ADDR \\ -data-dir=$ETCD_DATA_DIR \\ -name=$ETCD_NAME \\ -bind-addr=0.0.0.0 [Install] WantedBy=multi-user.target EOF systemctl daemon-reload systemctl enable etcd systemctl start etcd
对剩下的apiserver,controller-manager,schedulerçš„unitæ–‡äšgé…置的脚本,å¯ä»¥åœ¨github ä¸?a style="text-decoration: none; color: #286ab2; outline: none !important; margin: 0px; border: 0px; padding: 0px;">GetStartingKubernetes扑ֈ°åQŒåœ¨æ¤å°±ä¸ä¸€ä¸€åˆ—ä‹D。è¿è¡Œç›¸åº”的脚本åŽï¼Œåœ¨APIServer上etcd, apiserver, controller-manager, scheduleræœåŠ¡ž®Þpƒ½æ£å¸¸˜q行ã€?/p>
æ ÒŽ(gu¨©)®Kubernetes的设计架构,需è¦åœ¨minion上部¾|²docker, kubelet, kube-proxyåQŒåœ¨4.2节部¾|²APIServeræ—Óž¼Œæˆ‘们已绞®†kubeletå’Œkube-proxyå·²ç»åˆ†å‘åˆîC¸¤minion上,所以åªéœ€é…ç½®docker,kubelet,proxyçš„unitæ–‡äšgåQŒç„¶åŽå¯åЍæœåС就å›_¯åQŒå…·ä½“é…¾|®è§GetStartingKubernetesã€?/p>
ä¸ÞZº†æ–¹ä¾¿åQŒæˆ‘们ä‹É用Kubernetesæä¾›çš„例å?a style="text-decoration: none; color: #286ab2; outline: none !important; margin: 0px; border: 0px; padding: 0px;">Guestbookæ¥æ¼”½CºKubernetes½Ž¡ç†è·¨æœºå™¨è¿è¡Œçš„容器åQŒä¸‹é¢æˆ‘ä»¬æ ¹æ®Guestbookçš„æ¥éª¤åˆ›å»ºå®¹å™¨åŠæœåŠ¡ã€‚åœ¨ä¸‹é¢çš„过½E‹ä¸å¦‚果是第一‹Æ¡æ“作,å¯èƒ½ä¼šæœ‰ä¸€å®šçš„½{‰å¾…æ—‰™—´åQŒçжæ€å¤„于pendingåQŒè¿™æ˜¯å› 为第一‹Æ¡ä¸‹è½½images需è¦ä¸€ŒD‰|—¶é—´ã€?/p>
[root@kubernetes ~]# cd /tmp/kubernetes/kubernetes/examples/guestbook [root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 -c redis-master.json create pods [root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 -c redis-master-service.json create services
完æˆä¸Šé¢çš„æ“ä½œåŽåQŒæˆ‘们å¯ä»¥çœ‹åˆ°å¦‚下redis-master Pod被调度到192.168.230.4ã€?/p>
[root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 list pods Name Image(s) Host Labels Status ---------- ---------- ---------- ---------- ---------- redis-master dockerfile/redis 192.168.230.4/ name=redis-master Running
但除了å‘现redis-masterçš„æœåŠ¡ä¹‹å¤–ï¼Œ˜q˜æœ‰ä¸¤ä¸ªKubernetes¾pÈ»Ÿé»˜è®¤çš„æœåŠ¡kubernetes-roå’Œkubernetes。而且我们å¯ä»¥çœ‹åˆ°æ¯ä¸ªæœåŠ¡éƒ½æœ‰ä¸€ä¸ªæœåŠ¡IPåŠç›¸åº”的端å£åQŒå¯¹äºŽæœåŠ¡IPåQŒæ˜¯ä¸€ä¸ªè™šæ‹Ÿåœ°å€åQŒæ ¹æ®a(ch¨£n)piserverçš„portal_net选项讄¡½®çš„CIDR表示的IP地倌D‰|¥é€‰å–åQŒåœ¨æˆ‘们的集¾Ÿ¤ä¸è®„¡½®ä¸?0.10.10.0/24ã€‚äØ“æ¤æ¯æ–°åˆ›å»ÞZ¸€ä¸ªæœåŠ¡ï¼Œapiserver都会在这个地倌Dµä¸éšæœºé€‰æ‹©ä¸€ä¸ªIPä½œäØ“è¯¥æœåŠ¡çš„IP地å€åQŒè€Œç«¯å£æ˜¯äº‹å…ˆ¼‹®å®šçš„。对redis-masteræœåŠ¡åQŒå…¶æœåŠ¡åœ°å€ä¸?0.10.10.206åQŒç«¯å£äØ“6379ã€?/p>
[root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 list services Name Labels Selector IP Port ---------- ---------- ---------- ---------- ---------- kubernetes-ro component=apiserver,provider=kubernetes 10.10.10.207 80 redis-master name=redis-master name=redis-master 10.10.10.206 6379 kubernetes component=apiserver,provider=kubernetes 10.10.10.161 443
[root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 -c redis-slave-controller.json create replicationControllers [root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 -c redis-slave-service.json create services
ç„¶åŽé€šè¿‡list命ä×oå¯çŸ¥æ–°å¾çš„redis-slave Podæ ÒŽ(gu¨©)®è°ƒåº¦½Ž—法调度åˆîC¸¤å°minion上,æœåŠ¡IPä¸?0.10.10.92åQŒç«¯å£äØ“6379
[root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 list pods Name Image(s) Host Labels Status ---------- ---------- ---------- ---------- ---------- redis-master dockerfile/redis 192.168.230.4/ name=redis-master Running 8c0ddbda-728c-11e4-8233-000c297db206 brendanburns/redis-slave 192.168.230.5/ name=redisslave,uses=redis-master Running 8c0e1430-728c-11e4-8233-000c297db206 brendanburns/redis-slave 192.168.230.4/ name=redisslave,uses=redis-master Running [root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 list services Name Labels Selector IP Port ---------- ---------- ---------- ---------- ---------- redisslave name=redisslave name=redisslave 10.10.10.92 6379 kubernetes component=apiserver,provider=kubernetes 10.10.10.161 443 kubernetes-ro component=apiserver,provider=kubernetes 10.10.10.207 80 redis-master name=redis-master name=redis-master 10.10.10.206 6379
在创å»ÞZ¹‹å‰ä¿®æ”¹frontend-controller.jsonçš„Replicasæ•°é‡ä¸?åQŒè¿™æ˜¯å› 为我们的集群ä¸åªæœ?å°minionåQŒå¦‚果按照frontend-controller.jsonçš„Replicas默认å€?åQŒé‚£ä¼šå¯¼è‡´æœ‰2个Pod会调度到åŒä¸€å°minion上,产生端å£å†²çªåQŒæœ‰ä¸€ä¸ªPod会一直处于pending状æ€ï¼Œä¸èƒ½è¢«è°ƒåº¦ã€?/p>
[root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 -c frontend-controller.json create replicationControllers [root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 -c frontend-service.json create services
通过查看å¯çŸ¥Frontend Pod也被调度åˆîC¸¤å°minionåQŒæœåŠ¡IPä¸?0.10.10.220åQŒç«¯å£æ˜¯80ã€?/p>
[root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 list pods Name Image(s) Host Labels Status ---------- ---------- ---------- ---------- ---------- redis-master dockerfile/redis 192.168.230.4/ name=redis-master Running 8c0ddbda-728c-11e4-8233-000c297db206 brendanburns/redis-slave 192.168.230.5/ name=redisslave,uses=redis-master Running 8c0e1430-728c-11e4-8233-000c297db206 brendanburns/redis-slave 192.168.230.4/ name=redisslave,uses=redis-master Running a880b119-7295-11e4-8233-000c297db206 brendanburns/php-redis 192.168.230.4/ name=frontend,uses=redisslave,redis-master Running a881674d-7295-11e4-8233-000c297db206 brendanburns/php-redis 192.168.230.5/ name=frontend,uses=redisslave,redis-master Running [root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 list services Name Labels Selector IP Port ---------- ---------- ---------- ---------- ---------- kubernetes-ro component=apiserver,provider=kubernetes 10.10.10.207 80 redis-master name=redis-master name=redis-master 10.10.10.206 6379 redisslave name=redisslave name=redisslave 10.10.10.92 6379 frontend name=frontend name=frontend 10.10.10.220 80 kubernetes component=apiserver,provider=kubernetes 10.10.10.161 443
除æ¤ä¹‹å¤–åQŒä½ å¯ä»¥åˆ 除Podã€ServiceåŠæ›´æ–°ReplicationControllerçš„Replicas数釽{‰æ“ä½œï¼Œå¦‚åˆ é™¤FrontendæœåŠ¡åQ?/p>
[root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 delete services/frontend Status ---------- Success
˜q˜å¯ä»¥æ›´æ–°ReplicationControllerçš„Replicas的数é‡ï¼Œä¸‹é¢æ˜¯æ›´æ–°Replicas之å‰ReplicationController的信æ¯ã€?/p>
[root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 list replicationControllers Name Image(s) Selector Replicas ---------- ---------- ---------- ---------- redisSlaveController brendanburns/redis-slave name=redisslave 2 frontendController brendanburns/php-redis name=frontend 2
现在我们æƒÏxŠŠfrontendControllerçš„Replicasæ›´æ–°ä¸?åQŒåˆ™˜q™è¡Œå¦‚下命ä×oåQŒç„¶åŽå†é€šè¿‡ä¸Šé¢çš„命令查看frontendControllerä¿¡æ¯åQŒå‘现Replicaså·²å˜ä¸?ã€?/p>
[root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 resize frontendController 1 [root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 list replicationControllers Name Image(s) Selector Replicas ---------- ---------- ---------- ---------- redisSlaveController brendanburns/redis-slave name=redisslave 2 frontendController brendanburns/php-redis name=frontend 1
完æˆä¸Šé¢çš„æ“ä½œåŽåQŒæˆ‘们æ¥çœ‹å½“å‰Kubernetes集群ä¸è¿è¡Œç€çš„Podä¿¡æ¯ã€?/p>
[root@kubernetes guestbook]# kubecfg -h http://192.168.230.3:8080 list pods Name Image(s) Host Labels Status ---------- ---------- ---------- ---------- ---------- a881674d-7295-11e4-8233-000c297db206 brendanburns/php-redis 192.168.230.5/ name=frontend,uses=redisslave,redis-master Running redis-master dockerfile/redis 192.168.230.4/ name=redis-master Running 8c0ddbda-728c-11e4-8233-000c297db206 brendanburns/redis-slave 192.168.230.5/ name=redisslave,uses=redis-master Running 8c0e1430-728c-11e4-8233-000c297db206 brendanburns/redis-slave 192.168.230.4/ name=redisslave,uses=redis-master Running
通过上é¢çš„结果å¯çŸ¥å½“剿供å‰ç«¯æœåŠ¡çš„PHPå’Œæä¾›æ•°æ®å˜å‚¨çš„åŽç«¯æœåŠ¡Redis masterçš„Pod分别˜q行åœ?92.168.230.5å’?92.168.230.4上,å›_®¹å™¨è¿è¡Œåœ¨ä¸åŒä¸ÀLœºä¸Šï¼Œ˜q˜æœ‰Redis slave也è¿è¡Œåœ¨ä¸¤å°ä¸åŒçš„主æœÞZ¸ŠåQŒå®ƒä¼šä»ŽRedis masteråŒæ¥å‰ç«¯å†™å…¥Redis master的数æ®ã€‚䏋颿ˆ‘们从两方é¢éªŒè¯Kubernetes能æä¾›è·¨æœºå™¨é—´å®¹å™¨çš„通信åQ?/p>
在æµè§ˆå™¨æ‰“å¼€http://${IPAddress}:8000åQŒIPAddress为PHP容器˜q行的minionçš„IP地å€åQŒå…¶æš´æ¼çš„端å£äØ“8000åQŒè¿™é‡ŒIP_Addressä¸?92.168.230.5。打开‹¹è§ˆå™¨ä¼šæ˜„¡¤ºå¦‚下信æ¯åQ?/p>
ä½ å¯ä»¥è¾“入信æ¯åƈæäº¤åQŒå¦‚"Hello Kubernetes"ã€?Container"åQŒç„¶åŽSubmit按钮下方会显½CÞZ½ 输入的信æ¯ã€?/p>
从上é¢çš„¾l“æžœåQŒå¯å¾—知已ç»å®žçŽ°äº†è·¨æœºå™¨çš„é€šä¿¡åQŒçŽ°åœ¨æˆ‘ä»¬ä»ŽåŽç«¯æ•°æ®å±‚验è¯ä¸åŒæœºå™¨å®¹å™¨é—´çš„é€šä¿¡ã€‚æ ¹æ®ä¸Šé¢çš„输出¾l“æžœå‘现Redis slaveå’ŒRedis master分别调度åˆîC¸¤åîC¸åŒçš„minion上,åœ?92.168.230.4ä¸ÀLœºä¸Šæ‰§è¡Œdocker exec -ti c41711cc8971 /bin/shåQŒc41711cc8971是Redis master的容器IDåQŒè¿›å…¥å®¹å™¨åŽé€šè¿‡redis-cli命ä×o查看从æµè§ˆå™¨è¾“入的信æ¯å¦‚下:
如果我们åœ?92.168.230.5上è¿è¡Œçš„Redis slave容器里查到跟Redis master容器里相åŒçš„ä¿¡æ¯åQŒé‚£è¯´æ˜ŽRedis masterå’ŒRedis slave之间的数æ®åŒæ¥æ£å¸¸å·¥ä½œï¼Œä¸‹é¢æ˜¯ä»Ž192.168.230.5上è¿è¡Œçš„Redis slave容器查询到的信æ¯åQ?/p>
ç”±æ¤å¯è§Redis masterå’ŒRedis slave之间数æ®åŒæ¥æ£å¸¸åQŒOVS GREéš§é“æŠ€æœ¯ä‹É得跨机器间容器æ£å¸”R€šä¿¡ã€?/p>
本文主è¦ä»‹ç»å¦‚何在本地环境部¾|²Kubernetes集群和演½Cºå¦‚何通过Kubernetes½Ž¡ç†é›†ç¾¤ä¸è¿è¡Œçš„容器åQŒåƈ通过OVS½Ž¡ç†é›†ç¾¤ä¸åŒminionçš„Pod之间的网¾lœé€šä¿¡ã€‚接下æ¥ä¼šå¯¹Kuberneteså„个¾l„äšgæºç ˜q›è¡Œè¯¦ç»†åˆ†æžåQŒé˜˜q°Kubernetes的工作原ç†ã€?/p>
æ¨ç« 显,现就èŒäºŽCiscoåQŒä¸»è¦ä»Žäº‹WebEx SaaSæœåŠ¡˜qç»´åQŒç³»¾lŸæ€§èƒ½åˆ†æž½{‰å·¥ä½œã€‚特别关注云计算åQŒè‡ªåŠ¨åŒ–˜qç»´åQŒéƒ¨¾|²ç‰æŠ€æœ¯ï¼Œž®¤å…¶æ˜¯Goã€OpenvSwitchã€DockeråŠå…¶ç”Ÿæ€åœˆæŠ€æœ¯ï¼Œå¦‚Kubernetesã€Flocker½{‰Docker相关开æºé¡¹ç›®ã€‚Email: yangzhangxian@gmail.com
感谢éƒè•¾å¯ÒŽ(gu¨©)œ¬æ–‡çš„½{–åˆ’å’Œå®¡æ ¡ã€?/p>
除了Linux/amd64åQŒé»˜è®¤è¿˜ä¼šäؓ其他òq›_°åšäº¤å‰ç¼–è¯‘ã€‚äØ“äº†å‡ž®‘编译时é—ß_¼Œå¯ä»¥ä¿®æ”¹hack/lib/golang.shåQŒæŠŠKUBE_SERVER_PLATFORMSåQ?KUBE_CLIENT_PLATFORMSå’ŒKUBE_TEST_PLATFORMSä¸é™¤linux/amd64以外的其他åã^å°æ³¨é‡ŠæŽ‰
Kubernetes在创建Pod的时候,需è¦ä»Žgcr.io下è²ä¸€ä¸ªhelper镜åƒåQˆç›®å‰æ˜¯ gcr.io/google_containers/pause-amd64:3.0 åQ‰ã€?/p>
但是目å‰å›½å†…æ— æ³•è®‰K—®gcr.ioåQŒè¿™ä¸ªé—®é¢˜ä¼šå¯ÆD‡´æ— 法下è²è¯¥é•œåƒï¼Œç„¶åŽPod一直处于ContainerCreating状æ€ã€?/p>
ä¸é€šè¿‡¿U有registryä¸è{åQŒè€Œæ˜¯ä½¿ç”¨Docker save/load应该也å¯ä»¥ï¼Œåªæ˜¯è¦æŠŠsave导出的文件å¤åˆ¶åˆ°æ‰€æœ‰èŠ‚ç‚?
˜q™é‡Œè¯´çš„集群外是指K8s集群以外的主机,比如使用nginx/HAProxyæå¾çš„è´Ÿè½½å‡è¡¡ä¸»æœºã€‚è¿™äº›ä¸»æœø™·ŸK8s集群部çÖvåœ¨ä¸€èµøP¼Œåˆ°K8s¾|‘络å¯è¾¾ã€?/p>
å¯¹äºŽä¸æ˜¯éƒ¨çÖv在GCE以åŠAWS½{‰äº‘òq›_°çš„K8såQŒæˆ‘们一般需è¦è‡ªå·±æå»ø™´Ÿè½½å‡è¡¡ï¼Œç„¶åŽåˆ†å‘è¯äh±‚到到Serviceã€?/p>
使用NodePortæ–¹å¼å‘布æœåŠ¡åQŒé‚£ä¹ˆè´Ÿè½½å‡è¡¡ä¸»æœÞZ¸Šä¸éœ€è¦é¢å¤–é…¾|®ï¼›ä½¿ç”¨ClusterIPæ–¹å¼åQŒäؓ了能够访问Serviceçš„ClusterIPåQ?需è¦åœ¨˜q™äº›ä¸ÀLœºä¸Šå®‰è£…Flanneldå’Œkube-proxy