DNS插件在Kubernetes中輸入crashloopbackoff 1.4.5

作爲kubernetes的新手，我使用「packstack」羣集跟隨了以下教程。本教程使用kubernetes 1.2.1版，我還算了解是很老：DNS插件在Kubernetes中輸入crashloopbackoff 1.4.5

http://kubernetes.io/docs/getting-started-guides/coreos/coreos_multinode_cluster/

一切似乎好，我是能夠推出吊艙，所以我雖然我會嘗試安裝DNS插件作爲kubernetes服務/ RC。我搜索了一下，看到DNS添加需要kubernetes版本1.3或更高版本。

我在我的主節點雲節點中將Kubernetes版本撞到了1.4.5，然後再次嘗試。再次，一切似乎工作 - 除了當我嘗試啓動DNS複製控制器（和服務），我在日誌中看到的錯誤開始：

Expected to load root CA config from  /var/run/secrets/kubernetes.io/serviceaccount/ca.crt, but got err: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory

我不能確定如何進行。我試圖創建另一個服務帳戶，但祕密中沒有證書。我的雲配置沒有引用任何根證書，所以我想知道這是否是問題？

我已經附加了我的主節點和節點雲配置，我的DNS附加yaml文件和DNS服務器日誌。

雲少爺配置：

#cloud-config 

--- 
write-files: 
    - path: /etc/conf.d/nfs 
    permissions: '0644' 
    content: | 
     OPTS_RPC_MOUNTD="" 
    - path: /opt/bin/wupiao 
    permissions: '0755' 
    content: | 
     #!/bin/bash 
     # [w]ait [u]ntil [p]ort [i]s [a]ctually [o]pen 
     [ -n "$1" ] && \ 
     until curl -o /dev/null -sIf http://${1}; do \ 
      sleep 1 && echo .; 
     done; 
     exit $? 

hostname: master 
coreos: 
    etcd2: 
    name: master 
    listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001 
    advertise-client-urls: http://$private_ipv4:2379,http://$private_ipv4:4001 
    initial-cluster-token: k8s_etcd 
    listen-peer-urls: http://$private_ipv4:2380,http://$private_ipv4:7001 
    initial-advertise-peer-urls: http://$private_ipv4:2380 
    initial-cluster: master=http://$private_ipv4:2380 
    initial-cluster-state: new 
    fleet: 
    metadata: "role=master" 
    units: 
    - name: etcd2.service 
     command: start 
    - name: generate-serviceaccount-key.service 
     command: start 
     content: | 
     [Unit] 
     Description=Generate service-account key file 

     [Service] 
     ExecStartPre=-/usr/bin/mkdir -p /opt/bin 
     ExecStart=/bin/openssl genrsa -out /opt/bin/kube-serviceaccount.key 2048 2>/dev/null 
     RemainAfterExit=yes 
     Type=oneshot 
    - name: setup-network-environment.service 
     command: start 
     content: | 
     [Unit] 
     Description=Setup Network Environment 
     Documentation=https://github.com/kelseyhightower/setup-network-environment 
     Requires=network-online.target 
     After=network-online.target 

     [Service] 
     ExecStartPre=-/usr/bin/mkdir -p /opt/bin 
     ExecStartPre=/usr/bin/curl -L -o /opt/bin/setup-network-environment -z /opt/bin/setup-network-environment https://github.com/kelseyhightower/setup-network-environment/releases/download/v1.0.0/setup-network-environment 
     ExecStartPre=/usr/bin/chmod +x /opt/bin/setup-network-environment 
     ExecStart=/opt/bin/setup-network-environment 
     RemainAfterExit=yes 
     Type=oneshot 
    - name: fleet.service 
     command: start 
    - name: flanneld.service 
     command: start 
     drop-ins: 
     - name: 50-network-config.conf 
      content: | 
      [Unit] 
      Requires=etcd2.service 
      [Service] 
      ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{"Network":"10.244.0.0/16", "Backend": {"Type": "vxlan"}}' 
    - name: docker.service 
     command: start 
    - name: kube-apiserver.service 
     command: start 
     content: | 
     [Unit] 
     Description=Kubernetes API Server 
     Documentation=https://github.com/kubernetes/kubernetes 
     Requires=setup-network-environment.service etcd2.service generate-serviceaccount-key.service 
     After=setup-network-environment.service etcd2.service generate-serviceaccount-key.service 

     [Service] 
     EnvironmentFile=/etc/network-environment 
     ExecStartPre=-/usr/bin/mkdir -p /opt/bin 
     ExecStartPre=/usr/bin/curl -L -o /opt/bin/kube-apiserver -z /opt/bin/kube-apiserver https://storage.googleapis.com/kubernetes-release/release/v1.4.5/bin/linux/amd64/kube-apiserver 
     ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-apiserver 
     ExecStartPre=/opt/bin/wupiao 127.0.0.1:2379/v2/machines 
     ExecStart=/opt/bin/kube-apiserver \ 
     --service-account-key-file=/opt/bin/kube-serviceaccount.key \ 
     --service-account-lookup=false \ 
     --admission-control=NamespaceLifecycle,NamespaceAutoProvision,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota \ 
     --runtime-config=api/v1 \ 
     --allow-privileged=true \ 
     --insecure-bind-address=0.0.0.0 \ 
     --insecure-port=8080 \ 
     --kubelet-https=true \ 
     --secure-port=6443 \ 
     --service-cluster-ip-range=10.244.0.0/16 \ 
     --etcd-servers=http://127.0.0.1:2379 \ 
     --public-address-override=${DEFAULT_IPV4} \ 
     --logtostderr=true 
     Restart=always 
     RestartSec=10 
    - name: kube-controller-manager.service 
     command: start 
     content: | 
     [Unit] 
     Description=Kubernetes Controller Manager 
     Documentation=https://github.com/kubernetes/kubernetes 
     Requires=kube-apiserver.service 
     After=kube-apiserver.service 

     [Service] 
     ExecStartPre=/usr/bin/curl -L -o /opt/bin/kube-controller-manager -z /opt/bin/kube-controller-manager https://storage.googleapis.com/kubernetes-release/release/v1.4.5/bin/linux/amd64/kube-controller-manager 
     ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-controller-manager 
     ExecStart=/opt/bin/kube-controller-manager \ 
     --service-account-private-key-file=/opt/bin/kube-serviceaccount.key \ 
     --master=127.0.0.1:8080 \ 
     --logtostderr=true 
     Restart=always 
     RestartSec=10 
    - name: kube-scheduler.service 
     command: start 
     content: | 
     [Unit] 
     Description=Kubernetes Scheduler 
     Documentation=https://github.com/kubernetes/kubernetes 
     Requires=kube-apiserver.service 
     After=kube-apiserver.service 

     [Service] 
     ExecStartPre=/usr/bin/curl -L -o /opt/bin/kube-scheduler -z /opt/bin/kube-scheduler https://storage.googleapis.com/kubernetes-release/release/v1.4.5/bin/linux/amd64/kube-scheduler 
     ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-scheduler 
     ExecStart=/opt/bin/kube-scheduler --master=127.0.0.1:8080 
     Restart=always 
     RestartSec=10 
    update: 
    group: alpha 
    reboot-strategy: off

節點雲配置

#cloud-config 
write-files: 
    - path: /opt/bin/wupiao 
    permissions: '0755' 
    content: | 
     #!/bin/bash 
     # [w]ait [u]ntil [p]ort [i]s [a]ctually [o]pen 
     [ -n "$1" ] && [ -n "$2" ] && while ! curl --output /dev/null \ 
     --silent --head --fail \ 
     http://${1}:${2}; do sleep 1 && echo -n .; done; 
     exit $? 
coreos: 
    etcd2: 
    listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001 
    advertise-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001 
    initial-cluster: master=http://10.10.1.31:2380 
    proxy: on 
    fleet: 
    metadata: "role=node" 
    units: 
    - name: etcd2.service 
     command: start 
    - name: fleet.service 
     command: start 
    - name: flanneld.service 
     command: start 
    - name: docker.service 
     command: start 
    - name: setup-network-environment.service 
     command: start 
     content: | 
     [Unit] 
     Description=Setup Network Environment 
     Documentation=https://github.com/kelseyhightower/setup-network-environment 
     Requires=network-online.target 
     After=network-online.target 

     [Service] 
     ExecStartPre=-/usr/bin/mkdir -p /opt/bin 
     ExecStartPre=/usr/bin/curl -L -o /opt/bin/setup-network-environment -z /opt/bin/setup-network-environment https://github.com/kelseyhightower/setup-network-environment/releases/download/v1.0.0/setup-network-environment 
     ExecStartPre=/usr/bin/chmod +x /opt/bin/setup-network-environment 
     ExecStart=/opt/bin/setup-network-environment 
     RemainAfterExit=yes 
     Type=oneshot 
    - name: kube-proxy.service 
     command: start 
     content: | 
     [Unit] 
     Description=Kubernetes Proxy 
     Documentation=https://github.com/kubernetes/kubernetes 
     Requires=setup-network-environment.service 
     After=setup-network-environment.service 

     [Service] 
     ExecStartPre=/usr/bin/curl -L -o /opt/bin/kube-proxy -z /opt/bin/kube-proxy https://storage.googleapis.com/kubernetes-release/release/v1.4.5/bin/linux/amd64/kube-proxy 
     ExecStartPre=/usr/bin/chmod +x /opt/bin/kube-proxy 
     # wait for kubernetes master to be up and ready 
     ExecStartPre=/opt/bin/wupiao 10.10.1.31 8080 
     ExecStart=/opt/bin/kube-proxy \ 
     --master=10.10.1.31:8080 \ 
     --logtostderr=true 
     Restart=always 
     RestartSec=10 
    - name: kube-kubelet.service 
     command: start 
     content: | 
     [Unit] 
     Description=Kubernetes Kubelet 
     Documentation=https://github.com/kubernetes/kubernetes 
     Requires=setup-network-environment.service 
     After=setup-network-environment.service 

     [Service] 
     EnvironmentFile=/etc/network-environment 
     ExecStartPre=/usr/bin/curl -L -o /opt/bin/kubelet -z /opt/bin/kubelet https://storage.googleapis.com/kubernetes-release/release/v1.4.5/bin/linux/amd64/kubelet 
     ExecStartPre=/usr/bin/chmod +x /opt/bin/kubelet 
     # wait for kubernetes master to be up and ready 
     ExecStartPre=/opt/bin/wupiao 10.10.1.31 8080 
     ExecStart=/opt/bin/kubelet \ 
     --address=0.0.0.0 \ 
     --port=10250 \ 
     --hostname-override=${DEFAULT_IPV4} \ 
     --api-servers=10.10.1.31:8080 \ 
     --allow-privileged=true \ 
     --logtostderr=true \ 
     --cadvisor-port=4194 \ 
     --healthz-bind-address=0.0.0.0 \ 
     --healthz-port=10248 
     Restart=always 
     RestartSec=10 
    update: 
    group: alpha 
    reboot-strategy: off

DNS附加組件YAML

apiVersion: v1 
kind: Service 
metadata: 
    name: kube-dns 
    namespace: kube-system 
    labels: 
    k8s-app: kube-dns 
    kubernetes.io/cluster-service: "true" 
    kubernetes.io/name: "KubeDNS" 
spec: 
    selector: 
    k8s-app: kube-dns 
    clusterIP: 10.244.0.5 
    ports: 
    - name: dns 
    port: 53 
    protocol: UDP 
    - name: dns-tcp 
    port: 53 
    protocol: TCP 


--- 


apiVersion: v1 
kind: ReplicationController 
metadata: 
    name: kube-dns-v20 
    namespace: kube-system 
    labels: 
    k8s-app: kube-dns 
    version: v20 
    kubernetes.io/cluster-service: "true" 
spec: 
    replicas: 1 
    selector: 
    k8s-app: kube-dns 
    version: v20 
    template: 
    metadata: 
     labels: 
     k8s-app: kube-dns 
     version: v20 
     annotations: 
     scheduler.alpha.kubernetes.io/critical-pod: '' 
     scheduler.alpha.kubernetes.io/tolerations: '[{"key":"CriticalAddonsOnly", "operator":"Exists"}]' 
    spec: 
     containers: 
     - name: kubedns 
     image: gcr.io/google_containers/kubedns-amd64:1.8 
     resources: 
      limits: 
      memory: 170Mi 
      requests: 
      cpu: 100m 
      memory: 70Mi 
     livenessProbe: 
      httpGet: 
      path: /healthz-kubedns 
      port: 8080 
      scheme: HTTP 
      initialDelaySeconds: 60 
      timeoutSeconds: 5 
      successThreshold: 1 
      failureThreshold: 5 
     readinessProbe: 
      httpGet: 
      path: /readiness 
      port: 8081 
      scheme: HTTP 
      initialDelaySeconds: 3 
      timeoutSeconds: 5 
     args: 
     - --domain=cluster.local. 
     - --dns-port=10053 
     ports: 
     - containerPort: 10053 
      name: dns-local 
      protocol: UDP 
     - containerPort: 10053 
      name: dns-tcp-local 
      protocol: TCP 
     - name: dnsmasq 
     image: gcr.io/google_containers/kube-dnsmasq-amd64:1.4 
     livenessProbe: 
      httpGet: 
      path: /healthz-dnsmasq 
      port: 8080 
      scheme: HTTP 
      initialDelaySeconds: 60 
      timeoutSeconds: 5 
      successThreshold: 1 
      failureThreshold: 5 
     args: 
     - --cache-size=1000 
     - --no-resolv 
     - --server=127.0.0.1#10053 
     - --log-facility=- 
     ports: 
     - containerPort: 53 
      name: dns 
      protocol: UDP 
     - containerPort: 53 
      name: dns-tcp 
      protocol: TCP 
     - name: healthz 
     image: gcr.io/google_containers/exechealthz-amd64:1.2 
     resources: 
      limits: 
      memory: 50Mi 
      requests: 
      cpu: 10m 
      memory: 50Mi 
     args: 
     - --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1 >/dev/null 
     - --url=/healthz-dnsmasq 
     - --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1:10053 >/dev/null 
     - --url=/healthz-kubedns 
     - --port=8080 
     - --quiet 
     ports: 
     - containerPort: 8080 
      protocol: TCP 
     dnsPolicy: Default

DNS插件日誌

E1118 17:33:10.140677  1 config.go:265] Expected to load root CA config from /var/run/secrets/kubernetes.io/serviceaccount/ca.crt, but got err: open /var/run/secrets/kubernetes.io/serviceaccount/ca.crt: no such file or directory 
I1118 17:33:10.141079  1 server.go:94] Using https://10.244.0.1:443 for kubernetes master, kubernetes API: <nil> 
I1118 17:33:10.141596  1 server.go:99] v1.5.0-alpha.0.1651+7dcae5edd84f06-dirty 
I1118 17:33:10.141728  1 server.go:101] FLAG: --alsologtostderr="false" 
I1118 17:33:10.141840  1 server.go:101] FLAG: --dns-port="10053" 
I1118 17:33:10.141931  1 server.go:101] FLAG: --domain="cluster.local." 
I1118 17:33:10.142073  1 server.go:101] FLAG: --federations="" 
I1118 17:33:10.142171  1 server.go:101] FLAG: --healthz-port="8081" 
I1118 17:33:10.142260  1 server.go:101] FLAG: --kube-master-url="" 
I1118 17:33:10.142345  1 server.go:101] FLAG: --kubecfg-file="" 
I1118 17:33:10.142433  1 server.go:101] FLAG: --log-backtrace-at=":0" 
I1118 17:33:10.142522  1 server.go:101] FLAG: --log-dir="" 
I1118 17:33:10.142605  1 server.go:101] FLAG: --log-flush-frequency="5s" 
I1118 17:33:10.142688  1 server.go:101] FLAG: --logtostderr="true" 
I1118 17:33:10.142771  1 server.go:101] FLAG: --stderrthreshold="2" 
I1118 17:33:10.142853  1 server.go:101] FLAG: --v="0" 
I1118 17:33:10.142932  1 server.go:101] FLAG: --version="false" 
I1118 17:33:10.143056  1 server.go:101] FLAG: --vmodule="" 
I1118 17:33:10.143247  1 server.go:138] Starting SkyDNS server. Listening on port:10053 
I1118 17:33:10.143455  1 server.go:145] skydns: metrics enabled on : /metrics: 
I1118 17:33:10.143556  1 dns.go:166] Waiting for service: default/kubernetes 
I1118 17:33:10.144214  1 logs.go:41] skydns: ready for queries on cluster.local. for tcp://0.0.0.0:10053 [rcache 0] 
I1118 17:33:10.144358  1 logs.go:41] skydns: ready for queries on cluster.local. for udp://0.0.0.0:10053 [rcache 0] 
I1118 17:33:10.154429  1 dns.go:172] Ignoring error while waiting for service default/kubernetes: Get https://10.244.0.1:443/api/v1/namespaces/default/services/kubernetes: x509: failed to load system roots and no roots provided. Sleeping 1s before retrying. 
E1118 17:33:10.159852  1 reflector.go:214] pkg/dns/dns.go:155: Failed to list *api.Service: Get https://10.244.0.1:443/api/v1/services?resourceVersion=0: x509: failed to load system roots and no roots provided 
E1118 17:33:10.171051  1 reflector.go:214] pkg/dns/dns.go:154: Failed to list *api.Endpoints: Get https://10.244.0.1:443/api/v1/endpoints?resourceVersion=0: x509: failed to load system roots and no roots provided 
I1118 17:33:11.157527  1 dns.go:172] Ignoring error while waiting for service default/kubernetes: Get https://10.244.0.1:443/api/v1/namespaces/default/services/kubernetes: x509: failed to load system roots and no roots provided. Sleeping 1s before retrying. 
E1118 17:33:11.166217  1 reflector.go:214] pkg/dns/dns.go:155: Failed to list *api.Service: Get https://10.244.0.1:443/api/v1/services?resourceVersion=0: x509: failed to load system roots and no roots provided 
E1118 17:33:11.181840  1 reflector.go:214] pkg/dns/dns.go:154: Failed to list *api.Endpoints: Get https://10.244.0.1:443/api/v1/endpoints?resourceVersion=0: x509: failed to load system roots and no roots provided

來源

2016-11-18 Gapmeister66

我會嘗試刪除機密，或乾脆服務帳戶： Kubernetes如果沒有，將創建一個默認的服務帳戶。但是，如果您這樣做，則需要重新啓動所有窗格，以便獲取新的服務帳戶信息。 – MrE

我也會確保你的集羣有適當的容器版本。檢查回購正確的圖像版本，因爲這些事情發生了很大的變化，DNS是這個難題的重要部分。 – MrE

出於禮貌我張貼我的解決方案，這可能也有助於某人在相同的情況。我正在使用cloud-init啓動K8S服務並在一個pod中運行DNS。我意識到pod運行在不同的網絡中，因此修改我的dns-addon.yaml以將正確的網絡地址傳遞給kubedns容器：「 - --kube-master-url = http://10.10.1.31:8080」。錯誤消息有點誤導！

來源

2016-11-22 17:56:05 Gapmeister66

儘管您可以像您在答案中提出的那樣，與不安全的端口通話，從而繞過任何身份驗證並解決了kubeDNS問題，但這不會解決集羣中使用服務帳戶的其他任何內容的祕密。

祕密中沒有包含CA的原因是您沒有告訴控制器管理器包含一個。您可以爲根CA提供標誌--root-ca-file。

從kube-controller-manager documentation

如果設置，這根證書頒發機構將包含在服務帳戶的令牌祕密。這必須是有效的PEM編碼的CA bundle.`

我會強烈建議採取看看current version of the CoreOS Kubernetes Step-by-Step documentation，這是最新的最新與Kubernetes 1.4.3。已經有一些變化，它包括如何適當地生成和使用自簽名的證書，以確保您的集羣，並提供有效的服務帳戶令牌文檔

來源

2016-11-23 11:39:46 ChaosAffe

DNS插件在Kubernetes中輸入crashloopbackoff 1.4.5

回答

相關問題