K8s deployment problems I met lufy March 16, 2022 <p>systemctl start kubelet failed, journalctl -xeu kubelet show the fail message:</p> <p>Failed to run kubelet" err="failed to run Kubelet: misconfiguration: kubelet cgroup></p> <p>The whole log should be</p> <p>misconfiguration: kubelet cgroup driver: "cgroupfs" is different from docker cgroup driver: "systemd"</p> <p>I learn from <a href="https://github.com/kubernetes/kubernetes/issues/43805">https://github.com/kubernetes/kubernetes/issues/43805</a></p> <p>and change docker cgroup to systemd by editing /usr/lib/systemd/system/docker.service, add --exec-opt native.cgroupdriver=systemd after the dockerd cmdline.</p> <p>Then systemctl daemon-reload; systemctl start kubelet successfully.</p> <p>By the way, docker info | grep -i cgorup whould show what cgroup driver docker is using.</p> <p> </p> <p>Node not ready, reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized</p> <p>This is about I need a network plugin. By reading from</p> <p><a href="https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/">https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/</a></p> <p>and</p> <p><a href="https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/">https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/</a></p> <p>and execute the installation steps from</p> <p><a href="https://projectcalico.docs.tigera.io/getting-started/kubernetes/quickstart">https://projectcalico.docs.tigera.io/getting-started/kubernetes/quickstart</a></p> <p>I still get the NotReady message. Finally I found it's that I have not get the meaning. The quickstart say: you may need to change the default IP pool CIDR to match your pod network CIDR, which I have missed.</p> <p>So I wget <code id="codeblock-4"><a href="https://projectcalico.docs.tigera.io/manifests/custom-resources.yaml">https://projectcalico.docs.tigera.io/manifests/custom-resources.yaml</a> and edit the cidr, then after waiting about 20min, I observed the nodes are all ready.</code></p> <p><code></code></p> <p>k logs -p -n kubeapps kubeapps-internal-apprepository-controller-685685876b-kk2kl met</p> <p>Error from server: Get "https://192.168.0.204:10250/containerLogs/kubeapps/kubeapps-internal-apprepository-controller-685685876b-kk2kl/controller?previous=true": dial tcp 192.168.0.204:10250: i/o timeout</p> <p>I already checked firewalld was disabled.</p> <p>Some said flush iptables would work, but not really worked for me by:</p> <pre><code class="language-shell hljs"><span class="language-bash">systemctl stop kubelet<br /></span></code><code class="language-shell hljs">systemctl stop docke</code><code class="language-shell hljs"><br />iptables --flush iptables -tnat --flush<br /></code><code id="copy_target_1" class="language-shell hljs">systemctl start docker<br />systemctl start kubelet<br /><br />I solved this by re-stop firewalld with:<br />systemctl start firewalld ## here met unit file firewalld.service is masked, solved by systemctl unmask firewalld<br />firewall-cmd --add-port=10250/tcp ## then kubectl logs appears ok.<br />systemctl stop firewalld ## also, kubectl logs worked fine.<br />I think it's that iptables not flushed correctly.<br />I finally find out the reason of --<br />Why kubectl logs ok while re-stop firewalld but failed when os reboot:<br />when os reboot, iptables rules is just the same as when you disable firewalld. The rules added before disabling firewalld cannot be saved. <br />The differences hided in /etc/sysconfig/nftables.conf or /etc/sysconfig/iptables-config<br />And I solved this by empty /etc/sysconfig/nftables.conf by echo > /etc/sysconfig/nftables.conf</code><code id="copy_target_1" class="language-shell hljs"></code></pre> <pre><code id="copy_target_1" class="language-shell hljs"></code><br /><br /></pre> <pre><code id="copy_target_1" class="language-shell hljs"></code></pre>
Comments (0)
Leave a Comment
No comments yet. Be the first to comment!