Install Kubernetes

Setup

Prerequisites

Access as root user or user having root privileges
Create new user to run kubernetes
# useradd kadmin
Setup SUDO access for "kadmin user"
# vi /etc/sudoers kadmin ALL=(ALL) NOPASSWD:ALL
Setup hosts file with the master and worker node entries
# vi /etc/hosts 192.168.50.174 kube-master1 kube-master1.linuxtechspace.com 192.168.50.175 kube-worker1 kube-worker1.linuxtechspace.com 192.168.50.176 kube-worker2 kube-worker2.linuxtechspace.com
Disable SWAP on servers
# swapoff -a
# vi /etc/fstab Comment entry containing swap keyword

Install and Setup

Note: Perform following steps on all nodes (Master & Slave(s))

Install Container Runtime Environment

We will be using "containerd" as container runtime environment. We will use the version available at Docker repository.

Setup YUM repo file for containerd
# vi /etc/yum.repos.d/docker-ce.repo [docker-ce-stable] name=Docker CE Stable - $basearch baseurl=https://download.docker.com/linux/centos/$releasever/$basearch/stable enabled=1 gpgcheck=1 gpgkey=https://download.docker.com/linux/centos/gpg [centos-extra] name=CentOS extra baseurl=http://mirror.centos.org/centos/7/extras/x86_64/ enabled=1 gpgcheck=0
Install containerd package
# yum -y install containerd.io Loaded plugins: product-id, search-disabled-repos, subscription-manager This system is not registered with an entitlement server. You can use subscription-manager to register. centos-extra | 2.9 kB 00:00:00 docker-ce-stable | 3.5 kB 00:00:00 kubernetes | 1.4 kB 00:00:00 localrepo | 3.7 kB 00:00:00 (1/5): localrepo/group_gz | 144 kB 00:00:00 (2/5): docker-ce-stable/7Server/x86_64/updateinfo | 55 B 00:00:00 (3/5): centos-extra/primary_db | 246 kB 00:00:00 (4/5): docker-ce-stable/7Server/x86_64/primary_db | 78 kB 00:00:00 (5/5): kubernetes/x86_64/primary | 108 kB 00:00:00 kubernetes 797/797 Resolving Dependencies --> Running transaction check ---> Package containerd.io.x86_64 0:1.6.4-3.1.el7 will be installed --> Processing Dependency: container-selinux >= 2:2.74 for package: containerd.io-1.6.4-3.1.el7.x86_64 --> Running transaction check ---> Package container-selinux.noarch 2:2.119.2-1.911c772.el7_8 will be installed --> Processing Dependency: policycoreutils-python for package: 2:container-selinux-2.119.2-1.911c772.el7_8.noarch --> Running transaction check ---> Package policycoreutils-python.x86_64 0:2.5-29.el7 will be installed --> Processing Dependency: setools-libs >= 3.3.8-4 for package: policycoreutils-python-2.5-29.el7.x86_64 --> Processing Dependency: libsemanage-python >= 2.5-14 for package: policycoreutils-python-2.5-29.el7.x86_64 --> Processing Dependency: audit-libs-python >= 2.1.3-4 for package: policycoreutils-python-2.5-29.el7.x86_64 --> Processing Dependency: python-IPy for package: policycoreutils-python-2.5-29.el7.x86_64 --> Processing Dependency: libqpol.so.1(VERS_1.4)(64bit) for package: policycoreutils-python-2.5-29.el7.x86_64 --> Processing Dependency: libqpol.so.1(VERS_1.2)(64bit) for package: policycoreutils-python-2.5-29.el7.x86_64 --> Processing Dependency: libcgroup for package: policycoreutils-python-2.5-29.el7.x86_64 --> Processing Dependency: libapol.so.4(VERS_4.0)(64bit) for package: policycoreutils-python-2.5-29.el7.x86_64 --> Processing Dependency: checkpolicy for package: policycoreutils-python-2.5-29.el7.x86_64 --> Processing Dependency: libqpol.so.1()(64bit) for package: policycoreutils-python-2.5-29.el7.x86_64 --> Processing Dependency: libapol.so.4()(64bit) for package: policycoreutils-python-2.5-29.el7.x86_64 --> Running transaction check ---> Package audit-libs-python.x86_64 0:2.8.4-4.el7 will be installed ---> Package checkpolicy.x86_64 0:2.5-8.el7 will be installed ---> Package libcgroup.x86_64 0:0.41-20.el7 will be installed ---> Package libsemanage-python.x86_64 0:2.5-14.el7 will be installed ---> Package python-IPy.noarch 0:0.75-6.el7 will be installed ---> Package setools-libs.x86_64 0:3.3.8-4.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================== Installing: containerd.io x86_64 1.6.4-3.1.el7 docker-ce-stable 33 M Installing for dependencies: audit-libs-python x86_64 2.8.4-4.el7 localrepo 76 k checkpolicy x86_64 2.5-8.el7 localrepo 295 k container-selinux noarch 2:2.119.2-1.911c772.el7_8 centos-extra 40 k libcgroup x86_64 0.41-20.el7 localrepo 66 k libsemanage-python x86_64 2.5-14.el7 localrepo 113 k policycoreutils-python x86_64 2.5-29.el7 localrepo 456 k python-IPy noarch 0.75-6.el7 localrepo 32 k setools-libs x86_64 3.3.8-4.el7 localrepo 620 k Transaction Summary ============================================================================================================================================================== Install 1 Package (+8 Dependent packages) Total download size: 35 M Installed size: 130 M Downloading packages: (1/9): audit-libs-python-2.8.4-4.el7.x86_64.rpm | 76 kB 00:00:00 (2/9): libcgroup-0.41-20.el7.x86_64.rpm | 66 kB 00:00:00 (3/9): libsemanage-python-2.5-14.el7.x86_64.rpm | 113 kB 00:00:00 (4/9): checkpolicy-2.5-8.el7.x86_64.rpm | 295 kB 00:00:00 (5/9): python-IPy-0.75-6.el7.noarch.rpm | 32 kB 00:00:00 (6/9): policycoreutils-python-2.5-29.el7.x86_64.rpm | 456 kB 00:00:00 (7/9): setools-libs-3.3.8-4.el7.x86_64.rpm | 620 kB 00:00:00 (8/9): container-selinux-2.119.2-1.911c772.el7_8.noarch.rpm | 40 kB 00:00:00 warning: /var/cache/yum/x86_64/7Server/docker-ce-stable/packages/containerd.io-1.6.4-3.1.el7.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 621e9f35: NOKEY Public key for containerd.io-1.6.4-3.1.el7.x86_64.rpm is not installed (9/9): containerd.io-1.6.4-3.1.el7.x86_64.rpm | 33 MB 00:00:02 -------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 13 MB/s | 35 MB 00:00:02 Retrieving key from https://download.docker.com/linux/centos/gpg Importing GPG key 0x621E9F35: Userid : "Docker Release (CE rpm) " Fingerprint: 060a 61c5 1b55 8a7f 742b 77aa c52f eb6b 621e 9f35 From : https://download.docker.com/linux/centos/gpg Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : audit-libs-python-2.8.4-4.el7.x86_64 1/9 Installing : setools-libs-3.3.8-4.el7.x86_64 2/9 Installing : python-IPy-0.75-6.el7.noarch 3/9 Installing : libsemanage-python-2.5-14.el7.x86_64 4/9 Installing : checkpolicy-2.5-8.el7.x86_64 5/9 Installing : libcgroup-0.41-20.el7.x86_64 6/9 Installing : policycoreutils-python-2.5-29.el7.x86_64 7/9 Installing : 2:container-selinux-2.119.2-1.911c772.el7_8.noarch 8/9 setsebool: SELinux is disabled. Installing : containerd.io-1.6.4-3.1.el7.x86_64 9/9 Verifying : libcgroup-0.41-20.el7.x86_64 1/9 Verifying : checkpolicy-2.5-8.el7.x86_64 2/9 Verifying : libsemanage-python-2.5-14.el7.x86_64 3/9 Verifying : policycoreutils-python-2.5-29.el7.x86_64 4/9 Verifying : 2:container-selinux-2.119.2-1.911c772.el7_8.noarch 5/9 Verifying : python-IPy-0.75-6.el7.noarch 6/9 Verifying : containerd.io-1.6.4-3.1.el7.x86_64 7/9 Verifying : setools-libs-3.3.8-4.el7.x86_64 8/9 Verifying : audit-libs-python-2.8.4-4.el7.x86_64 9/9 Installed: containerd.io.x86_64 0:1.6.4-3.1.el7 Dependency Installed: audit-libs-python.x86_64 0:2.8.4-4.el7 checkpolicy.x86_64 0:2.5-8.el7 container-selinux.noarch 2:2.119.2-1.911c772.el7_8 libcgroup.x86_64 0:0.41-20.el7 libsemanage-python.x86_64 0:2.5-14.el7 policycoreutils-python.x86_64 0:2.5-29.el7 python-IPy.noarch 0:0.75-6.el7 setools-libs.x86_64 0:3.3.8-4.el7 Complete!
Modify containerd configuration to enable CRI

FInd the following line in the file and comment the line.

# vi /etc/containerd/config.toml #disabled_plugins = ["cri"]
Restart and Enable containerd service
# systemctl restart containerd
# systemctl enable containerd
Set the kernel tuning parameter
# modprobe br_netfilter
vi /etc/sysctl.conf net.bridge.bridge-nf-call-iptables = 1 net.ipv4.ip_forward = 1 # sysctl -p

Install Kubernetes

Setup YUM repository file
# vi /etc/yum.repos.d/kubernetes.repo #[kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch enabled=1 gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg exclude=kubelet kubeadm kubectl
Install kubernetes packages
# yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes Loaded plugins: product-id, search-disabled-repos, subscription-manager This system is not registered with an entitlement server. You can use subscription-manager to register. Resolving Dependencies --> Running transaction check ---> Package kubeadm.x86_64 0:1.24.0-0 will be installed --> Processing Dependency: kubernetes-cni >= 0.8.6 for package: kubeadm-1.24.0-0.x86_64 --> Processing Dependency: cri-tools >= 1.19.0 for package: kubeadm-1.24.0-0.x86_64 ---> Package kubectl.x86_64 0:1.24.0-0 will be installed ---> Package kubelet.x86_64 0:1.24.0-0 will be installed --> Processing Dependency: socat for package: kubelet-1.24.0-0.x86_64 --> Processing Dependency: conntrack for package: kubelet-1.24.0-0.x86_64 --> Running transaction check ---> Package conntrack-tools.x86_64 0:1.4.4-4.el7 will be installed --> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.1)(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64 --> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.0)(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64 --> Processing Dependency: libnetfilter_cthelper.so.0(LIBNETFILTER_CTHELPER_1.0)(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64 --> Processing Dependency: libnetfilter_queue.so.1()(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64 --> Processing Dependency: libnetfilter_cttimeout.so.1()(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64 --> Processing Dependency: libnetfilter_cthelper.so.0()(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64 ---> Package cri-tools.x86_64 0:1.23.0-0 will be installed ---> Package kubernetes-cni.x86_64 0:0.8.7-0 will be installed ---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed --> Running transaction check ---> Package libnetfilter_cthelper.x86_64 0:1.0.0-9.el7 will be installed ---> Package libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7 will be installed ---> Package libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 will be installed --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================== Package Arch Version Repository Size ============================================================================================================================================================== Installing: kubeadm x86_64 1.24.0-0 kubernetes 9.5 M kubectl x86_64 1.24.0-0 kubernetes 9.9 M kubelet x86_64 1.24.0-0 kubernetes 20 M Installing for dependencies: conntrack-tools x86_64 1.4.4-4.el7 localrepo 186 k cri-tools x86_64 1.23.0-0 kubernetes 7.1 M kubernetes-cni x86_64 0.8.7-0 kubernetes 19 M libnetfilter_cthelper x86_64 1.0.0-9.el7 localrepo 18 k libnetfilter_cttimeout x86_64 1.0.0-6.el7 localrepo 18 k libnetfilter_queue x86_64 1.0.2-2.el7_2 localrepo 23 k socat x86_64 1.7.3.2-2.el7 localrepo 290 k Transaction Summary ============================================================================================================================================================== Install 3 Packages (+7 Dependent packages) Total download size: 66 M Installed size: 288 M Downloading packages: (1/10): conntrack-tools-1.4.4-4.el7.x86_64.rpm | 186 kB 00:00:00 warning: /var/cache/yum/x86_64/7Server/kubernetes/packages/4d300a7655f56307d35f127d99dc192b6aa4997f322234e754f16aaa60fd8906-cri-tools-1.23.0-0.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 3e1ba8d5: NOKEY Public key for 4d300a7655f56307d35f127d99dc192b6aa4997f322234e754f16aaa60fd8906-cri-tools-1.23.0-0.x86_64.rpm is not installed (2/10): 4d300a7655f56307d35f127d99dc192b6aa4997f322234e754f16aaa60fd8906-cri-tools-1.23.0-0.x86_64.rpm | 7.1 MB 00:00:01 (3/10): dda11ee75bc7fcb01e32512cefb8f686dc6a7383516b8b0828adb33761fe602e-kubeadm-1.24.0-0.x86_64.rpm | 9.5 MB 00:00:02 (4/10): 0c7a02e05273d05ea82ca13546853b65fbc257dd159565ce6eb658a0bdf31c9f-kubectl-1.24.0-0.x86_64.rpm | 9.9 MB 00:00:00 (5/10): libnetfilter_cthelper-1.0.0-9.el7.x86_64.rpm | 18 kB 00:00:00 (6/10): libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm | 23 kB 00:00:00 (7/10): socat-1.7.3.2-2.el7.x86_64.rpm | 290 kB 00:00:00 (8/10): libnetfilter_cttimeout-1.0.0-6.el7.x86_64.rpm | 18 kB 00:00:00 (9/10): db7cb5cb0b3f6875f54d10f02e625573988e3e91fd4fc5eef0b1876bb18604ad-kubernetes-cni-0.8.7-0.x86_64.rpm | 19 MB 00:00:01 (10/10): 363f3fbfa8b89bb978e2d089e52ba59847f143834f8ea1b559afa864d8c5c011-kubelet-1.24.0-0.x86_64.rpm | 20 MB 00:00:03 -------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 12 MB/s | 66 MB 00:00:05 Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg Importing GPG key 0x6B4097C2: Userid : "Rapture Automatic Signing Key (cloud-rapture-signing-key-2022-03-07-08_01_01.pub)" Fingerprint: e936 7157 4236 81a4 7ec3 93c3 7325 816a 6b40 97c2 From : https://packages.cloud.google.com/yum/doc/yum-key.gpg Importing GPG key 0x307EA071: Userid : "Rapture Automatic Signing Key (cloud-rapture-signing-key-2021-03-01-08_01_09.pub)" Fingerprint: 7f92 e05b 3109 3bef 5a3c 2d38 feea 9169 307e a071 From : https://packages.cloud.google.com/yum/doc/yum-key.gpg Importing GPG key 0x836F4BEB: Userid : "gLinux Rapture Automatic Signing Key (//depot/google3/production/borg/cloud-rapture/keys/cloud-rapture-pubkeys/cloud-rapture-signing-key-2020-12-03-16_08_05.pub) " Fingerprint: 59fe 0256 8272 69dc 8157 8f92 8b57 c5c2 836f 4beb From : https://packages.cloud.google.com/yum/doc/yum-key.gpg Retrieving key from https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg Importing GPG key 0x3E1BA8D5: Userid : "Google Cloud Packages RPM Signing Key " Fingerprint: 3749 e1ba 95a8 6ce0 5454 6ed2 f09c 394c 3e1b a8d5 From : https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : socat-1.7.3.2-2.el7.x86_64 1/10 Installing : cri-tools-1.23.0-0.x86_64 2/10 Installing : libnetfilter_cthelper-1.0.0-9.el7.x86_64 3/10 Installing : libnetfilter_queue-1.0.2-2.el7_2.x86_64 4/10 Installing : kubectl-1.24.0-0.x86_64 5/10 Installing : libnetfilter_cttimeout-1.0.0-6.el7.x86_64 6/10 Installing : conntrack-tools-1.4.4-4.el7.x86_64 7/10 Installing : kubelet-1.24.0-0.x86_64 8/10 Installing : kubernetes-cni-0.8.7-0.x86_64 9/10 Installing : kubeadm-1.24.0-0.x86_64 10/10 Verifying : kubernetes-cni-0.8.7-0.x86_64 1/10 Verifying : kubeadm-1.24.0-0.x86_64 2/10 Verifying : libnetfilter_cttimeout-1.0.0-6.el7.x86_64 3/10 Verifying : kubectl-1.24.0-0.x86_64 4/10 Verifying : libnetfilter_queue-1.0.2-2.el7_2.x86_64 5/10 Verifying : libnetfilter_cthelper-1.0.0-9.el7.x86_64 6/10 Verifying : cri-tools-1.23.0-0.x86_64 7/10 Verifying : conntrack-tools-1.4.4-4.el7.x86_64 8/10 Verifying : socat-1.7.3.2-2.el7.x86_64 9/10 Verifying : kubelet-1.24.0-0.x86_64 10/10 Installed: kubeadm.x86_64 0:1.24.0-0 kubectl.x86_64 0:1.24.0-0 kubelet.x86_64 0:1.24.0-0 Dependency Installed: conntrack-tools.x86_64 0:1.4.4-4.el7 cri-tools.x86_64 0:1.23.0-0 kubernetes-cni.x86_64 0:0.8.7-0 libnetfilter_cthelper.x86_64 0:1.0.0-9.el7 libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7 libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 socat.x86_64 0:1.7.3.2-2.el7 Complete!
Start and Enable kubelet service
# systemctl start kubelet
# systemctl enable kubelet

Setup Cluster

Note: Perform following steps only on Master or Control Plane server

Login as "kadmin" user
# su - kadmin
Create kubernetes cluster
# sudo kubeadm init --control-plane-endpoint=192.168.50.174 --pod-network-cidr=192.168.0.0/16 --apiserver-advertise-address=192.168.50.174 [init] Using Kubernetes version: v1.24.0 [preflight] Running pre-flight checks [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using 'kubeadm config images pull' [certs] Using certificateDir folder "/etc/kubernetes/pki" [certs] Generating "ca" certificate and key [certs] Generating "apiserver" certificate and key [certs] apiserver serving cert is signed for DNS names [kube-master1.linuxtechspace.com kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.50.174] [certs] Generating "apiserver-kubelet-client" certificate and key [certs] Generating "front-proxy-ca" certificate and key [certs] Generating "front-proxy-client" certificate and key [certs] Generating "etcd/ca" certificate and key [certs] Generating "etcd/server" certificate and key [certs] etcd/server serving cert is signed for DNS names [kube-master1.linuxtechspace.com localhost] and IPs [192.168.50.174 127.0.0.1 ::1] [certs] Generating "etcd/peer" certificate and key [certs] etcd/peer serving cert is signed for DNS names [kube-master1.linuxtechspace.com localhost] and IPs [192.168.50.174 127.0.0.1 ::1] [certs] Generating "etcd/healthcheck-client" certificate and key [certs] Generating "apiserver-etcd-client" certificate and key [certs] Generating "sa" key and public key [kubeconfig] Using kubeconfig folder "/etc/kubernetes" [kubeconfig] Writing "admin.conf" kubeconfig file [kubeconfig] Writing "kubelet.conf" kubeconfig file [kubeconfig] Writing "controller-manager.conf" kubeconfig file [kubeconfig] Writing "scheduler.conf" kubeconfig file [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env" [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml" [kubelet-start] Starting the kubelet [control-plane] Using manifest folder "/etc/kubernetes/manifests" [control-plane] Creating static Pod manifest for "kube-apiserver" [control-plane] Creating static Pod manifest for "kube-controller-manager" [control-plane] Creating static Pod manifest for "kube-scheduler" [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s [apiclient] All control plane components are healthy after 22.054591 seconds [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace [kubelet] Creating a ConfigMap "kubelet-config" in namespace kube-system with the configuration for the kubelets in the cluster [upload-certs] Skipping phase. Please see --upload-certs [mark-control-plane] Marking the node kube-master1.linuxtechspace.com as control-plane by adding the labels: [node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers] [mark-control-plane] Marking the node kube-master1.linuxtechspace.com as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule node-role.kubernetes.io/control-plane:NoSchedule] [bootstrap-token] Using token: l95f5d.88i5o5mvne9sfsus [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to get nodes [bootstrap-token] Configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials [bootstrap-token] Configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token [bootstrap-token] Configured RBAC rules to allow certificate rotation for all node client certificates in the cluster [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace [kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key [addons] Applied essential addon: CoreDNS [addons] Applied essential addon: kube-proxy Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config Alternatively, if you are the root user, you can run: export KUBECONFIG=/etc/kubernetes/admin.conf You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ You can now join any number of control-plane nodes by copying certificate authorities and service account keys on each node and then running the following as root: kubeadm join 192.168.50.174:6443 --token l95f5d.88i5o5mvne9sfsus \ --discovery-token-ca-cert-hash sha256:6929793f07e8348c9366795583e3721f8365f36eb855c78e92a6a1b21d2f2702 \ --control-plane Then you can join any number of worker nodes by running the following on each as root: kubeadm join 192.168.50.174:6443 --token l95f5d.88i5o5mvne9sfsus \ --discovery-token-ca-cert-hash sha256:6929793f07e8348c9366795583e3721f8365f36eb855c78e92a6a1b21d2f2702
Copy cluster files to local kadmin user
# mkdir -p $HOME/.kube
# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
# sudo chown $(id -u):$(id -g) $HOME/.kube/config

Note: The Node state is "NotReady". This is because it depends on Network service which we still haven't setup yet.

Check the status of Node and Pods
# kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME kube-master1.linuxtechspace.com NotReady control-plane 110s v1.24.0 192.168.50.174 Red Hat Enterprise Linux Server 7.6 (Maipo) 3.10.0-957.el7.x86_64 containerd://1.6.4
# kubectl get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-6d4b75cb6d-k5lqv 0/1 Pending 0 4m28s kube-system coredns-6d4b75cb6d-whddj 0/1 Pending 0 4m28s kube-system etcd-kube-master1.linuxtechspace.com 1/1 Running 0 4m46s kube-system kube-apiserver-kube-master1.linuxtechspace.com 1/1 Running 0 4m44s kube-system kube-controller-manager-kube-master1.linuxtechspace.com 1/1 Running 0 4m48s kube-system kube-proxy-c4bfd 1/1 Running 0 4m28s kube-system kube-scheduler-kube-master1.linuxtechspace.com 1/1 Running 0 4m46s
Setup Network Service using Calito
# sudo kubectl --kubeconfig=/etc/kubernetes/admin.conf create -f https://projectcalico.docs.tigera.io/manifests/tigera-operator.yaml namespace/tigera-operator created customresourcedefinition.apiextensions.k8s.io/bgpconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/bgppeers.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/blockaffinities.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/caliconodestatuses.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/clusterinformations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/felixconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/globalnetworkpolicies.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/globalnetworksets.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/hostendpoints.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamblocks.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamconfigs.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipamhandles.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ippools.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/ipreservations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/kubecontrollersconfigurations.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/networkpolicies.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/networksets.crd.projectcalico.org created customresourcedefinition.apiextensions.k8s.io/apiservers.operator.tigera.io created customresourcedefinition.apiextensions.k8s.io/imagesets.operator.tigera.io created customresourcedefinition.apiextensions.k8s.io/installations.operator.tigera.io created customresourcedefinition.apiextensions.k8s.io/tigerastatuses.operator.tigera.io created Warning: policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+ podsecuritypolicy.policy/tigera-operator created serviceaccount/tigera-operator created clusterrole.rbac.authorization.k8s.io/tigera-operator created clusterrolebinding.rbac.authorization.k8s.io/tigera-operator created deployment.apps/tigera-operator created