Ubuntu 22 部署 kubernetes 集群

0x0. 环境准备

本文服务器的公网IP：192.168.56.101

• OS version: Ubuntu 22
• CPU Architecture: x86_64
• K8s version: v1.29

0x1. 安装依赖

apt install -y \
  curl \
  wget \
  systemd \
  lrzsz \
  bash-completion \
  gpg

0x2. 安装前准备

开启必要的端口

开启端口或直接关闭防火墙

sudo ufw disable

同步服务器时间

sudo timedatectl set-timezone Asia/Shanghai && sudo timedatectl set-local-rtc 0
sudo systemctl restart rsyslog
sudo systemctl restart crond

修改主机名

方便通过主机名访问对于的服务器

# 主节点
sudo hostnamectl set-hostname k8s-master
# 从节点
sudo hostnamectl set-hostname k8s-node1
sudo hostnamectl set-hostname k8s-node2

修改hosts

cat >/etc/hosts <<EOF
192.168.56.101    k8s-master
192.168.56.102    k8s-node1
192.168.56.103    k8s-node2
EOF

关闭selinux

setenforce 0 && sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

关闭swap分区或者禁用swap文件

swapoff -a && sed -ri 's/.*swap.*/#&/' /etc/fstab

0x3. 容器运行时

https://v1-29.docs.kubernetes.io/zh-cn/docs/setup/production-environment/container-runtimes/

转发IPv4并让iptables看到桥接流量

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

sudo modprobe overlay
sudo modprobe br_netfilter

# 设置所需的 sysctl 参数，参数在重新启动后保持不变
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

# 应用 sysctl 参数而不重新启动
sudo sysctl --system

# 通过运行以下指令确认 br_netfilter 和 overlay 模块被加载
lsmod | grep br_netfilter
lsmod | grep overlay
# 通过运行以下指令确认 net.bridge.bridge-nf-call-iptables、net.bridge.bridge-nf-call-ip6tables 和 net.ipv4.ip_forward 系统变量在你的 sysctl 配置中被设置为 1
sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward

安装容器运行时

https://docs.docker.com/engine/install/ubuntu/

# Add Docker's official GPG key:
sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install containerd.io

sudo mkdir -p /etc/containerd
sudo containerd config default >/etc/containerd/config.toml
sudo sed -i "s#SystemdCgroup = false#SystemdCgroup = true#g" /etc/containerd/config.toml
sudo systemctl daemon-reload
sudo systemctl enable containerd && sudo systemctl restart containerd

0x4. 安装k8s

apt update
apt install -y apt-transport-https ca-certificates curl gpg
sudo mkdir -p -m 755 /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.29/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.29/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
apt update
apt install -y kubelet-$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable-1.29.txt) \
  kubeadm-$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable-1.29.txt) \
  kubectl-$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable-1.29.txt)
# 锁定版本

cat >/etc/default/kubelet <<EOF
KUBELET_EXTRA_ARGS="--cgroup-driver=systemd"
EOF
# 设置容器运行时（仅容器运行时为containerd才需要进行以下设置，容器运行时为Docker则不需要）
crictl config runtime-endpoint unix:///var/run/containerd/containerd.sock
crictl config image-endpoint unix:///var/run/containerd/containerd.sock
sed -i '/KUBELET_KUBEADM_ARGS/s/"$/ --container-runtime=remote --container-runtime-endpoint=unix:\/\/\/run\/containerd\/containerd.sock"/' /var/lib/kubelet/kubeadm-flags.env

# kubelet开机自启
systemctl enable --now kubelet
# 查看kubelet状态
systemctl status kubelet
# 如果报错，查询错误信息
journalctl -xe

0x5. 运行k8s

https://v1-29.docs.kubernetes.io/zh-cn/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/

https://kubernetes.io/zh/docs/reference/setup-tools/kubeadm/kubeadm-init/

mkdir -p /k8sdata/log/
kubeadm init \
  --apiserver-advertise-address=192.168.56.101 \
  --kubernetes-version=v1.29.8 \
  --service-cidr=10.96.0.0/12 \
  --pod-network-cidr=10.244.0.0/16 \
  --cri-socket=unix:///var/run/containerd/containerd.sock | tee /k8sdata/log/kubeadm-init.log

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

提示:

1. 如果是搭建的服务器是主节点，则服务器至少2核2G，如果没有达到该配置但是仍想安装，则可以在kubeadm init 命令行中使用–ignore-preflight-errors=CpuNum即可忽略报错。
2. 如果初始化失败，通过kubeadm reset进行重设

0x6. 安装网络系统

• flannel

mkdir -p /k8sdata/network/
wget --no-check-certificate -O /k8sdata/network/flannelkube-flannel.yml https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl create -f /k8sdata/network/flannelkube-flannel.yml

• calico

mkdir -p /k8sdata/network/
wget --no-check-certificate -O /k8sdata/network/calico.yml https://docs.projectcalico.org/manifests/calico.yaml
kubectl create -f /k8sdata/network/calico.yml

0x7. k8s命令行补全

! grep -q kubectl "$HOME/.bashrc" && echo "source /usr/share/bash-completion/bash_completion" >>"$HOME/.bashrc"
! grep -q kubectl "$HOME/.bashrc" && echo "source <(kubectl completion bash)" >>"$HOME/.bashrc"
! grep -q kubeadm "$HOME/.bashrc" && echo "source <(kubeadm completion bash)" >>"$HOME/.bashrc"
! grep -q crictl "$HOME/.bashrc" && echo "source <(crictl completion bash)" >>"$HOME/.bashrc"
source "$HOME/.bashrc"

0x8. k8s常用命令

# 获取节点
kubectl get nodes -o wide
# 实时查询nodes状态
watch kubectl get nodes -o wide
# 获取pod
kubectl get pods --all-namespaces -o wide
# 查看镜像列表
kubeadm config images list
# 节点加入集群
kubeadm token create --print-join-command
# 描述node
kubectl describe node k8s-master
# 描述pod
kubectl describe pod kube-flannel-ds-hs8bq --namespace=kube-flannel