Config all nodes environment

Letting iptables see bridged traffic

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward=1
EOF
sudo sysctl --system

Turn off firewall

1 2	sudo systemctl stop ufw sudo systemctl disable ufw

Turn off swap

1 2	sudo swapoff -a && sudo sysctl -w vm.swappiness=0 sudo sed -ri '/^[^#]*swap/s@^@#@' /etc/fstab

Install docker

Please refer How Install Docker and Docker Compose On Ubuntu

Change docker cgroup driver to “systemd”

1	sudo vi /etc/docker/daemon.json

Add content

1
2
3

{
  "exec-opts": ["native.cgroupdriver=systemd"]
}

Restart docker

1 2	sudo systemctl restart docker sudo systemctl enable docker

Install K8S

sudo curl -s https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | sudo apt-key add -
sudo tee /etc/apt/sources.list.d/kubernetes.list <<-'EOF'
deb https://mirrors.aliyun.com/kubernetes/apt kubernetes-xenial main
EOF
sudo apt-get update
sudo apt-get install -y kubeadm kubectl kubelet

Init K8S on master

1	sudo kubeadm init --image-repository registry.aliyuncs.com/google_containers --pod-network-cidr=10.244.0.0/16

Once K8S initialize successfully, you will see output like this

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.2.8:6443 --token 8borel.phprbzxwjupl3cuy \
	--discovery-token-ca-cert-hash sha256:c441bbee5846711e3c00260e6daebc31258fcf8b9dbf7bb7b8b3b2c10f6ffbdf

Create new token with the following command

1	kubeadm token create --print-join-command

Copy config file to HOME folder

1
2
3

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Install flannel

1
2
3

curl https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml -o kube-flannel.yaml

kubectl apply -f kube-flannel.yaml

Check node status

1	kubectl get nodes -o wide

Add nodes to cluster

Run the command on worker node

1 2	kubeadm join 192.168.2.8:6443 --token 8borel.phprbzxwjupl3cuy \ --discovery-token-ca-cert-hash sha256:c441bbee5846711e3c00260e6daebc31258fcf8b9dbf7bb7b8b3b2c10f6ffbdf

Config GPU node

If your worker nodes are GPU servers, you should install nvidia-docker2 on it

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
   && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \
   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update
sudo apt-get install -y nvidia-docker2

Change container default runtime

1	sudo vi /etc/docker/daemon.json

Change default docker runtime to nvidia

{
    "exec-opts": ["native.cgroupdriver=systemd"],
    "default-runtime": "nvidia",
    "runtimes": {
        "nvidia": {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    }
}

Restart docker service

1 2	sudo systemctl daemon-reload sudo systemctl restart docker

Install nvidia device plugin on master

1 2	curl https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/master/nvidia-device-plugin.yml -o nvidia-device-plugin.yml kubectl apply -f nvidia-device-plugin.yml

Enable kubectl bash completion and alias

1 2	source <(kubectl completion bash) alias k=kubectl

Install kubesphere

```
kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.2.1/kubesphere-installer.yaml

kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.2.1/cluster-configuration.yaml
``

知秋

Install K8S on Ubuntu 18.04