Setup Rancher and Spinup K8s Cluster
1. Prepare 4 CentOS 7 VMs
Install CentOS 7 min and assign IPs for each of them. I prepare 4 VMs, 1 for Rancher2.0 Server and 3 for K8s cluster nodes.- Rancher OS
- K8s Master
- K8s Slave 1
- K8s Slave 2
2. Open ports/protocol for firewalld or Disable firewalld
Here is the command for open the port for firewalld in centos.
https://rancher.com/docs/rancher/v2.x/en/installation/references/
$ sudo firewall-cmd --list-all
$ sudo firewall-cmd --zone=public --add-port=2376/tcp --add-port=2379/tcp --add-port=2380/tcp --add-port=6443/tcp --add-port=8472/tcp --add-port=9099/tcp --add-port=10250/tcp --add-port=10254/tcp --add-port=30000-32767/tcp --add-port=30000-32767/udp --permanent
$ sudo firewall-cmd --reload
https://rancher.com/docs/rancher/v2.x/en/installation/references/
$ sudo firewall-cmd --list-all
$ sudo firewall-cmd --zone=public --add-port=2376/tcp --add-port=2379/tcp --add-port=2380/tcp --add-port=6443/tcp --add-port=8472/tcp --add-port=9099/tcp --add-port=10250/tcp --add-port=10254/tcp --add-port=30000-32767/tcp --add-port=30000-32767/udp --permanent
$ sudo firewall-cmd --reload
3. Install Docker
Install Docker on all of 4 nodes via the command as below. Ref - https://docs.docker.com/install/linux/docker-ce/centos/#install-using-the-repository
$ sudo yum install -y yum-utils \
device-mapper-persistent-data \
lvm2
$ sudo yum-config-manager \
--add-repo \
https://download.docker.com/linux/centos/docker-ce.repo
$ sudo yum install docker-ce
$ sudo systemctl start docker
4. Install Rancher OS 2.0
ssh 1st node ( rancher node ) to docker run rancher docker
$ docker run -d --restart=unless-stopped -p 80:80 -p 443:443 rancher/rancher:latest
5. Create K8s Cluster
First login to update admin password and create a user ss/ss - e.g. https://192.168.xx.xx
Seconds, Global --> Add Cluster --> Custom
Thirds choose etcd and Control Plane, worker is the default.
Last, ssh into 3 nodes we prepared for k8s cluster then copy and paste command line as below.
sudo docker run -d --privileged --restart=unless-stopped --net=host -v /etc/kubernetes:/etc/kubernetes -v /var/run:/var/run rancher/rancher-agent:v2.0.8 --server https://192.168.xx.xx --token gfs8s9hlbqgtf76n6bhkbst5t9gjlprcbhs8kn658rsxvjp5lrmw64 --ca-checksum 40356f4341b0bd08c1c7581cba224c2cf4aa9585f876870620d8e81e4817cc85 --etcd --controlplane --worker
When your k8s ready you can start to log into your Gitlab for hook up k8s cluster as your Gitlab runner for your unit test, integration test or build.
Configure K8s as Gitlab Runner in Gitlab Project
1. Login into GitLab and jump into your project from Menu --> Operations --> Kubernetes2. Add Kubernetes Cluster --> add existing cluster
You might see the Kubernetes cluster detail -> expend
a. For Kubernetes cluster name:
You can name whatever you want.b. For API url:
For API url, you need to ssh to your rancher 2.0 server then install jq$ yum install epel-release -y
$ yum install jq -y
$ jq --version
$ sudo chmod +x get_kubeconfig_custom_cluster_rancher2.sh
$ ./get_kubeconfig_custom_cluster_rancher2.sh <your cluster name>
In the output from get_kubeconfig_custom_cluster_rancher2.sh, server: "https://192.168.xx.xx:6443" is your API URL.
c. For CA Certificate
You need to copy certificate-authority-data:'s content to url (https://www.base64decode.org/) to convert to CA Certificate. Then copy and paste Decode files from Base64 format result back to CA Certificate field.
d. For Token:
Before generating a token you need to create a namespace gitlab-managed-apps via running
$ kubectl apply -f http://x.co/rm082018
in rancher web portal cluster k8s shell
or you have config file download then run
$ kubectl –kubeconfig apply -f http://x.co/rm082018
(This will create a service account and create a token for it, which is the token that we need to specify in the GitLab Kubernetes configuration pane.)
To obtain the secret token we can execute
$ kubectl describe secrets/gitlab-secret -n gitlab-managed-apps | grep token:
in rancher web portal cluster k8s shell.
Finally, copy and paste it into your Gitlab token field.
e. For Project namespace:
Last, for the namespace, I follow Rancher guide which is gitlab-managed-appsSave Changes!
f. Install the required containers.
Install Helm Tiller ( Helm streamlines installing and managing Kubernetes applications. Tiller runs inside of your Kubernetes Cluster, and manages releases of your charts.)
Install GitLab Runner ( GitLab Runner connects to this project's repository and executes CI/CD jobs, pushing results back and deploying, applications to production. )
If you fail while you are install GitLab Runner, then try to re-install runner got values-content-configuration-runner exist error. you can run in rancher web console k8s shell.
> kubectl --namespace gitlab-managed-apps get configmap
NAME DATA AGE
ingress-nginx-ingress-controller 1 49m
ingress.v1 1 49m
prometheus-prometheus-server 3 48m
prometheus.v1 1 48m
values-content-configuration-helm 3 1h
values-content-configuration-ingress 4 49m
values-content-configuration-prometheus 4 49m
values-content-configuration-runner 4 1h
> kubectl --namespace gitlab-managed-apps delete configmap values-content-configuration-runner
configmap "values-content-configuration-runner" deleted
> kubectl --namespace gitlab-managed-apps get configmap
NAME DATA AGE
ingress-nginx-ingress-controller 1 50m
ingress.v1 1 50m
prometheus-prometheus-server 3 50m
prometheus.v1 1 50m
values-content-configuration-helm 3 1h
values-content-configuration-ingress 4 51m
values-content-configuration-prometheus 4 51m
g. Final Result:
Last if you go to from menu Setting --> CICD --> Runner, you should be able to see your k8s runner there.
Reference
