Skip to content
Snippets Groups Projects
Select Git revision
  • master default protected
1 result
Blame Permalink
README.md 5.55 KiB

Kubernetes Cluster Federation (KubeFed)

Documentation

Below some useful links for the main tools of a Kubernetes federation:

Environment configuration

Before starting it is necessary create several Kubernetes clusters to make the federation. One cluster will be the Host (Host Cluster), in which will be install the KubeFed (Control Plane), and the remaining clusters will be the Members (Member Cluster). To create the clusters on Openstack, it is possible to use the follow Juju bundle: https://jaas.ai/u/csd-garr/kubernetes-openstack-integrator/bundle.

Once all the clusters for federation have been created, it is useful to collect all cluster access configurations (.kube/config) in a single config file. In this way you can easily switch from one cluster to another. Below is an example of a compound file config:

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJT...
    server: https://<ip>:443
  name: k8s-cluster-host
- cluster:
    certificate-authority-data: LS0tLS1CRUdJT...
    server: https://<ip>:443
  name: k8s-cluster-member1
- cluster:
    certificate-authority-data: LS0tLS1CRUdJT...
    server: https://<ip>:443
  name: k8s-cluster-member2
contexts:
- context:
    cluster: k8s-cluster-host
    user: admin-host
  name: k8s-context-host
- context:
    cluster: k8s-cluster-member1
    user: admin-member1
  name: k8s-context-member1
- context:
    cluster: k8s-cluster-member2
    user: admin-member2
  name: k8s-context-member2
current-context: k8s-context-host
kind: Config
preferences: {}
users:
- name: admin-host
  user:
    password: 8Rvl5qW...
    username: admin
- name: admin-member1
  user:
    password: 0JefxZq...
    username: admin
- name: admin-member2
  user:
    password: hJLiVLI...
    username: admin

In this way you have access to all the clusters by the context switch, below are the commands for the context switch:

list contexts:

$ kubectl config get-contexts

switch context:

$ kubectl config use-context <context-name>

Installation

KubeFed client (kubefedctl)

To install the KubeFed client run the following commands in your local machine:

$ VERSION=0.2.0-alpha.1
$ OS=linux
$ ARCH=amd64
$ curl -LO https://github.com/kubernetes-sigs/kubefed/releases/download/v${VERSION}/kubefedctl-${VERSION}-${OS}-${ARCH}.tgz
$ tar -zxvf kubefedctl-*.tgz
$ chmod u+x kubefedctl
$ sudo mv kubefedctl /usr/local/bin/ # make sure the location is in the PATH

N.B.: the versions we tested are: 0.1.0-rc6 and 0.2.0-alpha.1, you can try the latest version.

KubeFed server (Control Plane)

To install the Controll Plane, switch to the host context and configuring the RBAC role for Helm (in this case we use Helm v2):

$ cat << EOF | kubectl apply -f -
	apiVersion: v1
	kind: ServiceAccount
	metadata:
	  name: tiller
	  namespace: kube-system
	---
	apiVersion: rbac.authorization.k8s.io/v1
	kind: ClusterRoleBinding
	metadata:
	  name: tiller
	roleRef:
	  apiGroup: rbac.authorization.k8s.io
	  kind: ClusterRole
	  name: cluster-admin
	subjects:
	  - kind: ServiceAccount
	    name: tiller
	    namespace: kube-system
	EOF

helm init command:

$ helm init --service-account tiller

Now it is possible install the Control Plane on the Host Cluster:

$ helm repo add kubefed-charts https://raw.githubusercontent.com/kubernetes-sigs/kubefed/master/charts
$ helm install kubefed-charts/kubefed --name kubefed --version=0.2.0-alpha.1 --namespace kube-federation-system

N.B.: use the same version for client and server.

N.B.: in this way the Control Plane can find Federated resources in each namespace, to limit this action in a specific namespace add the parameter --set global.scope=Namespaced.

Create the federation

To create a Kubernetes Federation it is necessary federate Member Clusters. Use the KubeFed join command:

$ kubefedctl join fed-cluster-member1 --cluster-context k8s-context-member1 --host-cluster-context k8s-context-host --v=2
$ kubefedctl join fed-cluster-member2 --cluster-context k8s-context-member2 --host-cluster-context k8s-context-host --v=2

Remove cluster to the federation, unjoin command:

$ kubefedctl unjoin fed-cluster-member1 --cluster-context k8s-context-member1 --host-cluster-context k8s-context-host --v=2
$ kubefedctl unjoin fed-cluster-member2 --cluster-context k8s-context-member2 --host-cluster-context k8s-context-host --v=2

Command to show federated clusters:

$ kubectl -n kube-federation-system get kubefedclusters

Enable API

To enable a resource to be federated it is necessary to use the enable command of kubefedctl:

$ kubefedctl enable <api-resources-name.api-group> --kubefed-namespace kube-federation-system

To know the list of resourses:

$ kubectl api-resources -o wide

Some examples:

$ kubefedctl enable deployments.apps --kubefed-namespace kube-federation-system
$ kubefedctl enable services --kubefed-namespace kube-federation-system
$ kubefedctl enable ingresses.extensions --kubefed-namespace kube-federation-system

Command to disable federated API:

$ kubefedctl disable services --kubefed-namespace kube-federation-system

N.B.: Do not federate the “kind: pod” resource. In this case, federating a whole namespace will federate pods and deployments at the same time. Deployments will create replicasets which in turn create pods. It will result a duplication of the pods resources.