Adding an Ingress to on premises Kubernetes cluster without load balancer

Published: 2 September 2019

The following is part of a series of posts called "Repurposing old equipment by building a Kubernetes cluster".

While old equipment by itself is in general not very useful unless you find a particular use case, by combining a number of old devices you can build a more powerful system that can span perhaps a number of use cases. Kubernetes is a perfect candidate to be able to do this. I had a number of old laptops laying about and decided to test this theory out.

As I’m not running my cluster in the cloud behind a bunch of data center equipment, putting a load balancer in front of the cluster is quite a challenge, especially given that I’m on just a standard home internet connection and I don’t have any equipment able to do load balancing. Realistically, to serve requests in the given context, utilizing NodePorts is the best option and I can add an Ingress controller behind that.

MetalB is an option for doing load balancing in this scenario (on premises) but it requires a router with BGP capabilities, and while I may have one of those types of routers laying around somewhere, I just want to keep things simple at the moment either way.

To get things up and running I apply the ingress-nginx bare-metal configuration found here

# Mandatory Command for all deployments
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created

# Baremetal command
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/service-nodeport.yaml
service/ingress-nginx created

I can then verify the installation by checking out the status of the pods

$ kubectl get pods --all-namespaces -l app.kubernetes.io/name=ingress-nginx --watch
NAMESPACE       NAME                                        READY   STATUS    RESTARTS   AGE
ingress-nginx   nginx-ingress-controller-79f6884cf6-qnc52   1/1     Running   0          53s

Right, so now lets see if I can expose Weavescope that I installed via Helm in a previous post.

$ cat ingress.yml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-service
  annotations:
    kubernetes.io/ingress.class: nginx
spec:
  rules:
    - http:
        paths:
          - path: /
            backend:
              serviceName: weave-scope-test-weave-scope
              servicePort: 80

$ kubectl apply -f ingress.yml
ingress.extensions/ingress-service created

Ingress is set up, and Kubernetes has created the service. Now lets see if its working.

$ kubectl get svc --namespace ingress-nginx
NAME            TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx   NodePort   10.103.255.176   <none>        80:31942/TCP,443:32158/TCP   36m

Looks good. So now I visit https://10.0.0.200:32158 and after getting the self signed SSL error, the Weave Scope dashboard comes up. To test that the node port is working, I also check https://10.0.0.201:32158 through to https://10.0.0.205:32158 (the IP addresses for all nodes in the cluster which I set earlier) and everything is up and running nicely.

At this point, I could port-forward any required ports from my router to any of these nodes and be able to serve any containers globally on the internet. I’m always a bit weary of opening my home LAN to the outside world though, so for the moment, just going to leave it. At least at this point I can add an ingress for any service that I want to use locally without having to rely on kubectl to forward a port on my workstation.