Kubernetes Pitfalls

Ingress does not require an LB

For a while I assumed that ingress required some Load Balancing service to assign it an external IP. This is how it's done in most managed cloud k8s services.

But without a Load Balancer an ingress is still valuable because it's literally the only ingress to your service mesh. So you don't have to expose 50 different NodePort services to keep track of.

So even in my humble homelab I can make good use of an ingress controller.

Troubleshooting cert-manager

Not enough IPs

Following the official docs to setup kubeadm you end up using the option --cluster-cidr=10.0.12.0/24 which is enough for getting started but eventually you will run out of IPs to allocate and you might see an event say Node stage-app04 status is now: CIDRNotAvailable.

First of all you need to resolve this in your kubeadm command if you plan on setting up any new control nodes, but the quick workaround is to edit /etc/kubernetes/manifests/kube-controller-manager.yaml and change the cluster-cidr line like this;

spec:
  containers:
  - command:
    - kube-controller-manager
    ...
    - --cluster-cidr=10.0.12.0/22
    ...

This will use four subnets, 10.0.12.0/24, 10.0.13.0/24, 10.0.14.0/24 and 10.0.15.0/24. Then just restart kubelet and wait a while.

sudo systemctl restart kubelet

You can also verify the podCIDR used with this command;

kubectl get nodes -o jsonpath='{.items[*].spec.podCIDR}'

Kubernetes Pitfalls

Ingress does not require an LB

Troubleshooting cert-manager

Not enough IPs

See also

Kustomize edit set image