#metallb — Public Fediverse posts

# Bare-metal Kubernetes на 5 VM: Calico IPIP + MetalLB + GitOps — честный опыт с граблями

Поднял Kubernetes кластер на 5 VM с нуля на VMware: Calico IPIP, MetalLB, GitOps через ArgoCD, PostgreSQL HA. Три неочевидные проблемы которые съели много времени — MTU и TLS, нестабильный BGP на VMware, конфликты git push в GitOps.

https://habr.com/ru/articles/1041356/

#kubernetes #devops #ansible #gitops #argocd #calico #metallb #prometheus #cicd

Балансировка входящего трафика на железе: как надёжно вывести K8s наружу с MetalLB, BGP и L2 (подход от Deckhouse)

На bare metal внешний доступ к Kubernetes часто становится головной болью: NodePort с рандомными нестандартными портами не для продакшена, а облачного балансировщика нет. MetalLB превращает обычные узлы кластера в полноценный балансировщик с автоматическим failover. Разбираем режимы BGP и L2, а ещё показываем фишку Deckhouse Kubernetes Platform, которая спасает активные соединения при падении узла.

https://habr.com/ru/companies/flant/articles/1039392/

#nodeport #kubernetes #bgp #metallb #deckhouse_kubernetes_platform #baremetal #балансировка_нагрузки #l2 #nlb #loadbalancer

I've been working on the infrastructure for Launchpad023. Everything declaratively configured because servers should be cattle not pets.

I've put it up on Codeberg if anyone wants to take a look.
https://codeberg.org/launchpad023/launchpad023-infra

@launchpad023

#kluctl #kustomize #metallb #talosLinux #selfhosting #kubernetes #envoy_gateway #externalsecrets #stalwart #JMAIL #haarlem

I've been working on the infrastructure for Launchpad023. Everything declaratively configured because servers should be cattle not pets.

I've put it up on Codeberg if anyone wants to take a look.
https://codeberg.org/launchpad023/launchpad023-infra

@launchpad023

#kluctl #kustomize #metallb #talosLinux #selfhosting #kubernetes #envoy_gateway #externalsecrets #stalwart #JMAIL #haarlem

So edge nginx > proxy_pass > tunnel > routed into the compute network. proxy_pass and any other method im aware of needs an IP target, which means using #metalLb or something else that implements LoadBalancer and manages IPAM.

MetalLb last time I used it was actually v nice and worked well by giving it an ip range to work with.

Keeping the nginx config in sync with the services can probably be mostly set and forget but that feels like dumpster fire kindling

2/2

#homelab

So edge nginx > proxy_pass > tunnel > routed into the compute network. proxy_pass and any other method im aware of needs an IP target, which means using #metalLb or something else that implements LoadBalancer and manages IPAM.

MetalLb last time I used it was actually v nice and worked well by giving it an ip range to work with.

Keeping the nginx config in sync with the services can probably be mostly set and forget but that feels like dumpster fire kindling

2/2

#homelab

Катастрофоустойчивый Kubernetes: как использовать балансировщик MetalLB с BGP-anycast

Клиенты часто спрашивают, как построить геораспределенный и катастрофоустойчивый

https://habr.com/ru/companies/selectel/articles/992092/

#k8s #metallb #anycast #frrouting #selectel

Had an incredible session with #ClaudeCode yesterday debugging why my #homelab #k8s ingress was having trouble after every restart.
It went through my #argo/#helm config, used cli tools like arp etc, ran #netshoot on specific nodes etc to finally diagnose it as an #metallb stale arp cache problem.

Just watching it go through the diagnostics was sometimes so educative!
Using claude code (or any llm client agent for that matter) for infra troubleshooting is underrated!

Updated #Orked, my collection of scripts to help set up a production-ready #RKE2 #Kubernetes cluster in your #homelab. This update brings general improvements to the scripts, improved documentation, #HAProxy load balancer support for load balancing multiple Master nodes, and upgraded all components including RKE2, #Longhorn, #Nginx Ingress, #Cert-manager, #MetalLB, #Rancher, etc. to their latest versions.

I still hope someday to support more Kubernetes distributions like #k3s, but haven't gotten around to it. I've also been planning to support more #Linux distros as the base too, instead of only #RockyLinux/#RHEL, but that'll have to wait as well for now. Regardless, I am quite happy with how mature and stable these scripts have turned out to be. If you'd like to set up a cluster of your own, maybe check it out!

🔗 https://github.com/irfanhakim-as/orked

🔗 https://github.com/irfanhakim-as/orked/pull/41

Updated #Orked, my collection of scripts to help set up a production-ready #RKE2 #Kubernetes cluster in your #homelab. This update brings general improvements to the scripts, improved documentation, #HAProxy load balancer support for load balancing multiple Master nodes, and upgraded all components including RKE2, #Longhorn, #Nginx Ingress, #Cert-manager, #MetalLB, #Rancher, etc. to their latest versions.

I still hope someday to support more Kubernetes distributions like #k3s, but haven't gotten around to it. I've also been planning to support more #Linux distros as the base too, instead of only #RockyLinux/#RHEL, but that'll have to wait as well for now. Regardless, I am quite happy with how mature and stable these scripts have turned out to be. If you'd like to set up a cluster of your own, maybe check it out!

🔗 https://github.com/irfanhakim-as/orked

🔗 https://github.com/irfanhakim-as/orked/pull/41

TIL #Mikrotik won't ECMP iBGP learned routes. So using layer-3 BGP mode on #MetalLB has no advantage over layer-2 ARP-based mode. #homelab

TIL #Mikrotik won't ECMP iBGP learned routes. So using layer-3 BGP mode on #MetalLB has no advantage over layer-2 ARP-based mode. #homelab

Kubernetes: шестимесячный марафон по прокачке dBrain.cloud

За последние полгода наши инженеры проделали значительную работу по развитию и оптимизации платформы dBrain.cloud , ключевым инструментом которой является Kubernetes. В этой статье мы подробно расскажем об изменениях, произошедших в инфраструктуре, о внедрении новых функций и фишках, которые, мы надеемся, будут интересны DevOps-сообществу.

https://habr.com/ru/companies/dbraincloud/articles/931798/

#gpu #slicing #s3 #cilium #metallb #ipam #cgroup_v2 #kubernetes #dbrain #devops

#TIL that #metallb service announcement via #BGP4 not only is not recommend when using #calico in bgp mesh mode (which is default) but just plain does not work. If I had read the docs properly I would have known that. So for anyone building their own #kubernetes cluster: you HAVE TO configure callico to handle the routing, otherwise the routes to your nodes will show up in your external router, but your cluster will not receive any routes from the nodes to rest of your network. #homelab #k8s

Well that was an interesting one to debug... My blocky DNS service was down after a cluster restart

A given #metallb speaker won’t advertise the service if:
- the service has externalTrafficPolicy=local and there are no running endpoints on the speaker’s node

To use externalTrafficPolicy=local, the tolerations on metallb pods must match the tolerations on the destination pods

For once it wasn't DNS!

https://metallb.universe.tf/troubleshooting/

#homelab

Well that was an interesting one to debug... My blocky DNS service was down after a cluster restart

A given #metallb speaker won’t advertise the service if:
- the service has externalTrafficPolicy=local and there are no running endpoints on the speaker’s node

To use externalTrafficPolicy=local, the tolerations on metallb pods must match the tolerations on the destination pods

For once it wasn't DNS!

https://metallb.universe.tf/troubleshooting/

#homelab

@arichtman yeah using overlay at scale does work it just requires a lot of planning and design. Ideally with some network engineers. My last experience was contrail, years ago, and it was not great.

I’ve used flannel and cilium for one-off personal things without issue. In fact #metallb worked really really well for me with #k3s in the past as well.

#Talos #kubernetes single node cluster up and running with #Calico, #MetalLB, #Traefik and a test #whoami deployment

In no way scientific but it feels much more responsive that #microk8s

Next step to rebuild a clean node and then migrate some services

#Talos #kubernetes single node cluster up and running with #Calico, #MetalLB, #Traefik and a test #whoami deployment

In no way scientific but it feels much more responsive that #microk8s

Next step to rebuild a clean node and then migrate some services

wawaweewa #cilium #ingress adventure time!

https://docs.cilium.io/en/stable/network/servicemesh/ingress/
https://docs.cilium.io/en/stable/network/lb-ipam/

look ma! no #MetalLB !

wawaweewa #cilium #ingress adventure time!

https://docs.cilium.io/en/stable/network/servicemesh/ingress/
https://docs.cilium.io/en/stable/network/lb-ipam/

look ma! no #MetalLB !

Note to self: When converting a Kubernetes cluster with Cilium as CNI to replace MetalLB with Cilium's new L2 announcements, you need to tweak some settings in your Cilium installation. Especially enabling Cilium to act as a kube-proxy replacement (if you are not already doing so) and enabling the l2 announcements. Which means kube-proxy needs to be disabled in k3s.

In other words: k3s on my Raspi4 is now providing a loadbalancer to blocky using Cilium's L2 announcements...

#kubernetes #k3s #cilium #homelab #metalLB #loadbalancer #dns #blocky #hellyeah

Note to self: When converting a Kubernetes cluster with Cilium as CNI to replace MetalLB with Cilium's new L2 announcements, you need to tweak some settings in your Cilium installation. Especially enabling Cilium to act as a kube-proxy replacement (if you are not already doing so) and enabling the l2 announcements. Which means kube-proxy needs to be disabled in k3s.

In other words: k3s on my Raspi4 is now providing a loadbalancer to blocky using Cilium's L2 announcements...

#kubernetes #k3s #cilium #homelab #metalLB #loadbalancer #dns #blocky #hellyeah

Those who've been reading my toots, might have picked up on the fact that I'm building a #kubernetes cluster from scratch (yes, I like pain). After figuring out #cri_o #calico #certmanager #metallb #traefik and #cloudnativepg I finally deployed my first actual application: #nextcloud ! Wueeh! Extremely stocked! Now I need to figure out how I rope in my ZFS box for persistence, and then I'm ready for a deployment in testing! #k8s #selfhosting

#Kubernetes at home in old and used #minipc driven by #Talos #Linux and #proxmox. #selfhosted #quorum #Loadbalancer #Metallb #Devops 🖥️12 CPU - 48 GB RAM. Low #Power comsumption.

#Kubernetes at home in old and used #minipc driven by #Talos #Linux and #proxmox. #selfhosted #quorum #Loadbalancer #Metallb #Devops 🖥️12 CPU - 48 GB RAM. Low #Power comsumption.

@vwbusguy I am curious about what you use as #Kubernetes Stack, especially which load balancer you prefer for Pods that need their unique IP address. We played around with #k3s and #MetalLB, but I would like to know if you prefer other options.

@vwbusguy I am curious about what you use as #Kubernetes Stack, especially which load balancer you prefer for Pods that need their unique IP address. We played around with #k3s and #MetalLB, but I would like to know if you prefer other options.

I have this ~weird intermittent issue accessing services on my #k3s box over #tailscale w/ Subnet routing. I'm running #metallb as the Service type: LoadBalancer. But like sometimes things are fine, others not. If I SSH into the server and use SOCKS all works fine. I can access other services via the subnet router fine too.

Mostly rambling for when I figure out wtf is going on and either fix or replace it.

Tbh, metallb might be replaced soon with Cilium's L2 Annoucements

Has anyone used #talos #kubernetes with #cilium as #CNI and successfully gotten the IPAM part of cilium working? This basically should do what #metallb does, announce "additional" IPs via L2 or BGP and provide them as loadbalancer IPs inside the cluster.

I have the machine running, cilium is running, I defined the Cilium IPPool and my #traefik service got an external IP address from the ippool's range. I have ingressroutes ready and DNS set up for the ingressroute hostnames to point to that IP address.

But I cannot reach anything on that IP, curl tells me that it could not connect to the server...

Looks like nothing is listening on that IP.

Has anyone used #talos #kubernetes with #cilium as #CNI and successfully gotten the IPAM part of cilium working? This basically should do what #metallb does, announce "additional" IPs via L2 or BGP and provide them as loadbalancer IPs inside the cluster.

I have the machine running, cilium is running, I defined the Cilium IPPool and my #traefik service got an external IP address from the ippool's range. I have ingressroutes ready and DNS set up for the ingressroute hostnames to point to that IP address.

But I cannot reach anything on that IP, curl tells me that it could not connect to the server...

Looks like nothing is listening on that IP.

The good news is that my core problem seems to be fixed: powering down the k8s nodes and powering them back up will not break MetalLB. All the pods (eventually) become healthy in their own sequence and pass traffic.

#homelab #kubernetes #metallb #networking

The good news is that my core problem seems to be fixed: powering down the k8s nodes and powering them back up will not break MetalLB. All the pods (eventually) become healthy in their own sequence and pass traffic.

#homelab #kubernetes #metallb #networking

Found the easy way to fix this sequencing issue since Kubernetes is an orchestration engine after all. By using a priority class for each piece, I can ask Kubernetes to run the controller first followed by the speaker pods.

That being said, I think the scope for these priorities is on a per-node basis. If other nodes are ready to run the DaemonSet pods, they'll run. I might have accidentally got it working by having the controller start before a speaker node for whichever node is running both of them.

Scheduling info: https://kubernetes.io/docs/tasks/administer-cluster/guaranteed-scheduling-critical-addon-pods/

#homelab #kubernetes #metallb #networking

Well, I'm part of the way there. It seems there's a leftover 'speaker' process as part of the MetalLB service and there's one of these pods that runs on each node. 5 nodes, 5 pods, 1:1 relationship. This is normal.

When I boot up my k8s home lab cluster, all 5 of the k8s nodes have unhealthy MetalLB pods with the same error. I don't leave the ESXi host running all the time because it's a whole HPE DL380 Gen9 server, loud fans and all :)

Using netstat, I found the offending leftover process using my port, killed that process, deleted the pod (forcing k8s to redeploy it) and then everything was healthy again. This process was also called 'speaker' which tells me it is a part of MetalLB itself.

Now what I can't tell is why there seems to be some 'abandoned' process using :7946 when the pod going away means the port usage should also go away. Two processes can't bind to the same port of course, so it has to be something leftover from the last time MetalLB services were running.

I might be able to figure out what's going on via k8s or via Linux's own internal mechanisms. I'm not sure which one is the better approach yet.

The next question I have is this: if I reboot one of the k8s nodes, does it come up unhealthy again? If so, I'm just going to automate this repair with Ansible, at least in the short term.

To really debug this, I would need to track processes as they are launched, ports as they are consumed, line that up with Kubernetes events, and logs from MetalLB containers. This would be a really good use case for a dedicated syslog endpoint that everything sends to.

#metallb #kubernetes #opensource #golang #ansible #linux #troubleshooting #syslog

Well, I'm part of the way there. It seems there's a leftover 'speaker' process as part of the MetalLB service and there's one of these pods that runs on each node. 5 nodes, 5 pods, 1:1 relationship. This is normal.

When I boot up my k8s home lab cluster, all 5 of the k8s nodes have unhealthy MetalLB pods with the same error. I don't leave the ESXi host running all the time because it's a whole HPE DL380 Gen9 server, loud fans and all :)

Using netstat, I found the offending leftover process using my port, killed that process, deleted the pod (forcing k8s to redeploy it) and then everything was healthy again. This process was also called 'speaker' which tells me it is a part of MetalLB itself.

Now what I can't tell is why there seems to be some 'abandoned' process using :7946 when the pod going away means the port usage should also go away. Two processes can't bind to the same port of course, so it has to be something leftover from the last time MetalLB services were running.

I might be able to figure out what's going on via k8s or via Linux's own internal mechanisms. I'm not sure which one is the better approach yet.

The next question I have is this: if I reboot one of the k8s nodes, does it come up unhealthy again? If so, I'm just going to automate this repair with Ansible, at least in the short term.

To really debug this, I would need to track processes as they are launched, ports as they are consumed, line that up with Kubernetes events, and logs from MetalLB containers. This would be a really good use case for a dedicated syslog endpoint that everything sends to.

#metallb #kubernetes #opensource #golang #ansible #linux #troubleshooting #syslog

Contemplating digging in to the metallb source code tonight so I can figure out why:

1) it always works the first time I install it
2) never works after a shutdown/startup of my homelab k8s environment no matter what

There's a pretty obvious error about port 7946 being in use so at least it makes sense why it's not starting up. The weird part I still need to figure out is why.

Sample part of the error:
listen tcp 10.0.1.103:7946: bind: address already in use

#homelab #kubernetes #kubernetesnetworking #metallb

Contemplating digging in to the metallb source code tonight so I can figure out why:

1) it always works the first time I install it
2) never works after a shutdown/startup of my homelab k8s environment no matter what

There's a pretty obvious error about port 7946 being in use so at least it makes sense why it's not starting up. The weird part I still need to figure out is why.

Sample part of the error:
listen tcp 10.0.1.103:7946: bind: address already in use

#homelab #kubernetes #kubernetesnetworking #metallb

I've just merged a huge PR to my #Orked (O-tomated RKE Distribution - GREAT NAME I KNOW) that makes it easier than ever for anyone to set up a production-ready #RKE2 #Kubernetes cluster in their #homelab.

With this collection of scripts, all you need to do is just provision the nodes required, including a login/management node, and run the scripts right from the login node to configure all of the other nodes to make up the cluster. This setup includes:

- Configuring the Login node with any required or essential dependencies (such as #Helm, #Docker, #k9s, #kubens, #kubectx, etc.)

- Setup passwordless #SSH access from the Login node to the rest of the Kubernetes nodes

- Update the hosts file for strictly necessary name resolution on the Login node and between the Kubernetes nodes

- Necessary, best practice configurations for all of the Kubernetes nodes including networking configuration, disabling unnecessary services, disabling swap, loading required modules, etc.

- Installation and configuration of RKE2 on all the Kubernetes nodes and joining them together as a cluster

- Installation and configuration of #Longhorn storage, including formatting/configuring their virtual disks on the Worker nodes

- Deployment and configuration of #MetalLB as the cluster's load-balancer

- Deployment and configuration of #Ingress #NGINX as the ingress controller and reverse proxy for the cluster - this helps manage external access to the services in the cluster

- Setup and configuration of #cert-manager to obtain and renew #LetsEncrypt certs automatically - supports both #DNS and HTTP validation with #Cloudflare

- Installation and configuration of #csi-driver-smb which adds support for integrating your external SMB storage to the Kubernetes cluster

Besides these, there are also some other helper scripts to make certain related tasks easy such as a script to set a unique static IP address and hostname, and another to toggle #SELinux enforcement to on or off - should you need to turn it off (temporarily).

If you already have an existing RKE2 cluster, there's a step-by-step guide on how you could use it to easily configure and join additional nodes to your cluster if you're planning on expanding.

Orked currently expects and supports #RockyLinux 8+ (should also support any other #RHEL distros such as #AlmaLinux), but I am planning to improve the project over time by adding more #Linux distros, #IPv6 support, and possibly even #K3s for a more lightweight #RaspberryPi cluster for example.

I've used this exact setup to deploy and manage vital services to hundreds of unique clients/organisations that I've become obsessed with sharing it to everyone and making it easier to get started. If this is something that interests you, feel free to check it out!

If you're wondering what to deploy on a Kubernetes cluster - feel free to also check out my #mika helm chart repo 🥳

🔗 https://github.com/irfanhakim-as/orked

🔗 https://github.com/irfanhakim-as/charts

I've just merged a huge PR to my #Orked (O-tomated RKE Distribution - GREAT NAME I KNOW) that makes it easier than ever for anyone to set up a production-ready #RKE2 #Kubernetes cluster in their #homelab.

With this collection of scripts, all you need to do is just provision the nodes required, including a login/management node, and run the scripts right from the login node to configure all of the other nodes to make up the cluster. This setup includes:

- Configuring the Login node with any required or essential dependencies (such as #Helm, #Docker, #k9s, #kubens, #kubectx, etc.)

- Setup passwordless #SSH access from the Login node to the rest of the Kubernetes nodes

- Update the hosts file for strictly necessary name resolution on the Login node and between the Kubernetes nodes

- Necessary, best practice configurations for all of the Kubernetes nodes including networking configuration, disabling unnecessary services, disabling swap, loading required modules, etc.

- Installation and configuration of RKE2 on all the Kubernetes nodes and joining them together as a cluster

- Installation and configuration of #Longhorn storage, including formatting/configuring their virtual disks on the Worker nodes

- Deployment and configuration of #MetalLB as the cluster's load-balancer

- Deployment and configuration of #Ingress #NGINX as the ingress controller and reverse proxy for the cluster - this helps manage external access to the services in the cluster

- Setup and configuration of #cert-manager to obtain and renew #LetsEncrypt certs automatically - supports both #DNS and HTTP validation with #Cloudflare

- Installation and configuration of #csi-driver-smb which adds support for integrating your external SMB storage to the Kubernetes cluster

Besides these, there are also some other helper scripts to make certain related tasks easy such as a script to set a unique static IP address and hostname, and another to toggle #SELinux enforcement to on or off - should you need to turn it off (temporarily).

If you already have an existing RKE2 cluster, there's a step-by-step guide on how you could use it to easily configure and join additional nodes to your cluster if you're planning on expanding.

Orked currently expects and supports #RockyLinux 8+ (should also support any other #RHEL distros such as #AlmaLinux), but I am planning to improve the project over time by adding more #Linux distros, #IPv6 support, and possibly even #K3s for a more lightweight #RaspberryPi cluster for example.

I've used this exact setup to deploy and manage vital services to hundreds of unique clients/organisations that I've become obsessed with sharing it to everyone and making it easier to get started. If this is something that interests you, feel free to check it out!

If you're wondering what to deploy on a Kubernetes cluster - feel free to also check out my #mika helm chart repo 🥳

🔗 https://github.com/irfanhakim-as/orked

🔗 https://github.com/irfanhakim-as/charts

Update: I've figured out that I've worse networking knowledge than I thought. For people like me that plans to spin up a secondary #Kubernetes cluster - also with #Ingress support to publish your service online, do the following:

1. Ensure #MetalLB and Ingress #NGINX controller have been set up on the cluster, no (port) customisation required - leave them as is

2. Register a domain name on your DNS server i.e. #Cloudflare pointing towards your public IP (router) - I've a #Helm chart that automates this which works with non-enterprise networks with dynamic public IP

3. Set up port forwarding on your router for the second cluster - HTTP (wan port: 8080, virtual host port: 80, lan host IP: Cluster 2 loadbalancer IP) and HTTPS (wan port: 8443, virtual host port: 443, lan host IP: Cluster 2 loadbalancer IP)

4. Deploy an ingress for said service as per usual

All of these steps are the same for your "primary" Cluster except the different WAN ports since 80 and 443 will have been taken by your primary cluster on your router/network.

NOW, the thing I've missed thruout this goose chase, is... when you visit that domain you've specified in your service's ingress (in your Cluster 2), please add the custom port you've used (i.e. :8443) at the back... 🙃 Otherwise, ofc you'll get 404 NGINX errors since when you don't append the custom port at the back of the address, it will reach the default/standard ports 80 (http) or 443 (https) instead.

Jesus, this could've all been avoided if I hadn't went autopilot mode too much in my networking classes years ago. Happy kubernetes-ing everyone.

Update: I've figured out that I've worse networking knowledge than I thought. For people like me that plans to spin up a secondary #Kubernetes cluster - also with #Ingress support to publish your service online, do the following:

1. Ensure #MetalLB and Ingress #NGINX controller have been set up on the cluster, no (port) customisation required - leave them as is

2. Register a domain name on your DNS server i.e. #Cloudflare pointing towards your public IP (router) - I've a #Helm chart that automates this which works with non-enterprise networks with dynamic public IP

3. Set up port forwarding on your router for the second cluster - HTTP (wan port: 8080, virtual host port: 80, lan host IP: Cluster 2 loadbalancer IP) and HTTPS (wan port: 8443, virtual host port: 443, lan host IP: Cluster 2 loadbalancer IP)

4. Deploy an ingress for said service as per usual

All of these steps are the same for your "primary" Cluster except the different WAN ports since 80 and 443 will have been taken by your primary cluster on your router/network.

NOW, the thing I've missed thruout this goose chase, is... when you visit that domain you've specified in your service's ingress (in your Cluster 2), please add the custom port you've used (i.e. :8443) at the back... 🙃 Otherwise, ofc you'll get 404 NGINX errors since when you don't append the custom port at the back of the address, it will reach the default/standard ports 80 (http) or 443 (https) instead.

Jesus, this could've all been avoided if I hadn't went autopilot mode too much in my networking classes years ago. Happy kubernetes-ing everyone.

I've #MetalLB + #NGINX #Ingress controller setup on my #RKE2 #Kubernetes cluster. How it should/would work is for me to:

1. Register a domain name on my DNS server i.e. #Cloudflare pointing towards my public IP (router)

2. Set up port forwarding on my router for HTTP (wan port: 80, virtual host port: 80, lan host IP: my loadbalancer IP) and HTTPS (wan port: 443, virtual host port: 443, lan host IP: my loadbalancer IP)

3. Deploy an ingress for my service

^ this setup works. but right now, I'm in a situation where i want to avoid using WAN ports; 80 and 443 - and possibly switch them with 8080 and 8443. What should I do to be able to do this, because as it is if I do just that, going to the address/domain specified in my service's ingress would return me a 404 NGINX error.

Why I want to do this is because that 80-443 port forwarding rule I've already used for my Cluster 1's loadbalancer IP. I'm now setting things up for my second cluster, Cluster 2, and I can't port forward the same 80-443 pair to Cluster 2's loadbalancer IP.

Please help ;(

Asking this again to #Kubernetes/#networking experts cos I still am not able to figure this out. I've 1 #RKE2 cluster previously, with #MetalLB and #NGINX Ingress configured. In this "Cluster 1" setup, I have 2 port forwarding rules on my router that looks something like this:

name: cluster1-http
wan start/end port: 80
lan host address: 192.168.0.88
virtual host port: 80

name: cluster1-https
wan start/end port: 443
lan host address: 192.168.0.88
virtual host port: 443

Hmm also the networking noob that I am just learned that I couldn't port forward the same port twice, to multiple local IP addresses on a network haha - should be obvious I guess.

I've successfully deployed a second #Kubernetes cluster to supplement the first, and the part where I was deploying the #MetalLB load balancer, I thought I'd need to configure it with a different IP range than the first cluster but then realised how that probably wouldn't work since I couldn't port forward it...

I tried with the same IP range but I'm pretty sure I got an error doing that when deploying the IPAddressPool/L2Advertisement. I guess these are my 2 hurdles for now trying to maintain and use 2 clusters at the same time.

Got the home lab Kubernetes environment working, end to end!

Yes, this is a 404, but the web server is reachable from outside the cluster so this means I can setup DNS forwarding and tell resources where to go. Yay!

Of course, I will have more details in my blog post soon.

#homelab #kubernetes #dns #nginx #metallb

Guess who did an emergency secondary #DNS server rebuilding today after something between #flannel, #unifi, and #metallb took out their primairy #DNS cluster. Yup this guy 🤬

Spent 20 minutes hunting down a Kubernetes issue. #MetalLB wasn't advertising any L2 services, causing my Unifi devices to disconnect from the panel. Turns out its because of a node label added by #kubeadm as by default, which now MetalLB takes note of: https://github.com/metallb/metallb/issues/2274

For people who run master only setups, you'll need to make note of this. I should really move away from MetalLB now that I have a better understanding of HAProxy, I really don't need it.

For everyone using #MetalLB: Version 14 now supports `node.kubernetes.io/exclude-from-external-load-balancers`.

This is announcement is certainly unrelated to any recent outages that my infrastructure just experienced. 👀

#kubernetes

#metallb #rke2 #homelab #kubernetes #k8s fixed! Misunderstanding on my part in there metallb docs around the kubelet config options.