#metallb — Public Fediverse posts

# Bare-metal Kubernetes на 5 VM: Calico IPIP + MetalLB + GitOps — честный опыт с граблями

Поднял Kubernetes кластер на 5 VM с нуля на VMware: Calico IPIP, MetalLB, GitOps через ArgoCD, PostgreSQL HA. Три неочевидные проблемы которые съели много времени — MTU и TLS, нестабильный BGP на VMware, конфликты git push в GitOps.

https://habr.com/ru/articles/1041356/

#kubernetes #devops #ansible #gitops #argocd #calico #metallb #prometheus #cicd

# Bare-metal Kubernetes на 5 VM: Calico IPIP + MetalLB + GitOps — честный опыт с граблями

Поднял Kubernetes кластер на 5 VM с нуля на VMware: Calico IPIP, MetalLB, GitOps через ArgoCD, PostgreSQL HA. Три неочевидные проблемы которые съели много времени — MTU и TLS, нестабильный BGP на VMware, конфликты git push в GitOps.

https://habr.com/ru/articles/1041356/

#kubernetes #devops #ansible #gitops #argocd #calico #metallb #prometheus #cicd

# Bare-metal Kubernetes на 5 VM: Calico IPIP + MetalLB + GitOps — честный опыт с граблями

Поднял Kubernetes кластер на 5 VM с нуля на VMware: Calico IPIP, MetalLB, GitOps через ArgoCD, PostgreSQL HA. Три неочевидные проблемы которые съели много времени — MTU и TLS, нестабильный BGP на VMware, конфликты git push в GitOps.

https://habr.com/ru/articles/1041356/

#kubernetes #devops #ansible #gitops #argocd #calico #metallb #prometheus #cicd

Балансировка входящего трафика на железе: как надёжно вывести K8s наружу с MetalLB, BGP и L2 (подход от Deckhouse)

На bare metal внешний доступ к Kubernetes часто становится головной болью: NodePort с рандомными нестандартными портами не для продакшена, а облачного балансировщика нет. MetalLB превращает обычные узлы кластера в полноценный балансировщик с автоматическим failover. Разбираем режимы BGP и L2, а ещё показываем фишку Deckhouse Kubernetes Platform, которая спасает активные соединения при падении узла.

https://habr.com/ru/companies/flant/articles/1039392/

#nodeport #kubernetes #bgp #metallb #deckhouse_kubernetes_platform #baremetal #балансировка_нагрузки #l2 #nlb #loadbalancer

Балансировка входящего трафика на железе: как надёжно вывести K8s наружу с MetalLB, BGP и L2 (подход от Deckhouse)

На bare metal внешний доступ к Kubernetes часто становится головной болью: NodePort с рандомными нестандартными портами не для продакшена, а облачного балансировщика нет. MetalLB превращает обычные узлы кластера в полноценный балансировщик с автоматическим failover. Разбираем режимы BGP и L2, а ещё показываем фишку Deckhouse Kubernetes Platform, которая спасает активные соединения при падении узла.

https://habr.com/ru/companies/flant/articles/1039392/

#nodeport #kubernetes #bgp #metallb #deckhouse_kubernetes_platform #baremetal #балансировка_нагрузки #l2 #nlb #loadbalancer

Балансировка входящего трафика на железе: как надёжно вывести K8s наружу с MetalLB, BGP и L2 (подход от Deckhouse)

На bare metal внешний доступ к Kubernetes часто становится головной болью: NodePort с рандомными нестандартными портами не для продакшена, а облачного балансировщика нет. MetalLB превращает обычные узлы кластера в полноценный балансировщик с автоматическим failover. Разбираем режимы BGP и L2, а ещё показываем фишку Deckhouse Kubernetes Platform, которая спасает активные соединения при падении узла.

https://habr.com/ru/companies/flant/articles/1039392/

#nodeport #kubernetes #bgp #metallb #deckhouse_kubernetes_platform #baremetal #балансировка_нагрузки #l2 #nlb #loadbalancer

I've been working on the infrastructure for Launchpad023. Everything declaratively configured because servers should be cattle not pets.

I've put it up on Codeberg if anyone wants to take a look.
https://codeberg.org/launchpad023/launchpad023-infra

@launchpad023

#kluctl #kustomize #metallb #talosLinux #selfhosting #kubernetes #envoy_gateway #externalsecrets #stalwart #JMAIL #haarlem

I've been working on the infrastructure for Launchpad023. Everything declaratively configured because servers should be cattle not pets.

I've put it up on Codeberg if anyone wants to take a look.
https://codeberg.org/launchpad023/launchpad023-infra

@launchpad023

#kluctl #kustomize #metallb #talosLinux #selfhosting #kubernetes #envoy_gateway #externalsecrets #stalwart #JMAIL #haarlem

I've been working on the infrastructure for Launchpad023. Everything declaratively configured because servers should be cattle not pets.

I've put it up on Codeberg if anyone wants to take a look.
https://codeberg.org/launchpad023/launchpad023-infra

@launchpad023

#kluctl #kustomize #metallb #talosLinux #selfhosting #kubernetes #envoy_gateway #externalsecrets #stalwart #JMAIL #haarlem

I've been working on the infrastructure for Launchpad023. Everything declaratively configured because servers should be cattle not pets.

I've put it up on Codeberg if anyone wants to take a look.
https://codeberg.org/launchpad023/launchpad023-infra

@launchpad023

#kluctl #kustomize #metallb #talosLinux #selfhosting #kubernetes #envoy_gateway #externalsecrets #stalwart #JMAIL #haarlem

I've been working on the infrastructure for Launchpad023. Everything declaratively configured because servers should be cattle not pets.

I've put it up on Codeberg if anyone wants to take a look.
https://codeberg.org/launchpad023/launchpad023-infra

@launchpad023

#kluctl #kustomize #metallb #talosLinux #selfhosting #kubernetes #envoy_gateway #externalsecrets #stalwart #JMAIL #haarlem

So edge nginx > proxy_pass > tunnel > routed into the compute network. proxy_pass and any other method im aware of needs an IP target, which means using #metalLb or something else that implements LoadBalancer and manages IPAM.

MetalLb last time I used it was actually v nice and worked well by giving it an ip range to work with.

Keeping the nginx config in sync with the services can probably be mostly set and forget but that feels like dumpster fire kindling

2/2

#homelab

So edge nginx > proxy_pass > tunnel > routed into the compute network. proxy_pass and any other method im aware of needs an IP target, which means using #metalLb or something else that implements LoadBalancer and manages IPAM.

MetalLb last time I used it was actually v nice and worked well by giving it an ip range to work with.

Keeping the nginx config in sync with the services can probably be mostly set and forget but that feels like dumpster fire kindling

2/2

#homelab

So edge nginx > proxy_pass > tunnel > routed into the compute network. proxy_pass and any other method im aware of needs an IP target, which means using #metalLb or something else that implements LoadBalancer and manages IPAM.

MetalLb last time I used it was actually v nice and worked well by giving it an ip range to work with.

Keeping the nginx config in sync with the services can probably be mostly set and forget but that feels like dumpster fire kindling

2/2

#homelab

So edge nginx > proxy_pass > tunnel > routed into the compute network. proxy_pass and any other method im aware of needs an IP target, which means using #metalLb or something else that implements LoadBalancer and manages IPAM.

MetalLb last time I used it was actually v nice and worked well by giving it an ip range to work with.

Keeping the nginx config in sync with the services can probably be mostly set and forget but that feels like dumpster fire kindling

2/2

#homelab

So edge nginx > proxy_pass > tunnel > routed into the compute network. proxy_pass and any other method im aware of needs an IP target, which means using #metalLb or something else that implements LoadBalancer and manages IPAM.

MetalLb last time I used it was actually v nice and worked well by giving it an ip range to work with.

Keeping the nginx config in sync with the services can probably be mostly set and forget but that feels like dumpster fire kindling

2/2

#homelab

Катастрофоустойчивый Kubernetes: как использовать балансировщик MetalLB с BGP-anycast

Клиенты часто спрашивают, как построить геораспределенный и катастрофоустойчивый

https://habr.com/ru/companies/selectel/articles/992092/

#k8s #metallb #anycast #frrouting #selectel

Катастрофоустойчивый Kubernetes: как использовать балансировщик MetalLB с BGP-anycast

Клиенты часто спрашивают, как построить геораспределенный и катастрофоустойчивый

https://habr.com/ru/companies/selectel/articles/992092/

#k8s #metallb #anycast #frrouting #selectel

Катастрофоустойчивый Kubernetes: как использовать балансировщик MetalLB с BGP-anycast

Клиенты часто спрашивают, как построить геораспределенный и катастрофоустойчивый

https://habr.com/ru/companies/selectel/articles/992092/

#k8s #metallb #anycast #frrouting #selectel

Had an incredible session with #ClaudeCode yesterday debugging why my #homelab #k8s ingress was having trouble after every restart.
It went through my #argo/#helm config, used cli tools like arp etc, ran #netshoot on specific nodes etc to finally diagnose it as an #metallb stale arp cache problem.

Just watching it go through the diagnostics was sometimes so educative!
Using claude code (or any llm client agent for that matter) for infra troubleshooting is underrated!

Had an incredible session with #ClaudeCode yesterday debugging why my #homelab #k8s ingress was having trouble after every restart.
It went through my #argo/#helm config, used cli tools like arp etc, ran #netshoot on specific nodes etc to finally diagnose it as an #metallb stale arp cache problem.

Just watching it go through the diagnostics was sometimes so educative!
Using claude code (or any llm client agent for that matter) for infra troubleshooting is underrated!

Had an incredible session with #ClaudeCode yesterday debugging why my #homelab #k8s ingress was having trouble after every restart.
It went through my #argo/#helm config, used cli tools like arp etc, ran #netshoot on specific nodes etc to finally diagnose it as an #metallb stale arp cache problem.

Just watching it go through the diagnostics was sometimes so educative!
Using claude code (or any llm client agent for that matter) for infra troubleshooting is underrated!

Had an incredible session with #ClaudeCode yesterday debugging why my #homelab #k8s ingress was having trouble after every restart.
It went through my #argo/#helm config, used cli tools like arp etc, ran #netshoot on specific nodes etc to finally diagnose it as an #metallb stale arp cache problem.

Just watching it go through the diagnostics was sometimes so educative!
Using claude code (or any llm client agent for that matter) for infra troubleshooting is underrated!

Updated #Orked, my collection of scripts to help set up a production-ready #RKE2 #Kubernetes cluster in your #homelab. This update brings general improvements to the scripts, improved documentation, #HAProxy load balancer support for load balancing multiple Master nodes, and upgraded all components including RKE2, #Longhorn, #Nginx Ingress, #Cert-manager, #MetalLB, #Rancher, etc. to their latest versions.

I still hope someday to support more Kubernetes distributions like #k3s, but haven't gotten around to it. I've also been planning to support more #Linux distros as the base too, instead of only #RockyLinux/#RHEL, but that'll have to wait as well for now. Regardless, I am quite happy with how mature and stable these scripts have turned out to be. If you'd like to set up a cluster of your own, maybe check it out!

🔗 https://github.com/irfanhakim-as/orked

🔗 https://github.com/irfanhakim-as/orked/pull/41

Updated #Orked, my collection of scripts to help set up a production-ready #RKE2 #Kubernetes cluster in your #homelab. This update brings general improvements to the scripts, improved documentation, #HAProxy load balancer support for load balancing multiple Master nodes, and upgraded all components including RKE2, #Longhorn, #Nginx Ingress, #Cert-manager, #MetalLB, #Rancher, etc. to their latest versions.

I still hope someday to support more Kubernetes distributions like #k3s, but haven't gotten around to it. I've also been planning to support more #Linux distros as the base too, instead of only #RockyLinux/#RHEL, but that'll have to wait as well for now. Regardless, I am quite happy with how mature and stable these scripts have turned out to be. If you'd like to set up a cluster of your own, maybe check it out!

🔗 https://github.com/irfanhakim-as/orked

🔗 https://github.com/irfanhakim-as/orked/pull/41

Updated #Orked, my collection of scripts to help set up a production-ready #RKE2 #Kubernetes cluster in your #homelab. This update brings general improvements to the scripts, improved documentation, #HAProxy load balancer support for load balancing multiple Master nodes, and upgraded all components including RKE2, #Longhorn, #Nginx Ingress, #Cert-manager, #MetalLB, #Rancher, etc. to their latest versions.

I still hope someday to support more Kubernetes distributions like #k3s, but haven't gotten around to it. I've also been planning to support more #Linux distros as the base too, instead of only #RockyLinux/#RHEL, but that'll have to wait as well for now. Regardless, I am quite happy with how mature and stable these scripts have turned out to be. If you'd like to set up a cluster of your own, maybe check it out!

🔗 https://github.com/irfanhakim-as/orked

🔗 https://github.com/irfanhakim-as/orked/pull/41

Updated #Orked, my collection of scripts to help set up a production-ready #RKE2 #Kubernetes cluster in your #homelab. This update brings general improvements to the scripts, improved documentation, #HAProxy load balancer support for load balancing multiple Master nodes, and upgraded all components including RKE2, #Longhorn, #Nginx Ingress, #Cert-manager, #MetalLB, #Rancher, etc. to their latest versions.

I still hope someday to support more Kubernetes distributions like #k3s, but haven't gotten around to it. I've also been planning to support more #Linux distros as the base too, instead of only #RockyLinux/#RHEL, but that'll have to wait as well for now. Regardless, I am quite happy with how mature and stable these scripts have turned out to be. If you'd like to set up a cluster of your own, maybe check it out!

🔗 https://github.com/irfanhakim-as/orked

🔗 https://github.com/irfanhakim-as/orked/pull/41

Updated #Orked, my collection of scripts to help set up a production-ready #RKE2 #Kubernetes cluster in your #homelab. This update brings general improvements to the scripts, improved documentation, #HAProxy load balancer support for load balancing multiple Master nodes, and upgraded all components including RKE2, #Longhorn, #Nginx Ingress, #Cert-manager, #MetalLB, #Rancher, etc. to their latest versions.

I still hope someday to support more Kubernetes distributions like #k3s, but haven't gotten around to it. I've also been planning to support more #Linux distros as the base too, instead of only #RockyLinux/#RHEL, but that'll have to wait as well for now. Regardless, I am quite happy with how mature and stable these scripts have turned out to be. If you'd like to set up a cluster of your own, maybe check it out!

🔗 https://github.com/irfanhakim-as/orked

🔗 https://github.com/irfanhakim-as/orked/pull/41

TIL #Mikrotik won't ECMP iBGP learned routes. So using layer-3 BGP mode on #MetalLB has no advantage over layer-2 ARP-based mode. #homelab

TIL #Mikrotik won't ECMP iBGP learned routes. So using layer-3 BGP mode on #MetalLB has no advantage over layer-2 ARP-based mode. #homelab

TIL #Mikrotik won't ECMP iBGP learned routes. So using layer-3 BGP mode on #MetalLB has no advantage over layer-2 ARP-based mode. #homelab

TIL #Mikrotik won't ECMP iBGP learned routes. So using layer-3 BGP mode on #MetalLB has no advantage over layer-2 ARP-based mode. #homelab

TIL #Mikrotik won't ECMP iBGP learned routes. So using layer-3 BGP mode on #MetalLB has no advantage over layer-2 ARP-based mode. #homelab

Kubernetes: шестимесячный марафон по прокачке dBrain.cloud

За последние полгода наши инженеры проделали значительную работу по развитию и оптимизации платформы dBrain.cloud , ключевым инструментом которой является Kubernetes. В этой статье мы подробно расскажем об изменениях, произошедших в инфраструктуре, о внедрении новых функций и фишках, которые, мы надеемся, будут интересны DevOps-сообществу.

https://habr.com/ru/companies/dbraincloud/articles/931798/

#gpu #slicing #s3 #cilium #metallb #ipam #cgroup_v2 #kubernetes #dbrain #devops

Kubernetes: шестимесячный марафон по прокачке dBrain.cloud

За последние полгода наши инженеры проделали значительную работу по развитию и оптимизации платформы dBrain.cloud , ключевым инструментом которой является Kubernetes. В этой статье мы подробно расскажем об изменениях, произошедших в инфраструктуре, о внедрении новых функций и фишках, которые, мы надеемся, будут интересны DevOps-сообществу.

https://habr.com/ru/companies/dbraincloud/articles/931798/

#gpu #slicing #s3 #cilium #metallb #ipam #cgroup_v2 #kubernetes #dbrain #devops

Kubernetes: шестимесячный марафон по прокачке dBrain.cloud

За последние полгода наши инженеры проделали значительную работу по развитию и оптимизации платформы dBrain.cloud , ключевым инструментом которой является Kubernetes. В этой статье мы подробно расскажем об изменениях, произошедших в инфраструктуре, о внедрении новых функций и фишках, которые, мы надеемся, будут интересны DevOps-сообществу.

https://habr.com/ru/companies/dbraincloud/articles/931798/

#gpu #slicing #s3 #cilium #metallb #ipam #cgroup_v2 #kubernetes #dbrain #devops

#TIL that #metallb service announcement via #BGP4 not only is not recommend when using #calico in bgp mesh mode (which is default) but just plain does not work. If I had read the docs properly I would have known that. So for anyone building their own #kubernetes cluster: you HAVE TO configure callico to handle the routing, otherwise the routes to your nodes will show up in your external router, but your cluster will not receive any routes from the nodes to rest of your network. #homelab #k8s

#TIL that #metallb service announcement via #BGP4 not only is not recommend when using #calico in bgp mesh mode (which is default) but just plain does not work. If I had read the docs properly I would have known that. So for anyone building their own #kubernetes cluster: you HAVE TO configure callico to handle the routing, otherwise the routes to your nodes will show up in your external router, but your cluster will not receive any routes from the nodes to rest of your network. #homelab #k8s

#TIL that #metallb service announcement via #BGP4 not only is not recommend when using #calico in bgp mesh mode (which is default) but just plain does not work. If I had read the docs properly I would have known that. So for anyone building their own #kubernetes cluster: you HAVE TO configure callico to handle the routing, otherwise the routes to your nodes will show up in your external router, but your cluster will not receive any routes from the nodes to rest of your network. #homelab #k8s

#TIL that #metallb service announcement via #BGP4 not only is not recommend when using #calico in bgp mesh mode (which is default) but just plain does not work. If I had read the docs properly I would have known that. So for anyone building their own #kubernetes cluster: you HAVE TO configure callico to handle the routing, otherwise the routes to your nodes will show up in your external router, but your cluster will not receive any routes from the nodes to rest of your network. #homelab #k8s

Well that was an interesting one to debug... My blocky DNS service was down after a cluster restart

A given #metallb speaker won’t advertise the service if:
- the service has externalTrafficPolicy=local and there are no running endpoints on the speaker’s node

To use externalTrafficPolicy=local, the tolerations on metallb pods must match the tolerations on the destination pods

For once it wasn't DNS!

https://metallb.universe.tf/troubleshooting/

#homelab

Well that was an interesting one to debug... My blocky DNS service was down after a cluster restart

A given #metallb speaker won’t advertise the service if:
- the service has externalTrafficPolicy=local and there are no running endpoints on the speaker’s node

To use externalTrafficPolicy=local, the tolerations on metallb pods must match the tolerations on the destination pods

For once it wasn't DNS!

https://metallb.universe.tf/troubleshooting/

#homelab

Well that was an interesting one to debug... My blocky DNS service was down after a cluster restart

A given #metallb speaker won’t advertise the service if:
- the service has externalTrafficPolicy=local and there are no running endpoints on the speaker’s node

To use externalTrafficPolicy=local, the tolerations on metallb pods must match the tolerations on the destination pods

For once it wasn't DNS!

https://metallb.universe.tf/troubleshooting/

#homelab

Well that was an interesting one to debug... My blocky DNS service was down after a cluster restart

A given #metallb speaker won’t advertise the service if:
- the service has externalTrafficPolicy=local and there are no running endpoints on the speaker’s node

To use externalTrafficPolicy=local, the tolerations on metallb pods must match the tolerations on the destination pods

For once it wasn't DNS!

https://metallb.universe.tf/troubleshooting/

#homelab

Well that was an interesting one to debug... My blocky DNS service was down after a cluster restart

A given #metallb speaker won’t advertise the service if:
- the service has externalTrafficPolicy=local and there are no running endpoints on the speaker’s node

To use externalTrafficPolicy=local, the tolerations on metallb pods must match the tolerations on the destination pods

For once it wasn't DNS!

https://metallb.universe.tf/troubleshooting/

#homelab

@arichtman yeah using overlay at scale does work it just requires a lot of planning and design. Ideally with some network engineers. My last experience was contrail, years ago, and it was not great.

I’ve used flannel and cilium for one-off personal things without issue. In fact #metallb worked really really well for me with #k3s in the past as well.

@arichtman yeah using overlay at scale does work it just requires a lot of planning and design. Ideally with some network engineers. My last experience was contrail, years ago, and it was not great.

I’ve used flannel and cilium for one-off personal things without issue. In fact #metallb worked really really well for me with #k3s in the past as well.

@arichtman yeah using overlay at scale does work it just requires a lot of planning and design. Ideally with some network engineers. My last experience was contrail, years ago, and it was not great.

I’ve used flannel and cilium for one-off personal things without issue. In fact #metallb worked really really well for me with #k3s in the past as well.

@arichtman yeah using overlay at scale does work it just requires a lot of planning and design. Ideally with some network engineers. My last experience was contrail, years ago, and it was not great.

I’ve used flannel and cilium for one-off personal things without issue. In fact #metallb worked really really well for me with #k3s in the past as well.

#Talos #kubernetes single node cluster up and running with #Calico, #MetalLB, #Traefik and a test #whoami deployment

In no way scientific but it feels much more responsive that #microk8s

Next step to rebuild a clean node and then migrate some services