#mstdndk — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #mstdndk, aggregated by home.social.
-
Numerous technical and security improvements on the infrastructure that supports https://mstdn.dk
- DNS simplified extensively by migrating public facing secondary nameservers to #NSD using #CatalogZones from PowerDNS + DNSDist.
- #DNSSEC reenabled
- #ExternalDNS and #CertManager configuration vastly simplified.
- #Ingress controller migrated from #Nginx to #Traefik
Bottom line: https://sikkerpånettet.dk/ now gives the site a 100% #security score. There are still improvements to be made (weirdly enough) - specifically I'm looking into supporting DANE for #TLS certificate signatures in #DNS.
Now that's off the TODO-list :-)
-
Numerous technical and security improvements on the infrastructure that supports https://mstdn.dk
- DNS simplified extensively by migrating public facing secondary nameservers to #NSD using #CatalogZones from PowerDNS + DNSDist.
- #DNSSEC reenabled
- #ExternalDNS and #CertManager configuration vastly simplified.
- #Ingress controller migrated from #Nginx to #Traefik
Bottom line: https://sikkerpånettet.dk/ now gives the site a 100% #security score. There are still improvements to be made (weirdly enough) - specifically I'm looking into supporting DANE for #TLS certificate signatures in #DNS.
Now that's off the TODO-list :-)
-
Numerous technical and security improvements on the infrastructure that supports https://mstdn.dk
- DNS simplified extensively by migrating public facing secondary nameservers to #NSD using #CatalogZones from PowerDNS + DNSDist.
- #DNSSEC reenabled
- #ExternalDNS and #CertManager configuration vastly simplified.
- #Ingress controller migrated from #Nginx to #Traefik
Bottom line: https://sikkerpånettet.dk/ now gives the site a 100% #security score. There are still improvements to be made (weirdly enough) - specifically I'm looking into supporting DANE for #TLS certificate signatures in #DNS.
Now that's off the TODO-list :-)
-
Numerous technical and security improvements on the infrastructure that supports https://mstdn.dk
- DNS simplified extensively by migrating public facing secondary nameservers to #NSD using #CatalogZones from PowerDNS + DNSDist.
- #DNSSEC reenabled
- #ExternalDNS and #CertManager configuration vastly simplified.
- #Ingress controller migrated from #Nginx to #Traefik
Bottom line: https://sikkerpånettet.dk/ now gives the site a 100% #security score. There are still improvements to be made (weirdly enough) - specifically I'm looking into supporting DANE for #TLS certificate signatures in #DNS.
Now that's off the TODO-list :-)
-
Numerous technical and security improvements on the infrastructure that supports https://mstdn.dk
- DNS simplified extensively by migrating public facing secondary nameservers to #NSD using #CatalogZones from PowerDNS + DNSDist.
- #DNSSEC reenabled
- #ExternalDNS and #CertManager configuration vastly simplified.
- #Ingress controller migrated from #Nginx to #Traefik
Bottom line: https://sikkerpånettet.dk/ now gives the site a 100% #security score. There are still improvements to be made (weirdly enough) - specifically I'm looking into supporting DANE for #TLS certificate signatures in #DNS.
Now that's off the TODO-list :-)
-
Some time this weekend, we'll be upgrading the #Mastodon #PostgreSQL cluster on #MSTDNDK. The cluster is being managed by #CloudNativePG and upgrades have been successfully tested on multiple other clusters - even upgrades directly from version 15 to 18.
While this is an incredibly smooth and easy process, major upgrades requires that #CNPG takes down the whole database cluster, meaning https://mstdn.dk will be unavailable for a period of time. Exactly how long is unknown, since we've never actually done this before with clusters of this size.
I'll keep you posted and give you fair warning 🙂
-
I've been a little rough and irresponsible with my #baremetal #Kubernetes cluster, especially when it comes to randomly rebooting nodes. Today I fixed that.
I'm running a bunch of somewhat delicate workloads, including database clusters with CSIs like #Longhorn and #OpenEBS. Checking if everything is in working order has been demanding task and often something I've skipped before rebooting or upgrading nodes - occasionally with horrific results.
Last night I finally took the time and wrote a pretty thorough script that checks that everything is working and healthy, before politely cordoning off a node, draining it and applying upgrades.
I felt so confident today that I tested it by running this new safe upgrade script for all the nodes in the cluster - and it worked! All nodes are now fully upgraded and running kernel 6.12.73 on Debian 13.
This also fixes the outstanding issue caused by #Hetzner no longer supporting obtaining IP addresses through DHCP.
-
I've been a little rough and irresponsible with my #baremetal #Kubernetes cluster, especially when it comes to randomly rebooting nodes. Today I fixed that.
I'm running a bunch of somewhat delicate workloads, including database clusters with CSIs like #Longhorn and #OpenEBS. Checking if everything is in working order has been demanding task and often something I've skipped before rebooting or upgrading nodes - occasionally with horrific results.
Last night I finally took the time and wrote a pretty thorough script that checks that everything is working and healthy, before politely cordoning off a node, draining it and applying upgrades.
I felt so confident today that I tested it by running this new safe upgrade script for all the nodes in the cluster - and it worked! All nodes are now fully upgraded and running kernel 6.12.73 on Debian 13.
This also fixes the outstanding issue caused by #Hetzner no longer supporting obtaining IP addresses through DHCP.
-
I've been a little rough and irresponsible with my #baremetal #Kubernetes cluster, especially when it comes to randomly rebooting nodes. Today I fixed that.
I'm running a bunch of somewhat delicate workloads, including database clusters with CSIs like #Longhorn and #OpenEBS. Checking if everything is in working order has been demanding task and often something I've skipped before rebooting or upgrading nodes - occasionally with horrific results.
Last night I finally took the time and wrote a pretty thorough script that checks that everything is working and healthy, before politely cordoning off a node, draining it and applying upgrades.
I felt so confident today that I tested it by running this new safe upgrade script for all the nodes in the cluster - and it worked! All nodes are now fully upgraded and running kernel 6.12.73 on Debian 13.
This also fixes the outstanding issue caused by #Hetzner no longer supporting obtaining IP addresses through DHCP.
-
I've been a little rough and irresponsible with my #baremetal #Kubernetes cluster, especially when it comes to randomly rebooting nodes. Today I fixed that.
I'm running a bunch of somewhat delicate workloads, including database clusters with CSIs like #Longhorn and #OpenEBS. Checking if everything is in working order has been demanding task and often something I've skipped before rebooting or upgrading nodes - occasionally with horrific results.
Last night I finally took the time and wrote a pretty thorough script that checks that everything is working and healthy, before politely cordoning off a node, draining it and applying upgrades.
I felt so confident today that I tested it by running this new safe upgrade script for all the nodes in the cluster - and it worked! All nodes are now fully upgraded and running kernel 6.12.73 on Debian 13.
This also fixes the outstanding issue caused by #Hetzner no longer supporting obtaining IP addresses through DHCP.
-
I've been a little rough and irresponsible with my #baremetal #Kubernetes cluster, especially when it comes to randomly rebooting nodes. Today I fixed that.
I'm running a bunch of somewhat delicate workloads, including database clusters with CSIs like #Longhorn and #OpenEBS. Checking if everything is in working order has been demanding task and often something I've skipped before rebooting or upgrading nodes - occasionally with horrific results.
Last night I finally took the time and wrote a pretty thorough script that checks that everything is working and healthy, before politely cordoning off a node, draining it and applying upgrades.
I felt so confident today that I tested it by running this new safe upgrade script for all the nodes in the cluster - and it worked! All nodes are now fully upgraded and running kernel 6.12.73 on Debian 13.
This also fixes the outstanding issue caused by #Hetzner no longer supporting obtaining IP addresses through DHCP.
-
One of the #MSTDNDK nodes is currently experiencing issues with an #NVMe disk that started reporting #SMART errors. It's still up, and we're talking to #Hetzner support about scheduling a replacement. Once we've agreed on a plan moving forward, the node will be taken down and hardware will be replaced. Hopefully you won't feel a thing.
-
Hård uge. Arbejder og sover. Heldigvis holder @leeleedee styr på #mstdndk imens jeg jamrer og klager! Tak for det! 😊
.. og med de ord vil jeg se dyner! 🤣
-
The #MSTDNDK Mastodon instance just migrated away from Bitnami's #Redis helm chart and container images to #Valkey.
#Bitnami and Redis both seem to be exiting the open source community, which could prevent us from staying current, meaning always running the latest versions of the software components that make up this instance.
We understand the need to monetize software, but see that as an opportunity to add premium paid features, not take away existing ones from open source. Doing so will make your potential future customers look for other solutions. Looking at you, #MinIO, Redis and Bitnami.
Please report any issues you might experience as a result of the move to Valkey. So far it looks peachy.
-
Velkommen til @anduin som - hvis ikke #nypåmastodon - så i hvertfald ny på #mstdndk 🙂
-
The #Matrix Homeserver loosely associated with #MSTDNDK - https://mtrix.dk/ - is currently undergoing maintenance. We'll probably be dropping #KeyCloak in favor of the new Matrix Authentication Service to simplify the setup. Stay tuned.
-
Hej #mstdndk, jeg er #nypåmastodon. Jeg er tysker og lærer dansk. Jeg håber jeg kan træne at skrive på dansk her, fordi jeg mangler muligheden at gøre det i mit hverdagsliv i Tyskland. 🙂
-
Velkommen til #mstdndk, @uldkaktus !
-
-
Kære medborgere på #mstdndk! Glædelig bagjul og godt nytår! Tag fordi I er her, og tak for jeres tålmodighed! Det er en rejse at drive et setup som dette, og engang imellem koster det lidt nedetid at blive klogere. Med fare for a jinxe det, så har det kørt rigtig godt et stykke tid nu, og vi satser på at det fortsætter sådan! 🙂 Antallet af aktive brugere er lidt dalende, men det er jo som bekendt ikke kvantitet men kvalitet der tæller! Det er en fornøjelse at følge med i det lokale feed, og mit indtryk er at vi har skabt et rimelig udholdeligt lille hjørne af Fødiverset her! Pas godt på jer selv derude, og vi ses i 2026! 🙂🎆🎇
-
-
-
Made a backup account over on dice.camp . Just in case our #mstdndk server is out for a bit and I need to provide situation updates. You can follow that here:
-
Monitoring is slowly getting up and running.. now if only I knew how to read all that! ;-)
-
-
One of the nodes of the Midgaard #Kubernetes cluster running #mstdndk is currently down, and awaiting a disk replacement. Luckily clustering works, and you shouldn't feel a thing ;-)
Resiliency for the win! :-)
-
Tonight is #Halloween, which also means it's the #birthday of #mstdndk! Three years ago exactly, the domain was registered and we deployed #Mastodon with my home grown Helm chart. A couple of horrible crashes and a lot of experience later, we're still alive and kicking! With backups and everything! :mastowink:
Have a great day! 🙂
-
#mstdndk and the underlying Kubernetes cluster died a couple of days ago. Working on recovering.
-
Considering switching the #MinIO backend of #mstdndk to #Garage by #deuxfleurs. 3 replicas on #Kubernetes. Anyone with real life experience and/or tips? :-)
-
I've been doing things I shouldn't with #Kubernetes. We're using a replicated #MinIO cluster as the storage backend on #mstdndk, which requires a boat load of storage, especially if you forget to specify any kind of retention. So far, the quick workaround for a full disk, was just to expand the filesystem. Since we're replicating across nodes, we're using #OpenEBS #LVM for local storage. Poor partitioning means we're running out of storage on the volume group, but even worse - PVCs sizes were increased before checking if we had space for it. Kubernetes is now stuck in a most unfortunate situation - it can't grow the local filesystem, as the volume group is full and you're not allowed to decrease the size request. What then? Cue https://github.com/etcd-io/auger - a tools that allows you to edit #K8s resources directly in #etcd. Obviously you should never do this, but with steady hands and clinical precision, you can get yourself out of a pickle like mine. Size was reverted and PVCs were unstuck.
-
I've been doing things I shouldn't with #Kubernetes. We're using a replicated #MinIO cluster as the storage backend on #mstdndk, which requires a boat load of storage, especially if you forget to specify any kind of retention. So far, the quick workaround for a full disk, was just to expand the filesystem. Since we're replicating across nodes, we're using #OpenEBS #LVM for local storage. Poor partitioning means we're running out of storage on the volume group, but even worse - PVCs sizes were increased before checking if we had space for it. Kubernetes is now stuck in a most unfortunate situation - it can't grow the local filesystem, as the volume group is full and you're not allowed to decrease the size request. What then? Cue https://github.com/etcd-io/auger - a tools that allows you to edit #K8s resources directly in #etcd. Obviously you should never do this, but with steady hands and clinical precision, you can get yourself out of a pickle like mine. Size was reverted and PVCs were unstuck.
-
I've been doing things I shouldn't with #Kubernetes. We're using a replicated #MinIO cluster as the storage backend on #mstdndk, which requires a boat load of storage, especially if you forget to specify any kind of retention. So far, the quick workaround for a full disk, was just to expand the filesystem. Since we're replicating across nodes, we're using #OpenEBS #LVM for local storage. Poor partitioning means we're running out of storage on the volume group, but even worse - PVCs sizes were increased before checking if we had space for it. Kubernetes is now stuck in a most unfortunate situation - it can't grow the local filesystem, as the volume group is full and you're not allowed to decrease the size request. What then? Cue https://github.com/etcd-io/auger - a tools that allows you to edit #K8s resources directly in #etcd. Obviously you should never do this, but with steady hands and clinical precision, you can get yourself out of a pickle like mine. Size was reverted and PVCs were unstuck.
-
I've been doing things I shouldn't with #Kubernetes. We're using a replicated #MinIO cluster as the storage backend on #mstdndk, which requires a boat load of storage, especially if you forget to specify any kind of retention. So far, the quick workaround for a full disk, was just to expand the filesystem. Since we're replicating across nodes, we're using #OpenEBS #LVM for local storage. Poor partitioning means we're running out of storage on the volume group, but even worse - PVCs sizes were increased before checking if we had space for it. Kubernetes is now stuck in a most unfortunate situation - it can't grow the local filesystem, as the volume group is full and you're not allowed to decrease the size request. What then? Cue https://github.com/etcd-io/auger - a tools that allows you to edit #K8s resources directly in #etcd. Obviously you should never do this, but with steady hands and clinical precision, you can get yourself out of a pickle like mine. Size was reverted and PVCs were unstuck.
-
I've been doing things I shouldn't with #Kubernetes. We're using a replicated #MinIO cluster as the storage backend on #mstdndk, which requires a boat load of storage, especially if you forget to specify any kind of retention. So far, the quick workaround for a full disk, was just to expand the filesystem. Since we're replicating across nodes, we're using #OpenEBS #LVM for local storage. Poor partitioning means we're running out of storage on the volume group, but even worse - PVCs sizes were increased before checking if we had space for it. Kubernetes is now stuck in a most unfortunate situation - it can't grow the local filesystem, as the volume group is full and you're not allowed to decrease the size request. What then? Cue https://github.com/etcd-io/auger - a tools that allows you to edit #K8s resources directly in #etcd. Obviously you should never do this, but with steady hands and clinical precision, you can get yourself out of a pickle like mine. Size was reverted and PVCs were unstuck.
-
@Molvorin Hej og velkommen til Mastodon og #mstdndk ! Det lyder som om du har nogle interessante hobbyer, og jeg glæder mig til at følge med på vores servers livefeed.
Den bedste måde at finde ligesindede her er at sørge for at bruge og følge hashtags, der beskriver dine interesser. Det skyldes, at fediverset ikke har nogen algoritme til at promovere dine indlæg, hvis de ikke har hashtags.
Når du har fundet de hashtags, der interesserer dig, vil du sandsynligvis finde en masse dejlige mennesker at følge og interagere med, som poster med disse hashtags. Følg så mange mennesker og hashtags, der interesserer dig, som muligt, og på den måde vil du finde dit fællesskab her.
Hav det sjovt 👍!
(Her er nogle hashtags, der kunne være interessante for dig baseret på dit indlæg: #linux #linuxmint #linux_gaming #EliteDangerous #NoMansSky #StarCitizen )
-
A node crashed today, and unfortunately the remaining three were overburdened with a seemingly spontaneous filesystem check of some rather large volumes, including the ones holding the three database replicas and three media storage replicas for #mstdndk. We're running #Longhorn on #Kubernetes for replicated block storage, but we're doing this on a bit of a budget, which means spinning metal and 1Gbit connections between servers. You can get quite far by very carefully prioritizing the I/O and CPU of certain processes, but once in a rare while it tumbles over. If we had the money, we'd get fit each server with 16TB NVMEs and a 10Gbit backbone, but unfortunately that's not currently the case.
Is there any benefit to running this on Kubernetes? I have no doubt about it, but I'm also convinced that our current problem is replicated block storage and requirements associated with it.
Our current setup is four i7-8700's with 64GB RAM, 1x10TB spinning metal and 2x512GB SSDs.
-
We're experiencing an issue with the media storage backend on #mstdndk today. We'll look into it as soon as possible, which unfortunately isn't for at least a couple of hours, as I have some stuff to take care of in the real world first ;-)
-
Jeg er ny her og nåede aldrig at falde til på #Twitter Med #musk blev det let at slette min Twitter konto. #neilhimself postede på FB at han er flyttet til Mastodon så det giver jeg en chance.
Psykoterapeut studerende ved ID i #aarhus. Bruger meget tid i naturen og poster billeder fra mine morgenture på Instagram og FB.
Nu bliver det spændende at se, hvordan det her kommer til at fungere.
Billede er fra dagens morgentur.
-
Jeg er ny her og nåede aldrig at falde til på #Twitter Med #musk blev det let at slette min Twitter konto. #neilhimself postede på FB at han er flyttet til Mastodon så det giver jeg en chance.
Psykoterapeut studerende ved ID i #aarhus. Bruger meget tid i naturen og poster billeder fra mine morgenture på Instagram og FB.
Nu bliver det spændende at se, hvordan det her kommer til at fungere.
Billede er fra dagens morgentur.
-
Jeg er ny her og nåede aldrig at falde til på #Twitter Med #musk blev det let at slette min Twitter konto. #neilhimself postede på FB at han er flyttet til Mastodon så det giver jeg en chance.
Psykoterapeut studerende ved ID i #aarhus. Bruger meget tid i naturen og poster billeder fra mine morgenture på Instagram og FB.
Nu bliver det spændende at se, hvordan det her kommer til at fungere.
Billede er fra dagens morgentur.
-
Jeg er ny her og nåede aldrig at falde til på #Twitter Med #musk blev det let at slette min Twitter konto. #neilhimself postede på FB at han er flyttet til Mastodon så det giver jeg en chance.
Psykoterapeut studerende ved ID i #aarhus. Bruger meget tid i naturen og poster billeder fra mine morgenture på Instagram og FB.
Nu bliver det spændende at se, hvordan det her kommer til at fungere.
Billede er fra dagens morgentur.