#postmortem — Public Fediverse posts

El análisis póstumo de la elección de Kamala Harris y su posterior desastre.

https://democrats.org/wp-content/uploads/2026/05/May-20-2026.pdf

#KamalaHarris #Postmortem #Democrats

El análisis póstumo de la elección de Kamala Harris y su posterior desastre.

https://democrats.org/wp-content/uploads/2026/05/May-20-2026.pdf

#KamalaHarris #Postmortem #Democrats

El análisis póstumo de la elección de Kamala Harris y su posterior desastre.

https://democrats.org/wp-content/uploads/2026/05/May-20-2026.pdf

#KamalaHarris #Postmortem #Democrats

El análisis póstumo de la elección de Kamala Harris y su posterior desastre.

https://democrats.org/wp-content/uploads/2026/05/May-20-2026.pdf

#KamalaHarris #Postmortem #Democrats

El análisis póstumo de la elección de Kamala Harris y su posterior desastre.

https://democrats.org/wp-content/uploads/2026/05/May-20-2026.pdf

#KamalaHarris #Postmortem #Democrats

Баги, которые нас воспитали: инженерные истории с Go Loto

Каждый разработчик помнит тот самый момент, когда система, которая только что работала идеально, вдруг начинает вести себя так, будто сошла с ума. Когда дашборд в Grafana показывает что-то страшное, а ты стоишь перед ним с кружкой остывшего кофе и не понимаешь, с чего начать. На нашем мероприятии Avito Go Loto разработчики поделились своим опытом без прикрас. О блоате в полтора терабайта, о девяти инстансах, которые передрались за один звонок, о бэкенд-разработчице, которая в пятницу вечером открыла чужой фронтовый проект, о нагрузочных тестах за несколько месяцев до большой рекламной кампании, и о транзакции, которую забыли закоммитить тоже в пятницу вечером. Спойлер: все выжили. Но стали другими людьми.

https://habr.com/ru/companies/avito/articles/1037846/

#go #bloat #vacuum #микросервисы #распределенные_системы #нагрузочное_тестирование #postmortem

Баги, которые нас воспитали: инженерные истории с Go Loto

Каждый разработчик помнит тот самый момент, когда система, которая только что работала идеально, вдруг начинает вести себя так, будто сошла с ума. Когда дашборд в Grafana показывает что-то страшное, а ты стоишь перед ним с кружкой остывшего кофе и не понимаешь, с чего начать. На нашем мероприятии Avito Go Loto разработчики поделились своим опытом без прикрас. О блоате в полтора терабайта, о девяти инстансах, которые передрались за один звонок, о бэкенд-разработчице, которая в пятницу вечером открыла чужой фронтовый проект, о нагрузочных тестах за несколько месяцев до большой рекламной кампании, и о транзакции, которую забыли закоммитить тоже в пятницу вечером. Спойлер: все выжили. Но стали другими людьми.

https://habr.com/ru/companies/avito/articles/1037846/

#go #bloat #vacuum #микросервисы #распределенные_системы #нагрузочное_тестирование #postmortem

Баги, которые нас воспитали: инженерные истории с Go Loto

Каждый разработчик помнит тот самый момент, когда система, которая только что работала идеально, вдруг начинает вести себя так, будто сошла с ума. Когда дашборд в Grafana показывает что-то страшное, а ты стоишь перед ним с кружкой остывшего кофе и не понимаешь, с чего начать. На нашем мероприятии Avito Go Loto разработчики поделились своим опытом без прикрас. О блоате в полтора терабайта, о девяти инстансах, которые передрались за один звонок, о бэкенд-разработчице, которая в пятницу вечером открыла чужой фронтовый проект, о нагрузочных тестах за несколько месяцев до большой рекламной кампании, и о транзакции, которую забыли закоммитить тоже в пятницу вечером. Спойлер: все выжили. Но стали другими людьми.

https://habr.com/ru/companies/avito/articles/1037846/

#go #bloat #vacuum #микросервисы #распределенные_системы #нагрузочное_тестирование #postmortem

Баги, которые нас воспитали: инженерные истории с Go Loto

Каждый разработчик помнит тот самый момент, когда система, которая только что работала идеально, вдруг начинает вести себя так, будто сошла с ума. Когда дашборд в Grafana показывает что-то страшное, а ты стоишь перед ним с кружкой остывшего кофе и не понимаешь, с чего начать. На нашем мероприятии Avito Go Loto разработчики поделились своим опытом без прикрас. О блоате в полтора терабайта, о девяти инстансах, которые передрались за один звонок, о бэкенд-разработчице, которая в пятницу вечером открыла чужой фронтовый проект, о нагрузочных тестах за несколько месяцев до большой рекламной кампании, и о транзакции, которую забыли закоммитить тоже в пятницу вечером. Спойлер: все выжили. Но стали другими людьми.

https://habr.com/ru/companies/avito/articles/1037846/

#go #bloat #vacuum #микросервисы #распределенные_системы #нагрузочное_тестирование #postmortem

When two Hetzner servers died at the same time

On May 12, 2026, two of my Arch Linux + LUKS servers at Hetzner became unreachable at the same moment. Both had been running for 4+ months without issue. Both had received the same pacman -Syyu the day before, but had stayed on the old kernel until the morning the websites stopped responding. I rebooted — SSH never came back. nmap -Pn -p 22 showed filtered from anywhere. No ping. No banner. The Hetzner Robot panel insisted the hardware was fine.

Several hours went into hypotheses that turned out to be wrong:

• The encryptssh initcpio hook referencing a /usr/lib/initcpio/udev/11-dm-initramfs.rules file that no longer exists. Real bug, no boot impact — the initramfs rebuilds anyway.
• PermitRootLogin no in sshd_config. Real misconfiguration, fixed it, didn’t help. A refusing sshd shows closed, not filtered.
• Predictable interface-naming drift after the systemd 260 upgrade. Patched the .network config to match by MAC. Useful hardening; not the cause.
• Stale GRUB stage1 + core.img in the MBR. Arch never re-runs grub-install after a grub package upgrade. Refreshed it. Still filtered.
• Kernel 7.0.5 regression. Downgraded to 6.18.3, the kernel that had run for 4 months. Still filtered. So the kernel itself wasn’t it either.

The clue was in the persistent journal: a single recorded boot from December 31 to May 12 10:13 UTC, and absolutely nothing after. Every reboot since the upgrade was failing before systemd-journald could flush to disk — so the failure had to be in the initramfs, before the root filesystem was even mounted.

What it almost certainly was

Hetzner Dedicated servers configure the initramfs network with ip=dhcp on the kernel command line. That depends on Hetzner’s DHCP server replying to whatever request format the current kernel sends. Somewhere between kernel 6.18 / iproute2 6.18 and kernel 7.0 / iproute2 7.0, the request format changed enough that Hetzner’s DHCP stopped responding. Effects:

• Old kernel at runtime kept the interface already configured (Phase A — 32 hours of healthy operation after the package upgrade).
• New kernel cold-boots, hits DHCP, never gets an IP, dropbear cannot listen, port 22 stays filtered.

Hetzner’s own documentation has been quietly moving away from ip=dhcp toward static IPv4 in the kernel command line. The fix is exactly that:

GRUB_CMDLINE_LINUX="cryptdevice=/dev/md1:cryptroot ip=A.B.C.D::GATEWAY:255.255.255.255:hostname:eth0:none"

One line in /etc/default/grub, grub-mkconfig, reboot. No more dependency on Hetzner’s DHCP responding to whatever your current kernel sends.

Why it matters for anyone running this stack

If you run Arch on Hetzner Dedicated with full-disk encryption and remote unlock via dropbear, the ip=dhcp shipped by installimage is a latent bug. It can keep working for years and then break overnight, on every machine you have, after a routine pacman -Syyu. The static-IP version is what Hetzner now recommends and removes the entire dependency.

Tooling

While debugging, I turned the whole rescue / chroot / diagnose / fix workflow into a Python CLI (hal) — including hal fix static-ip, which derives the static cmdline directly from your existing systemd-networkd .network file:

→ github.com/kevinveenbirkenbach/hetzner-arch-luks

Single command, idempotent, reversible (the original /etc/default/grub is backed up to .hal-backup). If you’re on this stack, switch to static IP before the next kernel upgrade catches you.

When two Hetzner servers died at the same time

On May 12, 2026, two of my Arch Linux + LUKS servers at Hetzner became unreachable at the same moment. Both had been running for 4+ months without issue. Both had received the same pacman -Syyu the day before, but had stayed on the old kernel until the morning the websites stopped responding. I rebooted — SSH never came back. nmap -Pn -p 22 showed filtered from anywhere. No ping. No banner. The Hetzner Robot panel insisted the hardware was fine.

Several hours went into hypotheses that turned out to be wrong:

• The encryptssh initcpio hook referencing a /usr/lib/initcpio/udev/11-dm-initramfs.rules file that no longer exists. Real bug, no boot impact — the initramfs rebuilds anyway.
• PermitRootLogin no in sshd_config. Real misconfiguration, fixed it, didn’t help. A refusing sshd shows closed, not filtered.
• Predictable interface-naming drift after the systemd 260 upgrade. Patched the .network config to match by MAC. Useful hardening; not the cause.
• Stale GRUB stage1 + core.img in the MBR. Arch never re-runs grub-install after a grub package upgrade. Refreshed it. Still filtered.
• Kernel 7.0.5 regression. Downgraded to 6.18.3, the kernel that had run for 4 months. Still filtered. So the kernel itself wasn’t it either.

The clue was in the persistent journal: a single recorded boot from December 31 to May 12 10:13 UTC, and absolutely nothing after. Every reboot since the upgrade was failing before systemd-journald could flush to disk — so the failure had to be in the initramfs, before the root filesystem was even mounted.

What it almost certainly was

Hetzner Dedicated servers configure the initramfs network with ip=dhcp on the kernel command line. That depends on Hetzner’s DHCP server replying to whatever request format the current kernel sends. Somewhere between kernel 6.18 / iproute2 6.18 and kernel 7.0 / iproute2 7.0, the request format changed enough that Hetzner’s DHCP stopped responding. Effects:

• Old kernel at runtime kept the interface already configured (Phase A — 32 hours of healthy operation after the package upgrade).
• New kernel cold-boots, hits DHCP, never gets an IP, dropbear cannot listen, port 22 stays filtered.

Hetzner’s own documentation has been quietly moving away from ip=dhcp toward static IPv4 in the kernel command line. The fix is exactly that:

GRUB_CMDLINE_LINUX="cryptdevice=/dev/md1:cryptroot ip=A.B.C.D::GATEWAY:255.255.255.255:hostname:eth0:none"

One line in /etc/default/grub, grub-mkconfig, reboot. No more dependency on Hetzner’s DHCP responding to whatever your current kernel sends.

Why it matters for anyone running this stack

If you run Arch on Hetzner Dedicated with full-disk encryption and remote unlock via dropbear, the ip=dhcp shipped by installimage is a latent bug. It can keep working for years and then break overnight, on every machine you have, after a routine pacman -Syyu. The static-IP version is what Hetzner now recommends and removes the entire dependency.

Tooling

While debugging, I turned the whole rescue / chroot / diagnose / fix workflow into a Python CLI (hal) — including hal fix static-ip, which derives the static cmdline directly from your existing systemd-networkd .network file:

→ github.com/kevinveenbirkenbach/hetzner-arch-luks

Single command, idempotent, reversible (the original /etc/default/grub is backed up to .hal-backup). If you’re on this stack, switch to static IP before the next kernel upgrade catches you.

When two Hetzner servers died at the same time

On May 12, 2026, two of my Arch Linux + LUKS servers at Hetzner became unreachable at the same moment. Both had been running for 4+ months without issue. Both had received the same pacman -Syyu the day before, but had stayed on the old kernel until the morning the websites stopped responding. I rebooted — SSH never came back. nmap -Pn -p 22 showed filtered from anywhere. No ping. No banner. The Hetzner Robot panel insisted the hardware was fine.

Several hours went into hypotheses that turned out to be wrong:

• The encryptssh initcpio hook referencing a /usr/lib/initcpio/udev/11-dm-initramfs.rules file that no longer exists. Real bug, no boot impact — the initramfs rebuilds anyway.
• PermitRootLogin no in sshd_config. Real misconfiguration, fixed it, didn’t help. A refusing sshd shows closed, not filtered.
• Predictable interface-naming drift after the systemd 260 upgrade. Patched the .network config to match by MAC. Useful hardening; not the cause.
• Stale GRUB stage1 + core.img in the MBR. Arch never re-runs grub-install after a grub package upgrade. Refreshed it. Still filtered.
• Kernel 7.0.5 regression. Downgraded to 6.18.3, the kernel that had run for 4 months. Still filtered. So the kernel itself wasn’t it either.

The clue was in the persistent journal: a single recorded boot from December 31 to May 12 10:13 UTC, and absolutely nothing after. Every reboot since the upgrade was failing before systemd-journald could flush to disk — so the failure had to be in the initramfs, before the root filesystem was even mounted.

What it almost certainly was

Hetzner Dedicated servers configure the initramfs network with ip=dhcp on the kernel command line. That depends on Hetzner’s DHCP server replying to whatever request format the current kernel sends. Somewhere between kernel 6.18 / iproute2 6.18 and kernel 7.0 / iproute2 7.0, the request format changed enough that Hetzner’s DHCP stopped responding. Effects:

• Old kernel at runtime kept the interface already configured (Phase A — 32 hours of healthy operation after the package upgrade).
• New kernel cold-boots, hits DHCP, never gets an IP, dropbear cannot listen, port 22 stays filtered.

Hetzner’s own documentation has been quietly moving away from ip=dhcp toward static IPv4 in the kernel command line. The fix is exactly that:

GRUB_CMDLINE_LINUX="cryptdevice=/dev/md1:cryptroot ip=A.B.C.D::GATEWAY:255.255.255.255:hostname:eth0:none"

One line in /etc/default/grub, grub-mkconfig, reboot. No more dependency on Hetzner’s DHCP responding to whatever your current kernel sends.

Why it matters for anyone running this stack

If you run Arch on Hetzner Dedicated with full-disk encryption and remote unlock via dropbear, the ip=dhcp shipped by installimage is a latent bug. It can keep working for years and then break overnight, on every machine you have, after a routine pacman -Syyu. The static-IP version is what Hetzner now recommends and removes the entire dependency.

Tooling

While debugging, I turned the whole rescue / chroot / diagnose / fix workflow into a Python CLI (hal) — including hal fix static-ip, which derives the static cmdline directly from your existing systemd-networkd .network file:

→ github.com/kevinveenbirkenbach/hetzner-arch-luks

Single command, idempotent, reversible (the original /etc/default/grub is backed up to .hal-backup). If you’re on this stack, switch to static IP before the next kernel upgrade catches you.

When two Hetzner servers died at the same time

On May 12, 2026, two of my Arch Linux + LUKS servers at Hetzner became unreachable at the same moment. Both had been running for 4+ months without issue. Both had received the same pacman -Syyu the day before, but had stayed on the old kernel until the morning the websites stopped responding. I rebooted — SSH never came back. nmap -Pn -p 22 showed filtered from anywhere. No ping. No banner. The Hetzner Robot panel insisted the hardware was fine.

Several hours went into hypotheses that turned out to be wrong:

• The encryptssh initcpio hook referencing a /usr/lib/initcpio/udev/11-dm-initramfs.rules file that no longer exists. Real bug, no boot impact — the initramfs rebuilds anyway.
• PermitRootLogin no in sshd_config. Real misconfiguration, fixed it, didn’t help. A refusing sshd shows closed, not filtered.
• Predictable interface-naming drift after the systemd 260 upgrade. Patched the .network config to match by MAC. Useful hardening; not the cause.
• Stale GRUB stage1 + core.img in the MBR. Arch never re-runs grub-install after a grub package upgrade. Refreshed it. Still filtered.
• Kernel 7.0.5 regression. Downgraded to 6.18.3, the kernel that had run for 4 months. Still filtered. So the kernel itself wasn’t it either.

The clue was in the persistent journal: a single recorded boot from December 31 to May 12 10:13 UTC, and absolutely nothing after. Every reboot since the upgrade was failing before systemd-journald could flush to disk — so the failure had to be in the initramfs, before the root filesystem was even mounted.

What it almost certainly was

Hetzner Dedicated servers configure the initramfs network with ip=dhcp on the kernel command line. That depends on Hetzner’s DHCP server replying to whatever request format the current kernel sends. Somewhere between kernel 6.18 / iproute2 6.18 and kernel 7.0 / iproute2 7.0, the request format changed enough that Hetzner’s DHCP stopped responding. Effects:

• Old kernel at runtime kept the interface already configured (Phase A — 32 hours of healthy operation after the package upgrade).
• New kernel cold-boots, hits DHCP, never gets an IP, dropbear cannot listen, port 22 stays filtered.

Hetzner’s own documentation has been quietly moving away from ip=dhcp toward static IPv4 in the kernel command line. The fix is exactly that:

GRUB_CMDLINE_LINUX="cryptdevice=/dev/md1:cryptroot ip=A.B.C.D::GATEWAY:255.255.255.255:hostname:eth0:none"

One line in /etc/default/grub, grub-mkconfig, reboot. No more dependency on Hetzner’s DHCP responding to whatever your current kernel sends.

Why it matters for anyone running this stack

If you run Arch on Hetzner Dedicated with full-disk encryption and remote unlock via dropbear, the ip=dhcp shipped by installimage is a latent bug. It can keep working for years and then break overnight, on every machine you have, after a routine pacman -Syyu. The static-IP version is what Hetzner now recommends and removes the entire dependency.

Tooling

While debugging, I turned the whole rescue / chroot / diagnose / fix workflow into a Python CLI (hal) — including hal fix static-ip, which derives the static cmdline directly from your existing systemd-networkd .network file:

→ github.com/kevinveenbirkenbach/hetzner-arch-luks

Single command, idempotent, reversible (the original /etc/default/grub is backed up to .hal-backup). If you’re on this stack, switch to static IP before the next kernel upgrade catches you.

When two Hetzner servers died at the same time

On May 12, 2026, two of my Arch Linux + LUKS servers at Hetzner became unreachable at the same moment. Both had been running for 4+ months without issue. Both had received the same pacman -Syyu the day before, but had stayed on the old kernel until the morning the websites stopped responding. I rebooted — SSH never came back. nmap -Pn -p 22 showed filtered from anywhere. No ping. No banner. The Hetzner Robot panel insisted the hardware was fine.

Several hours went into hypotheses that turned out to be wrong:

• The encryptssh initcpio hook referencing a /usr/lib/initcpio/udev/11-dm-initramfs.rules file that no longer exists. Real bug, no boot impact — the initramfs rebuilds anyway.
• PermitRootLogin no in sshd_config. Real misconfiguration, fixed it, didn’t help. A refusing sshd shows closed, not filtered.
• Predictable interface-naming drift after the systemd 260 upgrade. Patched the .network config to match by MAC. Useful hardening; not the cause.
• Stale GRUB stage1 + core.img in the MBR. Arch never re-runs grub-install after a grub package upgrade. Refreshed it. Still filtered.
• Kernel 7.0.5 regression. Downgraded to 6.18.3, the kernel that had run for 4 months. Still filtered. So the kernel itself wasn’t it either.

The clue was in the persistent journal: a single recorded boot from December 31 to May 12 10:13 UTC, and absolutely nothing after. Every reboot since the upgrade was failing before systemd-journald could flush to disk — so the failure had to be in the initramfs, before the root filesystem was even mounted.

What it almost certainly was

Hetzner Dedicated servers configure the initramfs network with ip=dhcp on the kernel command line. That depends on Hetzner’s DHCP server replying to whatever request format the current kernel sends. Somewhere between kernel 6.18 / iproute2 6.18 and kernel 7.0 / iproute2 7.0, the request format changed enough that Hetzner’s DHCP stopped responding. Effects:

• Old kernel at runtime kept the interface already configured (Phase A — 32 hours of healthy operation after the package upgrade).
• New kernel cold-boots, hits DHCP, never gets an IP, dropbear cannot listen, port 22 stays filtered.

Hetzner’s own documentation has been quietly moving away from ip=dhcp toward static IPv4 in the kernel command line. The fix is exactly that:

GRUB_CMDLINE_LINUX="cryptdevice=/dev/md1:cryptroot ip=A.B.C.D::GATEWAY:255.255.255.255:hostname:eth0:none"

One line in /etc/default/grub, grub-mkconfig, reboot. No more dependency on Hetzner’s DHCP responding to whatever your current kernel sends.

Why it matters for anyone running this stack

If you run Arch on Hetzner Dedicated with full-disk encryption and remote unlock via dropbear, the ip=dhcp shipped by installimage is a latent bug. It can keep working for years and then break overnight, on every machine you have, after a routine pacman -Syyu. The static-IP version is what Hetzner now recommends and removes the entire dependency.

Tooling

While debugging, I turned the whole rescue / chroot / diagnose / fix workflow into a Python CLI (hal) — including hal fix static-ip, which derives the static cmdline directly from your existing systemd-networkd .network file:

→ github.com/kevinveenbirkenbach/hetzner-arch-luks

Single command, idempotent, reversible (the original /etc/default/grub is backed up to .hal-backup). If you’re on this stack, switch to static IP before the next kernel upgrade catches you.

A quotation from George Washington

Though in reviewing the incidents of my Administration I am unconscious of intentional error, I am nevertheless too sensible of my defects not to think it probable that I may have committed many errors. Whatever they may be, I fervently beseech the Almighty to avert or mitigate the evils to which they may tend. I shall also carry with me the hope that my country will never cease to view them with indulgence, and that, after forty-five years of my life dedicated to its service with an upright zeal, the faults of incompetent abilities will be consigned to oblivion, as myself must soon be to the mansions of rest.

George Washington (1732–1799) American military leader, Founding Father, US President (1789–1797)
Letter (1796-09-17), "Farewell Address" [with J. Madison, A. Hamilton]

More about this quote: wist.info/washington-george/81…

#quote #quotes #quotation #qotd #georgewashington #selfcriticism #selfassessment #retirement #administration #blame #error #fault #forgiveness #honestmistake #humility #mistake #oblivion #postmortem #president #review #selfdeprecation

Someone shared a cool thread about his #vibe-coding experience :

https://forum.godotengine.org/t/post-mortem-of-my-failed-attempt-to-vibe-code-a-metroidvania-game/137567/9

#vibe_coding #ai #llm #slop #aislop #fraud #scam #dev #postmortem #gamedev

Someone shared a cool thread about his #vibe-coding experience :

https://forum.godotengine.org/t/post-mortem-of-my-failed-attempt-to-vibe-code-a-metroidvania-game/137567/9

#vibe_coding #ai #llm #slop #aislop #fraud #scam #dev #postmortem #gamedev

Someone shared a cool thread about his #vibe-coding experience :

https://forum.godotengine.org/t/post-mortem-of-my-failed-attempt-to-vibe-code-a-metroidvania-game/137567/9

#vibe_coding #ai #llm #slop #aislop #fraud #scam #dev #postmortem #gamedev

Someone shared a cool thread about his #vibe-coding experience :

https://forum.godotengine.org/t/post-mortem-of-my-failed-attempt-to-vibe-code-a-metroidvania-game/137567/9

#vibe_coding #ai #llm #slop #aislop #fraud #scam #dev #postmortem #gamedev

Someone shared a cool thread about his #vibe-coding experience :

https://forum.godotengine.org/t/post-mortem-of-my-failed-attempt-to-vibe-code-a-metroidvania-game/137567/9

#vibe_coding #ai #llm #slop #aislop #fraud #scam #dev #postmortem #gamedev

Post Mortem: axios NPM supply chain compromise

https://github.com/axios/axios/issues/10636

#HackerNews #PostMortem #axios #NPM #supplyChain #Compromise #CyberSecurity #OpenSource

Post Mortem: axios NPM supply chain compromise

https://github.com/axios/axios/issues/10636

#HackerNews #PostMortem #axios #NPM #supplyChain #Compromise #CyberSecurity #OpenSource

Post Mortem: axios NPM supply chain compromise

https://github.com/axios/axios/issues/10636

#HackerNews #PostMortem #axios #NPM #supplyChain #Compromise #CyberSecurity #OpenSource

Post Mortem: axios NPM supply chain compromise

https://github.com/axios/axios/issues/10636

#HackerNews #PostMortem #axios #NPM #supplyChain #Compromise #CyberSecurity #OpenSource

Post Mortem: axios NPM supply chain compromise

https://github.com/axios/axios/issues/10636

#HackerNews #PostMortem #axios #NPM #supplyChain #Compromise #CyberSecurity #OpenSource

Как анализировать инциденты. История об ошибках

Стоимость минуты простоя в iGaming может приносить миллионы упущенной прибыли и более тяжелые репутационные потери. Когда real‑time ставки замирают, а букмекерские терминалы уходят в ступор — это не просто баг. Это экзамен на зрелость команды и процессов. Что мы делаем после — определяет, повторится ли он снова.

https://habr.com/ru/articles/933964/

#rca #postmortem #incident #problem #highload #itil #itsm

Awesome! 👌🏽

“How Complex Systems Fail” [1998], Richard Cook (https://how.complexsystems.fail/).

On HN: https://news.ycombinator.com/item?id=25550685

On Lobsters: https://lobste.rs/s/smzguc/how_complex_systems_fail

#Complexity #Failure #Reliability #SRE #RCA #PostMortem #RootCauseAnalysis #Fragility

Раскатил учётные записи с помощью Ansible playbook и... чёрт побери! В плейбуке пользователям задан /bin/bash, а на серверах с FreeBSD его нет. Разумеется, зайти уже не получалось по SSH и консольно.

Выручило то, что у хостера есть cloud-init. Добавил в запуск
runcmd:
- chsh -s /bin/sh root
и после перезагрузки смог зайти.

Дальше уже в плейбуке поменял порядок стадий, чтобы сначала ставился bash и другие пакеты, а потом уже по пользователям побежали настройки.

#devops #freebsd #postMortem

🚀 Oh, look! Yet another grandiloquent prize fest that ends with a whimper and a "blameless #postmortem." 🙄 Congratulations, QDay Prize organizers! You've achieved the remarkable feat of discovering that nobody cares. 🎉
https://algassert.com/post/2601 #grandiloquentprize #QDayPrize #nobodycares #prizefest #HackerNews #ngated

·· azkenak airean - last broadcast ··

cock sparrer
speed
post mortem
the blood

Entzun / Listen (streaming):
https://punkirratia.net:8443/punk

#cocksparrer #speed #postmortem #theblood #FediRadio #punk #streaming #NowPlaying #music #musica #musique #musika #MastoRadio

Claude postmortem: Details on why the AI acted dumb the last few weeks
https://www.anthropic.com/engineering/april-23-postmortem
#postmortem #claudecode #claude #bugs #llm #ai #+

Matrix.org - Post-mortem of the September 2 outage

"Matrix, the open protocol for secure decentralised communications"

Link: https://matrix.org/blog/2025/10/post-mortem/

#linkdump #blogpost #matrix #postmortem #story

Matrix.org - Post-mortem of the September 2 outage

"Matrix, the open protocol for secure decentralised communications"

Link: https://matrix.org/blog/2025/10/post-mortem/

#linkdump #blogpost #matrix #postmortem #story

Blameless post‑mortem: как разбирать инциденты так, чтобы они не повторялись

Инциденты случаются у всех. Разница между командами — не в их отсутствии, а в том, повторяются ли одни и те же сбои снова и снова. Часто разбор заканчивается быстро и предсказуемо: нашли виноватого, сделали вывод «надо быть внимательнее» и пошли дальше — до следующего инцидента. В этой статье разберём, почему такой подход не работает, что на самом деле стоит за blameless post‑mortem и как выстроить разбор инцидентов так, чтобы он приводил не к формальным выводам, а к конкретным изменениям в системе.

https://habr.com/ru/companies/otus/articles/1017054/

#team_lead #postmortem #разбор_инцидентов #blameless_культура #DevOps_практики #мониторинг_систем #инциденты_в_продакшене #root_cause_анализ

Bluesky April 2026 Outage Post-Mortem

https://pckt.blog/b/jcalabro/april-2026-outage-post-mortem-219ebg2

#HackerNews #Bluesky #Outage #PostMortem #April2026 #SocialMedia #Analysis #TechNews

Эпические баги: как переиспользование вызова функции принесло убыток в $450.000.000

Продолжаем тему эпических багов. В прошлый раз мы говорили про AT&T, положивших свою ультранадежную сеть одним "Break" в коде. Сегодня на очереди Knight Capital Group, решивших переиспользовать старый флаг в бинарном протоколе, затем там был мёртвый код, который забыли удалить и деплой, проверенный на семи серверах из восьми. Итог: уход в минус 450 миллионов долларов за 45 минут. На Хабре этот инцидент упоминался несколько раз, но даже в самой большой статье (к слову, переводу, со всеми странными атрибутами инопрессы, вроде фраз "Атака зомби из «Кода убийцы»" и пространным вступлением) инцидент рассматривался скорее как финансовый. А нас же больше интересуют именно технические детали.

https://habr.com/ru/companies/beget/articles/1012834/

#эпические_баги #Knight_Capital #высокочастотная_торговля #деплой #мёртвый_код #бинарный_протокол #ошибка_DevOps #postmortem #переиспользование_кода #тестирование

NEW POST MORTEM video dropping! Here’s the link to the YouTube premiere. It’s this Saturday (tomorrow! 😱)

► https://www.youtube.com/watch?v=CPN1SzJE8KU 🥳

#postmortem #youtube #video #ldjam #documentary #gamedev #ludumdare

Rendering the first release candidate! 🤞🥹

#ludumdare #ldjam #postmortem #youtuber

Почему серийной беспилотной логистики в России все еще нет?

Post-Mortem лонгрид о том, как я пытался быть первопроходцем на рынке беспилотной логистики в России Писал софт для дронов, провел десятки интервью и переговоров в промышленности, построил и защитил финмодели, собрал команды, привлек инвестиции, провел пилоты в нефтегазе — и в итоге закрыл проект . Расскажу, где хайповая доставка дронами дает реальный экономический эффект, а где остаётся красивой презентацией для инвесторов и СМИ. А также о всем моем опыте запуска этой истории после сытого найма

https://habr.com/ru/articles/1008146/

#беспилотники #дрон #стартап #нефтегаз #доставка_дронами #postmortem #факап #drone #бизнес

Управление рисками в GameDev. Управление проектом (Project Management). Риск срыва сроков, бюджета и выгорания команды

Начиная писать данную статью, меня не покидало ощущение, что я открываю “Ящик пандоры”. Холивар. Так как ну кто признается, что он плохой проектный менеджер? Кто скажет - я плохо управляю проектами? Ну я же… (дальше сами подставьте необходимый спич:)) Тем не менее, в рамках цикла статей по управлению рисками в GameDev данную область просто необходимо рассмотреть. Четкое понимание целей, целевого состояния проекта, продукта, позволяет лучше понять присущие риски. Эффективное управление игровым проектом и присущими рисками, дает конкурентное преимущество тому, кто знает, как управлять и эффективно этим пользуется. Мы все часто слышали: “Нужно задержаться”, “Нужно выйти в субботу”, “Вся команда пашет, а ты домой собрался”, “У нас дедлайн, нужно сделать и все”, “It is the Crunch time , baby!” и т.д. в подобном ключе. Я сам, бывало, работал и по выходным, и до утра. Иногда это было дико интересно, а иногда у тебя был просто выбор - либо поработать как просят, либо уволиться. Давайте разберемся, почему так происходит?

https://habr.com/ru/articles/991532/

#проектное_управление #анализ #риски #геймдев #геймдизайн #postmortem #индиразработка #gamedev #gamedevelopment #crunch

Управление рисками в GameDev. Геймдизайн (Game Design). Риск того, что игра не даст увлекательный и запоминающийся опыт

Большое количество игровых проектов терпят неудачу. Но что, если это не вопрос удачи, а следствие ошибок управления, рисков, не выявленных во время, или выявленных, но оставленных без должного внимания? Провал игрового проекта — это чаще всего не случайность.

https://habr.com/ru/articles/986168/

#анализ #риски #геймдев #postmortem #инди #gamedev #indiedev #gamedesign #indiegamedev #rpg

@andrewstroehlein

3/

16 – Religion was hijacked to serve the regime.

17 – Americans’ image of their country’s democracy was too unquestioning

18 – US allies abroad, particularly in Europe, never really tried to help

19 – The regime accelerated the country’s demise by expanding its use of violence.

https://www.internationalaffairsfornormalpeople.eu/why-the-us-collapsed/

#US
#PostMortem

@andrewstroehlein

Main points:

1 – Impunity at the top had devastating consequences.

2 – The presidency became more like a monarchy.

3 – White supremacy was central to the fall of the US.

4 – For decades, the US had been a country plagued with violence, especially gun violence.

5 – The rise of the security state.

6 – The increasing political focus on upping anti-immigrant sentiment

7 – Wealth disparities in the US grew to extreme levels

https://www.internationalaffairsfornormalpeople.eu/why-the-us-collapsed/

1/

#US
#PostMortem

‘Power Book IV: Force’ Series Finale Eliminates More Enemies While Setting Up Possible Continuation With The Return of *Spoiler*
#News #PostMortem #PowerBookIVForce #Starz

https://deadline.com/2026/01/power-book-iv-force-series-finale-explanation-1236682045/

RE: https://openbiblio.social/@literarymachine/115899939849723408

lesenswertes #postmortem zu einem läääängerem verfügbarkeitsproblem. #infosec #mastoadmin

CNAME vs A record: Cloudflare outage report talks about a subtle wrinkle in DNS, where the order of the records matters when it probably shouldn't
https://blog.cloudflare.com/cname-a-record-order-dns-standards/
#postmortem #cloudflare #outage #cname #dns #+

@ellieinspace

Part 2/2 Ep

BREAKING: We Finally Know Why Starship Flight 8 Failed! SpaceX Reveals All

https://youtu.be/51yeJOm4rr8?feature=shared

#spacex #starship #ift8 #rootcause #postmortem #ellieinspace

#LGBTQ #rights in #HongKong: #legal #milestones in the long #battle for #equality.

The #CourtofFinalAppeal has this week #ruled #government must formulate #framework for #recognising #samesex #partnerships in two years’ time. For decades, members of #LGBTQ #community have fought #policies #restricting #sex, #marriage and #postmortem #arrangements

#Women #Transgender #LGBTQ #LGBTQIA #Asia #HongKong #SameSexMarriage #MarriageEquality #Progress

https://www.scmp.com/news/hong-kong/law-and-crime/article/3233618/lgbtq-rights-hong-kong-legal-milestones-long-battle-equality

Lagos orders postmortem as twins die after vaccination

The Lagos State Government has ordered a postmortem on nine-month-old identical twins, Testimony and Timothy Alozie, who reportedly…
#NewsBeep #News #Healthcare #Health #healthcare #immunisation #infantdeath #infantmortality #investigation #lagos #LagosState #medicalinvestigation #postmortem #primaryhealthcare #Publichealth #SamuelAlozie #Twindeath #Twins #UK #UnitedKingdom #Vaccination
https://www.newsbeep.com/uk/374685/

How reality crushed Ÿnsect, the French startup that had raised over $600M for insect farming

https://web.brid.gy/r/https://techcrunch.com/2025/12/26/how-reality-crushed-ynsect-the-french-startup-that-had-raised-over-600m-for-insect-farming/