#monitorama — Public Fediverse posts

@obfuscurity hi Jason, do you have dates for #Monitorama 2025?

More awesomeness from hashtag #monitorama — @austinlparker combines excellent storytelling with his first-hand knowledge of the creation of #opentelemetry to give us a better understanding of what an open standard could and should be: https://youtu.be/yJFYNTq3uCs?si=tanX7y7jQ2JCrgm3

I'm finally getting to catch up on the #monitorama talks from this year. First up is this deeply informative talk from Colin Douch on @Cloudflare's adventures in using #ClickHouse to replace #Prometheus:

https://www.youtube.com/watch?v=-wIjq_dyzDg

Please enjoy in full my Monitorama 2024 presentation, "No Observability Without Theory"! You'll get:

* A bunch of made-up medical jargon
* Tips on facilitating a shared mental model in your graph dashboards and incident response processes
* 24% off my Leading Incidents video course (https://d2e.engineering/leading-incidents) with the discount code at the end!

https://www.youtube.com/watch?v=aJkynVk1k6M #observability #o11y #monitorama #devops #ops #softwareengineering #sre

@reyjrar Agreed, this is a key difference and really supports building a “tribe/community” of practitioners. I’ve spoken at #Monitorama at least five times and attended without speaking several times as well (like this year) because I want to be part of it.

In single track conferences, the speakers see other speakers' talks and they reference those talks in their talks. This unwittingly creates links from different ideas making both of those talks more memorable.

This year, almost every speaker referenced one of their peers. A few even called back to talks from past #monitorama events.

This makes the event more fun, but also makes the lessons learned signifcantly more impactful!

Heading home from #Monitorama now. It was great, as always. It's great on purpose. @obfuscurity puts a lot of effort into making the conference be the conference he always wanted. We are all benefactors of his discerning tastes.

See this thread for some of my favorite things about #Monitorama

Managed to make yoga after #monitorama.

I rarely win things…

But I won a Playdate @ #monitorama

Gathering data from lots of different sources, providing it to the model then processing the output through post processing steps? That sounds like an asynchronous workflow, USE TRACING ~ @cartermp lauding the benefits of tracing for deploying and iterating AI applications #monitorama #monitorama24 #observability

“Disintegration is a feature not a bug, of asynchronous systems, until you introduce telemetry and monitoring” ~ Johannes Tax from @grafana #monitorama #monitorama24 #observability

Johannes Tax at @grafana describing the pain of distributed tracing in asynchronous systems “Disintegrated telemetry: The pains of monitoring asynchronous workflows” #monitorama #monitorama24 #observability

sage words from @danslimmon on how to better develop you/your team's #o11y theory by getting practical practice #monitorama #observability

“Name your metrics, alerts and dashboards with the language you would use if you were having a conversation about the system at lunch, not the cryptic defaults or uuid hostnames” ~ @danslimmon #monitorama #monitorama24 #observability

We’ve all heard the definition for observability, but, who is one? What is involved in determining? ~ @danslimmon presenting “No observability without theory” You need a theory of the system to build valid inferences
#monitorama #monitorama24 #observability

Julia Thoreson at Bloomberg sharing “Incident Management: Lessons from Emergency Services” breaking down how the lessons learned in emergency services can apply to incident management in technical systems #monitorama #monitorama24 #incidentmanagement

Pete Fritchman’s Takeaways on managing internal services effectively:
Internal Services impact customers
Leverage your observability tools
Talk to your internal customers
*APPLY SRE PRINCIPLES*

#Monitorama #monitorama24 #observability #SRE

“New hires are super value able in your internal customer interviews, they actually expect things to work and aren’t bitter yet” ~ Pete Fritchman #Monitorama #monitorama24 #observability #sre

“Treat your internal tooling outages like the most critical production outages, because they’ll always hit when you’re trying to recover from a critical production outage” ~ Pete Fritchman #monitorama #monitorama24 #observability

“The shoemaker’s children have no shoes - why SRE teams must help themselves” Pete Fritchman making the case for investing in watching the watchmen, and techniques for accomplishing it. #monitorama #monitorama24 #observability #sre

Hashmaps to counts work great for small sets, but what happens when you need to count sets larger than memory? You need HyperLogLog or Disjunctive Normal Form (CVM) ~ @phredmoyer #monitorama #monitorama24 #observability

“Use counters to count things” @phredmoyer providing some examples where counting things by processing petabytes of log or trace data is prohibitively expensive and justify spending a bit more on dealing with higher cardinality metrics.
#monitorama #monitorama24 #observability

"Shoutout to Brad, who's still using Perl" #Monitorama

Baggage is bad for your relationships, good for your service graphs. @kalyanaj makes the case for an arbitrary key value metadata store (baggage) to propagate through your services to enable controllability and observability use cases. #monitorama #monitorama24 #observability

Nice #monitorama #w3c #baggage

“Distributed Context Propagation: How you can use it to Improve Observability, Test in Production, and more...” @kalyanaj explaining the importance of context in interpreting observability data #monitorama #monitorama24

“Every team has a different answer for discovering what the dependencies of their services are, some say firewall rules, some look at network flows, tracing gives us a uniform answer to this” ~ Sudeep Kumar #Monitorama #monitorama24 #observability #tracing

“We have so many microservices, people are always looking for an excuse to create more, and no one knows which ones they’re already dependent on” ~ Sudeep Kumar from Salesforce with “Tracing Service Dependencies at Salesforce”

#Monitorama #monitorama24 #observability #tracing

“Low cardinality in Prometheus and low cardinality in Clickhouse are vastly different things” - @colind in his talk “Experiments in Backing Prometheus with Clickhouse” #Monitorama #Monitorama24 #observability

drinking our own 🥂 with #elastic observability (it's not dog food): when something is up in your app and you let the automatic correlation do the work for you
PS: lots of talk about cost, OTel,... at #monitorama — not so much actually getting value out of all the labor...