#dtrace — Public Fediverse posts
Live and recent posts from across the Fediverse tagged #dtrace, aggregated by home.social.
-
Starting to get back into illumos/arm64. Current weekend project is getting DTrace working. Some initial progress:
# dtrace -n 'profile-997{@[stack()] = count();}' | tail
dtrace: description 'profile-997' matched 1 probe
genunix`taskq_thread+0x2e8
genunix`thread_exit
60xf9400e80
3910xd53b4221
genunix`thread_exit
3779Working fbt provider will be a bit harder...
-
-
After fixing up the pfsense gateway (forgot some log rotation, blew out the root dataset) I discovered that my home infra needs a little better observability.
I’d start with a timeseries db, and why not run under the new podman-on-freebsd-jails system? Should be quick and easy.
Anyway, I’ve discovered how to do things with jails that the manual says are explicitly forbidden, and I’m finally learning corners of dtrace I’ve never touched. Pray for me.
-
If you are using #DTrace on #FreeBSD, then I've got good news for you.
I've added a new manual to FreeBSD. It's called d(7) and it is a short reference of the #D scripting language. Currently, it covers only a subset of all variables and functions, but we'll add more iteratively.
Here are some previews for you:
HTML: https://people.freebsd.org/~0mp/d.7.html
MDOC (view with man(1)): https://people.freebsd.org/~0mp/d.7
Have fun tracing your systems!
-
@monospace Sounds pretty cool. It might be interesting to add a similar functionality as a #dwatch profile to #FreeBSD.
-
@monospace Sounds pretty cool. It might be interesting to add a similar functionality as a #dwatch profile to #FreeBSD.
-
@monospace Sounds pretty cool. It might be interesting to add a similar functionality as a #dwatch profile to #FreeBSD.
-
@monospace Sounds pretty cool. It might be interesting to add a similar functionality as a #dwatch profile to #FreeBSD.
-
@monospace Sounds pretty cool. It might be interesting to add a similar functionality as a #dwatch profile to #FreeBSD.
-
everytime someone mentions #dtrace on the internet @briancantrell drinks a diet coke
-
I've just committed a new manual page to #FreeBSD: tracing(7), an introduction to #tracing and #performance #monitoring facilities.
It'll take a moment before it is available for readers on man.freebsd.org, so for the time being you can grab a copy from here (https://cgit.freebsd.org/src/plain/share/man/man7/tracing.7?id=c29459f901dc73e1ebe1e654da2677cc0b7bd296) and open it with man(1).
#DTrace #dwatch #ktrace #osdev #foss #ptrace #utrace #truss #boottrace #tslog
-
I've just committed a new manual page to #FreeBSD: tracing(7), an introduction to #tracing and #performance #monitoring facilities.
It'll take a moment before it is available for readers on man.freebsd.org, so for the time being you can grab a copy from here (https://cgit.freebsd.org/src/plain/share/man/man7/tracing.7?id=c29459f901dc73e1ebe1e654da2677cc0b7bd296) and open it with man(1).
#DTrace #dwatch #ktrace #osdev #foss #ptrace #utrace #truss #boottrace #tslog
-
I've just committed a new manual page to #FreeBSD: tracing(7), an introduction to #tracing and #performance #monitoring facilities.
It'll take a moment before it is available for readers on man.freebsd.org, so for the time being you can grab a copy from here (https://cgit.freebsd.org/src/plain/share/man/man7/tracing.7?id=c29459f901dc73e1ebe1e654da2677cc0b7bd296) and open it with man(1).
#DTrace #dwatch #ktrace #osdev #foss #ptrace #utrace #truss #boottrace #tslog
-
I've just committed a new manual page to #FreeBSD: tracing(7), an introduction to #tracing and #performance #monitoring facilities.
It'll take a moment before it is available for readers on man.freebsd.org, so for the time being you can grab a copy from here (https://cgit.freebsd.org/src/plain/share/man/man7/tracing.7?id=c29459f901dc73e1ebe1e654da2677cc0b7bd296) and open it with man(1).
#DTrace #dwatch #ktrace #osdev #foss #ptrace #utrace #truss #boottrace #tslog
-
I've just committed a new manual page to #FreeBSD: tracing(7), an introduction to #tracing and #performance #monitoring facilities.
It'll take a moment before it is available for readers on man.freebsd.org, so for the time being you can grab a copy from here (https://cgit.freebsd.org/src/plain/share/man/man7/tracing.7?id=c29459f901dc73e1ebe1e654da2677cc0b7bd296) and open it with man(1).
#DTrace #dwatch #ktrace #osdev #foss #ptrace #utrace #truss #boottrace #tslog
-
@mhoye
#SystemTap or #DTrace might offer what you are looking for
https://sourceware.org/systemtap/wiki
https://www.brendangregg.com/dtrace.html -
@mhoye
#SystemTap or #DTrace might offer what you are looking for
https://sourceware.org/systemtap/wiki
https://www.brendangregg.com/dtrace.html -
@mhoye
#SystemTap or #DTrace might offer what you are looking for
https://sourceware.org/systemtap/wiki
https://www.brendangregg.com/dtrace.html -
@mhoye
#SystemTap or #DTrace might offer what you are looking for
https://sourceware.org/systemtap/wiki
https://www.brendangregg.com/dtrace.html -
@mhoye
#SystemTap or #DTrace might offer what you are looking for
https://sourceware.org/systemtap/wiki
https://www.brendangregg.com/dtrace.html -
Enabling DTrace in Python on FreeBSD (again)
https://antranigv.am/posts/2025/02/dtrace-python-freebsd-again/
Last night I was running our usual Greybeard AMA on FreeBSD’s Discord server, when someone asked “I’ve been using Linux for years, but I also like FreeBSD. what can I do for FreeBSD, and what can FreeBSD do for me, as a Python Full Stack Developer?”
I started talking about FreeBSD Jails, ZFS, Boot Environments and more, but I also wanted to focus on DTrace. The ability to dynamically trace on production still sounds like magic to me. I know it isn’t, I’ve read the code and the papers, but the fact that I can ask the operating system questions on the fly, without recompiling, is amazing.
So when I was demoing DTrace, I also wanted to show the DTrace integrations with Python. Little did I know that the Python package on FreeBSD was not compiled with the
--with-dtraceoption.As a sane person, I setup a Jail (using Jailer, of course), cloned the FreeBSD ports tree and added the
--with-dtraceoption back. It did not compile.The first issue that we encountered was the following:
--- Include/pydtrace_probes.h ---dtrace: option requires an argument -- s
ah yes, looking over the Makefile we see the following
Include/pydtrace_probes.h: $(srcdir)/Include/pydtrace.d $(MKDIR_P) Include $(DTRACE) $(DFLAGS) -o $@ -h -s $< : sed in-place edit with POSIX-only tools sed 's/PYTHON_/PyDTrace_/' $@ > [email protected] mv [email protected] $@
(you can also view the source here)
As far as I can tell, the
$<thingie does not work with BSD Make, so how about if we try usinggmakeinstead?I did the following changes to the
Makefilein the FreeBSD Ports tree. Basically addinggmakeintoUSES=.USES= compiler:c11 cpe ncurses pathfix pkgconfig \- python:${PYTHON_DISTVERSION:R},env readline shebangfix ssl tar:xz+ python:${PYTHON_DISTVERSION:R},env readline shebangfix ssl tar:xz gmakeNow let’s try compiling again.
We get the following errors now:
ld: error: undefined symbol: __dtraceenabled_python___function__entry
Ah yes, linking issues.
After an hour of digging we learned that the
DTRACE_OBJS=variable is set to… nothing. But it needs to be set toPython/pydtrace.o. I was not able to fix this issue properly (inconfigure,configure.ac, or whatever madness that GNU Autotools use), so I just changed the line manually in theMakefile.Now let’s try compiling again.
We get the following error:
./Python/sysmodule.c:223:24: warning: passing 'const char *' to parameter of type 'char *' discards qualifiers [-Wincompatible-pointer-types-discards-qualifiers]
Someone in the Discord chat recommended that I disable the LTO (Link-Time Optimization) option.
I did
make clean, I patched that line again, and here we go, one more time.It’s compiling, it’s compiling, it’s compiling… aaaaand we’re done! Let’s do
make install===> Installing for python311-3.11.11===> Checking if python311 is already installed===> Registering installation for python311-3.11.11[python.srv0.hackerspace.am] Installing python311-3.11.11...
good! now let’s try DTrace with Python!
In one terminal, I ran Python, and in another one I got the following
# dtrace -l -n 'python*:::' ID PROVIDER MODULE FUNCTION NAME85302 python8737 libpython3.11.so.1.0 sys_audit audit85303 python8737 libpython3.11.so.1.0 sys_audit_tstate audit85304 python8737 libpython3.11.so.1.0 _PyEval_EvalFrameDefault function-entry85305 python8737 libpython3.11.so.1.0 dtrace_function_return function-return85306 python8737 libpython3.11.so.1.0 _PyEval_EvalFrameDefault function-return85307 python8737 libpython3.11.so.1.0 gc_collect_main gc-done85308 python8737 libpython3.11.so.1.0 gc_collect_main gc-start85309 python8737 libpython3.11.so.1.0 PyImport_ImportModuleLevelObject import-find-load-done85310 python8737 libpython3.11.so.1.0 PyImport_ImportModuleLevelObject import-find-load-start85311 python8737 libpython3.11.so.1.0 _PyEval_EvalFrameDefault line
Woohoo! Now I’m happy!
Okay, so what did we learn?
- There’s a little bit of GNUMake-ism in Python’s Makefile. Either we have to use
gmakein FreeBSD when building Python or we need to submit an alternative to the upstream - Die GNU Autotools… I mean… the GNU Autotools usage in Python has an issue, where the generated Makefile does not set the
DTRACE_OBJScorrectly. Can anyone help with this? As a Pascal guy, reading GNU configure files makes me wanna die. - LTO is problematic. Solutions?
The real question is, how can we enable DTrace by default in FreeBSD packages for Python. DTrace is one of our market advantages and we should find a way to enable it everywhere we can.
This was a fun Greybeard AMA, where we touched multiple parts of the system. Looking forward for next week!
That’s all folks…
Reply via email.
- There’s a little bit of GNUMake-ism in Python’s Makefile. Either we have to use
-
Hard to believe it’s been 20 years since the first publication of #OpenSolaris source code with the #DTrace source release:
-
It arrived! Huge thanks to @danmcd for arranging shipping to the UK.
-
dtrace.conf is back as dtrace.conf(24)
Woke up middle of the night to grab a cup of water, decided to check Mastodon, and what do I see?
dtrace.conf(24)Tickets, Wed, Dec 11, 2024 at 9:00 AMThis makes me very happy! I love seeing DTrace in the wild, and having more DTrace content out there is beneficial to everyone in the DTrace community.
Obviously, being a Syrian with passport issues, I will not be able to attend, but hopefully everything will be recorded and published online. I’ll try to make it to
dtrace.conf(28).Have fun everyone!
Reply via email.
https://antranigv.am/posts/2024/10/dtrace-conf-24/
#debugging #DTrace #dtraceConf #FreeBSD #illumos #OxideComputerCompany
-
dtrace.conf is back as dtrace.conf(24)
Woke up middle of the night to grab a cup of water, decided to check Mastodon, and what do I see?
dtrace.conf(24)Tickets, Wed, Dec 11, 2024 at 9:00 AMThis makes me very happy! I love seeing DTrace in the wild, and having more DTrace content out there is beneficial to everyone in the DTrace community.
Obviously, being a Syrian with passport issues, I will not be able to attend, but hopefully everything will be recorded and published online. I’ll try to make it to
dtrace.conf(28).Have fun everyone!
Reply via email.
https://antranigv.am/posts/2024/10/dtrace-conf-24/
#debugging #DTrace #dtraceConf #FreeBSD #illumos #OxideComputerCompany
-
dtrace.conf is back as dtrace.conf(24)
Woke up middle of the night to grab a cup of water, decided to check Mastodon, and what do I see?
dtrace.conf(24)Tickets, Wed, Dec 11, 2024 at 9:00 AMThis makes me very happy! I love seeing DTrace in the wild, and having more DTrace content out there is beneficial to everyone in the DTrace community.
Obviously, being a Syrian with passport issues, I will not be able to attend, but hopefully everything will be recorded and published online. I’ll try to make it to
dtrace.conf(28).Have fun everyone!
Reply via email.
https://antranigv.am/posts/2024/10/dtrace-conf-24/
#debugging #DTrace #dtraceConf #FreeBSD #illumos #OxideComputerCompany
-
dtrace.conf is back as dtrace.conf(24)
Woke up middle of the night to grab a cup of water, decided to check Mastodon, and what do I see?
dtrace.conf(24)Tickets, Wed, Dec 11, 2024 at 9:00 AMThis makes me very happy! I love seeing DTrace in the wild, and having more DTrace content out there is beneficial to everyone in the DTrace community.
Obviously, being a Syrian with passport issues, I will not be able to attend, but hopefully everything will be recorded and published online. I’ll try to make it to
dtrace.conf(28).Have fun everyone!
Reply via email.
https://antranigv.am/posts/2024/10/dtrace-conf-24/
#debugging #DTrace #dtraceConf #FreeBSD #illumos #OxideComputerCompany
-
dtrace.conf is back as dtrace.conf(24)
Woke up middle of the night to grab a cup of water, decided to check Mastodon, and what do I see?
dtrace.conf(24)Tickets, Wed, Dec 11, 2024 at 9:00 AMThis makes me very happy! I love seeing DTrace in the wild, and having more DTrace content out there is beneficial to everyone in the DTrace community.
Obviously, being a Syrian with passport issues, I will not be able to attend, but hopefully everything will be recorded and published online. I’ll try to make it to
dtrace.conf(28).Have fun everyone!
Reply via email.
https://antranigv.am/posts/2024/10/dtrace-conf-24/
#debugging #DTrace #dtraceConf #FreeBSD #illumos #OxideComputerCompany
-
@sjmulder The answer is #DTrace, start with something like execsnoop but keep a track of parent/child relationships and how long the processes ran for, then use the tools in the #FlameGraph repository to convert the text output to an SVG.
-
#FreeBSD30 timeline - a few things that happened during these 30 #FreeBSD years:
#UNIX #BSD #MULTICS #1BSD #2BSD #42BSD #43BSD #386BSD #ipfw #FreeBSDCon #BSDCON #ports #jails #FreeBSDFoundation #kqueue #EuroBSDCon #CoreTeam #AsiaBSDCon #BSDCAN #pf #OpenBSD #ZFS #DTrace #VNET #Capsicum #Cheri #Poudriere #clang #llvm #subversion #OpenZFS #git
-
It's a pity that Mastodon lacks quote tweets, but in addition to the post I boosted yesterday, I just wanted to say my own thanks for #DTrace reaching 20.
I use it all the time, and struggle when working on other systems that do not have it. Only yesterday I wrote a quick script to verify a code path that lead to this fix: https://github.com/NetBSD/pkgsrc/commit/def71d1c1f988e3a2cf3bb86716a2937ce6f48fc.
It has found significant performance wins in #pkgsrc and #pkgin, fixed numerous bugs, and helped me understand systems.
Thanks @bcantrill and @ahl!
-
As part of my recent #pkgin work I came across another optimisation for "pkg_admin rebuild-tree" using #DTrace that makes it a further 12x faster on my test system.
http://mail-index.netbsd.org/tech-pkg/2023/08/07/msg027952.html
This is in addition to the 13x speedup I committed a few years ago:
https://gist.github.com/jperkin/98550d5bd07f4179ebfeea825fc3ec20
-
After some more #DTrace analysis and code reading, found another easy optimisation win in the pattern matching code, for a further 2x speedup.
Next up some database optimisations which will hopefully get us to around 8x combined speedup, as well as helping with future development work.
-
After maintaining #pkgin for a few years, there's still a bunch of low-hanging fruit for cleanup and optimisation.
While working on SUPERSEDES support, I was curious as to why the dependency calculation takes so long.
Re-ran it with #DTrace, saw break_depends() taking up 55% of the runtime. What does it do?
Turns out, nothing. Various changes over the years, notably refresh support, have made its calculations obsolete.
Nice easy 2x performance win and much easier-to-understand code path.
-
So things are getting bad, and #cloud is being considered more complex to manage and more costly.
I have to admit that while I really believed in the cloud, the problems that my customers were facing started to break my beliefs.
Ops costs increased, because they still hired the IT guy, but now a #SiteReliabilityEngineer costs way more than a traditional #SystemAdministrator, and often he has an #infra knowledge way inferior, since everything is not in house anymore. Everything is hosted and you don't access to the technology internals anymore.
Also, they are dealing with a level of complexity unheard of.
Before the cloud there were only physical and virtualized layers of complexity.
Now we also have then #containerorchestration and #servicemesh layers.
We moved from putting a stupid #Java class in classpath to run #JBoss with the #APM profiling to running #Prometheus, #Grafana, #Jaeger and #Kiali to obtain the same level of informations.The only good thing is that now in Linux engineers are using #eBPF to profiling network and software execution.
But this is NOT innovation at all. You could do that already with #DTrace in #Solaris and #FreeBSD with way less complexity, and the same level of profiling since #Java, #Ruby and lot of frameworks included its probes to support advanced tracing.
Apple's #macOS used to provide a good #DTrace frontend called #Instruments that was really cool to work with.https://world.hey.com/dhh/why-we-re-leaving-the-cloud-654b47e0
-
While we could add a bunch of CHECK_PORTABILITY_SKIP entries to skip these files, I prefer to fix things properly, and so have committed a general fix for this which should speed up most packages.
https://github.com/TritonDataCenter/pkgsrc/commit/df1af49c2953c235c8af501a53f13cdfa5bbf91c
Nice reduction in runtime from 86 seconds down to just 1!
If anyone has access to really old systems and are able to tell me if they do not have "read -n" then that'd be hugely appreciated, though it's likely we'd bootstrap #mksh on them anyway.
-
The year is 2024 and bpftrace crashes when there's an extra whitespace.
bpftrace is DTrace 2.0? DTrace my ass. I've never seen DTrace crash because of an extra whitespace.
Hell, I've never seen DTrace crash, ever.
-
Today I used #bpftrace on a #Linux system to investigate a performance issue. Really took me back to my days working on #Solaris at Sun Microsystems. I used #dtrace quite a bit at that time, and it's good to see the same good ideas thriving in Linux.
I worked in #AIX development at #IBM for many years, and I was always a little sad that people didn't use #probevue more. Perhaps this is because AIX has a good system trace infrastructure and the culture grew up using that. Hard to say.
-
While we could add a bunch of CHECK_PORTABILITY_SKIP entries to skip these files, I prefer to fix things properly, and so have committed a general fix for this which should speed up most packages.
https://github.com/TritonDataCenter/pkgsrc/commit/df1af49c2953c235c8af501a53f13cdfa5bbf91c
Nice reduction in runtime from 86 seconds down to just 1!
If anyone has access to really old systems and are able to tell me if they do not have "read -n" then that'd be hugely appreciated, though it's likely we'd bootstrap #mksh on them anyway.
-
While we could add a bunch of CHECK_PORTABILITY_SKIP entries to skip these files, I prefer to fix things properly, and so have committed a general fix for this which should speed up most packages.
https://github.com/TritonDataCenter/pkgsrc/commit/df1af49c2953c235c8af501a53f13cdfa5bbf91c
Nice reduction in runtime from 86 seconds down to just 1!
If anyone has access to really old systems and are able to tell me if they do not have "read -n" then that'd be hugely appreciated, though it's likely we'd bootstrap #mksh on them anyway.
