#krabby — Public Fediverse posts

An update on Krabby: There's a Zulip chat now! I wrote a little status update on the blog: https://bal-e.org/speed/krabby/hi-zulip-also-licenses/. Take a look!

#programming #rust #compilers #krabby

An update on Krabby: There's a Zulip chat now! I wrote a little status update on the blog: https://bal-e.org/speed/krabby/hi-zulip-also-licenses/. Take a look!

#programming #rust #compilers #krabby

An update on Krabby: There's a Zulip chat now! I wrote a little status update on the blog: https://bal-e.org/speed/krabby/hi-zulip-also-licenses/. Take a look!

#programming #rust #compilers #krabby

An update on Krabby: There's a Zulip chat now! I wrote a little status update on the blog: https://bal-e.org/speed/krabby/hi-zulip-also-licenses/. Take a look!

#programming #rust #compilers #krabby

The first post is up! You can read about my adventures with identifier interning in 2025: https://bal-e.org/speed/krabby/interning-in-2025/.

#programming #compilers #krabby

It's been one year and one week since I started my very-very-WIP Rust compiler, Krabby. While I used to blog about it relatively frequently, the last eight months slipped by somehow; but I was working on it the whole time! I wrote a little catch-up post (https://bal-e.org/speed/krabby/a-year/) highlighting the big adventures you might have missed out on; expect follow-up posts that dive into each one in detail.

#programming #compilers #krabby

For Krabby (my very-very-WIP Rust compiler), I did find a sensible use case for my parallel name resolver; a query on thread A ("look up core::iter::Iterator") can depend on a query being processed by thread B ("parse core/iter.rs"). Small atomics + a concurrent hash table lets me detect such conflicts, but I don't want to block the thread when this happens (parsing can take a while); instead, the hash table lets thread A pass on its work to thread B, to execute when its query finishes. Thread A will simply report "the query would block, come back later" and move on to other stuff. Once thread B is done, it will re-enqueue the blocked work. I'm excited to implement this, it's going to be so cool!

#programming #rust #compilers #krabby

My resolutions for this year are:

• name
• dependency
• macro, if you want to count it
• maybe a bit of type if I'm feeling cheeky

#programming #shitpost #krabby #rust #compilers

It's time for a little update on my very-very-WIP Rust compiler, Krabby! I haven't posted about it much recently, but I've been working at it behind the scenes. Slowly but surely, name resolution is coming together.

• I've figured my approach for local nameres, and have all the right infrastructure set up. Global nameres was trickier, but I have a satisfactory plan for the underlying database. You can check it out at https://codeberg.org/bal-e/krabby/pulls/18.

• A random musing about parking_lot somehow led me to rethinking the way tasks are defined and distributed across Krabby. Implementing it is going to take some time -- expect a blog post!

• My next step is dependency resolution, so that the name resolver will have more data to play with. I'm just going to read Cargo.lock and resolve feature flags for now.

I've been having a lot of fun, and I'm excited to see where things go in 2026 (and in time for RustWeek!)

#compilers #krabby

How can I ever explain the joy of sitting at my desk before the morning light, pulling out three whiteboards covered (on both sides!) with notes, and asking myself "what am I going to do with my compiler in the next 8 hours?".

#compilers #krabby

My current side-side project is phonebook, a fast multi-threaded identifier interner for Rust. I've been banging my head against the wall for the last few weeks, trying to solve concurrent reallocation; and for the last week, I've been taking a "break" by trying to build a reader-writer split API for rustls. And while phonebook is still not feeling very appealing, the project I needed it for -- Krabby -- has plenty of other work to do. So I spent today catching up on it. I hope I can find the inspiration to tackle that concurrent reallocation issue again.

I did end up building some nice stuff; a friend had contributed progress bar support a while back, and I've revamped it using ratatui. It also includes a lot of cool detail now. Have a look!

I expect the next few days to be pretty busy, so I plan to pick up phonebook next week; I'll focus on Krabby and rustls until then.

#programming #rust #compilers #krabby

#pokemon #pokemonoftheday #krabby

Here's some simple math to help calculate performance changes when you're profiling some code. Suppose you're rewriting some function F within your program -- before the change, perf told you it took up N% of your runtime, and after the change it's M%.

• The overall speedup is (100 - M) / (100 - N).
• The speedup of the function is (1 - 100 / M) / (1 - 100 / N).

I'm defining "speedup" here as the ratio between the old and new runtimes, i.e. how many times faster the new code is.

For example, I was optimizing my very-very-WIP Rust compiler, and I fixed a nasty performance bug in identifier interning. According to perf, interning went from 53% of my total runtime to 12%. By the above formulae, that implies a ~1.87x speedup of total runtime, and a ~8.3x speedup of interning itself.

#rust #programming #optimization #krabby

One of my issues with Rust is the "weakly typed" nature of conditional compilation. The compiler can't tell you whether your code will compile under all combination of cfgs (including feature flags), or suggest whether you missed a cfg somewhere.

I spent an hour trying to design a name resolution algorithm that is cfg-independent (i.e. it returns something to which any combination of cfgs can be applied to get a resolved AST), and I don't think it's practical. cfgs are too flexible; conditionally-compiled declarations can override each other in weird ways. Such an algorithm could have provided those "strong typing" features for cfgs, so I don't think there's much hope for them.

At least for Krabby (my very-very-WIP Rust project), I'm going to write a standard cfg-dependent name resolver. I have a pretty thorough design for it already.

#rust #compilers #krabby

Update on my very-very-WIP Rust compiler: I'm in the process of my third rewrite of the task queue system internals, this time as a separate crate that I'll publish on crates.io. Krabby isn't the only highly parallel, CPU-bound application where work is divided into small, synchronous tasks, so I may as well make this useful to others. I hope to get a blog post out over this week.

#rust #compilers #krabby

Fun note about my very-very-WIP Rust compiler: almost 9% of the runtime is spent in calling std::io::stdio::_print (i.e. println! and friends), even when I'm redirecting stdout to /dev/null. While a little bit of that time is spent in contended locks, because I'm printing from many threads, I'm not even calling it that often! Lesson learnt: watch out for string formatting if you're writing high-performance Rust code.

Edit: Look at my reply for some concrete numbers.

#rust #performance #krabby

I tried out Samply for profiling my very-very-WIP Rust compiler, and I loved the results. I tried both samply record and perf record + samply import; I've attached an image from the latter. It's so cool! You can see Krabby spending a lot of time on I/O at the start of compilation, while it's loading source files, and quickly moving into lexing (where it currently spends ~80% of the runtime).

About 20% of the runtime is spent in the kernel, and about half of that is spent loading source files and directories. The other half is, for the most part, just the kernel waiting on contended mutex locks; optimizing these away will be quite easy, and should speed up compilation by 10%.

There are a few potential approaches for optimizing the I/O for loading source files, primarily mmap(2) and Linux's io_uring. It would be really interesting, but there are (a lot of) higher-priority things I have to attend to first.

#rust #compiler #krabby

I'm in the process of implementing a new task queue architecture for my very-very-WIP Rust compiler, and I seem to be achieving some gorgeous speedups over crossbeam-deque (which, to be fair, wasn't really fit for my use case). My task architecture is capable of prioritizing tasks, while still implementing work-stealing. I previously had ~20% of the compiler runtime stuck in the task queues; it's now around 7.5%. The compiler's become about 25% faster overall (~650ms => ~450ms for my main benchmark target, over 4x2 threads).

There's still further improvements waiting: the 7.5% is spent in futexes, because the current data structures use locking, and I have a sketch of a lock-free version to try out. Near the end of compilation, threads are going to sleep and not being woken up correctly. Even so, there's more optimizations I had planned that are yet to come, such as using a better global data structure to reduce contention even further.

Blog post coming soon (ETA 1 week?).

#rust #compiler #krabby

A little bit later than planned, but I've published a blog post on my work for my very-very-WIP Rust compiler over the last two weeks. There isn't anything much that I haven't already posted here, actually. TL;DR: I wrote a fast TOML parser, I have a slightly nicer lexer, krabby build terminates now, and I'm going to spend this week (and possibly beyond) optimizing concurrent data structures. It's going to be fun!

https://bal-e.org/speed/krabby/for-context/

#rust #compilers #krabby

Update on my very-very-WIP Rust compiler: after a few weeks of banging my head against the wall, I've managed to implement the critical part of my highly-parallel task architecture, a concept of context for expressing the control flow between compilation steps. It's an important but fairly abstract change: the most visible effect is that my compiler can now determine when to exit, instead of running forever :P

This makes it way easier to benchmark and profile things, so I've set out to fix some silly bottlenecks (e.g. 35% of the runtime was stuck interning identifiers). Now, the compiler's able to lex about 1 million lines of code per second, per thread; I'm hoping to slowly drag that number up to 10 million.

I'll write a blog post sometime today or tomorrow, and I'll link that here when it's done. Overall, I'm making good progress!

#rust #compilers #krabby

Progress update on the very-very-WIP Rust compiler: I gave up on doing fancier TOML parsing and I'm back to writing a lexer and a parser. Some currently open questions:

• How to represent matched meta-variables that are output by a macro expansion, when they fall within a new macro invocation and thus fall within token streams?

• Is it possible to represent sequences of token trees more efficiently during macro expansion, so that tt-munching macros can be executed in linear time?

• Should macro invocations in an AST be expanded in-place from multiple threads, or should expansions within a single AST be collected together to be executed on a single thread?

• What granularity (per reference, per item, or per module) is the most appropriate for incremental name resolution?

These don't affect the ongoing lexer or parser implementations, but I'm giving myself plenty of time to think about them so that I'm reasonably confident heading into nameres.

#rust #compilers #krabby

Final update on the TOML shenanigans: it works! My prediction regarding the latency was correct, average runtime for the entire "find and load a Cargo manifest" task has dropped from 180-200µs to just 20-21µs. There's definitely some measurement overhead to tracing (which I will probably address with something custom later), but given that these times include I/O, I think I've achieved a clear 10x speedup over the cargo_toml crate.

Most importantly, my trace logs no longer start with 50+ lines of "thread X is waiting for a task to do"; every thread is getting work within ~50µs. I wouldn't have spent quite as much time on this optimization if the logs weren't bugging me so much.

You can find the updated code here. The main reason for the speedup is probably the simpler control-flow and the reduction in memory allocations.

Pending any other tangents, my next step is to get back to parsing source code.

#rust #compilers #krabby

Looks like my homegrown TOML parser is coming along well.

$ cargo run --example parse-toml < Cargo.toml
Ok(CloseSection([]))
Ok(Set([], "workspace", Table))
Ok(Set([Field("workspace")], "resolver", String("3")))
Ok(Set([Field("workspace")], "members", Array))
Ok(Push([Field("workspace"), Field("members")], String("src/cli")))
Ok(Push([Field("workspace"), Field("members")], String("src/cargo")))
<...>
Ok(CloseArray([Field("workspace"), Field("members")]))
Ok(CloseSection([Field("workspace")]))
Ok(Set([Field("workspace")], "package", Table))
Ok(Set([Field("workspace"), Field("package")], "authors", Array))
Ok(Push([Field("workspace"), Field("package"), Field("authors")], String("arya dradjica <[email protected]>")))
Ok(CloseArray([Field("workspace"), Field("package"), Field("authors")]))
<...>

Now I just need to teach it booleans and the unspeakable horrors of floating-point numbers.

#rust #compilers #krabby

My very-very-WIP Rust compiler currently has one mode of operation, where it checks the Cargo projects in all the directories specified on the command-line. The first step, for every project, is to find and load the manifest file. When you're only working with one project (even if it's a workspace), nothing can happen until that manifest is loaded.

I'm currently using the cargo_toml crate to parse the manifest, but I've noticed that it's a bit too slow for my use case. Opening the manifest file takes ~4.5µs, and reading its contents into memory takes ~4.8µs. But parsing the file -- an operation that needs no I/O -- takes a whopping 174µs! That's about 10 times longer than I expected.

The latency is probably due to the number of allocations performed, overhead due to serde, or inefficiency in the parser itself. I'm going to try writing a simple streaming TOML parser to make do instead, with a target latency of 5µs. Let's see whether that's achievable.

#rust #compilers #krabby #optimization

Somebody pointed out that my very-very-WIP Rust compiler's attempts to increase data parallelism also directly improved its incrementality. By being very specific about the dependencies of a task (e.g. I can find and load the source files in a Cargo package without resolving the crates it depends upon), I'm also ensuring that I don't re-run tasks (which I would do when their dependencies change) unnecessarily. I haven't given much thought to incrementality thus far, but it's a nice affirmation that I'm on the right path.

#rust #compilers #krabby

I have a basic proof of concept which demonstrates my very-very-WIP Rust compiler's task architecture. It's quite different from rustc's query system, and I'm really curious to see how it will hold up in performance! You can try it out yourself, and in the meantime, I'm going to be preparing for implementing name resolution.

https://bal-e.org/speed/krabby/proof-of-concept/

#rust #compilers #krabby

Updates on my very-very-very-WIP project to write a Rust compiler: I have a high level architecture planned out, and I know what I need to do for the next few weeks. I was going to work on it some more today, but I think I'll just post this and go for a walk instead.

https://bal-e.org/speed/krabby/the-architecture/

#rust #compilers #krabby

I'm writing a Rust compiler, for some reason! I've written a bit about why this is happening, and a first post about local name resolution is up now.

If that piques your interest: https://bal-e.org/speed/krabby

#rust #compilers #krabby

Neue Jahreszeit „Duales Schicksal“ für Pokémon GO enthüllt

Details zur nächsten Jahreszeit im Spiel wurden vorgestellt.

Zur News: https://news.bisafans.de/11129

#Event #Lapras #Hoopa #Regice #IOS #Android #Reshiram #Regirock #Registeel #Zekrom #Machollo #Saison #Kyurem #PokémonGO #Krabby #Necrozma #Abendmähne #Morgenschwingen #RaidTag #Mortipot #Fatalitee #CryptoRaid #DynaKampf #DualesSchicksal #DualDestiny

Neue Jahreszeit „Duales Schicksal“ für Pokémon GO enthüllt

Details zur nächsten Jahreszeit im Spiel wurden vorgestellt.

Zur News: https://news.bisafans.de/11129

#Event #Lapras #Hoopa #Regice #IOS #Android #Reshiram #Regirock #Registeel #Zekrom #Machollo #Saison #Kyurem #PokémonGO #Krabby #Necrozma #Abendmähne #Morgenschwingen #RaidTag #Mortipot #Fatalitee #CryptoRaid #DynaKampf #DualesSchicksal #DualDestiny

Neue Jahreszeit „Duales Schicksal“ für Pokémon GO enthüllt

Details zur nächsten Jahreszeit im Spiel wurden vorgestellt.

Zur News: https://news.bisafans.de/11129

#Event #Lapras #Hoopa #Regice #IOS #Android #Reshiram #Regirock #Registeel #Zekrom #Machollo #Saison #Kyurem #PokémonGO #Krabby #Necrozma #Abendmähne #Morgenschwingen #RaidTag #Mortipot #Fatalitee #CryptoRaid #DynaKampf #DualesSchicksal #DualDestiny

Gestern ein shiny krabby, vorgestern ein hundo Balgoras
#shinyluck #shiny #pokemongo #pokemongoapp #krabby #balgoras #hundo #tyrunt

Gestern ein shiny krabby, vorgestern ein hundo Balgoras
#shinyluck #shiny #pokemongo #pokemongoapp #krabby #balgoras #hundo #tyrunt

Gestern ein shiny krabby, vorgestern ein hundo Balgoras
#shinyluck #shiny #pokemongo #pokemongoapp #krabby #balgoras #hundo #tyrunt