Gave my self-hosted Qwen 3.8 Flash Next, running a pi agent, attached to the billion-context's session store and an LLm wiki, a very large project to chew on, a ton of context, some fundamental engineering principles, and let it rip.
It's been 18 hours. I went to check its reasoning.
Right now, it's thinking about the persistence layer. It's been weighing those engineering principles against a few short dialog and nudges **that I gave it last night** in a clear attempt to align well with my engineering philosophy.
It has high level guidance like:
- adhere to SOLID principles
- use TDD
- red, green, refactor
- the best code is the code that you don't have to write
It seemed to begin from a philosophy of minimizing dependencies. On one hand, interesting. On the other, I have remained skeptical that it may be choosing to write code instead of adopt because it is more trained on writing.
When discussing the database seeding code, I encouraged it to look for prior art. That aligns with not having to write code. However, it had just written a 120-line database seeder. On discussion, we agreed that 120 lines of robust enough code is better than introducing the several layers of dependencies that would come from any of the extra tools we would add for seeding.
What fascinated me, 18 hours in, when I checked on it is this: it was noting and weighing the tension between "not writing code" and "prior art" versus "keeping dependencies lightweight".
Having previously fed it an inclination toward PoEAA patterns, particularly the Data Mapper, Table Data Gateway, and Repository patterns, and its recognition hat it is working in a highly normalized RDBMS, it has been consistently turning its nose up at adopting SQLAlchemy. It pattern-matched its way to my taste for non-magical code and a distaste for the Active Record pattern!
The models themselves evidently only need to be just so capable. The tooling they have at their disposal, and how much they are nudged to use them, seems to make an astonishing difference.
I've done some light code review. It's written slightly heavy but not hefty classes. I'm sure I could find some trivial refactors into private methods; it could follow SRP better but it's not bad. It made a few obviously silly choices, like hard coding some not-quite-but-almost static lookup tables. We had a little chat and then it adapted. It learned from me that these tables would occasionally get new data, a little at a time. It realized that changing the code at runtime is undesirable and that changing a hard-codes db schema at runtime is even worse. It then took my advice to extract this data into lookup tables with the understanding that new document types would cause it to grow these tables.
Disclaimer: I'm only a believer in the AR pattern for low complexity databases. I've never seen it scale well. Also, admittedly, I rather suck at SQL. Hell of a combination.
All of this for about $1.25 of electricity (so far) running on hardware I already own less the 64gb to 128gb DDR4 upgrade I made a few days ago.
This is all using a community-developed runtime that lends itself to solid performance despite using GPU, CPU+RAM, and even SSD to a limited extent: https://github.com/Niko1221/Strata.
I'm increasingly tempted to sell some of my still-overpriced (thanks to shortages) former gaming hardware to fund ~2/3 of a slightly janky using older retired data center GPUs. It'd be a helluva thing to go from 24gb to 128gb VRAM with GPUs of an older architecture but a commensurate speed!
You may wonder why.
Strata is amazing but it parallelizes for shit. One request at a time: terrific! Two is not so good. Just forget about 3 or more. That overhead of the bandwidth differences between VRAM, PCIe, RAM, and SSD catch up with you quickly. Any of those mediums gets clogged up and the whole machine slows down fast.
Now, if you get that whole machine into VRAM? LLM goes BBBBBRRRR! Think parallelizing 4 requests (or more) with *better* performance per request than Strata. That would be 4 requests just as capable as my currently sole agent.
It would also be about 3x the electricity. And idle at about 5x the electricity...so there is that.
This is all the while leaning on a cloud-hosted model that certainly collects my data. Instead, I own the data. And that data goes into the machine to make it more capable (so long as it doesn't poison itself by creating bad examples that would hurt the feedback loop) and more aligned with my perspectives.
DISCLAIMER: if you are inclined to attack or criticize my use of self-hosted AI, please don't bother. You will only serve to aggravate yourself and maybe me. I've reflected deeply on my choices here and their moral implications. I'm not going to argue about them with some rando. However, if you wish to engage in a *civil* philosophical discourse, I am *always* open to this.
After years of disillusionment and depression, I've come back to seeing how technology, wielded responsibly, can still be a force for good. At some point, optimism has to be a choice—not chosen blindly but partly because the alternative is so much more agonizing. I can't live that way anymore.
