From aristocracy to ‘assetocracy’ – how wealth has transformed the House of Lords https://theconversation.com/from-aristocracy-to-assetocracy-how-wealth-has-transformed-the-house-of-lords-293492
Hundreds of peers enjoy extraordinary wealth, new research has found.
---
56 uses from 55 accounts #toplink
Search
160 results for “hund”
-
-
Geography Of Groundwater Use And Depletion In Internationally Shared Aquifers
--
https://doi.org/10.1073/pnas.2619357123 <-- shared paper
--
https://unu.edu/publication/why-groundwater-use-location-matters-transboundary-aquifers <-- shared #UNU policy brief
--
https://www.propublica.org/article/water-aquifers-groundwater-rising-ocean-levels <-- shared technical media article using spatial approach
--
https://doi.org/10.1038/s41586-023-06879-8 <-- shared paper
--
@kyle Davis | Associate Professor at University of Delaware
“More than 1/3 of the world's groundwater-irrigated cropland depends on aquifers that cross national borders, yet few countries have agreements on how to share them.
In a new… paper [link above], [the authors] looked at all 426 known transboundary aquifers. [They] show while they not being drained faster than aquifers lying within a country's border, pumping in many transboundary aquifers tends to cluster near the borders.
In these cases, one country's water use is much more likely to affect its neighbor, and those are the places where cooperation matters most…”
--
“SIGNIFICANCE: Transboundary aquifers supply irrigation water to feed hundreds of millions of people, yet formal cooperative arrangements remain rare. [The authors] assess groundwater depletion and demand across internationally shared aquifers and find no greater average depletion than in comparable domestic systems. Instead, groundwater depletion and use are more strongly concentrated toward international borders and vary substantially among systems. This distinction matters for governance: many shared aquifers have little groundwater demand near borders, while others concentrate demand where cross-border impacts are more plausible. A spatial typology characterizes these contrasting configurations and shows how the geography of groundwater demand shapes the relevance and options for international coordination..."
#groundwater #depletion #subsidence #waterresources #watersecurity #transboundary #borders #aquifer #irrigation #farming #agriculture #foodsecurity #international #global #pumping #overuse #water #watermanagement #extraction #agreements #coordination #waterrights #governance #GIS #spatial #mapping #spatialtypology #geography #dyad #typology #spatialanalysis #spatiotemporal #policy #planning #model #modeling
@United Nations University Institute for Water, Environment and Health (UNU-INWEH) -
Report: Mass UK protests are planned for Oct 31 against the ban on Palestine Action.
Outside the US embassy in London & the consulate in Edinburgh, protestors will be asking if Trump pressured the UK to outlaw the group.
Days later, the UK Supreme Court will hear an appeal by Palestine Action's Huda Ammori against the ban.
More than 3,850 have been arrested since the ban.
#UKPol #EUPol #USPol #palestine #PalestineAction #Gaza #GazaGenocide #news .
-
From the Anthropic prospectus, first page:
> For the first time in history, intelligence is something humans can build. The cognitive capability to reason, analyze, synthesize, decide, and create is producible at scale and accessible on demand.
"intelligence is something humans can build" - unlike a human learning a skill or developing knowledge?
"producible at scale and accessible on demand" - unlike the labour market of the past hundreds of years?
-
#Brasil #Bolsonaro no reconoce la #derrota de las #elecciones https://orf.at/stories/3446187/ Alrededor de 159 millones de brasileños fueron llamados a las elecciones - y muchos de ellos se han beneficiado del éxito social y económico Del Partido de los Trabajadores de Izquierda Pt de #Lula. Bajo su gobierno, el #empleo se hundió a un mínimo histórico, la #pobreza declinó, el salario mínimo aumentó. Además, el Parlamento decidió introducir gradualmente la semana de cinco días. La #deforestación en la región #amazónica, la selva tropical más grande de la Tierra, también se desaceleró.
-
"Why hasn’t PHP2Shell turned into mass exploitation?
Because no public exploit works reliably on a default PHP setup, and attackers are still trying to build or find one. As Sansec puts it after testing the code circulating online: “The public exploits do not work against a default PHP installation.” VulnCheck counted hundreds of proof-of-concept repositories on GitHub, very few of them legitimate; one got past a standard memory protection, but only on PHP 8.4.26 in one specific setup. Reliable exploits exist, but not in public, and VulnCheck reports no confirmed successful exploitation in the wild.The CrowdSec data from October 21 to 25 shows attackers testing, not harvesting:
Half of the 1,187 sources sent a single request in five days, and 82% sent three or fewer.
Known PHP2Shell techniques need several requests against the same server, so a real attempt repeats itself. Only 24 sources ever sent ten or more requests to the same sensor.
78% of the sources that sent a user agent posed as a web browser."
#php2shell #IT #ITSecurity #PHP -
Tokens Over Hires
In the middle of September I ran ccusage against every Claude Code session on my laptop since May. It walks the local logs, counts the tokens, and prices them at what the API would have charged. I expected a big number. What came back was about twenty billion tokens and just under fourteen thousand dollars, across every project I’d touched in four months, with most of it going to Pepper, which is now open for signups, so go check it out. Nineteen billion of those tokens were cached reads.
I sat with that for a minute. Fourteen thousand dollars is real money. At my current pace it works out to roughly thirty thousand a year. I could triple it and spend a hundred thousand dollars a year on tokens, and that still isn’t one engineer. In exchange, Chris and I have shipped more code than I’ve shipped in any stretch of my career. I wrote in September that the two of us merged 657 pull requests in our first thirty days with a budget instead of a platform team. When I put the ccusage number in those terms on a panel, it came to about a dollar fifty a pull request.
That number is what my laptop would have cost at API prices, and it isn’t what we actually pay. Here’s the whole bill for both of us. Two Codex subscriptions, one each, are $200 a month. Anthropic is $550 a month: a Max plan for me and Chris, plus two Vycari team seats, one for Chris and one that runs our automation. I have an Ollama subscription at $200 a year. Greptile is about $1,500 a month. All in, that’s a little under $2,300 a month, or about $28,000 a year, for both founders and every pipeline we run. It still isn’t one engineer.
The obvious conclusion is the one a friend reached when he heard the figure: if code is that cheap, just get more tokens. I thought so too, for about a day. Then I looked at what was actually slowing us down, and the token budget wasn’t on the list.
The Constraint Is Attention
Every agent I run needs things from me that tokens can’t buy. It needs a plan approved before it starts. It needs its pull request read when it finishes. When it hits something it can’t decide, it escalates, and somebody has to be there to answer. None of those steps is expensive on its own. The problem is what happens when I run enough agents that the steps overlap.
When I run too many agents, the cost goes up a little and the risk goes up a lot. The biggest thing that’s slipped past us so far was how we managed database connections. It got out of hand, a little at a time, while my attention was on other pull requests, and eventually it caused an outage. Chris had to roll forward while I was out of town, and I spent the following week building a proper database abstraction layer. He put it more bluntly than I would have: the thing that kills us is splitting our attention. That’s when stuff slips through.
When people ask how we decide what to spend, I give them the order. Optimize for human time, because that’s the scarcest thing we have. Then money, because that’s the second scarcest. Then everything around agent time. “I can waste agent time all day. I don’t care.”
So the number that limits us is how many agents one person can supervise well at the same time. I don’t have a precise figure for that ceiling. I know I’ve been over it, because I can feel the difference between a day when every pull request got a real read and a day when I was skimming to keep up.
Another friend told me recently about engineers at his company running up bills in the thousands to build simple internal copies of SaaS tools. His reluctant guess was that they just aren’t as good at using agents yet, and that it’s a skill people have to develop. I don’t know what they’re doing differently. It doesn’t sound like token maxxing, which I’ve never really understood either. I’d love to sit with engineers who run up giant bills and just watch. My guess is they’re using the agents inefficiently, and that what’s missing is supervision, which a bigger budget won’t fix.
Building for More Agents per Person
Almost everything I’ve built since the summer has been an attempt to raise that ceiling, and almost none of it has been about spending less.
The pipeline I described in The Agent That Never Merges is the first piece. Agents open pull requests continuously and never merge them, which means the supervision cost per pull request is bounded at one careful read. The pipeline absorbs the rest: triage, planning, the first round of review comments, the retries. I only see the work once it’s in a state worth my attention.
The second piece is where the work lives. I wrote about pulling Pepper apart into small, bounded repositories to keep CI fast and give agents less to load. It mattered just as much for my attention. An agent working in a repository with one job produces a pull request I can evaluate in one sitting.
We’re up to fourteen repositories now. Last week I moved our iOS app out of the monorepo into its own. That was 150,000 lines of code, and nobody noticed except Chris, who had to wait a day to land his next batch of iOS changes. Agents make refactors that size easy. Having the CLI and the iOS client in their own repositories also forces us to keep good API definitions between the server and its clients.
The third piece is delegation inside the agent layer itself. I run two Claude Opus 5.5 orchestrators on medium effort. One is the UTL, for uber tech lead, and its job is to spawn subagents that build features. The other is what I call the focused worker, and its job is working with me. It catches the manual tasks that come out of the UTL’s pull requests and keeps them in a ledger. Then we work through them together: systems changes, bringing up new platforms, migrating data, anything that needs my level of access and oversight. Everything else goes to a subagent under the UTL, Opus for design docs and code reviews, Sonnet for implementing a feature from a design doc. Every one of these agents can consult a model a tier above it as an advisor, and they all share a wiki for notes.
Anthropic is now advertising much the same pattern with Claude Projects, and I’m happy to see others arriving at the same answer. Chris has been building a loop that babysits open pull requests through review until they score well enough to be worth a human’s time. Each layer like that turns a job I used to do by hand into something I check. The ceiling goes up one delegated task at a time.
The Token Budget Was the Easy Problem
There was a week in September when the meter really was the constraint. We ran through our free credits and paid about $120 in overage. Then AutoDev ate the whole Vycari quota, and I did server cleanup by hand because I couldn’t do anything else.
Most of that was self-inflicted. I’d started with Fable as the orchestrator, with no cap on context, and burned a week’s quota in three days. The setup above fixed it. The orchestrators run on medium effort, and the cheaper model does the implementing. Opus answers the reviews because Sonnet is too literal about them, and a literal reading turns two review cycles into twelve. Every agent auto-compacts at 200K tokens, because Anthropic charges more for context past that point. With that setup, and the work spread across parallel sessions, I find it hard to spend all of my quota. My goal now is to hit 99% of it every week just before it resets. I keep a list of research projects, and if I have quota left 24 hours before the reset, I spin one up. That isn’t the same as maxing out tokens. I’m not trying to use as many as possible. I’m trying to use all the ones I’ve already paid for.
There’s also a second account on the same machine. Claude Code reads a
CLAUDE_CONFIG_DIRenvironment variable, so two shells with two config directories give you two accounts side by side. The second one runs the AutoDev and code review pipelines, and those use up its quota every week, so it isn’t a fallback for my own sessions. When I do run short, I supplement with other labs’ plans. Every one of those fixes took an afternoon or less. None of the supervision problems above has had an afternoon fix.Where the Money Actually Goes
While I had the numbers in front of me, I looked at where development spend actually lands, because it isn’t where I expected either.
Development tokens currently run about ten times what the product itself consumes, even with real people using it. Building the thing costs more than running it. That won’t last. The feedback from our private beta has been really good, and we now open Pepper to a limited number of new signups every day, so product usage is rising quickly.
Most of what we actually pay goes to code review. Subscriptions are the cheap way to buy tokens if you’re willing to hit limits, so they’re the smaller line. Greptile reviews every pull request we open, and at about $1,500 a month it’s our biggest single expense. We both think it’s worth it, because it’s the layer that lets the never-merge pipeline run without me reading every first draft.
In October I tested that. I spent a few days building a cheaper reviewer to replace it. I came out the other side saying, “I think we should just pay for it because I can’t do it cheaper.” What I could change was how often we call it. The agents kept asking for a re-review before they’d fixed anything. Because every repository shares a common dev environment, the fix went in once: our pull-request skill now has agents run
/simplifyand then/code-reviewbefore they’re allowed to open a pull request. On many of our repositories, our own reviewer also does the first round and only then calls Greptile. My goal is a pipeline where Greptile’s reviews come back with nothing to say. We’re not there yet, but we get a little better every week.I’d rather pay for review than have a second person do it, and the reason isn’t salary. The person would need supervising too, because review is a job I’d have to check, not one I can hand off.
What a Hire Is For Now
Five years ago, if I’d been getting a product like this ready for market, I’d have been hiring twenty to fifty engineers. The work would have needed that many hands, and most of what they’d have done was write code. Today the code is the cheap part. If I bring someone in to write code, I’ve added a person who needs plans approved and pull requests read, which is the exact resource I’m short on. A hire whose job is output adds coordination faster than it adds capacity.
I said on a podcast that came out last week that the code I’ve written would have cost about fifteen thousand dollars at API prices, and that I couldn’t hire engineers for that money. That’s true. The engineers I’d still want are a different kind from the ones I hired at Google. We need a few more people who can herd agents, not fifty.
The engineering hire that adds capacity is a different shape. It’s someone who can own a bounded service whole. Here’s the interface, here’s everything it needs to do, let me know when it’s done. That person brings their own supervision. They run their own agents, read their own pull requests, and answer their own escalations, and the only thing that crosses the boundary between us is the contract. Supervision divides along the interface instead of piling up on one person. That’s a tech lead’s job, and it’s the only kind of engineering hire I can see that raises the ceiling instead of lengthening the queue.
That raises a fair question about how anyone grows into that job now, which I tried to answer in The Questions Are the Job. The other advice I give is to work on open source. I’ve met people who showed they could work really efficiently with agents through open source projects, Gemini CLI in particular, and some of them ended up being hired onto my teams. Open source has its own problems right now. But if you show up as a thoughtful engineer with features or bug fixes that are well reviewed, well tested, and bring value to the project, I think you’ll be welcome. That’s one of the reasons I decided to open source the Pepper CLI.
The first hire probably isn’t an engineer at all. I’ve said this a few times now, including on stage at a panel in September. Neither Chris nor I are go-to-market people, and I may hire one before I hire another engineer. I can create piles and piles of code, even relatively good code. What I can’t do is agentically acquire customers. (Did I mention that Pepper is open for signups now?) Someone in marketing came up after the panel and told me I’d validated her existence.
The Third Trend
I’ve watched this happen twice before. Open source developer tooling made it so a small team didn’t need to buy or build its own compilers, databases, and frameworks. Cloud infrastructure made it so that team didn’t need to own hardware or a data center. Each time, the number of people you needed to start something real dropped, and the skills that mattered shifted toward the people who understood systems well enough to compose what was suddenly free.
Agentic tools are the third one. Anyone with a good idea can now get from concept to production on a budget that would have been a rounding error in a Google planning cycle. But the constraint didn’t vanish. It moved, the same way it did the last two times, toward the people who understand systems design well enough to drive the agents and notice when something has slipped through. I wrote a year ago that management skills were going to carry over to working with agents. I didn’t expect to be proving it with a two-person org chart and fourteen thousand dollars of tokens.
This isn’t only a small-company problem. At Google it was hard to get more than a few changes in a day, because each one waited on code review. I’ve talked to engineers there who now do all their work in experimental directories that don’t require review, then port it back to their team’s repository at a pace the review process can keep up with. That’s what happens when you don’t manage for human attention. Productive people spend their cycles waiting on the process.
When other companies ask me what to change, my answer has become: drive until human attention is the scarce resource, then build for that. If you’re reading your own ccusage number and wondering whether the answer is more tokens, look at where your attention went first. For us the answer has been more interfaces. Those are the places where one person’s supervision ends and another’s begins, or where an agent takes over a job one of us used to do by hand.
#Agents #AI #Developer #Startups -
Report: Mass UK protests are planned for Oct 31 against the ban on Palestine Action.
Outside the US embassy in London & the consulate in Edinburgh, protestors will be asking if Trump pressured the UK to outlaw the group.
Days later, the UK Supreme Court will hear an appeal by Palestine Action's Huda Ammori against the ban.
More than 3,850 have been arrested since the ban.
#UKPol #EUPol #USPol #palestine #PalestineAction #Gaza #GazaGenocide #news .
-
After a few days at Eskdale it was time to head down to Lakes Entrance, via the Omeo Highway. Absolutely fabulous motorcycle roads, although a bit chilly at altitude.
AND, I enjoyed the privilege of an experience that may never happen again. Heading south on the Omeo Hwy the day after the Philip Island MotoGP has spilled hundreds (thousands?) of bikies heading in the opposite direction.
Was cool to see vastly more two-wheelers than cars for a change.
-
#CatastrophicFlooding on #NavajoNation and in #NewMexico
By Brenda Norrell, Censored News, Sunday, Oct. 25, 2026
"The Navajo Nation Police Department warned of rising water in the #SanJuanRiver in #Utah this morning at 5 a.m., as catastrophic flooding continues with helicopter evacuations on the Navajo Nation and far reaching flooding in New Mexico.
"New Mexico Gov. Michelle Lujan Grisham said on Saturday that there were hundreds of water rescues between Bloomfield on the Navajo Nation, and #FarmingtonNM.
"In #RockPointAZ, on the Navajo Nation, the tribe's all terrain vehicle reached stranded #Dine' families waiting for evacuation with food and water, but the need continues, especially for food for
livestock."Family members in Rock Point said today that Dine' are still stranded and need food, water and blankets. They need heavy duty trucks to reach isolated family members.
"Dine' on #BlackMesa in #Arizona struggle to reach their homes as dirt roads are washed away, and turned to seas of mud.
#NavajoTimes reported from #DennehotsoAZ on the Navajo Nation on Friday.
"Among the residents affected is Phyllis Yazzie, 74, who slept in her car after floodwater reached her home. Mary Lou Johnson, 102, was rescued by raft and transported by helicopter to the chapter house as flooding forced families from their homes."
Read more:
https://bsnorrell.blogspot.com/2026/10/catastrophic-flooding-on-navajo-nation.html#ElNino #ClimateChange #Navajo #Dine #ClimateEmergency #FlashFlooding #IndigenousNews #NativeAmericans #ClimateDiary #MutualAid
-
Eisenkot will "alle Israelis wieder zusammenbringen"
Zwei Tage vor der Parlamentswahl in Israel hat der führende Oppositionspolitiker Gadi Eisenkot bei seiner Abschlusskundgebung versprochen, die Spaltung des Landes zu überwinden. "Unser Ziel ist nicht, dass eine Seite die andere besiegt", sagte der frühere Generalstabschef am Sonntag vor Hunderten Anhängern in Tel Aviv. Er wolle (…)
https://www.sn.at/politik/weltpolitik/eisenkot-will-alle-israelis-wieder-zusammenbringen-art-680481
-
Artcraft, Mindscaper, et al: Can we really just vibecode creative software?
Over the past couple days, massive hype has built up around Artcraft, a collection of literal clones of Adobe’s CC, Microsoft Office and other apps, entirely vibecoded and open source. Artists and creative folks battered by subscriptions costing hundreds and thousands of dollars a year finally smell relief that’s not the poorly designed open source jank of yesteryear, and the project receives dozens of similarly vibecoded pull requests each hour. At the same time, I’ve been building Mindscaper, practically with the same goal, but without the hype. As such, I have some thoughts and bias on this subject. And probably typos, this is 100% human-written.
Background
Artcraft is developed by Learning Machines LLC, which appears to be a one-man company by Brandon Thomas. Thomas previously developed the deepfake services “vo.codes” and “fakeyou.com” under this company, and developed VTubing tech. Judging by the blurb on “the.storyteller.company”, he appears to believe in “controllable AI” trivializing (“democratizing”) art creation, but with Artcraft opted to primarily clone existing professional software, putting AI on the backburner (although an AI subscription service is available).
By a lucky coincidence, I founded my company Koytek Wattenberg Media at a similar point in time, with similarly limited success. One of my biggest clients became Audacity, at which I worked on “democratizing” audio creation, and wouldn’t you know I’m also into VTubers. I started my foray into vibecoding about half a year after Artcraft, focusing on Audacity-shaped audio and video editors, Soundscaper and Framescaper.
On technical matters
I have nothing new to say here that hasn’t been already said a billion times. Modern coding robots (I used Opus 5 and GPT 5.6 Sol mostly) are perfectly capable of making something that Just Works. They just happen to be lying, lazy bastards that will happily claim something exists when it doesn’t. Test-driven development principles keep them reasonably honest, though they occasionally do manage to find a way to write tests so unspecific that a pixel-by-pixel spectrogram in the Roseus color scheme and a spectrogram with much coarser buckets using a blue-to-yellow color scheme pass the same test.
I am under no illusion that the code they’ve produced isn’t ugly and suboptimal. But then, the Audacity codebase is 25 years old and also extremely ugly and suboptimal. The Audacity team – all extremely talented software engineers and beautiful people – have been unable to make Audacity 4 feature-equivalent to Audacity 3 in over a year, despite that “only” being a UI migration with some UX changes attached. With Soundscaper, I managed to get an app going featuring all of Audacity 3’s and 4’s features within 3 months, plus a plethora of additional DAW features and video editor on top of it. Similarly, Artcraft managed to get a Photoshop (and general Adobe CC) clone going in half a year.
On practical matters
While the robots are perhaps good enough to get functionality implemented, they suck at figuring out the finer details – and pretty much all of the design.
Robots suck at progressive disclosure
Progressive disclosure is the part of UX where, instead of showing everything at once, you show things as they’re needed. A trivial example for this would be a menu; the “Format” menu is reduced to that single word on your screen until you, the user, decide that now is the time to format something and click on it, revealing a dozen or so options.
Robots suck at this. If you tell them to add a feature, they’ll generally attempt to place it on screen with a dedicated button. This is fine for an app with 6 controls in total, but less so for a complex creation app with 500. Forcing them to chuck everything into menus and dialogs helps, but then we’re straight back into design paradigms from the 80s.
This is where Artcraft and Mindscaper both have an advantage over a pure vibecoded approach: Artcraft already knows where everything goes because they’re busy copying existing design. I know where my stuff goes because I am the designer, and thus have a very strong idea on what these features are, how they should work, where they should go and when they should show up.
Robots suck at information density
In a similar vein to the thing above, robots love to implement stuff as “Control title – control – explanation sentence”. Again, this is fine for a small dialog with 2 controls in it, but for any sufficiently complex effect it just either overwhelms people with too many words and/or requires things to be spaced so far apart to fit all this extra text that the dialog becomes massive.
And again, cloning existing dialogs works here, as does being the original designer to make relevant decisions. Purely vibecoding this from scratch is going to be much more difficult though.
Robots suck at asking the right questions
The biggest difference between working with humans and working with robots is what questions I’m being asked. With the Audacity team, I’d get asked about user edge cases as they popped up, I’d get asked about clarifications on my designs, I’d get asked about things by users. With the robots, there are a lot fewer of these kinds of questions; the only time the robot asks questions at all seems to be in plan mode. However, in plan mode, the questions that are being asked almost always are implementation-centric, ie the stuff I don’t really care about. So what ends up happening is that the robot will go ahead and do something, proudly proclaim it’s finished and then it’s up to me to QA whether it actually did the right stuff. And more often than not, it misses some design edge case.
On philosophical matters
In this section I’d want to talk about the should and shouldn’ts of vibecoding a creative software suite.
Copyright
Artcraft is copying various proprietary programs wholesale using AI. On Artcraft’s side, this has all the usual copyright questions attached (who is the author? is the code copyrightable?), though on Adobe’s side this becomes more interesting: From Tetris v. Xio, we know that the look and feel of a program (in that case, Tetris) can be copyrightable. Note that this is different from copyright on the code itself (which I don’t think Artcraft can even infringe given that their thing is in Rust, which Adobe’s stuff probably isn’t), and also is different from attempting to claim copyright on the app’s structure or underlying ideas (that’s patent stuff, not copyright).
Artcraft thus may be in for a rude awakening should Adobe feel threatened by them. Adobe may or may not win this case, but they also probably don’t need to in order to hobble Artcraft’s development for years to come. Funnily enough, the actual code being open source may be a saving grace here, as it likely can trivially be forked, copied and modified to avoid Adobe’s gaze, to a better degree than pirates ever could. Another point in favor of Artcraft is AI itself; any rulings in favor of copyright holders over AI developers is also a ruling against Adobe’s own products. It thus may be that Adobe opts to do… nothing, in order to avoid the backblast. I don’t think they’ll necessarily care about the possibility of a Streisand; lawsuits like this generally are boring enough for the wider press to not care beyond perhaps a single story in the margins.
Mindscaper meanwhile doesn’t have that problem, as Soundscaper is building upon an open source app and crediting it, and Framescaper is original.
Patents
Software patents unfortunately arguably exist, some of them on UI (eg the Ribbon pattern found in MS Office). Adobe, unsurprisingly, has a tens of thousands of patents.
Software patents are one level more devious than mere copyright: It doesn’t matter that you had the idea all on your own, or that the idea may be obvious now, all that matters is that someone else also had the idea before you. And if they have registered it, you now are in trouble. It’s because of software patents that loading screens don’t have minigames, it’s because of software patents that Audacity makes you jump through hoops to download FFmpeg, as there are still over 3000 patents in the AAC patent pool alone. And if just a singular patent survives litigation, which given the sheer number of patents involved is gonna take forever anyway, you’re on the hook for yet more damages.
Software patents are the ideal tool for corporations to shut down people they don’t like, hence why some corporations (including Adobe and Microsoft, the companies most likely to be upset by what Artcraft is doing here) engage in non-aggression pacts with each other, like the Allied Security Trust or Open Innovation Network – this prevents these companies being at each other’s throats for patent reasons all the time.
However, these defensive alliances generally don’t apply to people outside the alliance, and neither Artcraft nor Mindscaper are in one. Thus, whenever one of the incumbents ends up being upset or feel threatened by what’s happening, they can pull the litigation lever.
I don’t know which patents could be relevant here, and frankly, I also don’t wanna know – in the end it doesn’t really matter. Patents are a risk factor for any software, and typically best left ignored as patent holders tend to be more busy with actual business than going after one-man companies. However, when you sufficiently piss off a company – and I’d argue that wholesale cloning of a software might constitute that – you have a lot stacked against you.
Freeing users
Subscription services for software create incentives for the builder of the software to put as little work as possible into the software while charging as much as possible. This allows the Autodesks and Adobes of the world to charge ridiculous amounts, while providing software so bad that it regularly attracts user petitions, begging them to fix longstanding bugs degrading the experience.
Simply providing free software alternatives here may work (eg in the case of blender and musescore), or may not work, as free software does (or at least: used to) suffer from small development teams and quite frequently UX so weird that it wasn’t possible for some folks to figure out how to use the program in question. A famous example for this is the “make a circle in GIMP”. The solution is to make an elliptical selection, intuit that you can convert that to a path, and then use a panel to draw the path – as opposed to Photoshop, where you just use the circle tool.
Providing free software versions of a program would indeed free people of subscription software to use something else, depending on the copyright situation more or less easy compared to pirating, and this is what Artcraft does provide.
Freeing new users
However, I’ve found that the entire creative software field suffers from underdeveloped design. A lot of professional-grade software is modeled after pre-digital hardware and uses design metaphors that simply don’t make sense if you haven’t experienced said hardware. I remember when Lightworks, an Oscar-winning video editor, became a free download in 2010, I couldn’t for the life of me figure it out, not even typically simple things like rendering the video. This is despite me having had some experience in Sony Movie Maker (the cheap version of Vegas) and Powerdirector.
This is understandable somewhat: To attract professional users in the 80s and 90s, you’d need to replicate their workflows from their pre-digital tools. And once you have these users and they’ve learned how to use the program, they’d hate to have to adapt to a new design. However, this also leads to new users having to jump through enormous quantities of hoops trying to learn the program. If you want to “democratize” art creation, making things free and open source is one thing, but the barrier to entry of learning the tool remains the same.
This is where my work on Audacity (and now Soundscaper) becomes relevant: It takes quite a bit of design work, but it in principle should be possible to design a complex creation app in such a way to make it easy to use. Notably, not everything has to be equally easy – setting up a simple recording should be a one-button operation, while setting up a more complex multi-track recording may be a bit more involved, as we can expect the user to have a more clear task in mind on which input goes to which track.
Our vision for Audacity (and Musescore for that matter) was to completely starve the current incumbents of new users by just having a much more modern design and having an app that’s easier to get into. I still think this is the way things will shake out eventually; we’ve already seen Finale starve to death from this pressure.
Artists
Of course, AI bros would go further here and envision a future where skill is completely replaced by AI. And while for things that already were sloppy and primarily utilitarian to begin with (shitposts, local business signage, anything using Wordart/clipart in the past), AI certainly is going to replace more dedicated tools, I don’t see it doing so for artists. To draw a chess analogy, computers have been better than humans in chess for a long time, yet humans still play chess, for one simple reason: It’s fun. Similarly, artists don’t really care if a robot can do a hyperrealistic godzilla in mere seconds, as the fun part of making art is actually creating it, not just admiring the result (that would be mere consumption).
As an artist, you of course get to pick your battles. You can decide that shading or coloring is too annoying, and stick to strict black and white, or that mixing and mastering is too annoying and stick to making melodies and chords. From there you can hire someone to do the shading and coloring and additional work for you, or indeed use AI instead. In either case, your initial creative vision gets diluted by the other work; the more people are involved and the heavier the AI use, the more diluted your vision gets. This can be acceptable to you, especially in a world where art is traded as a commercial good, but if we assume that art as a good is going to lose worth, this tradeoff may become less worthwhile, perhaps making it so that you only wants tools strictly designed for you to express yourself.
For-profit open source
Having both worked at and seen for-profit open source, I’m perhaps a bit jaded:
- We see companies close access to their source the moment they feel like they can extract revenue out of it (various server software comes to mind)
- We see companies increase the amount of closed components in open source software, to guarantee only they can have a revenue stream (VSCode and Android come to mind)
- We see companies add user-hostile features to benefit only themselves (Google Chrome/Android is the poster-child here, to the point where regulators need to step in to it)
There might be a way to make a for-profit company be a good steward of an open source project, but the incentives always point in a different direction – and even if the current leadership can resist these incentives, the next one can flip towards enshittification so fast. I haven’t seen Thomas’ Artcraft project enough yet to know whether he’ll stay true to open source, though given he already has a subscription service (for cloud AI compute, of course) and a launcher ready, I gotta say that I’m worried.
It’s for this reason that I spun out Soundscaper from my company kw.media into Mindscaper, a non-profit association. It is now immune to me, or anyone else for that matter, deciding I would like it to make me tons of money, as long as its members (so, ideally, the users) don’t want to be milked. As an association (as opposed to a foundation), it also is way more democratic, giving users a real voice in what should happen to Soundscaper.
Resources & Sustainability
It’s impossible to discuss AI without touching upon the massive resource use it causes. Throughout my time developing Soundscaper, I have burnt through Codex’ and Claude’s 20x subscription quota almost every single reset cycle, including OpenAI’s ridiculous practice of vibing when a reset occurs and having a dude announce it on twitter. This has cost me hundreds of dollars, would have cost me tens of thousands of dollars in API cost, and has cost OpenAI and Anthropic likely thousands of dollars as well.
It is clear to me that the current pricing isn’t sustainable, and that slowly but surely, costs will go up. It is for this reason that Soundscaper uses local inference whenever useful and won’t be integrating with any cloud AI services. As far as I’m concerned, the entire point of open source is to fuck around with stuff on the machines you have access to and have it not tied to some service which decides what is and isn’t allowed.
It also is clear to me that there are real environmental impacts vibecoding this hard has; especially when comparing how long and hard my local setup has to work to achieve much simpler tasks (I let gemma and aya-expanse handle translation work locally, for example). Thus, I try to limit my vibecoding to the areas I really care about. I have a media background, so I shall vibecode media tools I’ll personally use – as opposed to token-maxxing just because I can.
Conclusion
So, can we vibecode creative software? On a technical level, yes. On a design level, yes, if we are a designer or follow someone else’s design. But should we? Depending on how adventurous we are with copyright and patent laws, especially if we are just re-implementing someone else’s design – maybe not.
We absolutely should open source and reverse engineer the things we care about, but we shouldn’t take a robot’s “one-shot” in an area we know nothing about as proof that it’s good now. It’s enormously easy to slop a whole software suite into existence but it actually takes some real work to get the software suite to actually be usable by real people – and surely, the benefit to real people has to be the goal.
If you’re interested in contributing to a non-profit making creative software, consider joining Mindscaper.
#ai #artcraft #artificialIntelligence #audacity #mindscaper #openSource #software #soundscaper #technology #vibecoding #writing -
IN BRIEF Pride flag demonstrators, Jason Lee removed, blood donation in Iceland, Iris Prize, Them’s Person of the Year, Colman Domingo, Bette Midler
NATIONAL In California, hundreds of demonstrators protested the Sonoma Valley Unified School District’s new policy that has forced…
#Iceland #IS #Europe #Europa #EU #Entertainmentnews #iceland #internationalnews #island #nationalnews
https://www.europesays.com/3331486/ -
On 26th October 1948, in Spencer, West Virginia, USA, a group of children, parents, teachers, and priests, publicly burned hundreds of what? Porn? Communist propaganda? No, it was comic books. There was a national fear of comic books corrupting the youth of America. #Comics
-
🐾 Conoce al lince de #Canadá, un maestro de la supervivencia adaptado a los fríos bosques boreales.
Lo más sorprendente de este felino son sus enormes patas, que funcionan como raquetas de nieve naturales distribuyendo su peso para caminar sobre la nieve profunda sin hundirse. Junto a su denso pelaje, es el depredador sigiloso perfecto del #Norte. ❄️🇨🇦
#Naturaleza #LinceDeCanada #MundoAnimal #Biologia #Curiosidades #FaunaSilvestre #FotografiaDeNaturaleza #SabiasQue #Animales -
Berichte und Analysen aus Frankreich, EU, Westfalenstadion und Borussia-Park
Die Reflexe stimmen. Sie bleiben so schlecht, wie sie immer waren. Die “alten” und “neuen” – in der Regel asozialen – Massenmedien springen an bei Sex&Crime, Mord&Totschlag – und wenn das alles nicht geboten wird, dann wenigstens Feuer&Sachbeschädigung, jedenfalls “geile Bilder” müssen es sein. Nichts davon, das wissen Sie, bekommen Sie hier. Hier nur Leseempfehlungen, Buchstaben, und davon viele hintereinander. Warnung: ganze Sätze und Gedanken!
Aus Frankreich berichtet Bernard Schmid/telepolis: “Frankreich: Der Protest bleibt, der Hass wächst – Hunderttausende demonstrieren weiter. Hetze und Angriffe verschärfen die Lage. In Orléans zeigt sich, dass es auch anders geht. Analyse. – Wenn bei einem Kräftemessen alle Beteiligten erklären, sie seien hochzufrieden, lohnt es sich, genauer hinzusehen.”
Volksentscheid – ist das wichtig?
Aus EU-Brüssel berichtet Falk Steiner/heise zur Zeitumstellung: “Missing Link: Vor, zurück, vor zurück – Seit 2018 will die EU die Zeitumstellung abschaffen – passiert ist nichts. Eine leicht satirische Spurensuche durch die Geschichte der europäischen Zeitpolitik.”
Es ist also nicht nur der Berliner Volksentscheid der den feinen Demokrat*inn*en der “Mitte” nun schon seit Jahren am verlängerten Rücken vorbeigeht. Bei diesem Demokratieverständnis muss sich dann auch heute niemand mehr über irgendwas wundern.
Sturzflutartiges Bier die “Süd” hinunter
So soll es gestern im Westfalenstadion, dem zentralen Versammlungsort der Stadt Dormund gewesen sein.
Dominik/schwatzgelb: “Geht doch! – Der BVB punktet endlich wieder in der Liga und zeigt trotz der Reisestrapazen eine wirklich gute, kämpferische Leistung – auch wenn nicht alles gelingt. Was gegen Frankfurt auf dem Rasen und den Rängen los war, lest ihr in unserem Spielbericht.”
In Mönchengladbach muss die Stadt 24 Stunden zuvor menschenleer gewesen sein. Weil sie alle, 5.000 sollen es gewesen sein, eine 400km-Reise in die Stadt mit dem abgerissenen Hauptbahnhof gewagt haben, todesmutig.
Christian Spoo/seitenwahl: “Das crazy! – Die Schröder-Challenge fällt aus. In Verzweiflungsrhetorik hatte sich Gladbachs Sportdirektor geübt, als er in Erwartung der siebten Niederlage in Folge gefragt wurde, ob es das dann womöglich schon gewesen sei, mit Borussia und der Bundesliga. Er freue sich auf den Negativrekord, weil die Challenge dann umso größer sei. Haarsträubender Unsinn, aber angesichts der desolaten Situation vor dem Auswärtsspiel beim VfB Stuttgart fast verständlich. Was soll der Mann denn sagen?”
So stelle ich mir sachlich-fachlichen Journalismus vor. Gut gemacht.
Über Martin Böttger:
Martin Böttger ist seit 2014 Herausgeber des Beueler-Extradienst. Sein Lebenslauf findet sich hier...
Sie können dem Autor auch via Fediverse folgen unter: @martin.boettger -
#PhantastikPrompts 25.10. Wie viele Charaktere hast du maximal in einer Szene? Gibt es da ein Limit für dich?
Für mich macht es einen Unterschied, wie viele Leute sprechen und wie viel schweigend zusehen.
Ich habe Versammlungen mit mehreren hundert Menschen, wo aber nur ein bis zwei Leute wirklich sprechen. Das ist locker machbar.
Dialog mit mehr als vier Personen? Habe ich in Zells Geschichte mehrfach, aber das wird dann auch anstrengend.
-
Erst Kommunen austrocknen, dann Investoren als Retter anbieten. Merz lockt Investoren wie BlackRock und KKR mit 30 Mrd. öffentlichem Geld, Reiche trifft mehrfach den KKR-Europachef. KKR will die Stromnetze Hunderter Stadtwerke in einer GmbH bündeln. Deren 8 bis 10 % Rendite zahlen wir über die Netzentgelte.
„Dabei handelt es sich um eine reine Umverteilung von der inländischen Realwirtschaft hin zu internationalen Finanzinvestoren“ -
I just had to do some lookup after viewing a clip on YouTube by Marjorie Taylor Greene. She was in the Permian basin of Texas, the largest reserve of petroleum in the US. She was right that Donald Trump and the oil companies are screwing the consumers, but she left out an important fact: You pump a finite amount of fossil liquid formed hundred of milions of years ago out of the ground and eventual it runs out. Shale oil in this reserve and two others in the western US have fueled our recent oil boom. They are in rapid decline. That in part explains Trump and his oligarch cronies' panicky move to steal oil where ever it exists, by hook or crook, kidnapping a president and starting a regional war. Their is no other developed power source than oil that is so amenable to our industrial society. Petroleum powers our country and the mechanical workings of empire, but it has also powered the individual for profit economy that has made a few extraordinarily rich. Addicts need their fix by any means. The end of peak oil is going to be hard on us all, but it is life or death for the Epstien class. And that may be a sign we are headed for a more egalitarian future and there will be a sigh of relief from all the other creatures on this planet.
https://www.artberman.com/blog/beginning-of-the-end-for-the-permian/
#oileconomylosingallitsgains
#marjorietaylorgreene
#peakoil -
Report: Mass UK protests are planned for Oct 31 against the ban on Palestine Action.
Outside the US embassy in London & the consulate in Edinburgh, protestors will be asking if Trump pressured the UK to outlaw the group.
Days later, the UK Supreme Court will hear an appeal by Palestine Action's Huda Ammori against the ban.
More than 3,850 have been arrested since the ban.
#UKPol #EUPol #USPol #palestine #PalestineAction #Gaza #GazaGenocide #news .
-
This video tells the story of Malone Lam, a 20-year-old from Singapore who helped steal over $230 million worth of Bitcoin through an elaborate social engineering scam. No sophisticated hacking tools or complex code, just deception, impersonation, and a convincing scheme.
From scamming Minecraft players to tricking a wealthy cryptocurrency investor into handing over access to his Bitcoin, Lam's activities escalated into one of the largest cryptocurrency thefts in history. The stolen fortune funded an extravagant lifestyle filled with luxury cars, lavish mansions, private jets, and nightclub bills reaching hundreds of thousands of dollars.
But the spending spree eventually attracted the attention of federal investigators, leading to arrests and a sprawling investigation into where the stolen Bitcoin went, much of which remains unaccounted for.
Video Credit: youtube.com/@InVoyceYT
-
https://www.youtube.com/shorts/M74adlmhe-g
“The eyes of others our prisons; their thoughts our cages.”
— Virginia Woolf, An Unwritten Novel (1920)
How much of our freedom do we surrender to the judgment of others?
A solitary woman stands surrounded by oppressive architecture, hundreds of watching eyes, and suffocating fog. The shadows gradually form a prison, yet no physical cage exists.
#VirginiaWoolf #LostQuotes #Philosophy #Psychology #Literature #Surrealism #SocialPressure #HumanNature #Freedom
-
An agent that feels instant on a laptop can feel broken the moment its model call crosses a network boundary, and the postmortem usually blames the model. The regression more often comes from three numbers nobody recorded before the move: cold-start latency, payload size leaving the device, and behaviour when the remote seat is unreachable. The scenario below is illustrative rather than a customer story, but the failure shape is common enough to plan around.
This article builds a small probe that records those three numbers for any endpoint pair and turns them into a residency decision. It assumes you already have an agent that runs end to end and you are choosing where the model call happens. Nothing here depends on one vendor, and every threshold below is a starting point you should retune for your own product.
Why three measurements beat one benchmark
Public leaderboards measure model quality on fixed prompts, and they say almost nothing about the machine your agent actually runs on. Residency is a property of your process, your network path and your prompt size, so it has to be measured in place. A single latency number is also easy to game: a warm local runtime can win on the median and lose badly on the first request after a laptop wakes up.The probe therefore records a distribution rather than a point estimate, and it pairs that distribution with two non-latency signals. The payload audit answers what content leaves the device, which matters when prompts carry repository context or customer records. The offline check answers whether the agent degrades into a queue or into an exception, and that difference decides how much of the fallback path you have to build.
Step 1: Fix one request shape for both endpoints
The comparison is only meaningful when both sides receive the same bytes, so define one request body and reuse it. Keep the system prompt and user prompt in files, and let environment variables supply the base URLs. This also makes the probe cheap to rerun whenever a runtime updates.mkdir -p prompts
printf 'Summarise the three functions in this diff.\n' > prompts/short.txtexport LOCAL_BASE_URL="http://127.0.0.1:8080/v1/complete"
export REMOTE_BASE_URL="https://your-endpoint.example/v1/complete"The two URLs are placeholders, and the script below does no vendor-specific shaping. If your local runtime expects a different JSON envelope, adapt
build_bodyonce and keep the rest identical. That single edit point is what makes the numbers comparable later.Step 2: Measure first body byte, with explicit honesty about it
Time to first token is the metric developers feel, but most HTTP clients only expose time to first body byte. The probe reads exactly one byte and stops, then labels the result as an approximation. It also reports p50 and p95 separately, because tail behaviour is where remote seats and cold local runtimes diverge.The script also counts failures instead of hiding them behind a mean. A run that times out is data, and a remote seat that fails two of seven attempts changes the architecture conversation more than a hundred milliseconds of median latency.
#!/usr/bin/env python3
"""residency_probe.py - reference implementation, not a benchmark suite.Records first-body-byte latency, payload size, secret-pattern hits and
forced-offline behaviour for up to two endpoints. Every number it prints
is yours to collect; no vendor performance is asserted here.
"""
from __future__ import annotationsimport argparse
import json
import os
import re
import socket
import statistics
import time
import urllib.error
import urllib.requestSECRET_PATTERNS = {
"provider_key": re.compile(r"sk-[A-Za-z0-9]{16,}"),
"bearer_value": re.compile(r"(?i)bearer[ ]+[A-Za-z0-9._-]{12,}"),
"aws_secret": re.compile(r"(?i)aws_secret_access_key[ ]*[:=]"),
}def build_body(prompt: str, system: str) -> bytes:
return json.dumps({"system": system, "prompt": prompt}).encode("utf-8")def scan_for_secrets(raw: bytes) -> list[str]:
text = raw.decode("utf-8", errors="replace")
hits = [name for name, pat in SECRET_PATTERNS.items() if pat.search(text)]
for key in ("API_KEY", "ACCESS_TOKEN", "SECRET"):
value = os.environ.get(key, "")
if len(value) >= 12 and value in text:
hits.append("env:" + key)
return hitsdef first_body_byte_ms(url: str, body: bytes, timeout: float) -> float:
req = urllib.request.Request(
url, data=body, headers={"Content-Type": "application/json"}
)
start = time.perf_counter()
with urllib.request.urlopen(req, timeout=timeout) as resp:
resp.read(1) # first body byte, NOT first token
return (time.perf_counter() - start) * 1000.0def probe(url: str, body: bytes, attempts: int, timeout: float) -> dict:
samples, failures = [], 0
for _ in range(attempts):
try:
samples.append(first_body_byte_ms(url, body, timeout))
except (urllib.error.URLError, socket.timeout, TimeoutError, ConnectionError):
failures += 1
ordered = sorted(samples)def pct(p: float) -> float | None:
if not ordered:
return None
idx = min(len(ordered) - 1, int(round(p * (len(ordered) - 1))))
return round(ordered[idx], 1)return {
"url": url,
"attempts": attempts,
"failures": failures,
"p50_ms": pct(0.50),
"p95_ms": pct(0.95),
"mean_ms": round(statistics.fmean(samples), 1) if samples else None,
}def offline_behaviour(fallback) -> dict:
"""Point at the discard port, then check the fallback still returns work."""
dead = "http://127.0.0.1:9/v1/complete"
started = time.perf_counter()
try:
first_body_byte_ms(dead, b"{}", timeout=2.0)
reached = True
except (urllib.error.URLError, socket.timeout, TimeoutError, ConnectionError):
reached = False
result = fallback()
return {
"endpoint_reachable": reached,
"fallback_returned": result is not None,
"fallback_seconds": round(time.perf_counter() - started, 3),
}def main() -> None:
ap = argparse.ArgumentParser()
ap.add_argument("--prompt", default="prompts/short.txt")
ap.add_argument("--attempts", type=int, default=7)
ap.add_argument("--timeout", type=float, default=20.0)
ap.add_argument("--out", default="residency_report.json")
args = ap.parse_args()prompt = open(args.prompt, encoding="utf-8").read().strip()
body = build_body(prompt, system="residency-probe")
report = {
"payload_bytes": len(body),
"payload_secret_hits": scan_for_secrets(body),
"endpoints": [],
"offline": offline_behaviour(lambda: {"queued": True}),
}for name in ("LOCAL_BASE_URL", "REMOTE_BASE_URL"):
url = os.environ.get(name)
if not url:
continue
row = probe(url, body, args.attempts, args.timeout)
row["name"] = name
report["endpoints"].append(row)with open(args.out, "w", encoding="utf-8") as fh:
json.dump(report, fh, indent=2)
print(json.dumps(report, indent=2))if __name__ == "__main__":
main()Run it twice: once with the local runtime freshly started, and once after it has served a few requests. Those two runs are the cheapest way to see cold-start cost without building a load generator.
Step 3: Audit what actually leaves the device
The payload audit is deliberately blunt, and it is not a data-loss-prevention product. It flags obvious credential shapes and any long environment value that leaked into the serialised body, which catches the common accident of pasting a token into a system prompt. A hit is a stop sign for a remote seat and a warning for a local one, because local runtimes log too.The byte count matters as much as the hits. A prompt carrying an entire diff can be orders of magnitude larger than a chat message, and upload time on a home connection scales with it. Record the byte count next to the latency figures, or you will attribute a bandwidth problem to model speed.
Step 4: Force the offline path on purpose
The probe points at a dead port and then calls the same fallback your agent would call. If the fallback raises instead of returning queued work, the agent needs a fallback contract before any residency change ships. If it returns queued work in a few seconds, the offline story is already credible.This check is cheap and it prevents the most expensive migration surprise. A remote seat that disappears during a demo is survivable when work queues locally and can be resumed. It is fatal when the whole run lives in a request handler that has already returned.
A decision table you can argue with
SignalWhat local-first tends to giveWhat a free server tends to giveRead it asWarm p95 first body byteLow and stable once the runtime is pinnedDepends on queueing at the shared endpointLocal wins on steady interactive loopsCold startLong first request after sleep or restartEndpoint is already warmRemote wins on sporadic usePayload bytes off deviceZero for the model callFull prompt, including contextSensitive context argues for localSecret-pattern hitsStill a bug, still loggedBlocks the send until redactedAny hit stops the remote optionOffline behaviourUsable if you persist stateFails closed without a queueQueue design is the real workConcurrency shapeBounded by one machineBursts absorbed elsewhereSpiky workloads favour remote
The thresholds below are operator-tunable starting points rather than published guidance. Fill the middle columns with your own probe output and use the last two as the interpretive lens.
The table is a discussion device, not a verdict. Two teams with identical numbers can land on different rows because their data classification and support burden differ.Where a free server genuinely wins
A recurring discussion in developer circles concerns how many intermediaries sit between a developer and a model, and residency is the concrete version of that question. The honest answer is that self-hosting is sometimes the wrong trade, especially when the local machine cold-starts constantly or has no accelerator at all. Spiky workloads, shared team artifacts and short-lived experiments all favour a remote seat that is already warm.MonkeyCode, an open-source coding agent project, offers free model access and a free server option, which makes it a reasonable second column for the probe above. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Token allowances and server terms change, so read the project page for the current limits before you design around them, and never assume a free tier is permanent.
The useful pattern is to run the probe against a free remote seat and against your laptop on the same day with the same prompt file. That produces a defensible comparison instead of an intuition, and it costs one environment variable.
Limitations and who should not use this
The probe measures first body byte, not time to first token, and the two diverge when a server streams headers early. It is a single-machine instrument, so it is not a load test and it will not reveal behaviour under ten concurrent runs. The secret scan is a regex tripwire, not a compliance control, and it will miss bespoke credential formats.Teams with hard data-residency obligations should keep the model call local and treat the remote column as unavailable rather than slow. Interactive products with tight latency budgets should not adopt a shared endpoint without a warm-up strategy and a measured tail. Anyone expecting deterministic throughput from a free tier should build the queue first and treat the server as an accelerator, not a guarantee.
If you want a second column without provisioning anything, point the probe at a free remote seat first and keep the local runtime as the fallback. The numbers you collect will tell you more than any migration plan written before them.#ai #localfirst #python #architecture #software #coding #development #engineering #inclusive #community
Measuring Before Migrating: A Residency Smoke Test for Local-First Agents and Free Servers -
One of the interesting things about using a chatbot is how naturally a conversation can continue.
I can ask a question, provide additional information, refer to something mentioned earlier, and expect the assistant to understand what I mean.
But while studying how chatbots work, I learned an important distinction:
A continuous conversation is something the application must create. It is not automatically provided by every individual LLM call.
What happens when I send a follow-up message?
Imagine this conversation:User: My food hasn't arrived.
Assistant: Would you like to wait or request a refund?
User: I want a refund.
The last message makes sense because of the earlier exchange.
But what happens if the application sends only “I want a refund” to the model?
The model may not have the information needed to understand the situation. The application needs to preserve the relevant conversation history and include it in the next request.
This was my first important takeaway: the application is responsible for maintaining the conversation.
Memory is not necessarily inside the model
Suppose I tell a chatbot that my preferred programming language is Java.Later, I ask which language I prefer.
It might seem that the model has permanently learned something about me.
However, the application could have stored that preference, retrieved it when needed, and supplied it as context for the new request.
The process could look like this:
User Information
↓
Application Memory
↓
Retrieve Relevant Information
↓
Add to Current Context
↓
LLM Generates ResponseThis is different from training the model and updating its learned parameters.
That distinction changed how I think about chatbot memory. The user experience may look like persistent memory even when the application is retrieving stored information and passing it to the model.
More context is not always better
Another concept that stood out to me was the context window.A model can process only a limited amount of information in a given interaction.
An application might store hundreds or thousands of messages, but it does not necessarily need to send every message with every request.
It can select recent messages, summarize older exchanges, or retrieve only information relevant to the current question.
Why does this matter?
Because repeatedly including a growing conversation can increase input token usage, which can affect cost and latency. It also consumes some of the model's available context capacity.
The important design question is not simply:
“How much context can I send?”
It is:
“Which information does the model actually need to answer this request well?”
Streaming is a user-experience decision
The lecture also connected next-token generation to streaming.Instead of waiting for an entire answer before showing anything, an application can display generated content progressively as it arrives.
This can make an application feel more responsive, although it does not necessarily make the underlying model generate the complete response faster.
It is another reminder that the experience users see depends on application design, not just model capabilities.
And then there is hallucination
A model can generate an answer that sounds convincing without the answer being factually correct.Fluent language and verified information are two different things.
For a real application, this matters enormously.
Imagine a support chatbot answering questions about an order. It should retrieve the actual order status rather than inventing one. If an action requires authorization, the application must enforce that authorization instead of trusting a prompt alone.
Depending on the use case, retrieval, external APIs, tool calling, validation, guardrails, and evaluation can improve reliability.
They reduce certain risks, but they do not guarantee that every answer will be correct.
What I'm taking away as a software engineer
I'm beginning to see chatbot engineering as more than sending prompts to a model.The application needs to manage conversation history, select context, retrieve relevant information, control access to tools, and handle responses appropriately.
These are familiar software engineering concerns applied to a new kind of capability.
My biggest takeaway from this lecture is:
A good chatbot does not merely generate responses. Its application must provide the right context and manage those responses responsibly.
As I continue studying Forward Deployed Engineering and Generative AI, I want to understand not only how models work, but also how to build reliable applications around them.#ai #llm #softwareengineering #software #coding #development #engineering #inclusive #community
How Does a Chatbot Remember? The Engineering Behind Context and Memory -
Anthropic officially filed its Form S-1 with the SEC on Monday, setting up a Nasdaq listing under the ticker ANT. For two years, the private artificial intelligence market operated on private marks, secondary desk whispers, and venture rounds structured around liquidation preferences. Now we have an audited registration statement. The numbers quantify what the foundation model business actually costs to operate.
For the first half of 2026, Anthropic reported $16.3 billion in revenue, up from $1.2 billion in the first half of 2025. That top line came alongside an operating loss of $693 million, narrowed from a $2.1 billion operating deficit a year earlier. The bottom-line net loss touched $80.7 billion, driven by non-cash mark-to-market adjustments on convertible debt instruments that convert upon the offering. At the customer level, Anthropic counts over 11,900 enterprise accounts generating more than $100,000 in annualized spend, 1,900 spending above $1 million, and roughly 250 paying over $10 million annually.
Enterprise software growth at this velocity usually commands extreme valuation multiples. The constraint sits lower on the statement: the off-balance-sheet commitments.
According to the filing, Anthropic has entered into at least $518 billion in multi-year infrastructure commitments with hyperscale cloud and hardware partners over the next decade. Crucially, roughly 80 percent of those contracts are non-cancelable or require fixed payments regardless of actual server utilization. That includes at least $111.1 billion committed to Alphabet, $110 billion to Amazon, $31.4 billion to Microsoft, and $161.2 billion in equipment lease structures tied to Broadcom silicon.
In classical corporate finance, software companies trade at high multiples because their cost structure scales smoothly: gross margins exceed 75 percent, marginal distribution costs approach zero, and capital expenditures stay minimal. Anthropic's cost profile resembles a capital-intensive utility or an industrial commodity refinery. By signing hundreds of billions in take-or-pay cloud obligations, the lab traded variable margin flexibility for dedicated physical capacity.
That commitment creates severe operating leverage. When enterprise inference demand expands, fixed clusters clear high incremental cash flow. But if demand plateaus, or if competing open-source model architectures depress token pricing faster than hardware amortizes, those fixed obligations do not adjust downward. Every empty megawatt of reserved compute drains gross profit straight through to operating cash flow.
The public markets will not price ANT as an asset-light SaaS business. They will price it as a high-beta spread between token real estate yields and non-cancelable debt-like power contracts. When public trading begins, the entire frontier laboratory cohort will finally get marked to market every single morning.#ai #economics #cloud #finance #software #coding #development #engineering #inclusive #community
The S-1 Reveals the Real AI Balance Sheet -
The Greenland Diaries: Day 77
Last night, I’m not sure what happened, or why there were people moving through the area in the darkness, but around midnight the shadows were flashing with popping gunfire and small explosions. I followed the sounds of the bullets, and they bounced off of everything in hollow glances and pings. A few even hit the shed. I covered Snowy with my backpack and put her around my chest.
I was so worried that a shot would pierce the shed and split one of us open.
I hugged up so tightly to her, she would lick my face through the backpack as the thunder of distant mortars and rockets threw tree branches everywhere. They sounded hollow and rocky, like skeletons were trying to swallow up the shed and pull it away. A few times I shifted to the other wall and buried us beneath the boxes of supplies and gallons of water.
I heard machines rolling through the streets, hundreds of them. The pavement cracked and moaned against their heavy treads. The grinding was so intense, I thought the earth was going to fall apart and my house, shed, and everything would be swallowed up. It’d be nice to go out that fast. It’d be nice to avoid the shadows and wild claws.
The Drum still beat behind the whirlwind of gunpowder. I stayed still the entire night as the shooting raged. Seldom did I hear a gun or cannon stop firing to reload. It was like they were spending every inch of ammunition on this battle. I know the Unnamed aren’t overt and present when you fight them. I’ve been lucky enough to fight them in close quarters. At a distance, they’re nearly impossible to see, especially in the dark. How could these people fighting them possibly know where they were hiding?
Around four in the morning, they finally stopped their fighting, and the world returned to the Drum and the silence between each beat. There were no illusions or Unnamed sulking around the empty spots of crumbled houses or caved-in walls. I stayed up all night watching them, but there was nothing. Only the bugs brewed about in aimless clouds.
In the morning, I scouted the area for traces of the battle. I looked for bits of fresh rubble, empty bullet casings glittering in the sun, and bits of torn clothes with inky blood beneath their layers. There was nothing though, not a single crater in the ground or pulverized wall from any of the nearby houses. In previous battles, I’d find this debris so fresh, it’d be shedding dust in the sun like wayward dandruff.
There is nothing though, no sign of the previous night’s battle.
I walked around the block three times with Snowy, looking for the smallest, most minuscule remnants of the battle. Nothing moved. Only the trees and sheets of thick grass shook against the breeze. The plants always look like they’re ready to bubble and burst.
I stopped searching in the late afternoon. It had been an illusion. It was all part of some massive trick by the Unnamed and their unseen allies. When I wandered back home with a worried Snowy, I saw the kid from before. He had been following me the entire time, dodging behind the houses like I used to do when I played with Nerf guns when I was his age.
I bet he regrets any imaginary nightmare he played war with now, the apocalypse was too real
“Don’t get tricked by the battle, they’re illusions too,” I yelled, as he ran away from me. I hope he listens. I don’t know why he always runs.
I’m not going to hurt anybody.
#apocalyptic #books #fiction #horror #journals #monsters #novels #reading #writing -
Context Engineering over Prompt Engineering: Architecting Codebases for AI Agents
TLDR: Agents don't fail because your prompt was badly worded. They fail because your codebase was never designed to be read. The engineers getting compounding returns from agents aren't the best prompt writers. They're the ones whose repos carry the architecture outside their heads: context standards in the repo, strongly typed contracts, and deliberately pruned retrieval. Prompt engineering is the visible 10%. Context engineering is the structural 90%.
The Prompt Ceiling
Somewhere around 2024, the industry's collective mental model of "working with AI" became a prompt box. The skill was asking well. Whole courses, cheat sheets, and prompt libraries flourished on the premise that the delta between a good and bad AI outcome lived in the wording of your request.For single-shot, single-file tasks, that was roughly true. It also quietly set everyone's expectations, and the expectations are now the problem.
Because in 2026 the work moved. Mid-level and senior engineers aren't asking a chatbot for a function anymore. They're dispatching agents across repositories of hundreds of thousands of lines, agents that read code, edit files, run tests, and open pull requests. And in this regime something counterintuitive happens that every serious team eventually notices: prompt quality stops being the constraint.
You can rewrite your prompt five times. You can role-play the model into a "senior staff engineer persona." Doesn't matter. The agent will still confidently duplicate an internal utility that already existed three directories away, import a dependency your org banned two quarters ago, and couple two modules your architecture has spent three years keeping apart.
Not because the prompt was weak. Because the codebase was opaque.
What an Agent Actually Sees
To understand why, look at what the average large repository exposes to a reader:- Implicit boundaries. "Don't call the data layer from the HTTP handler" exists as a norm, maybe a lint rule, maybe an ADR nobody reads. It does not exist anywhere a code reader encounters while working in the repo.
- Unwritten dependency policy. Which libraries are approved, which are banned, which need a security review. Usually a Slack thread. Sometimes a wiki page. Never in the path of work.
- Tribal domain models. What a "tenant" means in this system, which fields are PII, which service owns which concept. Spread across PR conversations, onboarding docs from 2019, and the memory of whoever's been here longest.
A new human engineer takes weeks to absorb all this, and that's with onboarding sessions and mentors and the ability to ask someone in the hallway. Then the industry hands an agent a repo checkout and a prompt, and acts surprised when the output violates constraints that were never written down anywhere the agent could read.
Here's the failure as it typically shows up, so common it's almost a template.
An agent is asked to "add a retry mechanism to the notification service." It produces a perfectly reasonable implementation. It also duplicates the exponential-backoff utility that already exists in an internal library, pulls in a third-party HTTP client the org specifically removed after a security audit, and places its new logic in a module that creates a circular dependency with the domain layer.
None of those are coding errors. All of them are context errors. And notice what makes them expensive: the code compiles, the tests pass, and the violations are only visible to someone who already knows the architecture. If that person exists, a senior just spent their verification budget (see the verification trap) catching things a document could have prevented.
Fix 1: Put the Architecture Where the Reader Is
The single highest-leverage change: stop storing architecture outside the repository.The pattern that's emerged across serious AI-adopting orgs is an in-repo context standard, a known, versioned location that every agent (and every new human) reads first. An
AGENTS.mdor.context/directory at the repo root holding the high-level domain model, the module boundaries and their rules, the explicit list of disallowed dependencies, the invariants ("never call X from Y," "all money fields are integers in minor units"), and pointers to the ADRs that matter.Treated like code, not documentation. Reviewed in PRs. Updated when the architecture changes, in the same commit as the change.
The test for whether this is working is embarrassingly simple. Could a competent engineer who has never met your team make a boundary-respecting change using only what's in the repo? If your honest answer is "no, they'd need to ask someone in Slack," then your agent never had a chance. Neither did your next hire.
The corollary that surprises teams: this investment isn't really for the agent. It pays back twice. Once when the agent's first-draft PR respects the boundaries, and again every time a new human onboards in days instead of weeks. The agent didn't create the need. It just made the cost of not meeting it immediate.
Fix 2: Strongly Typed Contracts
The second fix is about what kind of surface your system presents to a reader.Context files are necessary but they're advisory. Agents, like humans, follow probability, and prose is soft. Types are hard. The orgs getting the best agent behavior are the ones that make architectural constraints mechanically enforceable:
- Explicit interface definitions between modules: generated clients, protobuf schemas, OpenAPI specs for anything crossing a service boundary. An agent that hallucinates a function signature doesn't produce a subtly wrong call that compiles. It produces a type error, immediately.
- Strict types end to end. No
Anyescape hatches at the boundaries, no stringly-typed config. The failure mode shifts from "plausible code that violates the design" to "code that doesn't run," and that shift is the entire ballgame. An error the compiler catches at generation time costs seconds. The same error caught in review costs a senior's attention. Caught in production, it costs six hours or six figures. - Machine-checked policy where you can get it: lint rules and dependency analyzers that encode "this module may not import that one" as something the build fails on, not something the reviewer remembers.
The framing worth internalizing: every invariant you express only in prose is an invariant an agent will eventually violate, charmingly and with total confidence. Every invariant you express as a type or a lint rule is one less thing for the human reviewer to carry.
Fix 3: Prune the Context
The third fix is the counterintuitive one, because everything above has been "give the agent more context." But there's a failure mode on the other side, and mature teams hit it right around the time their context standards start working.Context dilution. If your instinct is "the agent should know everything," you dump half the repository into the window, and the model now treats your critical domain rules as one equal voice in a choir of 40,000 tokens of boilerplate. The invariant you care about gets the same statistical weight as a vendored dependency's CHANGELOG. The agent saw the rules. It just didn't prioritize them, because you gave it no structure to prioritize with.
The working pattern is deliberate, selective context:
- AST-based indexing and retrieval, tools that navigate the code's structure (definitions, call graphs, usages) instead of streaming its text. The agent gets "here are the three functions relevant to this task and their signatures," not "here's 200KB of the module they live in."
- The in-repo standard provides the constants: domain model, boundaries, banned deps. The retrieval layer provides the variables: exactly the code relevant to this task, nothing else.
- Budget context like a designed API surface, not an all-you-can-eat buffer. Teams that do report the same thing: smaller, sharper contexts outperform complete ones, on cost and quality, almost every time.
The principle underneath all three fixes is one the prompt-engineering era obscured: an AI agent is the most literal-minded new hire your team has ever onboarded. It will not infer your conventions. It will not ask a clarifying question in the hallway. It will follow exactly what's in front of it, at full speed, with total confidence.
Every year your team spent frustrated at new engineers for "not getting it," when the real problem was undocumented architecture and oral-tradition onboarding, is now doubled. Because the agent executes the misunderstanding immediately, at scale, in code.
The Question Behind the Question
If your codebase needs a person to already know your architecture in order to work safely in it, is that an asset or a liability? For twenty years, "the architecture lives in our senior staff's heads" sounded like job security. In an agent-heavy 2026, it sounds like a single point of failure with a Slack handle.
Your Turn
I'd genuinely like to hear how other teams are drawing this line, because the tradeoffs are real and I don't think there's a settled answer.- If a new human developer takes three weeks to understand your architecture, why do you expect an AI agent to infer it from raw files?
- Is your codebase structured to guide autonomous agents, or is your architecture stored purely in your senior engineers' heads? And if it's the second one: does that feel like security, or a countdown?
Comment with the ugliest thing an agent did in your repo that turned out to be your architecture's fault. Those stories are more useful than the prompts everyone keeps posting.
Next in the series: Deterministic Safety Gates for Agentic Workflows. What happens when agents stop writing code and start changing infrastructure, and why "the agent tested itself" isn't a safety argument.#discuss #ai #aiagents #softwareengineering #software #coding #development #engineering #inclusive #community
Context Engineering over Prompt Engineering: Architecting Codebases for AI Agents -
The Problem With "Free" Online PDF Converters
If you have ever used a standard online PDF compressor, converter, or merger, you have almost certainly uploaded your documents to a third-party server. In many cases, that is an unacceptable risk. Typical PDF payloads contain sensitive data:Executed contracts and legal agreements
Patient medical records and diagnoses
Banking, tax, and financial statements
Proprietary internal company documentation
When uploaded, that data travels across external networks, lands on third-party file systems (often indefinitely), and is processed by infrastructure you cannot audit.We set out to build a different model: a document processing suite where files never leave the user device. Here is the technical stack and architecture that makes client-side PDF execution possible.
Architecture Overview
┌─────────────────────────────────────────────┐
│ Browser │
│ │
│ ┌──────────┐ ┌──────────────────────┐ │
│ │ File API │───▶│ WebAssembly Module │ │
│ └──────────┘ │ (pdf-lib / PDF.js / │ │
│ │ custom WASM binary) │ │
│ ┌──────────┐ └──────────┬───────────┘ │
│ │ Canvas │◀──────────────┘ │
│ │ Blob │ │
│ │ URL │ │
│ └──────────┘ │
└─────────────────────────────────────────────┘
↕ Zero network requests for file data
All heavy tasks execute inside dedicated Web Workers to ensure the main UI thread stays at a fluid 60 FPS. Once processing finishes, the output is converted into an in-memory Blob URL and handed directly to the user for immediate download. Not a single byte of file content is sent over the wire.Core Stack
Layer Technology
PDF parsing & manipulation pdf-lib + PDF.js
Computation offloading Dedicated Web Workers
Image compression (WASM) @jsquash/mozjpeg + @jsquash/webp
Application framework Next.js (App Router)
State management React useReducer + useRef
Step 1: Reading PDFs with the HTML5 File API
The first milestone is reading document bytes into local browser memory without triggering network activity:javascript
// Local memory read — zero network interaction
async function readPDFFile(file) {
return new Promise((resolve, reject) => {
const reader = new FileReader();
reader.onload = (e) => resolve(e.target.result); // ArrayBuffer
reader.onerror = reject;
reader.readAsArrayBuffer(file);
});
}
// Event handler
const input = document.getElementById('file-input');
input.addEventListener('change', async (e) => {
const file = e.target.files[0];
const buffer = await readPDFFile(file);// buffer is an ArrayBuffer allocated strictly in local client memory
processPDF(buffer);
});
Using FileReader.readAsArrayBuffer() provides raw byte access inside JavaScript memory. From this point forward, every operation operates strictly as local computation.Step 2: Document Manipulation via pdf-lib
pdf-lib is a standalone JavaScript library that runs natively in the browser without any Node.js runtime or server dependencies. Here is how document merging is implemented client-side:javascript
import { PDFDocument } from 'pdf-lib';
async function mergePDFs(arrayBuffers) {
// Initialize a new empty PDF document
const mergedPdf = await PDFDocument.create();
for (const buffer of arrayBuffers) {
// Load each PDF from ArrayBuffer in memory
const sourcePdf = await PDFDocument.load(buffer);// Copy all pages into the merged target document
const pages = await mergedPdf.copyPages(
sourcePdf,
sourcePdf.getPageIndices()
);pages.forEach((page) => mergedPdf.addPage(page));
}
// Serialize to Uint8Array within client memory
const mergedBytes = await mergedPdf.save();// Create an instant download URL without remote servers
return URL.createObjectURL(
new Blob([mergedBytes], { type: 'application/pdf' })
);
}
This entire merge cycle runs synchronously in memory without a single outbound network packet.Step 3: Offloading Computation to Web Workers
PDF serialization and parsing can easily freeze the main JavaScript thread on documents with hundreds of pages. We isolate all computation inside a background Web Worker:javascript
// worker.js
import { PDFDocument } from 'pdf-lib';
self.onmessage = async ({ data }) => {
const { type, payload } = data;try {
switch (type) {
case 'COMPRESS': {
const result = await compressPDF(payload.buffer, payload.quality);
// Transfer the ArrayBuffer back (zero-copy memory transfer)
self.postMessage({ success: true, buffer: result }, [result]);
break;
}
case 'MERGE': {
const result = await mergePDFs(payload.buffers);
self.postMessage({ success: true, buffer: result }, [result]);
break;
}
}
} catch (err) {
self.postMessage({ success: false, error: err.message });
}
};
javascript// main thread
const worker = new Worker(new URL('./worker.js', import.meta.url));
function processPDFInWorker(buffer) {
return new Promise((resolve, reject) => {
worker.postMessage(
{ type: 'COMPRESS', payload: { buffer, quality: 0.8 } },
[buffer] // Transferable object: passes pointer without memory duplication
);worker.onmessage = ({ data }) => {
if (data.success) resolve(data.buffer);
else reject(new Error(data.error));
};});
}
By specifying [buffer] in the transfer list, ownership of the memory block is transferred instantly to the worker thread without invoking expensive structured clone copying.Step 4: PDF Compression with WebAssembly Codecs
File sizes in scanned or multi-page documents are predominantly caused by embedded raster images. Effective compression requires:Rendering each PDF page to an offscreen canvas
Re-encoding the canvas pixels using WebAssembly-compiled image encoders
Re-assembling the PDF with re-quantized image streams
javascriptimport { getDocument } from 'pdfjs-dist';
import * as mozjpeg from '@jsquash/mozjpeg';
async function compressPDFImages(buffer, quality = 0.75) {
const pdfDoc = await getDocument({ data: buffer }).promise;
const newPdf = await PDFDocument.create();
for (let i = 1; i <= pdfDoc.numPages; i++) {
const page = await pdfDoc.getPage(i);
const viewport = page.getViewport({ scale: 1.5 }); // Target 144 DPI// Render to OffscreenCanvas without needing DOM bindings
const canvas = new OffscreenCanvas(
viewport.width,
viewport.height
);
const ctx = canvas.getContext('2d');await page.render({ canvasContext: ctx, viewport }).promise;
// Extract raw image data
const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height);// Encode via MozJPEG WebAssembly
const compressedBytes = await mozjpeg.encode(imageData, { quality: quality * 100 });// Embed back into destination document
const jpgImage = await newPdf.embedJpg(compressedBytes);
const newPage = newPdf.addPage([viewport.width, viewport.height]);
newPage.drawImage(jpgImage, {
x: 0, y: 0,
width: viewport.width,
height: viewport.height,
});}
return await newPdf.save();
}
Why MozJPEG over Native Canvas Encoders?
Browser native canvas toBlob() implementations prioritize encoding speed over file size compression. MozJPEG compiled to WebAssembly consistently produces 30% to 50% smaller image files at equivalent perceptual quality metrics (SSIM and PSNR). The gzipped WASM module weighs only around 200 KB and runs near native execution speeds.Performance Benchmarks
Benchmark performed on a 10-page scanned color document (original payload: 8.2 MB):Processing Method Output Size Visual Fidelity Duration
Native canvas.toBlob() (quality 0.8) 4.1 MB Good 1.2s
MozJPEG WASM (quality 75) 2.3 MB Sharp 1.8s
MozJPEG WASM (quality 60) 1.4 MB Acceptable 1.8s
Server-side Ghostscript engine 1.1 MB Good ~3s + upload transit
The client-side WASM pipeline approaches within 30% of server-side Ghostscript performance while completely eradicating upload latency and security risks.Step 5: Memory Management and Streaming
When loading a 50 MB document, naïve implementations can easily inflate memory usage to over 200 MB due to multiple concurrent representations (ArrayBuffer, parsed AST, canvas pixel arrays).To prevent tab crashes on memory-constrained devices:
javascript
// Stream processing: parse and clean page-by-page
async function* streamPages(buffer) {
const pdfDoc = await getDocument({ data: buffer }).promise;for (let i = 1; i <= pdfDoc.numPages; i++) {
const page = await pdfDoc.getPage(i);
yield page;// Explicitly evict page cache from PDF.js internal store
page.cleanup();}
}
// Bounded heap iteration
for await (const page of streamPages(buffer)) {
await processPage(page);
}
For files exceeding 100 MB, the application advises splitting the document before applying compression or conversions — a pattern supported directly in KAOpdf Split PDF.Handling Edge Cases
Password-Protected Documents
javascriptasync function loadProtectedPDF(buffer, password) {
try {
return await PDFDocument.load(buffer, {
password,
ignoreEncryption: false
});
} catch (err) {
if (err.message.includes('password')) {
throw new Error('Incorrect password entered.');
}
if (err.message.includes('encrypted')) {
throw new Error('Unsupported encryption standard (256-bit AES).');
}
throw err;
}
}
Corrupted or Non-Standard PDF Streams
PDF.js is significantly more tolerant of damaged cross-reference tables than strict parsers. When structural modification fails in pdf-lib, we fall back to PDF.js rendering:javascript
async function robustLoad(buffer) {
try {
return await PDFDocument.load(buffer);
} catch {
// Fallback: render damaged pages via PDF.js and reconstruct
const pdfJsDoc = await getDocument({
data: buffer,
stopAtErrors: false
}).promise;
return rebuildFromRenderedPages(pdfJsDoc);
}
}
Security Model and HTTP Headers
Moving processing to the client eliminates data breach vectors on the backend, but requires strict client-side browser controls.Security Guarantees
Zero server breach exposure for uploaded files
Immunity to man-in-the-middle transmission interception
Zero regulatory storage liability (GDPR, HIPAA compliance by design)
Client Configuration Requirements
To enable high-performance multithreaded WASM and SharedArrayBuffer, modern browsers mandate Cross-Origin Isolation headers:javascript
// next.config.js — Cross-Origin Isolation headers
const nextConfig = {
async headers() {
return [
{
source: '/(.*)',
headers: [
{ key: 'Cross-Origin-Opener-Policy', value: 'same-origin' },
{ key: 'Cross-Origin-Embedder-Policy', value: 'require-corp' },
],
},
];
},
};
Production Implementation at KAOpdf
This architecture is currently running live in production across more than 35 web-based tools on KAOpdf:Compress PDF — MozJPEG WebAssembly quantization
Merge PDF — In-memory page aggregation
Split PDF — Client-side page extraction
OCR PDF — Offline Tesseract.js WASM character recognition
AI PDF Summarizer — On-device WebGPU model execution
Across all tools, the fundamental contract remains unchanged: zero file uploads, zero remote storage, and complete user privacy.Engineering Takeaways
1.pdf-lib for structure, PDF.js for rendering: pdf-lib handles fast document generation, while PDF.js is the most resilient parser for non-standard PDFs.
2.Transferables are mandatory: Always pass ArrayBuffers in worker transfer lists to prevent costly memory duplication.
- WebAssembly image codecs deliver real gains: WASM-compiled MozJPEG consistently beats native browser canvas encoders in compression efficiency.
- Enforce bounded memory loops: Always call .cleanup() after rendering pages to prevent tab crashes on mobile browsers.
Resources and Documentation
pdf-lib Documentation
pdfjs-dist on npm
@jsquash/mozjpeg on npm
KAOpdf Live ToolsIf you are experimenting with client-side document processing or running into WebAssembly memory bottlenecks, feel free to share your thoughts or questions in the discussion below.#webassembly #ai #pdf #webdev #software #coding #development #engineering #inclusive #community
How I Built a Privacy-First PDF Processor That Never Uploads Your Files -
The first week after launch is not about ranking. It is about confirming three things: crawlers can reach the site, they are actually fetching it, and the contact path works end to end. Everything else can wait until there is data.
Day 0, before you tell anyone: four things that block indexing silently
- A coming-soon or maintenance mode that is still switched on.
- A
noindexleft over from staging. There are two places: a meta tag in the head, and anX-Robots-Tagresponse header. Check both. Open the page source and search fornoindex, then request the page headers withcurl -I. The header is the one people forget. - HTTP auth or a site-wide password still enabled.
- The same page reachable on more than one hostname (with and without
www, or both protocols) with no redirect between them. Pick one, 301 the other.
One more, before launch rather than after: put the measurement tag on the site. Analytics installed on day 3 means day 1 and 2 have no data at all.
Test all of this from a phone on mobile data, not from the office wifi, and not while logged in as an administrator. Logged-in views hide caching and coming-soon behaviour.
Day 1: verification and submission
- Add a domain property, verified by a DNS TXT record. It covers every subdomain and both protocols in one property, which matters when you are not yet sure whether the canonical host is
wwwor not. - Add a URL-prefix property as well if you want per-protocol reporting. It is a second view of the same data, verified separately.
- Submit the sitemap once.
Before submitting, open the sitemap in a browser. It has to load, return 200, be valid XML, and contain only canonical URLs that themselves return 200. No redirects, no 404s, no noindexed pages inside it. A sitemap full of redirects teaches the crawler to distrust the whole file.
Then URL inspection on a handful of pages, not hundreds: home page, one category or service page, one product page, the contact page. "Request indexing" is a queue entry, not a switch. There is a daily quota and no way to buy more.
Bing: import from Search Console or verify directly, then submit the sitemap there too. Ten minutes, and it feeds the results that are not Google.
What not to do in week one: resubmitting the sitemap every morning, requesting indexing for every URL, or deleting and recreating the property because nothing shows on day 2.
Day 2-4: crawl your own site before the crawlers do
Check that every key URL returns the status you expect. One line per URL, status code first:while read -r u; do
printf '%s %s\n' "$(curl -s -o /dev/null -w '%{http_code}' -L "$u")" "$u"
done < urls.txturls.txtis one URL per line in a plain text file: home, top landing pages, contact, the sitemap, robots.txt.Then check, on each page:
- the canonical tag points at that page itself, not at the home page and not at the other hostname
- internal links use the canonical hostname; one mixed link is enough to create a duplicate
- the main text is in the HTML source, not injected by JavaScript. View source, not the rendered DOM.
Fix these now. Crawlers come back, but they come back on their own schedule, not yours.
Day 3-7: read the access log
This is the step people skip, and it is the place that shows what actually happened rather than what you submitted.Where the file is. On a managed host, the control panel, usually under access logs or raw logs, rotated daily. On your own server,
/var/log/nginx/access.logor/var/log/apache2/access.log. Behind a CDN, the origin log shows the CDN's IP addresses, so you need the CDN's own log or an origin header that carries the real client IP.Download a couple of days and work on the copy.
# Who is hitting the site at all
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -20# How many requests claim to be a crawler
grep -Eic 'googlebot|bingbot' access.log# Which paths the crawler actually fetched, busiest first
grep -Ei 'googlebot' access.log | awk '{print $7}' | sort | uniq -c | sort -rn | head -20# What we served it
grep -Ei 'googlebot' access.log | awk '{print $9}' | sort | uniq -c | sort -rnThe field numbers (
$1,$7,$9) assume the combined log format: client IP, identity, user, timestamp, request, status, bytes, referrer, user agent. Check the first line of the file and the server config before trusting them.A user agent proves nothing. Anyone can send the word Googlebot. Verify the IP by reverse lookup, then forward-resolve the name it returns and confirm it comes back to the same IP:
host 66.249.66.1
# the returned name should sit under googlebot.com or google.com
host <the-name-it-returned>If either lookup fails, or the forward lookup does not return the original IP, that hit was somebody else. Usually a scraper or a monitoring service.
What real crawling looks like: more than one hit, spread over several days; requests for HTML pages rather than only images and CSS; 200 responses; and a spread of URLs that includes internal pages, not just the home page.
What it does not look like: a single hit on the home page, a run of hits on the login path, or hits only on assets. Those are fetchers and scanners, not indexing.
The self-inflicted block. If you switched on a firewall bot rule, a rate limit, or a security feature that blocks suspicious traffic during launch week, check it before concluding anything about the index. The log will show 403s or 429s served to a crawler. That is your own rule, not a search engine decision.
Inquiries: prove the path works end to end
Submit the form yourself on launch day, from a network that is not yours, using a mailbox you control. Confirm four separate things:- the thank-you message appears
- the notification email arrives
- the entry is stored somewhere you can query
- the sender address passes SPF, DKIM and DMARC for your domain
The email is the fragile part. It can fail while the form still looks fine to the visitor. Check the spam folder, then check the host's mail log for the message ID. If the mail path turns out to be unreliable, at least store every submission so nothing is lost.
Server-side proof that a submission reached you, even when the email did not:
grep 'POST ' access.log | awk '{print $7, $9}' | sort | uniq -c | sort -rn | head
A 200 on the form handler means the data arrived somewhere. A 500 means the handler is failing and the visitor saw an error or a blank page. A 429 means your own rate limit ate a real inquiry.
Separating noise from a real inquiry. Noise usually looks like this: the same IP hitting the handler several times in a few minutes, fields that are empty or obviously generated, a message body that is only a link, a referrer from a directory site. A real one usually has a company name you can look up, a reply-to address that matches the domain in the message, and a specific question about a specific thing.
That is a set of heuristics, not a rule. Read the message.
The week-one sheet
WhatWhereNormal in week onePages known / indexedSearch Console pages reporta small number, growing slowlyCrawl requestsaccess log, filtered by verified crawler IPsa handful per day, spread across URLs4xx / 5xx on canonical URLsaccess logclose to zeroForm POSTsaccess logequals the number of inquiries you receivedInquiriesinbox or a shared sheetany number, including zeroTop referrersanalyticsmostly direct and your own links early on
One place, filled in daily. Five minutes a day.
The form POST count and the inquiry count should match. If they do not, you have a delivery problem, not a traffic problem.What not to conclude in week one
- Zero indexed pages on day 3 is not a penalty.
- Zero inquiries from fifteen visitors is not a conversion problem. It is a traffic problem, and it is too early to see either.
- Do not rewrite the site on five days of data.
- Do not resubmit anything twice. Once is the instruction; the second time is noise in your own process.
The whole first week is a verification exercise: is the door open, is anyone walking through it, and does the contact path actually deliver. Everything after that is content, which runs on a much slower clock.#seo #startup #webdev #software #coding #development #engineering #inclusive #community
Launch week: what to submit, what the logs should show, and how to tell a real inquiry from noise -
Saying “never bet against Elon” does not actually make you look smart
The world’s richest man has been more of a main character in the news cycle than usual over the past 10 days, which is saying a lot. We had Elon Musk’s showoff of a “flying car” version of the future Tesla Roadster (“jumping car” more accurately describes this exercise in cold-gas vaporware), more announcements of steps to turn Starlink Mobile into a full-fledged network (never mind SpaceX’s severe shortfall of terrestrial spectrum) and Musk hyping a Tesla-SpaceX merger (SpaceX shareholders might want to hold their applause).
Whatever Musk says, you can expect one reaction from many people who should know better: Never bet against Elon.
That saying seems to come from Musk’s former business partner Peter Thiel, but it’s become a bit of a mantra–especially on X, where Musk has drawn a weird Greek chorus of sycophantic superfans who either don’t mind or outright cheer his racist rants about immigrants.
I understand how that phrase can be easy to invoke from the legit accomplishments of SpaceX and Tesla. I’ve experienced them firsthand–watching one Falcon 9 and two Falcon Heavy launches at Cape Canaveral, then doing a nearly-1,700-mile road trip in a Model 3 in 2022–and I’ve written my share of appreciative things about both companies, SpaceX especially.
But as a catchall rationalization of everything Elon says he’ll make happen, “never bet against Elon” amounts to weak-minded foolishness. Of course you can bet against the guy–and in three obvious cases, your wager could have paid off well.
Cybertruck: Tesla’s metal trapezoid of a truck may seem ubiquitous because of how easy it is to spot one of these things from a quarter-mile away, but in reality it’s sold worse than Ford’s Edsel.
Tesla’s slumping sales could use a boost right now. But instead of shipping a lower-cost EV that might help Tesla compete with the affordable electric cars from BYD and other Chinese automakers that are now racking up sales outside the U.S., the company introduced this $74,990-and-up monstrosity that isn’t street-legal in Europe.
I imagine that the reaction to that poor choice–and to the not-really-flying Roadster and to the still-vaporware Optimus robot that for some reason graces Tesla’s Georgetown showroom in D.C.–among China’s EV industry remains whatever “LOL” is in Mandarin.
Hyperloop: Musk’s notion of vehicles whooshing through low-pressure tubes at hundreds of miles an hour deserved skepticism when he revealed it in 2013 as a supposedly superior alternative to building high-speed rail in California (not that there isn’t much to critique about CAHSR’s management and design failures).
Musk not committing his own money and his own Boring Company operation to build hyperloop networks should have been a tell. Instead, startups lined up investors to build versions of this design in places like Abu Dhabi, then failed to pour concrete because the design proved unworkable.
Much closer to Washington, former Maryland governor Larry Hogan (R.) fell for this nonsense in 2017 and then watched the concept of a D.C.-NYC hyperloop unravel over the next few years. Elsewhere around the world–notably China–high-speed rail has kept rolling.
Political donations: Yes, Musk spending more than a quarter of a billion dollars to help Donald Trump get elected in 2024 may have helped in swing states. But what happened months later when Musk spent another $20 million in just one of those states, Wisconsin?
Dropping that much money there–plus briefly touting an illegal $1 million sweepstakes for voters–did not swing an April 2025 election for a spot on the state Supreme Court. Despite Musk declaring that “Western civilization” was somehow at stake should Brad Schimel (R.) lose, the people of Wisconsin voted for the judge’s Democratic opponent, Susan Crawford, by an almost 10-point margin.
One possibly relevant factor: Musk’s approval ratings in polls are now almost as bad as Trump’s, and Musk can’t claim anything like Trump’s cult of personality.
The appropriate response to any version of “never bet against Elon” is “citation needed.” Musk has done some extraordinary things but now seems extraordinarily detached from the societal circumstances that result in most people proffering dumb ideas having their peers tell them “that’s a dumb idea.”
Too many media-industry folks, meanwhile, need to stop bringing a “never bet against Elon” mindset to covering Musk’s predictions or pledges in what my fellow tech scribe Karl Bode mocks as “CEO Said a Thing” stories. Journalists do their readers a disservice when they fail to remind them of Musk’s spotty record of delivering on his promises–and freelance journalists do their own incomes a disservice by passing on such an easy way to pad out their word counts.
#betAgainstElon #BoringCompany #CEOSaidAThing #citationNeeded #Cybertruck #DOGE #ElonMusk #Falcon9 #FalconHeavy #Hyperloop #MuskCampaignDonations #MuskWisconsin #OptimusRobot #SpaceX #StarlinkMobile #Tesla #TeslaModel3 #TeslaRoadster #TeslaSpaceXMerger