“dev_lenchen” — Fediverse search results on home.social

2026-10-19

Watch an AI agent "think" and you'll notice something wasteful: it writes out every intermediate thought in full English sentences, token by token, before it ever gets to the part that matters — calling a tool, running code, giving you an answer. That's not reasoning. That's narrating.

A paper and open-source release from Stanford, UIUC, and Together AI — dropped days ago, front-paged on Hacker News yesterday — argues the narration was never necessary. It's called HybridAgent, and the pitch is almost rude in its simplicity: let the model think in raw vectors instead of words, and only switch to text when something outside the model actually needs to read it.

The trick: two modes, one generation

Most "efficient reasoning" research picks a side — either the model reasons in latent space (fast, but opaque and bad at using tools) or it reasons in text (slow, token-hungry, but legible). HybridAgent refuses to pick. It moves between both within a single generation pass, using a small adapter called HybridLink that maps the model's last hidden state straight back into its own input-embedding space. That sidesteps the usual bottleneck — decode a token, re-embed it, feed it back in — and lets the model keep "thinking" in pure vector form for dozens of steps before it ever has to commit to a word.

Two special tokens, <latent> and </latent>, mark the on-ramp and off-ramp. Outside those tags, it's a normal language model making tool calls, reading search results, writing an answer. Inside them, it's compressing an entire chain of thought into something closer to a held breath than a sentence.

The part that should make skeptics less nervous: none of this is a black box by default. An "interpretable latent mode" renders every latent step back out as pseudo-text, so you can literally watch a HybridAgent's internal monologue as an HTML report — garbled, abbreviated, but legible. In one released trace, a span of 42 latent steps decodes to something like "the the speed of speed in air at 20° is about 343... should verify... let me search" — recognizably the ghost of a real thought, not noise.

How it learned to switch

Training is two stages. First, HybridSFT teaches the base model (built on Qwen3, at 8B and 14B) to coordinate with HybridLink at all — basic fluency in switching modes. Then HybridRL takes over and learns when latent mode is worth it, rewarded on correctness, output format, and whether tool calls actually succeed. The model isn't told to minimize tokens; it discovers that thinking quietly is often cheaper and just as accurate.

The numbers

Across 11 benchmarks spanning math, science, medicine, search, and agentic tasks, at three model scales, HybridAgent beats frontier text-based and latent-only agents by 10.4% on average — while cutting token usage 35.5–54.7%, running 1.8–2.6x faster, and costing 2.7–3.7x less. An ablation check matters more than the headline stats: swap a model's latent spans with ones from unrelated problems, and accuracy drops 45–51%. The latent thinking isn't filler — it's doing real work. Meanwhile tool-call execution stays reliable at 99.7%, so the efficiency gains aren't coming at the cost of the part that actually touches the outside world.

Try it

Everything's on GitHub (togethercomputer/HybridAgent, MIT-licensed) with runnable checkpoints, not just a paper:

git clone github.com/togethercomputer/Hy… && cd HybridAgent
pip install -r requirements.txt && pip install -e .
cp .env.example .env # add a TAVILY_API_KEY for the search tool
python -m hybrid_inference.run_batch --model togethercomputer/HybridAgent-8B-Agentic

You'll need a CUDA GPU; the Qwen3 tokenizer already ships with the anchor tokens baked in.

Where this sits

It's not alone — LatentMAS, from a separate group, applies a similar idea to multi-agent systems, passing latent thoughts between agents instead of within one. HybridAgent's distinction is that it's single-agent, fully inspectable, and already shipping usable weights rather than just a method description. That's probably why it's the one getting the HN traction right now: it's something you can git clone on a Tuesday night, not something you have to reimplement from a methods section.

The headline isn't that agents got smarter. It's that they finally stopped assuming "smarter" has to mean "more words." For anyone running agent pipelines at any real scale, that's the metric that was quietly bankrupting them the whole time.#tooling #ai #agents #software #coding #development #engineering #inclusive #community
Your AI Agent Has Been Thinking Out Loud This Whole Time

#ai #community #software #coding #development #engineering

Search