16 results for “localfirst”
An agent that feels instant on a laptop can feel broken the moment its model call crosses a network boundary, and the postmortem usually blames the model. The regression more often comes from three numbers nobody recorded before the move: cold-start latency, payload size leaving the device, and behaviour when the remote seat is unreachable. The scenario below is illustrative rather than a customer story, but the failure shape is common enough to plan around.
This article builds a small probe that records those three numbers for any endpoint pair and turns them into a residency decision. It assumes you already have an agent that runs end to end and you are choosing where the model call happens. Nothing here depends on one vendor, and every threshold below is a starting point you should retune for your own product.
Why three measurements beat one benchmark
Public leaderboards measure model quality on fixed prompts, and they say almost nothing about the machine your agent actually runs on. Residency is a property of your process, your network path and your prompt size, so it has to be measured in place. A single latency number is also easy to game: a warm local runtime can win on the median and lose badly on the first request after a laptop wakes up.
The probe therefore records a distribution rather than a point estimate, and it pairs that distribution with two non-latency signals. The payload audit answers what content leaves the device, which matters when prompts carry repository context or customer records. The offline check answers whether the agent degrades into a queue or into an exception, and that difference decides how much of the fallback path you have to build.
Step 1: Fix one request shape for both endpoints
The comparison is only meaningful when both sides receive the same bytes, so define one request body and reuse it. Keep the system prompt and user prompt in files, and let environment variables supply the base URLs. This also makes the probe cheap to rerun whenever a runtime updates.
mkdir -p prompts
printf 'Summarise the three functions in this diff.\n' > prompts/short.txt
export LOCAL_BASE_URL="http://127.0.0.1:8080/v1/complete"
export REMOTE_BASE_URL="https://your-endpoint.example/v1/complete"
The two URLs are placeholders, and the script below does no vendor-specific shaping. If your local runtime expects a different JSON envelope, adapt build_body once and keep the rest identical. That single edit point is what makes the numbers comparable later.
Step 2: Measure first body byte, with explicit honesty about it
Time to first token is the metric developers feel, but most HTTP clients only expose time to first body byte. The probe reads exactly one byte and stops, then labels the result as an approximation. It also reports p50 and p95 separately, because tail behaviour is where remote seats and cold local runtimes diverge.
The script also counts failures instead of hiding them behind a mean. A run that times out is data, and a remote seat that fails two of seven attempts changes the architecture conversation more than a hundred milliseconds of median latency.
#!/usr/bin/env python3
"""residency_probe.py - reference implementation, not a benchmark suite.
Records first-body-byte latency, payload size, secret-pattern hits and
forced-offline behaviour for up to two endpoints. Every number it prints
is yours to collect; no vendor performance is asserted here.
"""
from __future__ import annotations
import argparse
import json
import os
import re
import socket
import statistics
import time
import urllib.error
import urllib.request
SECRET_PATTERNS = {
"provider_key": re.compile(r"sk-[A-Za-z0-9]{16,}"),
"bearer_value": re.compile(r"(?i)bearer[ ]+[A-Za-z0-9._-]{12,}"),
"aws_secret": re.compile(r"(?i)aws_secret_access_key[ ]*[:=]"),
}
def build_body(prompt: str, system: str) -> bytes:
return json.dumps({"system": system, "prompt": prompt}).encode("utf-8")
def scan_for_secrets(raw: bytes) -> list[str]:
text = raw.decode("utf-8", errors="replace")
hits = [name for name, pat in SECRET_PATTERNS.items() if pat.search(text)]
for key in ("API_KEY", "ACCESS_TOKEN", "SECRET"):
value = os.environ.get(key, "")
if len(value) >= 12 and value in text:
hits.append("env:" + key)
return hits
def first_body_byte_ms(url: str, body: bytes, timeout: float) -> float:
req = urllib.request.Request(
url, data=body, headers={"Content-Type": "application/json"}
)
start = time.perf_counter()
with urllib.request.urlopen(req, timeout=timeout) as resp:
resp.read(1) # first body byte, NOT first token
return (time.perf_counter() - start) * 1000.0
def probe(url: str, body: bytes, attempts: int, timeout: float) -> dict:
samples, failures = [], 0
for _ in range(attempts):
try:
samples.append(first_body_byte_ms(url, body, timeout))
except (urllib.error.URLError, socket.timeout, TimeoutError, ConnectionError):
failures += 1
ordered = sorted(samples)
def pct(p: float) -> float | None:
if not ordered:
return None
idx = min(len(ordered) - 1, int(round(p * (len(ordered) - 1))))
return round(ordered[idx], 1)
return {
"url": url,
"attempts": attempts,
"failures": failures,
"p50_ms": pct(0.50),
"p95_ms": pct(0.95),
"mean_ms": round(statistics.fmean(samples), 1) if samples else None,
}
def offline_behaviour(fallback) -> dict:
"""Point at the discard port, then check the fallback still returns work."""
dead = "http://127.0.0.1:9/v1/complete"
started = time.perf_counter()
try:
first_body_byte_ms(dead, b"{}", timeout=2.0)
reached = True
except (urllib.error.URLError, socket.timeout, TimeoutError, ConnectionError):
reached = False
result = fallback()
return {
"endpoint_reachable": reached,
"fallback_returned": result is not None,
"fallback_seconds": round(time.perf_counter() - started, 3),
}
def main() -> None:
ap = argparse.ArgumentParser()
ap.add_argument("--prompt", default="prompts/short.txt")
ap.add_argument("--attempts", type=int, default=7)
ap.add_argument("--timeout", type=float, default=20.0)
ap.add_argument("--out", default="residency_report.json")
args = ap.parse_args()
prompt = open(args.prompt, encoding="utf-8").read().strip()
body = build_body(prompt, system="residency-probe")
report = {
"payload_bytes": len(body),
"payload_secret_hits": scan_for_secrets(body),
"endpoints": [],
"offline": offline_behaviour(lambda: {"queued": True}),
}
for name in ("LOCAL_BASE_URL", "REMOTE_BASE_URL"):
url = os.environ.get(name)
if not url:
continue
row = probe(url, body, args.attempts, args.timeout)
row["name"] = name
report["endpoints"].append(row)
with open(args.out, "w", encoding="utf-8") as fh:
json.dump(report, fh, indent=2)
print(json.dumps(report, indent=2))
if __name__ == "__main__":
main()
Run it twice: once with the local runtime freshly started, and once after it has served a few requests. Those two runs are the cheapest way to see cold-start cost without building a load generator.
Step 3: Audit what actually leaves the device
The payload audit is deliberately blunt, and it is not a data-loss-prevention product. It flags obvious credential shapes and any long environment value that leaked into the serialised body, which catches the common accident of pasting a token into a system prompt. A hit is a stop sign for a remote seat and a warning for a local one, because local runtimes log too.
The byte count matters as much as the hits. A prompt carrying an entire diff can be orders of magnitude larger than a chat message, and upload time on a home connection scales with it. Record the byte count next to the latency figures, or you will attribute a bandwidth problem to model speed.
Step 4: Force the offline path on purpose
The probe points at a dead port and then calls the same fallback your agent would call. If the fallback raises instead of returning queued work, the agent needs a fallback contract before any residency change ships. If it returns queued work in a few seconds, the offline story is already credible.
This check is cheap and it prevents the most expensive migration surprise. A remote seat that disappears during a demo is survivable when work queues locally and can be resumed. It is fatal when the whole run lives in a request handler that has already returned.
A decision table you can argue with
The thresholds below are operator-tunable starting points rather than published guidance. Fill the middle columns with your own probe output and use the last two as the interpretive lens.
The table is a discussion device, not a verdict. Two teams with identical numbers can land on different rows because their data classification and support burden differ.
Where a free server genuinely wins
A recurring discussion in developer circles concerns how many intermediaries sit between a developer and a model, and residency is the concrete version of that question. The honest answer is that self-hosting is sometimes the wrong trade, especially when the local machine cold-starts constantly or has no accelerator at all. Spiky workloads, shared team artifacts and short-lived experiments all favour a remote seat that is already warm.
MonkeyCode, an open-source coding agent project, offers free model access and a free server option, which makes it a reasonable second column for the probe above. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Token allowances and server terms change, so read the project page for the current limits before you design around them, and never assume a free tier is permanent.
The useful pattern is to run the probe against a free remote seat and against your laptop on the same day with the same prompt file. That produces a defensible comparison instead of an intuition, and it costs one environment variable.
Limitations and who should not use this
The probe measures first body byte, not time to first token, and the two diverge when a server streams headers early. It is a single-machine instrument, so it is not a load test and it will not reveal behaviour under ten concurrent runs. The secret scan is a regex tripwire, not a compliance control, and it will miss bespoke credential formats.
Teams with hard data-residency obligations should keep the model call local and treat the remote column as unavailable rather than slow. Interactive products with tight latency budgets should not adopt a shared endpoint without a warm-up strategy and a measured tail. Anyone expecting deterministic throughput from a free tier should build the queue first and treat the server as an accelerator, not a guarantee.
If you want a second column without provisioning anything, point the probe at a free remote seat first and keep the local runtime as the fallback. The numbers you collect will tell you more than any migration plan written before them.#ai #localfirst #python #architecture #software #coding #development #engineering #inclusive #community
Measuring Before Migrating: A Residency Smoke Test for Local-First Agents and Free Servers
TG (telegram-parser) — обзор проекта
Telegram-parser ( github ) — это открытый некоммерческий local-first инструмент для выгрузки и аналитики комментариев Telegram-каналов в структурированный, пригодный для анализа датасет прямо на компьютере пользователя, без облачных посредников. Большинство каналов в Telegram — это не только посты, но и оживлённая «параллельная» жизнь вокруг них в чатах тг-каналов, где агрегируются комментарии к постам, формируются ветки обсуждений с реакциями, стикерами, ответами, упоминаниями, ссылками и мультимедиа. Эта живая активность часто ценнее самого поста — это то, что делает каналы и коммьюнити в Телеграме по-настоящему живыми. Проблема в том, что Telegram не даёт удобных инструментов, чтобы забрать эти данные одним куском и проанализировать их. Штатный экспорт чатов ограничен, привязан к ручным действиям и не масштабируется на десятки каналов.
Next up in our awesome list of alternatives to Arc Browser: a completely free, open-source, minimalist Mac browser featuring sidebar tabs, grouped Rooms, and local-first privacy. You should definitely check it out! 👉 https://alternativeto.net/software/nook-browser/about/
We handed the agent a webhook client that signs each outbound payload with HMAC before sending. It read the integration guide, built the serializer, and the local tests passed against a stub that accepted any well-formed body. Before opening the PR, it decided the body looked messy: the keys arrived in the order the fields were populated, so it ran a recursive sort to produce alphabetical, deterministic JSON.
The tests still passed. CI went green. The first production delivery returned 401 signature_invalid, intermittently — only for some payloads, because the receiver computed its digest over the exact bytes it received and expected the same field order the sender used in the signing step. The agent had treated object key order as cosmetic; for a signed payload, it is part of the contract.
Why the tests stayed green
The test asserted that the body parsed and that the HMAC function was called with some secret. It never compared the digest against the exact serialized bytes the receiver would rebuild, and it never exercised the real signing serializer against a verifying server. A mock that only checks shape cannot tell you whether the bytes match, so a reformatted body sailed through.
What actually broke
JSON objects are unordered semantically: a parser treats {a, b} and {b, a} as the same value. HMAC does not compare values, it hashes bytes. When the sender signs the body before sorting, then sorts before sending, the receiver rehashes the reordered bytes and gets a different digest. Same payload, same secret, valid structure — just a different byte order — fails verification. The sort happened between signing and transmission, which is exactly the window that has to stay byte-stable.
Keep the signing step and the wire bytes identical
Tests that catch this
Where the contract lives
When the signed shape and field order are stated in the integration spec, that spec should stay the source of truth locally rather than a hand-kept serialization note. This is the local-first workflow we built around in Powerduck: keep the contract on disk, validate the actual outgoing bytes against it, and replay one real request against an enforcing endpoint instead of trusting a permissive mock.
Before you call the signing path done:
A normalized, pretty body reads better in review. On the wire between two parties that sign, byte stability is the product. Order is free inside a parser; it is expensive after a digest.#aiagents #aicode #apicontract #security #software #coding #development #engineering #inclusive #community
The agent sorted the JSON body keys to tidy it up, and the request signature stopped matching
I’m building EatNeat, a mobile app that helps people turn a nutrition plan into a manageable routine—before “What should I eat?” takes over the evening.
A lot of meal-planning tools focus on creating a plan. I’m focused on what comes after: remembering it, finding something practical when hunger clocks in, and tracking what you ate without turning dinner into a spreadsheet assignment.
EatNeat is still a prototype. Here’s what’s working, what I’m figuring out, and how the mascot Bitey provides moral support.
The core loop
The app is organized around a simple cycle:
1. Create a meal plan or bring in an existing one.
2. See the day’s meals and get reminders at their scheduled times.
3. Browse meal ideas that match selected constraints.
4. Capture or mark a meal as eaten.
5. Review progress over the day and across recent days.
The Today screen brings schedule and progress together. Meals can be marked eaten when their scheduled time arrives, and progress includes estimated nutrition metrics. The user stays in control; the app helps without handing out gold stars for lunch.
AI as a helper, not the whole app
EatNeat has AI-assisted flows for personalized plans, meal recommendations, recipes, and optional photos. Requests include selected filters and nutrition targets. Practical constraints are requirements: an “under 15 minutes” meal can’t hide a 40-minute oven step, and equipment filters should match the actual cooking method.
Food photos are optional and handled separately from meal text. Without a photo, the meal still appears with a bundled animation or illustration. The recipe shouldn’t vanish because its glamour shot is late.
The app can call configured service endpoints. During local development, some AI flows can call Gemini directly with a development environment key. A production app should route requests through an authenticated server and keep provider keys off the client.
AI nutrition values are estimates. EatNeat is not a medical tool, and the prototype does not treat a missed meal as a failure or make clinical recommendations.
Local-first state and reminders
Plans and meal logs are stored on-device with AsyncStorage. Writes are debounced and serialized, and local notifications remind users at planned meal times—because remembering breakfast shouldn’t require remembering to remember breakfast.
Private meal-log photos stay on-device. If a user uploads a plan photo for parsing, it goes to the configured plan-reading service, with that made clear before upload.
The stack
The current app uses:
What is still a prototype
The accountability feed displays sample posts and does not publish to a social service. Some preview screens use illustrative nutrition data. The app doesn’t ship a backend, so server-based AI features need a configured endpoint. For now, the social circle is supportive and fictional.
I’m also breaking larger screens into focused components. The Today screen’s progress visualization and schedule calculations now live outside its route file, making the daily flow easier to evolve.
What I’m learning
The most useful decision so far: keep the basic habit loop independent of AI. Meal ideas work without generated photos; logging works without image analysis; plans can be reviewed when a service is down. A flaky model shouldn’t get veto power over lunch.
AI can add convenience, but a slow request or missing key shouldn’t block the everyday action. The meal still exists even if the pixels don’t.
EatNeat is under active development. I’m currently refining the meal flow, reminders, and progress UI while building toward a more complete, privacy-conscious backend.
If you’re building a habit-focused app, I’d love to hear how you decide which parts should depend on AI and which should keep working on their own.#buildinpublic #mobile #showdev #software #coding #development #engineering #inclusive #community
# Building EatNeat: Meal Plans, Gentle Reminders, and a Supportive Mascot
EchoVault: I Built an Open-Source AI That Helps Forgotten Memories Live On
Some of the most valuable things we own aren't expensive devices or files stored in the cloud.
They're the stories we hear from our grandparents. The recordings we make during family gatherings. The letters we keep but rarely revisit. The little details that seem ordinary today but become priceless years later.
And yet, these memories are surprisingly easy to lose.
Recordings remain buried in folders. Audio files are difficult to search. Handwritten notes get separated from their context. Eventually, the people who remember these stories might no longer be around to tell them.
For Hacktoberfest Week 4, I wanted to explore a different interpretation of Ghost in the Machine.
What if AI could help preserve the voices, stories and memories that time might otherwise erase?
That idea became EchoVault.
What I Built
EchoVault is an AI-powered memory archive designed to help people preserve, organize and rediscover personal stories.
Instead of treating old recordings and written memories as files that simply sit on a device, EchoVault helps turn them into a searchable collection of stories.
The workflow is simple:
The goal isn't to make AI replace the people behind these memories. It's to make their stories easier to preserve, revisit and share.
EchoVault is about helping memories survive, not pretending that AI can bring someone back.
Demo
In the demonstration, I'll upload a sample recording, show how EchoVault transcribes and summarizes it, and retrieve a detail from the saved story.
I'll also demonstrate how the original recording and its generated content remain connected, so users can verify the archive entry against its source.
Code
The repository contains the application source code, setup instructions, model configuration, dependencies and documentation for running EchoVault.
The goal is to make the project reproducible so other developers can experiment with the models, improve the archive workflow or adapt it to their own use cases.
How I Built It
I built EchoVault around Python, Streamlit and open-source AI tools.
The architecture separates transcription, language-model processing, storage and retrieval so that each component can be tested independently.
1. Transcription
For supported audio recordings, EchoVault uses OpenAI Whisper running locally to convert speech into text.
The transcript makes a recording searchable and gives the next stage a text representation to work with.
Transcription accuracy still depends on recording quality, accents, background noise and the languages supported by the selected model.
2. Memory extraction
An open-weight Gemma model processes the transcript and generates structured metadata such as a title, summary, key events and people mentioned.
I designed this stage to distinguish the original story from AI-generated interpretations. The original transcript remains available for verification.
3. Searchable archive
EchoVault stores the original files and associated metadata, making it possible to revisit individual memories rather than repeatedly processing every recording.
4. Grounded question answering
The question-answering feature retrieves relevant material from the archive and uses it to formulate an answer.
Rather than treating the model as an unrestricted storyteller, the application is designed to ground responses in the saved records.
If a question cannot be answered from the available material, the system should say so instead of inventing a memory.
5. A human-centered interface
I used Streamlit to create an interface for uploading recordings, reviewing transcripts, exploring archived stories and generating remembrance pages.
I wanted the experience to feel more like a digital archive than a generic chatbot.
This part of the project taught me that building a useful AI application involves more than prompting a model. Data handling, validation, retrieval and the way users verify generated content all matter.
Why Does Open Innovation Matter?
For a project dealing with personal memories, the choice of AI infrastructure matters as much as the interface.
Privacy and control
Family recordings and personal stories can contain sensitive information.
Running transcription and language models locally can reduce the need to send this material to external AI services. In a fully local configuration, recordings and generated content can remain on the user's device.
This is an architectural goal, not an automatic guarantee: the complete application must be checked for external network calls, telemetry, cloud storage and other data transfers before claiming that it is fully offline.
Freedom to experiment
Open-weight models give developers more flexibility to choose models, change prompts, compare performance and adapt the application to their requirements.
For EchoVault, this means being able to experiment with different transcription and language models without designing the entire product around a single proprietary AI API.
Preservation beyond a single service
An archive should not become inaccessible simply because a hosted service changes its pricing, API or availability.
Keeping original recordings, transcripts and metadata in portable formats makes the collection easier to back up and migrate independently of a particular model provider.
Making the technology accessible
A local-first approach can reduce dependence on recurring hosted inference charges, although users still need compatible hardware, storage and electricity.
The larger point is that useful AI tools should be adaptable and understandable, rather than locked into an infrastructure that their users cannot control.
My Agent Session
I used python to implement and test EchoVault.
The session documents the decisions, experiments and debugging steps that shaped the final application.
The Spirit of It
The theme Ghost in the Machine made me think about how technology interacts with memory.
A recording is only a collection of bits until someone listens to it. A transcript is only text until someone recognizes the person, place or moment behind it.
AI cannot replace the human experience of remembering someone. It can, however, help make the material we leave behind easier to preserve and understand.
I want EchoVault to make it easier for someone to rediscover a story they had almost forgotten, find a detail in an old recording, or bring together memories that would otherwise remain scattered across devices.
The most meaningful result isn't an AI-generated story that sounds convincing.
It's helping someone find a real story worth remembering.
What's Next?
I'd like to explore better multilingual transcription, additional document formats, richer search, exportable remembrance pages and tools for organizing large collections of recordings.
I also want to improve source attribution so users can move directly from a generated summary or answer to the original passage that supports it.
EchoVault began with a simple question: what if we could use open-source AI to help preserve the stories that matter to us?
My answer is a small step toward making personal archives more accessible, searchable and useful.
Because sometimes the most valuable thing an AI can do isn't generate something new.
It's help us hold on to something we already have.
devchallenge #hf26challenge
#devchallenge #hf26challenge #software #coding #development #engineering #inclusive #community
EchoVault: I Built an Open-Source AI That Helps Forgotten Memories Live On
Once is more than enough. McClure, Pennsylvania's the "BEAN SOUP," is a popular yearly festival begun by a Civil War Cook, centered on the most insipid soup ever tasted. People are nuts. #thebeansoup #beansoup #civilwar #civilwarbeansoup #civilwarbeansoupreunion #reunion #localfestival #localfestivals #pennsylvania #pennsylvanialocalfestivals #local #localeventsforlocalfolks #localevents #localfolk #localfolks #insipidfood #insipidsoup #badsoup #soup #soupnazi #funwithfriends #outings
Weekly Wrap-Up
It's been a beautiful week here at the 'stone! We decided to reschedule our cemetery trip so Casey could work with Jr to bring down one of those menacing pear trees that was fixin' to fall on his shed. He spent every day doing prep - bringing down branches closer to the ground - and on Saturday they rented one of those cherry-picker-like things and Casey had the time of his life running it. Meanwhile, out in the garden, the Pineapple Sage, the One We've Been Waiting For as we continued to […]