Search

24 results for “munkeycop”

Dev Community @[email protected]
2026-10-11

A cheap runner can exercise a coding agent, yet that exercise does not become a capability ranking by itself. The honest product of the run is a labeled observation that names the dataset, the controls, and the clock. Readers should treat any single score from an unpaid lane as a lane note rather than a market claim. A publishable comparison starts only after those boundaries sit in a card that another engineer can replay.
Coding-agent notes often arrive with a polished percentage and almost no account of the task pack behind it. That pattern resembles a race result printed without the track length, the wind, or the class of car. A developer comparing tools then confuses access to a runner with proof that the agent solves a held-out class of work. The confusion grows when a free route lowers the cost of trying ideas while leaving the measurement problem untouched.
MonkeyCode can host that rehearsal when the operator uses its free model access and its free server option. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Those availability claims describe access only, and they do not state quotas, hardware, duration, or permanent terms. A card that hard-codes a token grant or a server size would invent facts the operator must read from current documentation at trial time.
The pack is the stick

The dataset is a small task pack whose manifest, content hash, and holdout split stay fixed after the first scored run. Each item carries a stable identifier, a language tag, a fixture directory, and a pass rule that a script can apply. Training-like prompts, leaked solutions, and tasks copied from a vendor demo stay in an excluded bucket outside the scored set. The pack works like a measuring stick, and sanding that stick after the number appears turns the trial into a story.
Useful metrics should describe completion, repair cost, and instability rather than collapsing the trial into one vanity percentage. Completion is the fraction of holdout items whose tests pass when the frozen command runs without manual edits. Repair cost is the median tool-call count, kept apart from median wall time so a fast failure is not a cheap success. Instability is the mean pass spread across repeats of one holdout item, so one lucky seed cannot stand for the lane.
Controls exist so the free route is not quietly compared with a different class of runner or clock. The same task hash, the same timeout, the same retry ceiling, and the same network policy should bind every arm. A free server may queue, throttle, or rotate capacity, so the card stores the start time and a lane label. If the route is selected dynamically, the log captures the route name returned at runtime rather than a name guessed in the draft.
A harness that refuses a rank

The workflow below is a proposed harness, not a result from an executed study and not a claim about any live quota. An author drops fixtures into a directory, writes a manifest, and lets the script refuse a score when required controls are missing. That refusal is the point, because a missing control is more informative than a confident table built on an unlocked pack. The same script can later sit beside a paid runner, which keeps the comparison about the card rather than about a brochure.
#!/usr/bin/env python3
"""Proposed, unexecuted lane card. It records controls and refuses a rank."""
from __future__ import annotations
import hashlib
import json
import statistics
import time
from pathlib import Path
REQUIRED = ("task_hash", "lane_label", "timeout_s", "retry_ceiling", "network_policy")
def hash_pack(root: Path) -> str:
digest = hashlib.sha256()
for path in sorted(p for p in root.rglob("*") if p.is_file()):
digest.update(path.relative_to(root).as_posix().encode())
digest.update(path.read_bytes())
return digest.hexdigest()
def summarize(trials: list[dict]) -> dict:
holdout = [item for item in trials if item.get("split") == "holdout"]
by_item: dict[str, list[int]] = {}
calls, walls = [],
[] for item in holdout:
by_item.setdefault(str(item.get("id", "unknown")), []).append(
1 if item.get("passed") else 0
)
if "tool_calls" in item:
calls.append(item["tool_calls"])
if "wall_ms" in item:
walls.append(item["wall_ms"])
spreads = [max(flags) - min(flags) for flags in by_item.values() if flags]
passed = sum(flags[-1] for flags in by_item.values()) if by_item else 0
return {
"holdout_items": len(by_item),
"completion": (passed / len(by_item)) if by_item else None,
"median_tool_calls": statistics.median(calls) if calls else None,
"median_wall_ms": statistics.median(walls) if walls else None,
"mean_item_pass_spread": statistics.mean(spreads) if spreads else None,
}
def build_card(root: Path, trials: list[dict], controls: dict) -> dict:
missing = [key for key in REQUIRED if not controls.get(key)]
if missing:
raise SystemExit(f"refusing score; missing controls: {', '.join(missing)}")
if controls["task_hash"] != hash_pack(root):
raise SystemExit("refusing score; task pack hash drifted")
return {
"generated_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
"task_hash": controls["task_hash"],
"lane_label": controls["lane_label"],
"controls": controls,
"metrics": summarize(trials),
"not_marketing": [
"not a general coding-ability claim",
"not a price or quota claim",
"not valid after the pack or route changes",
],
"rank_emitted": False,
}
if __name__ == "__main__":
root = Path("task_pack")
controls = json.loads(Path("controls.json").read_text())
trials = json.loads(Path("trials.json").read_text())
card = build_card(root, trials, controls)
Path("lane_card.json").write_text(json.dumps(card, indent=2) + "\n")
print(json.dumps({"status": "card_written", "rank_emitted": False}))
A companion control file keeps the route explicit, and the example values below are placeholders rather than observed product limits. The timeout, the retry ceiling, and the network policy are choices of the study, not features inferred from a vendor page. The lane label should quote the access class that the operator confirmed on the day of the run. Leaving those fields blank is preferable to filling them with a remembered grant from an older post.
{
"task_hash": "<sha256 of task_pack>",
"lane_label": "free-model-access/free-server; terms read at runtime",
"timeout_s": 120,
"retry_ceiling": 1,
"network_policy": "no egress beyond the fixture mirror"
}
The commands that follow are a local rehearsal sketch, and they do not contact a model or reserve a server. An author can create the pack, compute nothing remote, and confirm that a missing label stops the score. That dry run is useful because it teaches the refusal path before any token is spent on a live route. A later live run should append the route name returned by the server, then regenerate the card without editing the holdout fixtures.
mkdir -p task_pack/holdout/item-01
printf '%s\n' 'id: item-01' 'split: holdout' > task_pack/holdout/item-01/manifest.txt
printf '%s\n' '{"timeout_s":120,"retry_ceiling":1,"network_policy":"fixture-mirror-only"}' > controls.json
printf '%s\n' '[]' > trials.json
python3 lane_card.py
# expected: refusing score; missing controls: task_hash, lane_label
A written card that survives the checks still carries rank_emitted set to false, which blocks a casual sort of vendors. The metrics object may show a completion fraction, two cost medians, and a pass spread, each tied to the stored hash. A reader who cannot find the negative claims should treat the file as incomplete even if the fraction looks strong. That rule is the difference between a lab note and a sentence that could be pasted into an advertisement.
What the numbers are not allowed to say

These numbers are not marketing because they answer a narrow question under named constraints, and they expire when those constraints change. A higher completion rate on a small pack does not prove general coding ability, a lower price, or a permanent free grant. The card should state the negative claims in plain language, including what was not measured and which arms were absent. Publishing the hash next to the table lets a later reader see whether the pack moved after the headline was written.
Failure analysis belongs in the same narrative as the score, because a green fraction without its misses invites overreading. A timeout cluster may say more about queue behavior on a free server than about the agent's repair skill. A pass that appears only after a second retry tells a different story from a pass on the first frozen attempt. Recording those patterns beside the median keeps the result closer to a lab note than to a launch sentence.
Synthetic trials can teach the reader how to hold that line before a live route is involved at all. Suppose two repeats of one holdout item pass on the first seed and fail on the second, while wall time jumps from a quiet run to a queued run. The completion field may still look acceptable if only the last flag is counted, but the pass spread exposes the instability. A reviewer who sees that spread should keep the lane label attached and decline to promote the fraction into a ranking table.
Who should leave this method alone

This approach should not be used to certify a model for production, to claim a security review, or to rank vendors for a purchase committee. It also fails when the task pack is private, when tests depend on live paid APIs, or when the author edits fixtures after seeing failures. A free server option can disappear, slow down, or change routing, so a result from one afternoon is a sample of that afternoon. Teams that need a contractual service level should repeat the same card on an explicit environment before they trust the delta.
An engineer rehearsing this card can check MonkeyCode's current free model access and free server option before sharing a score. The live terms should be copied into the lane label before any score is shared with another reader. The useful habit is the refusal to rank an unlocked pack, and that habit remains valuable even if the hosting choice changes. A later reader should be able to reject the number without guessing which track, which wind, or which class of car produced it.#ai #testing #python #programming #software #coding #development #engineering #inclusive #community
A Cheap Run Needs a Fixed Holdout Pack

#usrbinenv #testing #inclusive #engineering #development #python
Dev Community @[email protected]
2026-10-11

A small platform team cloned a coding-agent repository on Thursday and started the host process before lunch. The health route returned a short success body, and a model call through the free access path also returned text. The channel treated those two green signals as a release note and moved the change toward review. Three days later the same sequence failed in a pull-request check because a tool field had been renamed.
The port still answered, which made the failure look like a flaky test rather than a broken tool trace. That confusion is common when a smoke curl is asked to stand in for a closed workflow loop. A curl proves that some process accepted bytes, much as a doorbell proves that a building has electricity. It does not prove who opened the door, which room was reached, or whether the same visitor would be recognized tomorrow.
MonkeyCode is relevant here only as one place where a free server option and free model access can host that early smoke path. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Only two product facts are used here: a free server option and free model access, with no model name or quota attached. Readers should confirm both facts in the current project docs, because access limits can change and this article does not freeze them.
Recent public discussion has noted that model quality can rise while everyday software practice stays loose. That observation fits this failure, because a better reply does not tighten an unpinned host, an unpinned schema, or an unpinned tree. The practical question is which claim each separate probe is allowed to support in a review. A useful workflow separates reachability, execution, and replay before anyone writes a summary sentence for the channel.
Myth: a listening port means the loop is closed

Developers repeat this claim because local demos reward the first successful status code more than the second differing one. The corrected model treats reachability as a gate, not as a verdict about tools, files, or repeated behavior. Record the process command, the bound port, and the status code, then stop the claim at that boundary. Anything past the boundary belongs to a different probe with its own fixture and its own failure name.
On Thursday the health body was enough to satisfy the channel, and that social fact is not technical evidence. A process can listen while its tool registry still points at yesterday's field names and yesterday's paths. The analogy is a shop with the lights on and the cash drawer disconnected from the inventory system. Customers can enter the shop, and the inventory books can still be wrong at the close of day.
Myth: a model paragraph means the tool actually ran

A fluent reply can describe a file change without the host ever having executed the described tool call. That gap resembles a meeting note that summarizes a decision nobody entered into the issue tracker. Evidence for execution is a structured tool result, a schema version, and a diff against the fixture repository. If any of those three artifacts is missing, the paragraph remains narration rather than an integration result.
Free model access does not change that standard, because availability of a model path is not identity of a schema. The same caution applies to the free server option, which can serve the process without freezing the registry inside it. Ask the run for a machine-readable tool result, and keep the prose reply in a separate field. Reviewers can then reject narration that lacks the structured sibling and the schema version it claims to match.
Myth: sending the same prompt again counts as a replay

A second prompt can hit a different tool schema, a different working tree, or a different unspoken default. Replay requires the same request body, the same repository revision, and the same recorded tool responses. Without those pins, the second run is a new experiment that happens to share some wording with the first. Store the pins beside the output so a later reader can see what was held constant and what was not.
Retry culture makes this myth sticky, because a green second attempt feels like confirmation rather than a new sample. Confirmation needs a recorded first sample and a diff of the pins, not a similar sentence in the terminal. If the schema version changed between attempts, label the second attempt as a migration check instead. If the revision changed, label it as a new build, even when the prompt text matches.
A corrected mental model

Think of the free server as a shared workbench and the free model path as a shared measuring tape. Both can be useful on the first afternoon, and neither becomes a calibrated instrument by being available. Calibration is the evidence file you write: revision, schema version, probe name, and a named outcome. When that file is absent, a green curl should stay in the lab notebook as a reachability note only.
The closed loop is the path from pinned tree, through a named tool call, to a stored result that a second machine can classify. Reachability opens the bench, the execution probe checks the measuring marks, and replay checks whether the marks still agree. Collapsing those three checks into one status code is how Thursday's note became Monday's review surprise. Naming the three checks separately is slower by a few minutes and cheaper than a reverted review thread.
An unexecuted classifier you can adapt

The following checker is a proposal, not a captured run from this account, and it invents no product metrics. It reads a local JSON fixture that you produce, then classifies each probe before any human summary is written. Adapt the field names to your host, and do not treat the sample paths as documented product endpoints. The script fails closed when a required pin is missing, which is the behavior a smoke curl usually lacks.
#!/usr/bin/env python3
"""Proposal only: classify a local evidence fixture. Not a recorded product run."""
import json
import sys
REQUIRED = ("git_revision", "schema_version", "probe", "outcome")
PROBES = {
"reachability": {"allowed_outcomes": {"listening", "refused", "timeout"}},
"execution": {"allowed_outcomes": {"executed_match", "executed_mismatch", "tool_missing"}},
"replay": {"allowed_outcomes": {"pins_match", "pins_drift", "incomplete"}},
}
def classify(row: dict) -> dict:
missing = [key for key in REQUIRED if not row.get(key)]
probe = row.get("probe")
outcome = row.get("outcome")
if missing or probe not in PROBES:
return {"verdict": "incomplete", "missing": missing or ["probe"], "claim_scope": "none"}
allowed = PROBES[probe]["allowed_outcomes"]
if outcome not in allowed:
return {"verdict": "invalid_outcome", "probe": probe, "claim_scope": "none"}
scope = {
"reachability": "port_only",
"execution": "tool_execution_only",
"replay": "pinned_comparison_only",
}
[probe] passed = outcome in {"listening", "executed_match", "pins_match"}
return {"verdict": "pass" if passed else "fail", "probe": probe, "claim_scope": scope}
def main() -> int:
document = json.load(sys.stdin)
rows = document.get("probes", [])
report = [classify(row) for row in rows]
scopes = {item["claim_scope"] for item in report if item["verdict"] == "pass"}
needed = {"port_only", "tool_execution_only", "pinned_comparison_only"}
closed = needed <= scopes
json.dump({"probes": report, "closed_loop": closed}, sys.stdout, indent=2)
sys.stdout.write("\n")
return 0 if report and all(item["verdict"] == "pass" for item in report) else 1
if __name__ == "__main__":
raise SystemExit(main())
A reader can generate the surrounding pins with ordinary commands, then fill the fixture by hand from observed responses. The commands below record identity of the tree and the clock, not the performance of any host or model. They are safe to run in a disposable clone, and they still do not prove a tool call happened. Paste their output into the fixture fields rather than into a chat summary that will be forgotten.
git rev-parse HEAD
git status --porcelain=v1
python3 --version
date -u +%Y-%m-%dT%H:%M:%SZ
{
"git_revision": "replace-with-git-rev-parse",
"schema_version": "replace-with-tool-schema-id",
"probes": [
{
"git_revision": "replace-with-git-rev-parse",
"schema_version": "replace-with-tool-schema-id",
"probe": "reachability",
"outcome": "listening"
},
{
"git_revision": "replace-with-git-rev-parse",
"schema_version": "replace-with-tool-schema-id",
"probe": "execution",
"outcome": "executed_mismatch"
},
{
"git_revision": "replace-with-git-rev-parse",
"schema_version": "replace-with-tool-schema-id",
"probe": "replay",
"outcome": "incomplete"
}
]
}
python3 classify_probes.py < evidence.json
Read the machine-readable report before you accept a prose recap of that same afternoon in the team channel. A pass on reachability with an executed_mismatch is the Thursday shape: the door opened, and the inventory field did not match. An incomplete replay means you may not claim that the second attempt confirmed the first recorded sample. The closed_loop flag becomes true only when all three scopes have passed, which the sample fixture above should refuse.
What the Thursday notes should have contained

The useful note would have said the port listened, the schema id was unread, and no replay pins were stored. That sentence is less exciting than a release claim, and it is the one a later reviewer can trust. Add the git revision from the disposable branch and the tool schema identifier from the host response. Leave model marketing names out of the note unless the current docs and the response metadata agree.
If the execution probe returns executed_mismatch, stop and diff the fixture schema against the live registry. Do not paper over that diff with a second prompt that asks the model to explain the error. An explanation can be helpful after the mismatch is recorded, and it is not a substitute for the mismatch record. The corrected habit is to file the structured failure first and to write the narrative only after that file exists.
Limitations and who should skip this

This approach does not load-test the host, price a loop, or prove that a warm process kept state stable. It also does not verify authentication, secret redaction, or multi-tenant isolation on a shared free server. Teams that need those guarantees should use their own audited environment and their own written acceptance tests. People debugging a one-off prompt with no repository fixture will get little value from the extra pins.
Skip this workflow if you cannot record a repository revision, because the classifier will refuse to pass. Skip it if your question is capacity or cost, since those questions need different instruments and different fixtures. Skip it if you need a vendor benchmark, because this article records no timings and names no model. The sample outcomes are labels for a local file, not measurements collected from any public or shared run.
If the free server option is already available to you, point the three probes at a disposable fixture branch before you announce a result. Read the current docs for access limits before you quote any allowance in a design review. That habit keeps a Thursday smoke signal from turning into an accidental release claim on the following Monday.#testing #devops #python #ai #software #coding #development #engineering #inclusive #community
A Smoke Curl Is Not a Closed Loop: A Myth-Busting FAQ

#usrbinenv #testing #inclusive #engineering #development #python
Dev Community @[email protected]
2026-10-13

Keep every working tree on the local disk. Send one redacted hunk only after local checks pass. A remote pass stays optional, never the default path.
Pasting a whole repository is the common failure. That habit spends latency and leaks secret lines. A filing cabinet should not travel for one page.
Local inference starts without a network round trip. Remote inference waits on DNS, TLS, and queue time. For a tiny rename, that wait rarely pays.
Measure the path before you trust a feeling. A short timed connect shows whether the route answers. Reachability is not model quality, and it is not a quota.
Secrets ride inside diffs more often than people think. Tokens, private keys, and customer rows do not belong on a free server. Scan the hunk on disk before any upload begins.
Offline work is a normal state, not an error page. Write the request to a local queue and keep editing. Promote that queue only after the link returns.
A free server wins on a narrow class of jobs. The hunk is small, clean, and hard for local models. You still keep tests and secrets on the laptop.
MonkeyCode can serve as the remote hop here. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator describes open-source code plus free model access and a free server.
This draft claims no quota, hardware, or permanent tier. Check current project docs before you plan around them. A blog post cannot freeze terms that later change.
Treat the free server like a reading room across town. You carry one page, never the whole cabinet. You leave when the room closes, and the cabinet stays.
Latency has three visible parts, even without a benchmark lab. Name lookup, handshake, and first token are separate waits. A local process skips all three, then spends only compute.
That split matters on flaky links more than on a campus LAN. A laptop on a train loses the path between keystrokes. The edit should not freeze while a socket times out.
Secret risk is not the same problem as slow risk. A fast server can still be the wrong destination. Speed does not wash a key that left the disk.
Offline risk is the third axis, and it is blunt. If the route is down, remote quality is irrelevant. The honest result is a queued file beside the diff.
The decision below uses those three axes and nothing fancier. Payload bytes stand in for the cabinet size. A secret hit or a dead route blocks the trip.
Thresholds in the script are example knobs, not measured optima. Eight thousand bytes is a starting fence, not a law. Change it after you inspect your own diffs.
Classify one hunk

The artifact below is a proposal, not a measured benchmark. It classifies one diff as local, queued, or remote. Run it on a sample file before you trust it.
#!/usr/bin/env python3
"""Proposal only. Routes one diff. Not a benchmark."""
import os
import re
import socket
import sys
from pathlib import Path
# Example fence, not a measured optimum.
MAX_REMOTE_BYTES = 8000
CONNECT_TIMEOUT_S = 1.5
SECRET_RULES = (
re.compile(r"AKIA[0-9A-Z]{16}"),
re.compile(r"-----BEGIN [A-Z ]*PRIVATE KEY-----"),
re.compile(r"(?i)\b(api[_-]?key|secret|token|password)\b\s*[:=]\s*\S+"),
)
def load_diff(path: Path) -> str:
return path.read_text(encoding="utf-8", errors="replace")
def secret_hits(text: str) -> int:
return sum(len(rule.findall(text)) for rule in SECRET_RULES)
def route_open(host: str, port: int) -> bool:
try:
with socket.create_connection((host, port), CONNECT_TIMEOUT_S):
return True
except OSError:
return False
def decide(text: str, host: str, port: int) -> str:
if secret_hits(text) > 0:
return "local"
if len(text.encode("utf-8")) > MAX_REMOTE_BYTES:
return "local"
if not text.strip():
return "local"
if not host:
return "local"
if not route_open(host, port):
return "queued"
return "remote"
def main() -> int:
if len(sys.argv) != 2:
print("usage: route_hunk.py DIFF_FILE", file=sys.stderr)
return 2
text = load_diff(Path(sys.argv[1]))
host = os.environ.get("REMOTE_HOST", "")
port = int(os.environ.get("REMOTE_PORT", "443"))
choice = decide(text, host, port)
print(choice)
if choice == "queued":
queue = Path("queue")
queue.mkdir(exist_ok=True)
target = queue / Path(sys.argv[1]).name
target.write_text(text, encoding="utf-8")
print(f"queued {target}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
Capture a hunk with git, then classify it. Keep the command local until the script prints remote. Do not pipe the diff to a network tool yourself.
git diff --unified=3 -- src/app.py > /tmp/hunk.diff
python3 route_hunk.py /tmp/hunk.diff
An unset host means the script stays local. That empty host is intentional, not a bug. Set a host only when you mean to test the path.
REMOTE_HOST=example.invalid REMOTE_PORT=443 python3 route_hunk.py /tmp/hunk.diff
Use a host you operate or the docs name. This article does not invent a product endpoint. The name example.invalid should land in the queue.
A clean sample looks like a few changed lines. No keys, no dumps, and no giant fixture. That shape is the page you might carry across town.
diff --git a/src/app.py b/src/app.py
--- a/src/app.py
+++ b/src/app.py
@@ -10,3 +10,3 @@
-def timeout_ms():
- return 200
+def timeout_ms():
+ return 350
That hunk is small and has no secret shape. If the route answers, the script may print remote. If the route fails, it prints queued and writes a copy.
A dirty sample should never print the remote word. Plant a fake key in a scratch diff and confirm the local result. Delete the scratch file when the check ends.
printf 'api_key = "not-a-real-secret"\n' > /tmp/dirty.diff
python3 route_hunk.py /tmp/dirty.diff
The expected printed word is local, not remote. The regex is a coarse net, not a full secret scanner. It misses some formats and flags some harmless lines.
False positives belong on the local side of the gate. A missed remote call is cheaper than a leaked row. Widen the rules only after you review real diffs.
False offline signals happen on slow DNS too. A one point five second timeout is an example, not a standard. Raise it if your network is slow but stable.
Do not treat a remote print as permission to upload the tree. Upload the same hunk the script read, or upload nothing. The queue file is still private local state.
When the choice is remote, ask for a bounded review. Name the function, the bug, and the test you already hold. Refuse a request that needs the whole history.
A useful prompt stays smaller than the hunk fence. You can paste the diff into a free-server session by hand. This script does not call an API, and it does not store a token.
That gap is deliberate, because upload would hide the last check. The printout is a gate, and you are the latch. Automatic shipping would train a careless copy habit.
Local remains better for rename sweeps and format noise. Those diffs are wide, dull, and full of names. A remote model adds delay and little new structure.
Remote remains better for a short logic knot. You want a second reading of a clean algorithm. Free model access matters only in that narrow slot.
If the free server is down, the queue still preserves the ask. Replay is a later command, not a background daemon. A daemon that retries blindly can ship a stale hunk.
python3 route_hunk.py queue/hunk.diff
Compare the queued bytes with the current git diff before replay. If the bytes differ, discard the queue file. Stale context produces confident notes that miss the edit.
There is no latency table in this piece, on purpose. No run was executed for publication, so no millisecond claim is honest. Time your own path if a number will guide a team rule.
The same restraint covers cost and capacity claims. Free model access can still meet a fair-use cap you have not read. A blog sentence cannot reserve capacity for your sprint.
Where the route stops

Limitations sit in the regex, the timeout, and the byte fence. Binary diffs may decode as noise and look small. Review binary changes by hand, because the scanner is text-biased.
A successful connect proves a socket, not a healthy model. Auth, region, and capacity can still fail after the port opens. Treat remote as eligible, then confirm with a manual prompt.
The script does not understand licenses or customer contracts. A clean regex is not a legal review. If policy forbids external models, the only result is local.
Do not use this approach for production incident logs. Those files mix tokens, hostnames, and personal data. Keep that material on disk even when the hunk looks short.
Do not use it when the local tree itself is untrusted. A malicious diff can try to confuse a reviewer. Remote reasoning will not purify a poisoned patch.
Do not use it as a silent team default. Each developer should see the printed choice first. Hidden routing trains people to forget where text went.
Skip the free server when you cannot read the current terms. Skip it when the docs disagree with a remembered quota. Memory is a poor source of capacity planning.
If your work is air-gapped, stop at the local branch. Leave the host unset and treat remote as dead code. The queue remains optional even in that setting.
The workflow is a loop you can say in one breath. You diff, scan, classify, and then edit. Cross the wire only for a page that earned the trip.
If you already work this way, the free server is a spare reading room. Read the live MonkeyCode docs, then try one clean hunk. Keep the cabinet at home either way you choose.#ai #python #devops #security #software #coding #development #engineering #inclusive #community
Route by Payload, Not by Habit

#usrbinenv #inclusive #engineering #development #python #coding
Dev Community @[email protected]
2026-10-13

A fluent patch is not a pass. A free server that happens to boot is not a pass either. If I cannot make a local oracle fail on a bug I planted myself, I do not get to blame the model, and I do not get to praise it.
That is the conclusion. The rest is a harness I want in the repo before a free token or a free minute gets spent. This draft does not report a live run, because I did not execute one for it.
You know the scene. The assistant nods, the diff looks tidy, and the slice is still wrong. Who gets the lecture, the model or the missing test?
Blame is a warm drink. It does not fail a build. I want the check to fail first, on purpose, with a command I can rerun when I am bored and suspicious. No leaderboard, and no borrowed authority. Just an exit code that was already red before any model was allowed to speak.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode is an open-source project, and two availability claims here are operator-supplied rather than measurements from 13 October 2026: free model access, and a free server option.
I was also told the free model side includes a grant on the order of 10 million tokens. Confirm that figure, which models it actually covers, and the server terms on the current project page before you plan a week around them. I will not invent model names, hardware, durations, or a win rate.
The call site still has to live in git, next to the plants, or the evaluation is a chat you cannot replay. Free access is a budget for that replay. It is not evidence the patch was right. Delete the product name and the stop-rule still applies to any client you already have.
People keep saying the models got sharper while the bugs stayed put. Maybe they did. Sharpness without a red oracle is a brighter flashlight in a room you never filled with smoke. I am not grading a summer of announcements. I am asking whether my own gate can fail a bug I already understand.
Plant something boring

Clever bugs flatter the person who planted them. Boring bugs audit the gate. I keep one function that returns one element too many, the kind a tired reviewer can spot, because that is the point.
If the oracle cannot see a bug a tired human can see, why would I trust it on the bugs I miss at 1 a.m.? The plant stays small so the failure is about the gate, not about my talent for hiding a defect in a maze.
# harness/plants.py
# Proposal. Not a recorded run.
from dataclasses import dataclass
@dataclass(frozen=True)
class Plant:
name: str
buggy: str
fixed: str
TAKE = Plant(
name='take-off-by-one',
buggy='''def take(xs, n):
return xs[: n + 1]
''',
fixed='''def take(xs, n):
return xs[:n]
''',
)
The test is equally dull. Dull is a feature, not a lack of ambition. It names one boundary and refuses to negotiate.
# tests/test_take.py
from take import take
def test_take_does_not_overshoot():
assert take(['a', 'b', 'c'], 2) == ['a', 'b']
A writer script keeps me from pasting the bug by hand and then forgetting which variant is on disk. Hands are how plants get quietly fixed before the model ever sees them.
# harness/write_plant.py
# Proposal. Overwrites take.py from the plant table.
import argparse
from plants import TAKE
BODIES = {'buggy': TAKE.buggy, 'fixed': TAKE.fixed}
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument('--name', required=True)
parser.add_argument('--variant', choices=sorted(BODIES), required=True)
args = parser.parse_args()
if args.name != TAKE.name:
raise SystemExit('unknown plant')
with open('take.py', 'w', encoding='utf-8') as fh:
fh.write(BODIES[args.variant])
if __name__ == '__main__':
main()
Write the buggy body into place and let pytest speak before you do. If you feel an urge to peek at a completion first, that urge is the experiment trying to cheat.
python harness/write_plant.py --name take-off-by-one --variant buggy
python -m pytest -q tests/test_take.py
echo oracle_exit=$?
If that exit is 0, stop. The plant is invisible. Asking a free model to review an invisible bug is a play, not a measurement. Fix the test, and do not touch the prompt.
A smoke alarm you have never held to smoke can still beep in a drill. The red pytest is the smoke. Without it I am admiring the plastic cover and calling it safety.
Only then may the model look

The prompt stays poorer than the test file. I do not paste the expected list. I do not ask whether the function is good, because that adjective is how a yes gets hired as an engineer.
The instruction I send is plain. One function is attached. A test named test_take_does_not_overshoot fails. Return either a line that starts with REJECT and one sentence, or a unified diff that changes only take. Do not claim the tests were run. Save the reply next to the plant name. Feeling is not a column.
# harness/classify.py
# Proposal. A malformed reply is a miss, not a cue to retry.
def classify(reply: str) -> str:
text = reply.strip()
if text.startswith('REJECT:'):
return 'reject'
if text.startswith('---') or text.startswith('diff '):
return 'diff'
return 'malformed'
Then the mechanical half, still on the laptop, still too boring to argue with:
python harness/extract_diff.py reply.txt > candidate.diff
patch -p0 < candidate.diff
python -m pytest -q tests/test_take.py
echo after_model_exit=$?
Two exits. They do not get to shake hands. The first had to be non-zero or the day is void. The second may be zero, and that is a different sentence.
A REJECT on a function the oracle already hates can be a real hit. The model refused to launder a bad slice. A minimal diff that goes green is a different hit. Would you average those and call the average quality? I will not. Unlike events do not become one fact just because they share a spreadsheet.
If patch refuses the diff, I do not retry inside the scorer. A diff that does not apply is a miss. Retry-until-apply is another study, and it will eat a grant while wearing a lab coat.
The ledger is the third gate

Free tokens disappear in the rephrase. You paste the traceback, you ask again, and you call it iteration. The meter calls it lunch.
This ledger is local, and it should stay proud of that. If the provider meter disagrees, the provider wins. Write the gap down. Do not litigate it in the README and pretend the argument is a measurement.
# harness/ledger.py
# Proposal. Counts what this process tried to spend.
class Ledger:
def __init__(self, ceiling: int):
self.ceiling = ceiling
self.spent = 0
def charge(self, prompt_tokens: int, completion_tokens: int) -> None:
self.spent += prompt_tokens + completion_tokens
if self.spent > self.ceiling:
raise SystemExit(
f'stop: spent {self.spent} > ceiling {self.ceiling}'
)
Set the ceiling to a slice, not to the folklore of the whole grant. A few thousand tokens will tell you whether the loop is bounded. Ten million, if the project page still says that when you look, is headroom. It is not a result I measured, and it is not a promise I can extend.
Quotas move. A method that only works if a published number never changes is not a method. Re-read the page the day you run the card, and write the number you actually saw into the notebook next to the date.
python harness/run_eval.py --plants harness/plants --ceiling 4000 --dry-run
The dry-run flag prints the prompt and charges nothing. I want it the way I want a seatbelt in a parking lot. An evaluation you cannot rehearse will get rehearsed on the grant, by accident, late, when you are sure this one extra call does not count.
The free server witnesses. It does not judge.

I may use the free server only after the local oracle is already red, and only to ask a narrower question. Does the same pytest exit match on that box? That is a witness statement about the room. It does not grade taste, tone, or how confident the completion sounded.
Laptop red and server green means portability, not genius. Both red means the patch missed, or the plant is still in the tree. Both green before the model spoke means the day is invalid, so go back and make smoke.
I am not asking a remote box whether a model is smart. Smart is not a command. Pytest is a command. If the documented runner is different from the placeholder below, keep the exit code and throw my hostname away.
# Placeholder only. Use the documented free-server run command.
# example.test is not a product host.
ssh [email protected] -- python -m pytest -q tests/test_take.py
echo witness_exit=$?
Keep the exit code. Ignore any story a boot splash tries to sell. A machine that started is a machine that started. That sentence feels obvious until a green banner lands in a screenshot and everybody in the thread relaxes.
Read the card like gauges

When the card comes back I read it the way a mechanic reads gauges, not the way a fan reads a scoreline. Oracle not red? The card is void. No argument, no partial credit, no interesting qualitative notes.
Oracle red and the reply malformed? A miss, and the ledger still charged, which is the lesson hiding in the annoyance. Oracle red, diff applies, pytest green, witness matches? That is the only quiet pass I trust, and only for this plant.
Oracle red, and the model calls the buggy function fine? False praise. That cell matters more to me than a pretty repair. A tool that blesses a known-bad slice will bless the next one when I am tired and late. Where does a free model tend to look strong on a pack like this? On the boring repair, once the oracle has already named the failure. Where does it break? On false praise, on diffs that do not apply, and on any day the local check was blind. The free server breaks the story when its exit disagrees with the laptop, and it proves nothing when you boot it first and call the boot a result.
Four counts sit side by side, and they never get blended. Oracle-fail rate on plants, which must be total or you throw the day out. Repair rate on plants the oracle actually caught. False-praise rate. Witness mismatches between the laptop and the free server.
I am leaving those cells empty on purpose. No timings. No win rate. No claim that one stack beat another. You fill them on your plants, and you keep the raw exits next to the commit. An empty card is stricter than a handy number I could have typed.
Who should walk away

Skip this if you need a procurement bake-off. A toy slice will flatter whoever has seen an off-by-one in training, and it will teach you almost nothing about a 400-file service. Skip it for auth, crypto, or any hazard a unit test does not name.
A green take() will not make a token check safe. Skip it if the free tier forbids scripted calls. A ledger is not a pardon, and I am not suggesting you route around a limit. Skip it if you wanted a leaderboard by Friday. You wanted a mood, and moods do not belong in the README.
Two limits sit in the design, and they will not politely stay small. Leakage is the first. Paste the expected list into the prompt and a model can patch the example instead of the function. Keep the prompt hungrier than the test file.
Scale is the second. One plant is a fire drill, not a map of the repo. Restage bugs you have already fixed, oracle included, names stripped so you are not grading memory of your own commit message. A bug you cannot restage is an anecdote with better lighting. Treat it as a story. Do not enter it on the card.
If you point this same stop-rule at MonkeyCode's free model, and only afterward at the free server, read the current docs first. Confirm the token grant and the server limits yourself. Run the dry flag before you spend anything. I would rather you distrust the grant than trust this paragraph.#testing #python #ai #programming #software #coding #development #engineering #inclusive #community
I Made the Oracle Fail First

#testing #inclusive #engineering #development #python #coding
Dev Community @[email protected]
2026-10-11

A nightly prompt eval that collapses every failure into one boolean will blame the model for a cut-off response. The useful split is smaller than a full scoring platform and stricter than a single green check. You should treat transport loss, truncation, refusal, malformed JSON, and a wrong answer as five different verdicts. Only a wrong answer, among those verdicts, should be allowed to move a pinned content baseline.
When a completion stops at a token limit, the tail of a JSON object disappears and the parser throws. A boolean grader records that throw as a miss, then a diff against last week looks like a reasoning regression. The same distortion appears when the connection drops, the server returns an empty body, or the model refuses a case it previously answered. Those events are operational facts, and they should not wear the label of a content failure.
This workflow assumes you already store each raw completion beside its case id for a later replay. Free model access and a free server option matter because the nightly job is repetitive, not because a free lane is an oracle. Disclosure: This article was prepared as part of MonkeyCode's product outreach. If those two options are available, use them for the scheduled completion calls, and do not treat that availability as a measured quality bar.
Classify before you score

The following grader is a proposal, not a log from a production fleet, and this draft has not executed it. It reads a manifest of golden cases, classifies each saved completion, and writes one typed verdict per case. Expected answers stay in the case file, while the grader returns a class before any numeric score exists. That order is the whole point, because a score without a class invites the wrong comparison later.
from __future__ import annotations
import hashlib
import json
from dataclasses import dataclass
from enum import Enum
from typing import Any
class Verdict(str, Enum):
OK = "ok"
WRONG = "wrong"
MALFORMED = "malformed"
REFUSED = "refused"
TRUNCATED = "truncated"
TRANSPORT = "transport"
UNKNOWN = "unknown"
CONTENT = {Verdict.OK, Verdict.WRONG}
@dataclass(frozen=True)
class Case:
case_id: str
prompt: str
expect: dict[str, Any]
@dataclass(frozen=True)
class Completion:
text: str
finish: str
error: str | None = None
def classify(case: Case, done: Completion) -> Verdict:
if done.error or done.finish == "error":
return Verdict.TRANSPORT
if done.finish == "length":
return Verdict.TRUNCATED
if done.finish == "refusal":
return Verdict.REFUSED
if done.finish != "stop":
return Verdict.UNKNOWN
try:
payload = json.loads(done.text)
except json.JSONDecodeError:
return Verdict.MALFORMED
if not isinstance(payload, dict):
return Verdict.MALFORMED
for key, expected in case.expect.items():
if payload.get(key) != expected:
return Verdict.WRONG
return Verdict.OK
def manifest_sha256(cases: list[Case]) -> str:
blob = json.dumps(
[{"id": c.case_id, "prompt": c.prompt, "expect": c.expect} for c in cases],
sort_keys=True,
separators=(",", ":"),
)
return hashlib.sha256(blob.encode()).hexdigest()
def content_rate(rows: list[Verdict]) -> float | None:
judged = [row for row in rows if row in CONTENT]
if not judged:
return None
hits = sum(row is Verdict.OK for row in judged)
return hits / len(judged)
def content_diff_allowed(rows: list[Verdict], min_judged: int = 8) -> bool:
judged = [row for row in rows if row in CONTENT]
if len(judged) < min_judged:
return False
blocked = len(rows) - len(judged)
return blocked * 5 <= len(rows)
def summarize(cases: list[Case], rows: list[Verdict]) -> dict[str, Any]:
counts = {item.value: sum(row is item for row in rows) for item in Verdict}
return {
"manifest_sha256": manifest_sha256(cases),
"counts": counts,
"content_rate": content_rate(rows),
"content_diff_allowed": content_diff_allowed(rows),
}
def diff_content(previous: dict[str, Any], current: dict[str, Any]) -> str:
if previous["manifest_sha256"] != current["manifest_sha256"]:
raise SystemExit("manifest hash changed; refuse the content diff")
if not current["content_diff_allowed"]:
raise SystemExit("operational mix too high; content diff suppressed")
prev = previous["content_rate"]
cur = current["content_rate"]
if prev is None or cur is None:
raise SystemExit("no judged content rows; nothing to compare")
return f"content_rate {prev:.3f} -> {cur:.3f}"
def demo() -> None:
expect = {"priority": "p2", "queue": "billing"}
prompt = "Return JSON with priority and queue."
cases = [
Case("ticket-14", prompt, expect),
Case("ticket-15", prompt, expect),
Case("ticket-16", prompt, expect),
]
saved = {
"ticket-14": Completion('{"priority": "p2"', "length"),
"ticket-15": Completion(
'{"priority": "p1", "queue": "billing"}', "stop"
),
"ticket-16": Completion("", "refusal"),
}
rows = [classify(case, saved[case.case_id]) for case in cases]
print(json.dumps(summarize(cases, rows), indent=2))
if __name__ == "__main__":
demo()
A content rate that ignores truncated rows will look healthier than the raw pass rate on the same run. That healthier figure is acceptable only when the same record also publishes the blocked count beside it. The helper refuses a content diff when fewer than eight cases were judged, or when non-content verdicts exceed one fifth of the run. Those thresholds are policy choices in this sketch, not measured optima, so retune them against your own manifest size.
The demo at the bottom of the module is also unexecuted in this draft, and its expected result follows from reading the branches. By inspection, one row is truncated, one is refused, and one is wrong, so the judged set contains only the wrong answer. The content rate on that judged set is zero, and the diff flag stays false because three rows cannot satisfy a minimum of eight. That closed flag is the desired outcome of the demo, and it is not a measurement of any hosted model.
A missing tray is not an empty tray

A postal scale that records a missing reading as zero kilograms will announce a famine every time the truck is late. The typed verdict is the practical difference between an empty tray and a tray that never reached the scale. A support-ticket case that expects priority p2 and queue billing is enough structure to show that split without a large fixture. When finish equals length and the object is only half written, the classifier returns truncated and that row stays outside the content rate.
A later clean completion that returns priority p1 for the same queue is wrong, and that row does enter the rate. Those two outcomes must not share a cell, or the weekly chart will narrate a quality drop that was a cut-off. Keep the raw text, the finish flag, and the case id in the incident file so a reviewer can confirm the class. Rerunning immediately would mix a new sample into the evidence, which is a different question from how this sample was graded.
Lock the classes offline

Before any nightly model call, lock the verdict taxonomy with saved completions that never leave the test process. A three-line fixture for truncated, refused, and wrong is enough to prove the classes do not collapse under a parser refactor. The proposed test below is unexecuted here, and it should fail if a later edit maps length onto wrong. Once that test is green, the nightly runner may call a model, but it must reuse the same classify function the test already pinned.
from verdicts import Case, Completion, Verdict, classify
def test_length_is_not_a_content_miss() -> None:
case = Case(
"ticket-14",
"Return JSON with priority and queue.",
{"priority": "p2", "queue": "billing"},
)
assert classify(case, Completion('{"priority": "p2"', "length")) is Verdict.TRUNCATED
assert (
classify(case, Completion('{"priority": "p1", "queue": "billing"}', "stop"))
is Verdict.WRONG
)
assert classify(case, Completion("", "refusal")) is Verdict.REFUSED
assert classify(case, Completion("not-json", "stop")) is Verdict.MALFORMED
assert classify(case, Completion("", "stop", error="reset")) is Verdict.TRANSPORT
[
{
"id": "ticket-14",
"prompt": "Return JSON with priority and queue.",
"expect": {"priority": "p2", "queue": "billing"}
}
]
python -m venv .venv
. .venv/bin/activate
python -m pip install pytest
python -m pytest tests/test_verdict_class.py -q
python verdicts.py
From a clean virtual environment, install only the test runner, then execute the taxonomy tests before you wire any client. The commands below match the two files in this draft, and they do not call a hosted model. A later wrapper around summarize and diff_content should refuse to start when the manifest hash cannot be computed. That wrapper should also exit non-zero when the hashes differ or the newer content-diff flag is false.
What a nightly record may claim

The run record is one JSON object with a manifest hash, class counts, a content rate, and a flag for the content diff. The hash binds expected JSON to the run, so a quiet edit to a golden case cannot look like a model change. Class counts sit beside the rate because a high rate can still hide a night when one fifth of the rows never finished. If the flag is false, keep the previous content series and store the new rows as an incident beside that series.
A refusal is closer to a policy outcome than to a wrong field, and folding it into the content rate punishes a declined answer. Transport errors sit on the path between processes, and they should not be narrated as text the model actually produced. Malformed JSON after a clean stop is the ambiguous middle, because the model finished and still broke the contract you requested. This sketch files that middle as malformed rather than wrong, so a format break stays visible when you later tighten the prompt.
Limits

This sketch assumes the provider tells you why the completion ended, and many clients simply never send that flag. If finish is missing, you cannot honestly separate truncation from malformed text, and you should label the row unknown rather than guess. Exact JSON equality is a narrow grader, because free-text drift will look wrong even when a human would accept it. Do not use this harness for open-ended essays, legal advice, or any case where partial credit is the real product requirement.
A free model lane can change how often responses are cut off without changing the answers that finish. That shift in mix is why the blocked share must gate the content diff rather than the incident log. That gate is not a benchmark of the vendor, and this draft records no latency, cost, or accuracy numbers. If your client cannot return a finish reason, or your cases are prose rather than a small JSON contract, skip this pattern.
If free model access and a free server are already on your MonkeyCode account, that pair can carry this nightly job. Keep the grading script in your own repo, and read the finish flag before you read any score. Nothing in this note claims a quota, a model name, or a permanent offer, because those facts were not supplied here as primary sources. The comparison worth keeping is the content rate beside the class counts, for one pinned manifest, on one pinned endpoint.#testing #python #ai #devops #software #coding #development #engineering #inclusive #community
A Truncation Is Not a Content Miss

#testing #inclusive #engineering #development #python #coding
Dev Community @[email protected]
2026-10-11

You should hold a prompt until you can name every room it will enter, because each room is a new trust boundary. A model service, a hosted server, and your own log pipeline are three different rooms even when one vendor offers all three. Free access does not shrink those rooms, and a friendly price does not change who can read what you paste. The useful habit is to label the payload, refuse the hot fields, and only then decide whether a remote run is acceptable.
The mistake usually starts with convenience, which feels like one door when the path is really a hallway. You paste a stack trace to explain a bug, and the trace still carries a connection string from the process environment. You attach a config snippet to show a path, and the snippet still carries a token that your linter never flagged. You then ask a hosted model to edit that same tree on a remote server, so one gesture crosses two boundaries.
Treat that gesture as a transfer, not as a chat, and the threat model becomes easier to draw on paper. Your laptop is the room you control, while the model endpoint is a room you only rent for a reply. The server is a third room, because it stores files and runs commands after the chat window has already closed. Logs form a fourth room, since retention and support access often outlive the session that created them.
If you cannot say which of those four rooms will see a given field, you are not ready to send the field. A request identifier can move, while the authorization header on the next line cannot, even though both arrived together. A file path can reveal a customer name or a private repository that you never intended to publish. You can keep the exception type and the line number, and you should drop the headers, the query string, and the body.
MonkeyCode enters this walkthrough only as one option where a free model path and a free server option may sit behind those doors. Disclosure: This article was prepared as part of MonkeyCode's product outreach. You should read the current terms yourself before you rely on either offer, because availability language goes stale faster than most posts. This article does not name models, quotas, hardware, or a promised duration, because those details were not verified from a primary page here.
If a product page advertises a token grant or a free machine, treat that page as the source of truth. Check the page on the day you start, because a number copied from an old thread is not evidence. A screenshot without a date is not a contract, and a recycled paragraph cannot enroll you. You can use the offer after you confirm it, and the security decision stays the same when the invoice is zero.
The fields you should never send are the ones that grant access, identify a person, or reconstruct a private system. A cloud key, a session cookie, a private key block, and a database URL all sit in the first class. Customer names inside a frame, raw request bodies, and unredacted support notes sit in the second class. Internal hostnames, tenant bucket names, and full environment dumps sit in the third class even when they look like ordinary debugging detail.
A useful analogy is a hotel hallway rather than a locked diary, because each door has a different reader behind it. The model door is read by a service that produces text, and that service may retain prompts under a policy you have not opened. The server door is read by a machine image that can copy your tree into a workspace you do not administer. The log door is read later by search tools, billing jobs, and sometimes a person who is debugging your debugging.
You need a local gate that fails closed, because a careful intention does not survive a tired paste at the end of the day. Copy the candidate text into a fixture, run the classifier, and read the verdict before you open a chat box. The classifier does not need to understand your product, since pattern refusal is enough to stop the obvious leaks. Rewrite the fixture until the verdict is clean, and allow only that clean fixture to cross any remote boundary.
A gate that fails closed

The script below is a proposed local gate, and you should run it on your own samples before you trust its patterns. It knows three outbound destinations, and it refuses to treat your laptop as a destination because the laptop is the source. It looks for assignment-style secrets, private key banners, common connection URLs, a familiar cloud key shape, bearer tokens, and email addresses. A match prints the rule name and returns a nonzero status, which is the signal your wrapper should treat as stop.
#!/usr/bin/env python3
"""Proposed local gate. It labels outbound text and never calls a model."""
import re
import sys
from pathlib import Path
DESTINATIONS = ("model", "server", "logs")
RULES = [
(
"assignment_secret",
re.compile(
r"(?i)\b(?:api[_-]?key|secret|token|password|passwd)\b\s*[:=]\s*\S+"
),
),
(
"private_key",
re.compile(r"-----BEGIN (?:RSA |OPENSSH |EC )?PRIVATE KEY-----"),
),
(
"connection_url",
re.compile(r"(?i)\b(?:postgres|mysql|mongodb|redis|amqp)://\S+"),
),
("cloud_key_shape", re.compile(r"\bAKIA[0-9A-Z]{16}\b")),
("bearer_token", re.compile(r"(?i)\bBearer\s+\S+")),
(
"email_address",
re.compile(r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b"),
),
]
def classify(text):
found =
[] for name, pattern in RULES:
if pattern.search(text):
found.append(name)
return found
def main():
if len(sys.argv) != 3:
print("usage: boundary_gate.py <model|server|logs> <file>")
return 2
destination = sys.argv
[1] path = Path(sys.argv[2])
if destination not in DESTINATIONS:
print("refuse destination=unknown")
return 2
text = path.read_text(encoding="utf-8")
hits = classify(text)
if hits:
for name in hits:
print(f"refuse field={name} destination={destination}")
return 1
size = len(text.encode("utf-8"))
print(f"allow destination={destination} bytes={size}")
return 0
if __name__ == "__main__":
sys.exit(main())
You can create two fixtures and compare them with the same command shape, which keeps the ritual boring enough to repeat. The hot fixture should fail for model, server, and logs, because a secret stays hot in every rented room. The clean fixture should pass those same three destinations, and the allow line should mention only a byte count. If the hot fixture ever exits zero, you have a broken pattern, and you should fix the gate before you fix the prompt.
mkdir -p fixtures
cat > fixtures/stack_hot.txt << 'EOF'
CheckoutError: timeout in charge_card
Authorization: Bearer ya29.example-not-a-real-token
[email protected]
DATABASE_URL=postgres://app:[email protected]:5432/shop
EOF
cat > fixtures/stack_clean.txt << 'EOF'
CheckoutError: timeout in charge_card
status=504
hint=upstream timed out before capture
EOF
python3 boundary_gate.py model fixtures/stack_hot.txt
python3 boundary_gate.py server fixtures/stack_hot.txt
python3 boundary_gate.py logs fixtures/stack_hot.txt
python3 boundary_gate.py model fixtures/stack_clean.txt
python3 boundary_gate.py server fixtures/stack_clean.txt
python3 boundary_gate.py logs fixtures/stack_clean.txt
Imagine a synthetic checkout trace that includes a bearer token, a customer email, and a postgres URL on three lines. You want help with the exception class, so you keep the class name, the status code, and the function name in the clean fixture. You delete the bearer line, you replace the email with a placeholder, and you replace the URL with the word database. You then run the gate against model, server, and logs, and you send the prompt only after all three runs exit zero.
That sequence is the whole method, and it is deliberately slower than pasting the terminal straight into a chat box. The slowness is the control, in the same way a badge reader is slower than an open lobby. You are not hiding the bug from the model, and you are hiding credentials from every room but your own. A model can still explain a timeout when the token and the mailbox are gone, because the failure shape remains.
There is a second decision after the gate, and it concerns what the server is allowed to hold even when the prompt is clean. A clean prompt can still travel with a dirty tree if you upload the real repository to a machine you do not run. Environment files, shell history, and editor swap files skip the prompt check unless you point the script at them too. Run the same script over those files before any sync, and keep the remote copy limited to what you already chose to share.
Where the gate stops

This gate has sharp limits, and you should say them out loud before you trust a green exit code. Regular expressions miss secrets that are split, encoded, or named in a way the rules do not expect. They also miss private meaning with no classic shape, and they can flag a harmless email you meant to keep. A clean result means those known patterns were absent, and it does not mean the text is safe for every legal regime.
You should not use this approach as your only control if you handle health records, payment data, or material under a written retention contract. You should not use it if a green script would persuade you to skip reading the host policy. You should not use it if your team needs centralized revocation, audit exports, or a reviewer who did not write the prompt. In those settings you need an approved data-loss process, and a local fixture check is only a rehearsal.
A free server option changes the cost conversation, and it does not change the custody conversation in any important way. Someone else still schedules the machine, images the disk, and decides how long a forgotten workspace remains mounted. You can enjoy the lower cost after the gate is green, and you should still assume the room can be inspected. If that assumption feels too strict for the data in front of you, keep the run on the laptop and send nothing.
When you want a remote practice room, confirm current free model access and the free server option on MonkeyCode's own page. Enroll only after that check, then send a fixture this gate has already marked clean for every destination. If the page no longer matches what you expected, believe the page and keep the payload local until the terms are clear. The boundary work remains useful even if you never create an account, because the four rooms exist on every host.#security #privacy #python #ai #software #coding #development #engineering #inclusive #community
Name Every Door Before the Prompt Leaves

#ai #security #privacy #community #software #coding
Dev Community @[email protected]
2026-10-11

You do not start an architecture review inside the diff. You start by naming the failure domain that can hurt you. A clean patch can still cross a boundary you never drew.
Think of the change as water in a set of pipes. The pipes matter more than the color of the water. If a pipe leaks, the patch quality will not save you.
Your first constraint is the scope of files. You name the files the change may touch. You also name the files it must not touch.
A suggestion that wanders past that fence fails review. That remains true even when the code compiles. Scope is a wall, not a hopeful wish.
Your second constraint is the path data takes. Prompts, file snippets, and test logs either stay on your machine or they leave. You write that choice down before you ask any model for help.
Your third constraint is where the patch lands. Apply stays on a branch that you control. Or it lands on a machine you do not administer.
Those two targets are different failure domains entirely. Do not blur them because the suggestion looks small. A small diff can still open the wrong door.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode offers free model access and a free server option. This review uses only those two supplied facts.
It does not assume a model name or a quota. It also skips hardware shape and any duration. You read the live plan yourself before you rely on either offer.
Treat the free server as a rented workbench, not as your archive. You can try a patch there when your laptop is the wrong place. You still keep the source of truth on a host you can inspect.
Here is the data flow you should draw before you paste anything. Your intent stays written in your own notes. The context you select then moves to the model.
The suggestion comes back to you as text. That text then sits in a review file. Apply happens only after the domain map says yes.
That flow has four hops, and each hop can fail alone. A failure in hop two does not excuse a failure in hop four. You review them as separate rooms in one building.
Hop one is the intent you already wrote. You write the constraint in a short file, not in a chat scroll. If the constraint is fuzzy, the model will invent a goal.
You cannot review an invented goal after the fact. A fuzzy intent is how drift enters the building. Write the goal in one sentence you can test.
Hop two is the egress of selected context. You choose snippets, not the whole tree, unless the tree is already public. Secrets, tokens, and customer paths do not ride along.
If you cannot redact, you do not send. Egress is a door you choose to open. Once the snippet leaves, you cannot pull the meaning back.
Hop three is the proposal text you received. The model returns a suggestion you can store. You store it as a patch file with a timestamp.
You do not apply it from the chat window. The chat is a transcript, not a release. A transcript can vanish, while a patch file can be diffed.
Hop four is where you apply the patch. You apply on a branch, or you apply inside the free server sandbox. You run the tests that belong to the touched files.
A green test on the wrong host is not evidence. Evidence is a file you can still open tomorrow. If that file is missing tomorrow, the hop already failed.
Failure domains are the rooms where a hop can burn you. Name each one before you admire the diff. You can use five names and keep them stable.
The egress domain fails when private data leaves. The drift domain fails when the patch solves a different problem. The apply domain fails when the patch lands on the wrong ref.
The custody domain fails when you cannot retrieve remote files. The evidence domain fails when no file can prove what ran. Five rooms share one building and no excuse.
The custody domain is about getting files back. It is not about how much room you were given. A free server may hold a workspace for a while.
That hold is not a backup you can trust. It is also not a promise you should invent. Read the current plan, then assume the workspace can vanish.
You can encode this review as a small gate. The gate reads a manifest you write by hand. It refuses apply when the map is incomplete or unsafe.
The script below is a proposal you can run locally. This proposal claims no benchmark and no vendor API. It only checks the map you already wrote.
Use hold when the map is fine but you are not ready. Hold stores the decision without landing any patch. Apply stays closed until you change that target on purpose.
#!/usr/bin/env python3
# Local domain gate. Proposal only. It makes no network calls.
import json
import sys
REQUIRED = (
"intent",
"touched",
"forbidden",
"egress",
"apply_target",
"evidence",
)
TARGETS = {"local-branch", "free-server-sandbox", "hold"}
EVIDENCE = {"patch-file", "patch-file+tests"}
def load(path):
with open(path, encoding="utf-8") as handle:
return json.load(handle)
def domain_failures(spec):
failures =
[] missing = [key for key in REQUIRED if not spec.get(key)]
if missing:
failures.append("map-incomplete:" + ",".join(missing))
return failures
if spec.get("egress") == "external" and spec.get("secrets") is True:
failures.append("egress-domain:secrets-would-leave")
overlap = set(spec.get("touched") or []) & set(spec.get("forbidden") or [])
if overlap:
failures.append("drift-domain:forbidden-overlap")
target = spec.get("apply_target")
if target not in TARGETS:
failures.append("apply-domain:unknown-target")
if target == "free-server-sandbox" and not spec.get("source_of_truth"):
failures.append("custody-domain:no-local-source-of-truth")
if spec.get("evidence") not in EVIDENCE:
failures.append("evidence-domain:no-local-artifact")
if target != "hold" and spec.get("waiver") != "reviewed":
failures.append("gate:apply-without-waiver")
return failures
def paths_in_diff(diff_text):
paths =
[] for line in diff_text.splitlines():
if line.startswith("+++ b/"):
paths.append(line[6:])
return paths
def extra_paths(diff_text, touched):
allowed = set(touched)
return [path for path in paths_in_diff(diff_text) if path not in allowed]
def main():
spec = load(sys.argv[1])
failures = domain_failures(spec)
if len(sys.argv) > 2:
with open(sys.argv[2], encoding="utf-8") as handle:
extra = extra_paths(handle.read(), spec.get("touched") or [])
if extra:
failures.append("drift-domain:patch-outside-touched")
if failures:
print("REFUSE")
for item in failures:
print(item)
return 2
print("ALLOW")
print("target=" + spec["apply_target"])
return 0
if __name__ == "__main__":
sys.exit(main())
Save that checker as domain_gate.py on your machine. Then write a manifest that matches a real change. Do not let a model fill the forbidden list.
That list is your constraint, not a suggestion. A model that edits the fence is already inside drift. You keep the pen when you write that list.
{
"intent": "Rename the retry helper without changing its public signature.",
"touched": ["src/retry.py", "tests/test_retry.py"],
"forbidden": ["src/secrets.py", ".env", "deploy/"],
"egress": "external",
"secrets": false,
"apply_target": "local-branch",
"source_of_truth": "git@origin/main",
"evidence": "patch-file+tests",
"waiver": "reviewed"
}
Run the domain gate before you apply anything. A refuse result is the review, not a nuisance. Read the exit code before you read the diff.
python3 domain_gate.py review.json suggestion.patch
status=$?
echo "exit=${status}"
test "${status}" -eq 0 || exit "${status}"
git switch -c review/retry-helper
git apply --check suggestion.patch
git apply suggestion.patch
python3 -m pytest tests/test_retry.py -q
If the gate prints an egress failure, stop and redact. If it prints a custody failure, keep the patch file local. Do not treat the free server as the archive.
Walk a concrete miss so the rooms feel real. You asked for a rename of a retry helper. The suggestion also edits the config loader to be helpful.
The touched paths and forbidden paths now overlap. The gate refuses, and that refuse is the architecture talking. You do not negotiate with the patch to stay green.
REFUSE
drift-domain:forbidden-overlap
You shrink the request and then ask again. You keep the same forbidden list in place. Helpful extra edits are drift, even when they look kind.
Walk a second miss on the free server. Tests pass in the sandbox during the afternoon. Next morning the workspace is gone, and so is your only copy.
The evidence domain failed when the patch never came home. The free server did not betray a quota you invented. It simply was never meant to be your archive.
You change the order of hops, not the slogan. Write the patch file locally before any remote run. Copy out logs you care about before you close the session.
A second manifest shows the safer sandbox path. You still name a source of truth you control. The free server is where you try the hop.
{
"intent": "Try the same rename where the local machine is busy.",
"touched": ["src/retry.py", "tests/test_retry.py"],
"forbidden": [".env", "secrets/"],
"egress": "external",
"secrets": false,
"apply_target": "free-server-sandbox",
"source_of_truth": "/work/repo",
"evidence": "patch-file+tests",
"waiver": "reviewed"
}
python3 domain_gate.py sandbox.json suggestion.patch
# copy only touched files, run the tests, bring the patch file back
You should change the gate next, not the model. Turn the waiver into a commit message line, not a loose flag. Add a check that the patch edits only touched paths.
Expire sandbox copies on a clock you choose. A vendor clock is not your own constraint. Bring the patch home before you call the hop done.
Do not change the model as your first move. A sharper model does not shrink an unnamed failure domain. It only writes a more confident patch in the same pipes.
Here is the analogy to keep on the wall. A building inspector does not start by praising the paint. The inspector walks the load path, the exits, and the fire spread.
Your domain map is the inspection walk itself. The diff is only the paint on the wall. Fresh paint on a cracked beam is still a cracked beam.
Limits sit in plain sight for this gate. This gate cannot see secrets you forgot to mark. It cannot prove the free server isolated your work.
It cannot tell you a quota or a region. It cannot tell you how long a workspace lives. Those facts change, so you read them at use time.
This article will not freeze those live facts for you. Do not use this approach for regulated data that must stay local. Do not use it when you need a durable build farm.
Do not use it if a tool should merge to the default branch. The gate is a review aid, not a release system. If the manifest and the patch disagree, believe the manifest.
Throw the patch away and ask for a smaller suggestion. You should not widen touched paths just to make the gate go green. That move hides drift instead of naming it.
A green gate on a lying map is worse than a red gate. Free model access helps when the hop is small and the map is complete. A free server helps when you need a disposable place to run that hop.
Neither offer replaces the walk through failure domains. If you try that walk with MonkeyCode, confirm the live plan first. Then run this gate on a scratch branch before you trust a suggestion.#architecture #testing #devops #ai #software #coding #development #engineering #inclusive #community
Name Failure Domains Before Apply

#usrbinenv #testing #inclusive #engineering #development #architecture
Dev Community @[email protected]
2026-10-11
I replayed a four-event trace on paper before I trusted a side-lane retry classifier at all. The planner had already sealed compensation when the side lane finally returned its late yes answer. A naive merger treated that late yes as fresh permission to reopen the already sealed attempt. Would you really let one delayed label rewrite a closed attempt after compensation had already landed?
That order is the whole review, not a footnote I can patch later with a longer timeout. I am treating retry authorization as an append that must carry an open generation fence token. If the fence does not match the live attempt, the log rejects the label and leaves the seal untouched. Why would a slower lane earn more authority than the path that already closed the books?
Assumptions I will not smuggle in later

Before I trust the diagram, I want the constraints written where a reviewer can attack them. If any assumption is false in your system, stop and do not borrow the acceptance rule at the bottom. Would you rather discover a missing generation id during review, or after a sealed attempt has already been reopened?
I assume delivery between planner, side lane, and decision log is at least once, so duplicates are normal rather than rare.
I assume the side lane may run on a free server and call a free model, with no latency promise I will write down.
I assume the primary path seals on its own clock and never waits unbounded for a classification to arrive.
I assume each attempt has a monotonic generation, and a seal is terminal for that generation only.
I am not claiming a measured quota, a named model, or a permanent free tier, because those facts were not given to me. I also refuse to invent hardware, duration, or a benchmark just to make the side lane sound faster. What I will claim is narrower: MonkeyCode offers free model access and a free server option for this shadow lane.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
I use that free model access only to draft advisory labels, and I host this desk harness on the free server option. A slow advisory label may inform the next generation, but it never becomes the component that seals the current one. If you remove every product name, the fence rule should still be worth keeping on your own bus.
The flow I would actually draw

planner --open(G)--> attempt_log
planner --classify(G)--> side_lane (free server + free model)
side_lane --label(G, yes|no|unknown)--> fence_gate
fence_gate --accept--> attempt_log only if G is open
fence_gate --reject--> stale_drop if G is sealed or unknown
planner --seal(G)--> attempt_log
Please read that diagram as an execution path, not as a loose slogan about separation of concerns. The fence gate is the only component allowed to interpret a side-lane label as an authorization. Everything else may propose a label, but nothing else may reopen a generation that has already been sealed. I would rather drop a late yes than let the diagram arrow skip the gate and write the log directly.
Numbered steps for one attempt

Open generation G, persist the open mark, and only then emit the classify request carrying that same generation.
Let the primary path proceed on its own clock, and never hold the seal timer hostage for a classification.
On any arriving label, compare generation G with the log before you read yes, no, or unknown.
If G is open and the label is yes, record one authorization; if G is sealed, append a stale drop and stop.
Seal G from the primary path alone, and ignore every later label that still names that same generation.
Does step three feel redundant when you already trust the queue to preserve your original emit order? I do not trust that, because a restart can deliver the classify response after the seal message has landed. You emitted classify first, yet the log can still observe seal first, and that swap is the bug. If your mental model forbids that swap, your mental model is describing a different bus than mine.
Failure domains I refuse to collapse

The side lane, the free server process, and the decision log are three failure domains I will not merge. A free server restart can drop an in-flight classification while the planner retries the same request under one generation. That retry is a duplicate delivery, not a fresh attempt, so it must not mint a new authorization by accident. A late yes after seal is a stale append, and replaying it would reopen work the compensation already undid.
An unknown label is not a soft yes, because ambiguity must not widen retries once the denominator is closed attempts. What about a yes that arrives before the seal, gets accepted, and then a duplicate yes arrives after the seal? The first append stands, and the second must hit the stale drop, or your exactly-once story is only a hope. I would rather reject the duplicate than compensate again, because a second compensation can undo a valid retry.
A minimal simulator, labeled as unexecuted

I have not run this fixture against a production bus, so treat the script as a desk proposal you can execute locally. It encodes the fence, the seal, and the three outcomes I care about: accept, reject, and duplicate after seal. If the assertions fail, the invariant is already broken, and no model score can talk you out of that. Would you ship a merger that cannot pass this trace, just because the free lane usually sounds confident?
# proposal: fence_sim.py - not a production claim
from dataclasses import dataclass, field
@dataclass
class Attempt:
generation: int
sealed: bool = False
auths: int = 0
drops: int = 0
@dataclass
class Log:
attempts: dict = field(default_factory=dict)
def open(self, g: int) -> None:
self.attempts[g] = Attempt(g)
def seal(self, g: int) -> None:
self.attempts[g].sealed = True
def on_label(self, g: int, label: str) -> str:
attempt = self.attempts.get(g)
if attempt is None or attempt.sealed or label != 'yes':
if attempt:
attempt.drops += 1
return 'reject'
attempt.auths += 1
return 'accept'
def permute() -> None:
orders = [('label', 'seal'), ('seal', 'label'), ('label', 'seal', 'label')]
for order in orders:
log = Log()
log.open(7)
auths_at_seal = None
for step in order:
if step == 'label':
log.on_label(7, 'yes')
else:
auths_at_seal = log.attempts[7].auths
log.seal(7)
assert log.attempts[7].auths == auths_at_seal
print('fence held across injected orders')
if __name__ == '__main__':
permute()
python3 fence_sim.py
If you want a property check rather than one happy trace, permute seal and label before you argue about scale. Assert that a sealed generation never increases its auth count, even when the same yes is delivered twice. I would add that permutation to the harness before I discussed throughput, because one printed line is not an acceptance rule. The fixture still counts two yes labels before seal as two auths, so idempotent accept is the next change, not a hidden feature.
Acceptance rule and the denominator

The denominator is sealed generations that later receive at least one label, not the total number of model calls. The numerator is those same generations whose authorization count stays unchanged after the seal has been written. I accept this design only when that ratio stays at one across label-then-seal, seal-then-label, and duplicate-yes-after-seal. A free model that is often right does not move this ratio, because the fence is about order, not about label quality.
Tradeoffs I would not hide
ChoiceLatencyCorrectnessCostWhat breaksWait for the side lane before sealHigherStronger if the lane always returnsFree-lane calls sit on the critical pathA slow free server stalls every attemptSeal without a fenceLowerWeakCheap until a late yes reopens workCompensation and retry both applyFence, reject stale, do not replayPrimary stays shortSeal is stableDrops late labelsA useful late yes is ignoredFence plus compensate on stale yesPrimary stays shortRiskyExtra writesSecond compensation undoes a valid retry

I would pick the third row for the decision log, and I would keep label quality on a separate scoreboard. Waiting for the side lane looks safer until a slow free host stalls every attempt that was ready to seal. Sealing without a fence looks cheap until a late yes and a compensation both apply to the same generation. Do you actually need that late yes inside the sealed generation, or only as input when the planner opens the next one?
What I would change next

I would split the authorization store from the decision log so a side-lane bug cannot obtain the seal writer. I would stamp every classify request with generation, attempt id, and a hash of the tool error being judged. I would refuse any label whose hash does not match, even when the generation is still open and the label says yes. I would canary those three failure classes, delayed label, duplicated label, and restarted side lane, and halt if any sealed generation gains an auth.
A rejected late yes may open generation G plus one only through the planner, never through the fence gate itself. Free model access can draft those advisory labels, and the free server can host the harness, within the limits already stated. If you want to try that split, start from the fixture and let only the planner seal, while the free host only proposes labels.
Who should not use this

Skip this pattern if your product requires the classifier to block the seal, because I deliberately removed that wait. Skip it if you have no generation id, since a fence without a stable name is only a comment beside the diagram. Skip it if a human must approve every retry, because this gate only decides whether a label may append at all. I would not use a free lane as the only record of a retry, since I promise neither retention nor uptime.
Which order should break it?

Which event order breaks the invariant: seal, then a duplicate yes, then a planner restart that re-emits the same classify call? I would reject that yes, refuse to replay it into generation G, and refuse to compensate the sealed attempt again. Should your system do the same, or do you still believe a late yes is safer than a dropped label? If you cannot name that order, you do not yet have an invariant, only a hope that the side lane stays fast.#architecture #distributedsystems #testing #ai #software #coding #development #engineering #inclusive #community
Make Authorization Fence an Invariant Before a Side Lane Reopens the Attempt
#distributedsystems #testing #inclusive #engineering #development #architecture
Dev Community @[email protected]
2026-10-11
I would open the notebook when a helper check fails and the only saved label is the word free. Had the suggestion drifted, or had the machine that ran the assertion changed underneath the same command? I could not answer either question from a log that says free, which means that log is a story. These notes are the forty-eight hour plan I would follow before I trust that kind of story again.
This notebook is a plan and a local recorder, not a measured study and not a quota report. If you need latency numbers, hardware names, or a ranked model list, you should stop here and go read primary docs. I would rather leave a field blank than borrow a number I did not print myself. Would you really ship a green check when the note cannot name the path that produced it?
Hours 0 to 12: one flag felt like enough

During the first half day I would write a helper that drafts a review comment and then runs one fixture test. My first sketch would use a single function called run_free, because both steps feel unpaid and therefore identical. That shortcut is the bug I already understand, even before I execute anything on a second machine. Can one boolean describe a model route and a server route without throwing away the diagnosis?
Hours 12 to 24: the replay could not choose a cause

On the second block of hours I would replay the same fixture and compare the two notes side by side. A different comment might mean the suggestion path answered differently than it did in the morning. A missing tool might mean the execution path was not the laptop I used when I wrote the fixture. If both explanations fit the same log line, I have not debugged anything, I have only renamed the confusion.
The split I would write down before hour thirty

Free model access means I may request a suggestion without paying for that request, under limits I still have to read. A free server option means I may run a job on a hosted machine without paying for that machine, under separate limits. I am taking those two availability claims as given for this exercise, and I am not treating them as a measurement. I still need the provider's current documentation before I treat either path as present on my own account.
Disclosure: This article was prepared as part of MonkeyCode's product outreach. I am using MonkeyCode only as the hosted example for those two choices, and I am not adding model names or hardware claims. If your account does not show one of those options, the card should store unavailable instead of a hopeful synonym. Would you rather pause the run than invent a path name just to keep the script moving?
Fields the card must answer
Keys I would require

I would keep each attempt in its own JSON file so a later diff cannot quietly overwrite the morning. The file has to answer a small set of questions before I allow myself to read the suggestion text. Boring keys are the point, because a clever schema is how I smuggle assumptions back into the note. Which of these could I still explain on Friday without reopening the chat that produced them?
suggestion_path: local, hosted-free-model, unused, or unavailable, and never the bare word free
execution_path: local, hosted-free-server, unused, or unavailable, chosen independently of the suggestion
provider_label: the label the tool printed, copied verbatim, or unreported when the tool stayed silent
command: the exact check I asked the execution path to run, stored as an argument list
started_at: a UTC timestamp from the machine that wrote the card, not a time I remembered later
exit_code: an integer, or null when the command never started because the card refused the paths
notes: one sentence about what I expected to learn, written before I look at the diff

Extras I would ban

I would also ban three tempting extras, because this notebook cannot honestly support any of them. No guessed model nickname, no pasted quota, and no hardware description belongs in the file. If the tool prints nothing, the provider label is unreported, which is a recorded fact rather than a hole. A hole invites me to fill it later from memory, and memory is how the word free returned last time.
The recorder I would run before I trust the note

The script below is a proposal you can run on your laptop, and it does not call a vendor API. I have not treated its output as a benchmark, and you should not paste its card into a status report as one. It refuses unknown path names, and it refuses unavailable, so a missing option cannot be blurred into local. I would store it as tools/path_card.py and aim it only at a fixture command I already trust.
#!/usr/bin/env python3
"""Write a two-path run card. Proposal: run locally; do not treat the file as a study."""
import argparse
import json
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path
ALLOWED = {
"local",
"hosted-free-model",
"hosted-free-server",
"unused",
"unavailable",
}
def parse_args():
parser = argparse.ArgumentParser(
description="Record suggestion path and execution path as separate fields."
)
parser.add_argument("--suggestion-path", required=True)
parser.add_argument("--execution-path", required=True)
parser.add_argument("--provider-label", default="unreported")
parser.add_argument("--command", nargs="+", required=True)
parser.add_argument("--out", default="run-card.json")
parser.add_argument(
"--dry-run",
action="store_true",
help="Write the card without starting the command.",
)
return parser.parse_args()
def require_known(name, value):
if value not in ALLOWED:
raise SystemExit(f"{name} must be one of {sorted(ALLOWED)}, got {value!r}")
def main():
args = parse_args()
require_known("suggestion_path", args.suggestion_path)
require_known("execution_path", args.execution_path)
if "unavailable" in (args.suggestion_path, args.execution_path):
raise SystemExit("unavailable path; refusing to blur it into local or unused")
if args.suggestion_path == "unused" and args.execution_path == "unused":
raise SystemExit("both paths are unused; refusing to write an empty card")
if args.suggestion_path == "hosted-free-server":
raise SystemExit("suggestion_path cannot take a server role")
if args.execution_path == "hosted-free-model":
raise SystemExit("execution_path cannot take a model role")
if args.suggestion_path == "local" and args.execution_path == "local":
print("note: both paths are local; that is allowed, and it is not a hosted run")
started = datetime.now(timezone.utc).isoformat()
exit_code = None
stdout, stderr = "", ""
if not args.dry_run:
completed = subprocess.run(args.command, text=True, capture_output=True)
exit_code = completed.returncode
stdout, stderr = completed.stdout, completed.stderr
card = {
"suggestion_path": args.suggestion_path,
"execution_path": args.execution_path,
"provider_label": args.provider_label or "unreported",
"command": args.command,
"started_at": started,
"exit_code": exit_code,
"stdout_tail": stdout[-400:],
"stderr_tail": stderr[-400:],
"dry_run": args.dry_run,
}
Path(args.out).write_text(json.dumps(card, indent=2) + "\n", encoding="utf-8")
print(f"wrote {args.out}")
return 0 if args.dry_run or exit_code == 0 else exit_code or 1
if __name__ == "__main__":
sys.exit(main())
A check that rejects the bare word

Before I trust the longer script, I would run this classifier in a Unix shell, with no network and no fixture file. The word free is intentionally absent from the allowed set, so a lazy label fails in the open. A swapped role fails too, because a server path must not pretend to be the suggestion path. If this check ever starts accepting free, the rest of the notebook has already lost the point.
python3 - <<'PY'
allowed = {
"local",
"hosted-free-model",
"hosted-free-server",
"unused",
"unavailable",
}
def classify(suggestion, execution):
if suggestion not in allowed or execution not in allowed:
return "reject-unknown"
if suggestion == "hosted-free-server" or execution == "hosted-free-model":
return "reject-swapped-role"
if "unavailable" in (suggestion, execution):
return "reject-unavailable"
if suggestion == "unused" and execution == "unused":
return "reject-empty"
if suggestion == execution:
return "named-pair-same"
return "named-pair-split"
assert classify("free", "local") == "reject-unknown"
assert classify("hosted-free-model", "unavailable") == "reject-unavailable"
assert classify("hosted-free-server", "local") == "reject-swapped-role"
assert classify("unused", "hosted-free-server") == "named-pair-split"
assert classify("hosted-free-model", "hosted-free-server") == "named-pair-split"
print("path split checks passed")
PY
Refusal, then a named local pair

I would prove the refusal before I prove the happy path, because a soft recorder is just a diary. These commands stay on one machine, and they do not require a hosted login to teach the habit. Watch the exit status, not the prose, since a friendly message can still hide a zero. Did the process actually stop, or did it write a card that flatters an unnamed path?
# Expect a non-zero exit, and expect no new card file afterward.
python3 tools/path_card.py \
--suggestion-path hosted-free-model \
--execution-path unavailable \
--out cards/should-not-exist.json \
--command python3 -m unittest tests/test_fixture.py
When both paths are intentionally local, I still pass both flags, because silence is not a stored value. The output file should sit under cards, with a unique name, so a refusal cannot leave yesterday's success in place. Replace the example unittest path with a test module you already have, and do not invent a fixture just to see green. A card without a timestamp is exactly how two separate evenings collapse into one false memory.
mkdir -p cards
python3 tools/path_card.py \
--suggestion-path local \
--execution-path local \
--provider-label unreported \
--out cards/local-local.json \
--command python3 -m unittest tests/test_fixture.py -q
If I later have a hosted free server available, I change only the execution flag and I copy the printed label. I do not rename the suggestion path so the pair looks tidy, and I do not reuse yesterday's file. Same command, different path, new file: that is the whole trick, and it is smaller than it sounds. Would a tidy matching pair really be worth more to you than a pair that stays true?
A gate table for the second day

Before the second day's runs, I would read a gate table instead of negotiating with a diff. The table is not a ranking, and it does not claim that any hosted path is faster or safer. It only asks whether I can name each path today, with the wording the product actually showed me. If I cannot answer, the row says stop, which is a result I am allowed to keep.
Question I can answer todaySuggestion pathExecution pathWhat I recordI only need a wording draft, and I will not run codehosted-free-model, if the account shows itunusedDraft text plus the printed label, and no exit codeI already wrote the patch, and I only need a machineunusedhosted-free-server, if the account shows itCommand, exit code, and the printed labelI need both a draft and a checkname it separatelyname it separatelyTwo fields, even when both values are localI cannot tell which path is offered right nowunavailableunavailableStop, and do not backfill a guess from memory

Notice that free never appears as a stored value, even though it is the word that started the confusion. The word can stay in conversation, but it should not survive into the artifact I will reread tomorrow. If a provider changes what an offer includes, older cards still name the path I believed I used. That is a weaker claim than a benchmark, and the weaker claim is the one I can defend.
Hours 36 to 48: what I would repeat

On the last block of hours I would replay three cases and write one plain sentence under each card. First I pass an unknown path string and confirm the script exits before it writes anything flattering. Next I run a fixture I already trust with both paths set to local, and I keep the card beside the output. Finally, only if a hosted option is actually offered to me, I rerun that same fixture and I refuse to merge cards.
The paper failure is already enough to change the habit, even if I never leave the laptop. A single free flag cannot tell me whether to reread a suggestion or to inspect the host that ran the check. I would repeat the split when both paths feel boring, because boredom is when I start abbreviating notes. Would a future version of me understand this file without the chat scroll that produced the wording?
I would also keep a rejection list beside the folder, written as checks rather than as slogans. Each line is a skip I would expect from myself when the evening runs long and the diff looks small. The list is short on purpose, because a long policy becomes another document I will not open. Which line would you be tempted to delete first, and what failure would that deletion invite back in?
Do not retry a failed hosted run until the card shows which path failed and which path was never asked.
Do not paste a suggestion into the execution command, because model text is untrusted input, not a shell program.
Do not compare two cards that lack started_at, even when the command strings look identical on the screen.
Do not treat a missing provider label as permission to invent one from a blog post or a memory.
Do not ignore a missing interpreter, because that failure belongs to the execution path, not the suggestion.

Who should not use this notebook

Skip this notebook if you need a compliance review, a capacity plan, or a number you can publish. The card does not measure queue time, retention, or whether a free option will still exist next week. Skip it when the command might touch production credentials, because a free server is still a remote machine. You still have to authorize that machine on purpose, and a friendly price does not change that duty.
Skip it if you wanted a prompt that quietly picks a best model, since this workflow only records a choice you already made. I would also skip hosted columns when the work includes secrets, personal data, or an unreviewed shell string. Local-only cards remain useful, and they keep the same keys so a later hosted run does not invent a format. A consistent boring card beats a clever one I will not reread on the following Monday.
One hosted name, then back to the habit

If I already have a MonkeyCode account, I fill a hosted value only after that account shows the option I am about to use. The outreach note above is why this product appears here, and deleting the name should not break the recorder. I would rather file one honest card than collect another evening of notes that say free and mean two machines. That is the habit I would repeat, and it does not require me to pretend I measured a platform.#python #testing #devops #ai #software #coding #development #engineering #inclusive #community
I Deleted Free From the Card and the Failure Picked a Side
#usrbinenv #testing #inclusive #engineering #development #python
Dev Community @[email protected]
2026-10-13
A flake freeze is valid only on one host, under one fixture epoch, and against an oracle the patch did not edit. A second machine can measure drift. It cannot open that freeze.
Agent patches fail in ways that look identical on a red CI line. The invariant can be false. The fixture lock can be stale. The owner host can disagree with itself, or a second host can disagree with the first.
Those four outcomes need different records. Collapsing them into one freeze hides the defect the next patch will trip over. The workflow below is a proposal. The snippets are unexecuted specification code, not a report from a live suite, and no pass rate, quota, or hardware figure is claimed.
What must match before a class exists

Treat the run as an equivalence class, not as a test name. Two failures that share a name and differ in their pins are different experiments. Write the pins down before the first replay, and refuse to classify a run that is missing any of them.
The pin set is small.
patch_id is the hash of the diff, not the branch name.
oracle_hash is the hash of the property module. The patch is not allowed to move it.
fixture_epoch is the hash of a manifest that lists every fixture file and that file's own hash.
seed is the integer the property checks actually used.
host_id names the machine that is allowed to own a freeze later.
replay_budget is chosen before the run. A red result does not raise it.
Change one pin and you start a new row. You do not amend the old one.
Build the epoch from bytes, not from directory mtime. A runner can unpack the same files and still change timestamps. Sort relative POSIX paths, write path digest lines, hash that text, and store the digest. If a clock, a temp path, or directory order can change the outcome, put that input in the manifest or stop calling the failure a flake.
Six classes, and only one may freeze

Score the observation with the table below. The freeze column is the policy. The other columns are bookkeeping.
ObservationClassFreeze on owner hostNext actionPatch changes the oracle hashoracle_tamperNoReject the diff. Restore the oracle file.Pinned property failsproperty_failNoKeep the counterexample. Do not replay for a freeze.Manifest hash differs from the lockfixture_driftNoRelock on purpose, or discard the run.Owner host still inside budget, not a clean passreplay_incompleteNoFinish the budget. Do not call host B yet.Owner host shows both pass and fail after the budgetflake_candidateYes, time-boxedInsert a freeze row bound to that host_id only.Owner host is stable, other host differsenvironmentNoFile drift. Do not copy the row into the other ledger.Owner host is stably red, other host agrees or was not calledregressionNoTreat it as a real failure of the patch.Owner host is stably greenpassNoContinue review.

A property failure stays a property failure even if a later run goes green. That green run is a different question, and it does not launder the counterexample into a flake. Stable red on the owner host is a regression when the oracle held. Mixed results after the budget are the only freeze input.
Host B is a measurement, including when host B is a free server rather than another CI runner. Its boolean does not get a vote on the owner ledger.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode's free model access matters here only as a source of candidate diffs. The free server option matters only as a possible host B. Neither claim adds a model name, a quota, or a performance number to the table, and none of those are required.
A green probe on the free server does not cancel a property failure on the owner host. A red probe does not open the owner host's freeze.
Seven steps

Hash the oracle path. If the diff touches it, emit oracle_tamper and stop.
Build the fixture manifest from the fixture root and compare it with the locked epoch.
Run property checks on the owner host with the stored seed. On failure, emit property_fail and stop.
Run the named regression once on the owner host. A pass skips the freeze path.
On failure, replay on the owner host until the budget is consumed and every outcome is recorded.
Only then, if the class is not already property_fail or fixture_drift, submit the same patch_id, seed, and fixture epoch to host B.
Emit one class from the table. Persist it. Do not edit the class after the ledger write.
Step 6 is deliberately late. An unfinished owner-host run cannot change the table, so calling host B early only spends a seat. If that seat is a free server, keep the probe off until step 5 has a complete vector.
Expiry of a freeze that already exists is a separate policy. This gate only decides whether a new row may be inserted, and on which host.
Classifier

This module is a reviewable proposal. It imports nothing outside the standard library. It encodes the table. It does not discover new invariants.
import hashlib
from pathlib import Path
def sha256_file(path: Path) -> str:
digest = hashlib.sha256()
digest.update(path.read_bytes())
return digest.hexdigest()
def fixture_epoch(root: Path) -> str:
lines =
[] for path in sorted(p for p in root.rglob("*") if p.is_file()):
rel = path.relative_to(root).as_posix()
lines.append(f"{rel} {sha256_file(path)}")
payload = ("\n".join(lines)).encode()
return hashlib.sha256(payload).hexdigest()
def classify(obs: dict) -> str:
if obs["oracle_hash"] != obs["oracle_lock"]:
return "oracle_tamper"
if obs["fixture_epoch"] != obs["fixture_lock"]:
return "fixture_drift"
if not obs["property_passed"]:
return "property_fail"
host_a = list(obs["host_a_outcomes"])
budget = int(obs["replay_budget"])
if budget < 1 or not host_a or len(host_a) > budget:
raise ValueError("owner-host outcomes are outside the declared budget")
distinct = set(host_a)
host_b = obs.get("host_b_outcome")
if len(host_a) < budget and distinct != {True}:
return "replay_incomplete"
if distinct == {True, False}:
return "flake_candidate"
if distinct == {True}:
return "environment" if host_b is False else "pass"
if host_b is True:
return "environment"
return "regression"
def freeze_allowed(kind: str, host_id: str, owner_host: str) -> bool:
return kind == "flake_candidate" and host_id == owner_host
freeze_allowed is the policy in one line. Environment drift returns false. A cheaper host does not become the owner by being cheaper. Host B is not consulted inside the mixed-result branch. Instability on the owner host is already enough for flake_candidate, and host B cannot veto that row or inherit it.
Three illustrations use placeholder ids. They are not measured runs.
Owner outcomes [False, False, False], budget 3, property passed, host B True. The class is environment. The owner host is stable. Freezing the test name would block later patches for a failure that did not reproduce on the other host, and it would hide the drift.
Owner outcomes [False, True, False], budget 3, host B False. The class is flake_candidate. A freeze may open, and only for the owner host_id. Host B's failure does not extend that row.
Owner outcomes [False], budget 3. The class is replay_incomplete. One red result is not a class you can freeze. Finish the vector first, and do not call host B on this row yet.
Specification checks

Pair the function with assertions that restate the table. They are part of the proposal. They fail if a later edit lets a second host open the owner ledger.
def test_cross_host_split_is_not_a_freeze():
kind = classify({
"oracle_hash": "aa",
"oracle_lock": "aa",
"fixture_epoch": "bb",
"fixture_lock": "bb",
"property_passed": True,
"host_a_outcomes": [False, False, False],
"host_b_outcome": True,
"replay_budget": 3,
})
assert kind == "environment"
assert freeze_allowed(kind, "ci-a", "ci-a") is False
def test_mixed_owner_host_can_freeze():
kind = classify({
"oracle_hash": "aa",
"oracle_lock": "aa",
"fixture_epoch": "bb",
"fixture_lock": "bb",
"property_passed": True,
"host_a_outcomes": [False, True, False],
"host_b_outcome": False,
"replay_budget": 3,
})
assert kind == "flake_candidate"
assert freeze_allowed(kind, "host-b", "ci-a") is False
assert freeze_allowed(kind, "ci-a", "ci-a") is True
The host check is the assertion reviewers skip. A flake on ci-a must not be writable by host B, even when both saw a failure. Host identity is part of the key.
A minimal persisted record, again with placeholder values, looks like this.
{
"patch_id": "example",
"oracle_hash": "aa",
"fixture_epoch": "bb",
"seed": 17,
"host_id": "ci-a",
"replay_budget": 3,
"class": "environment",
"freeze": false
}
Command order

Check the oracle path before the test runner starts. An assertion inside the test file is too late, because the patch may have weakened that assertion.
git diff --name-only "$BASE"..HEAD -- tests/properties/invariants.py
python -m gate.classify --record runs/patch.json --owner-host ci-a
The first command must print nothing. If it prints the oracle path, emit oracle_tamper and do not start the suite. The second command reads the observation file and prints one class. Exit non-zero for every class other than pass.
Host B gets the same pins and an empty ledger. The flags below belong to the proposed local probe, not to a vendor CLI.
python -m gate.probe \
--patch-id "$PATCH_ID" \
--fixture-epoch "$EPOCH" \
--seed 17 \
--host-id host-b \
--ledger none
--ledger none is the useful constraint. Host B may return true or false. It may not append a freeze row. Point --host-id at whatever second machine you already operate. A free server seat can fill that role. It does not become the merge gate.
Keep the agent off the oracle path in two places. Say so in the task instruction, then enforce it with the hash compare. The instruction is not the gate. The hash is.
Limitations

The classifier does not prove the patch is correct. It can only show that the stated properties held and that the named check was stable on one host. Unstated behavior can still be wrong. That is a coverage gap, not a reason to freeze a red name.
A same-host disagreement can still be an unlisted fixture. Wall clocks, temp paths, and directory order will mint false flake_candidate rows if they sit outside the manifest. Widen the manifest before you widen the budget.
The budget in the examples is a policy integer, not a calibrated flake rate. This article does not estimate how often agent patches flake. If you have a measured baseline, use it. If you do not, keep the integer small, and accept that a stable failure costs that many owner-host runs before it becomes regression.
Cross-host comparison here is a boolean. Two hosts can fail in different assertions and look like agreement. Store the exception type and the assertion id if you need a stricter probe. That extension still must not open a freeze by itself.
Who should skip it

Skip the workflow if you cannot name one owner host. A pool of ephemeral runners with no stable id will mark almost every split as environment, and the freeze path will never open. That can be the right policy. Write "no freeze on pooled hosts" instead of pretending the pool is one machine.
Skip it if the suite has no property module the patch is forbidden to edit. Named tests alone cannot separate tamper from regression. Add the oracle before you add host B.
Skip it if you need a vendor quota, a hardware spec, or a published benchmark before a second host is allowed to run. Those are not inputs to the table, and this note does not supply them.
Put the observation JSON in the pull request. If the class is environment, link the two host ids and leave the freeze ledger untouched. If a free MonkeyCode server is already on your bench, use that seat as host B with an empty ledger. The owner host, not the extra seat, still decides whether a freeze exists.#testing #python #devops #ai #software #coding #development #engineering #inclusive #community
A Second Host Measures Drift. It Does Not Open the Freeze.
#testing #inclusive #engineering #development #python #coding
Dev Community @[email protected]
2026-10-13

Reference documentation stays trustworthy when interface facts are extracted and locked before any model writes orientation prose around them. A block-role manifest records that split before drafting starts, so review does not depend on catching invented flags by eye. Deterministic extraction owns signatures, flags, environment variables, and error codes, while the model may fill only marked orientation slots. The checker later in this article fails the page when drafted prose names an interface token that the locked inventory does not contain.
Models that draft reference pages often smooth over a missing row by inventing a plausible flag, status, or default value. That failure is different from a weak explanation, because the sentence looks fluent while the contract itself is false. Full-page review does not scale once several endpoints share one pull request and each page mixes tables with narrative. A role tag on every block makes the allowed draft surface small enough for a script to check without rereading tables.
Heading-level ownership is not enough when one heading contains both a generated flag table and a paragraph of advice. The unit of control here is the block, not the file and not the heading, because those larger units hide mixed authority. A style promise about tone also does not stop a model from adding a flag that no release ever shipped. The manifest therefore names concrete tokens, and the checker compares drafted sentences against that closed token set.
What the model may draft

The model may draft orientation slots that explain sequencing, prerequisites in plain language, and common misreadings of a locked row. Those slots may compare two documented flags only when both flags already appear in the inventory for that page. The model may also propose a short note about when not to call the operation, if every concrete token is already listed. Generic words such as request, client, and retry stay allowed, while product-specific tokens must be listed before any draft.
Orientation text should use a surface from the locked row rather than restating the default, the type, or the status code. Restating those fields creates a second source of truth that will drift on the next release even if the first draft was careful. If a sentence needs a default, the page should render that default from the inventory, not from the drafted paragraph. The drafting prompt should receive slot ids and allowed surfaces, not the unfinished page as a free-form writing canvas.
What a human must own

A human must own the inventory source, whether that source is an OpenAPI document, a captured help file, or an error catalog. A human must own status mappings, default values, deprecation markers, and the decision that a given token is public. A human must approve each orientation slot before it exists, because an extra slot is an extra place to invent scope. A human must also merge extracted tables into the page, so a model cannot rewrite a locked block while polishing nearby prose.
Ownership is not the same thing as typing every sentence that a reader eventually sees on the published page. A human can delegate extraction to a script, then review the inventory diff before any prose job is allowed to run. What cannot be delegated is the judgment that a token is safe to document, stable enough to name, and correctly scoped. If that judgment is missing, a drafting pass will not create it, and the checker will only confirm that the invention was consistent.
Step-by-step workflow

Follow the six steps in order, and treat each artifact as an input to the next rather than as optional notes. Skipping the lock file and asking for a full page is how invented flags re-enter an otherwise careful repository. The commands and files below are a proposed workflow, not a record of a run against a production documentation set. Replace the sample command name with the binary or spec that the team already treats as the interface source.
1. Capture the interface inventory

Store one inventory file per reference page, and treat that file as the only list of public tokens the page may name. Capture help text or spec excerpts with a deterministic command, then convert the capture into a token file a human reviews. Do not ask a model to transcribe a specification into the inventory, even when the specification is long or awkwardly formatted. A transcription pass can drop a required field and still look complete to a reviewer who did not open the source.
mkdir -p docs/inventory docs/pages docs/slots docs/fixtures tools
# Proposed capture. Replace mycli with the binary the team actually ships.
mycli --help > docs/inventory/mycli-help.txt
{
"page_id": "export-job",
"source_capture": "docs/inventory/mycli-help.txt",
"note": "Proposed inventory. Defaults are null because this file is not a captured product.",
"tokens": [
{"id": "flag.out", "kind": "flag", "public": true, "default": null},
{"id": "flag.since", "kind": "flag", "public": true, "default": null},
{"id": "err.partial", "kind": "error", "public": true, "default": null}
]
}
If the source is OpenAPI, extract operation ids, parameters, and response codes with a parser the team already trusts. Keep the raw capture next to the token file so a reviewer can see what the parser dropped or renamed. Commit the raw capture and the token file in the same change, and reject token files that have no matching capture. This step is finished only when a human has marked each token public, internal, or omitted from the page.
2. Write the block-role manifest

The manifest is the contract between extraction and drafting, and it should be reviewed like a schema change rather than like prose. Each block has a role of extract or orient, plus the token ids that the role is allowed to mention. Extract blocks are rendered from the token file and are not sent to the model as writable text. Orient blocks are the only regions a drafting job may replace, and each one carries a word cap.
# docs/inventory/export-job.manifest.yaml
# Proposed example. Not a captured production file.
page_id: export-job
inventory_sha256: "<fill-after-step-3>"
blocks:
- id: flags
role: extract
source: docs/inventory/export-job.tokens.json
- id: sequencing
role: orient
max_words: 120
allowed_token_ids: [flag.out, flag.since, err.partial]
- id: skip-when
role: orient
max_words: 80
allowed_token_ids: [flag.since, err.partial]
tokens:
- id: flag.out
surface: ["--out"]
- id: flag.since
surface: ["--since"]
- id: err.partial
surface: ["EXPORT_PARTIAL", "partial export"]
The surface list is the set of spellings a sentence may use for a token, and the sample keeps those spellings distinctive. A multi-word surface, such as a named error phrase, belongs in that list so the checker can allow the phrase without allowing nearby inventions. Keep allowed_token_ids narrow, because a slot that may name every token on the page is almost as loose as a full-page prompt. If two slots need the same flag, list that flag in both, rather than creating one large slot that covers the whole page.
3. Lock the inventory hash

Compute a hash after a human accepts the token file, and store that hash in the manifest before any slot is drafted. The hash binds the prose job to a specific inventory, so a quiet edit cannot change the contract under an old draft. If the token file changes, the hash check fails until a human updates the manifest in the same commit on purpose. That failure is useful, because orientation jobs should stop when the interface moves instead of rewriting themselves around the change.
python3 - <<'PY'
import hashlib
import pathlib
p = pathlib.Path("docs/inventory/export-job.tokens.json")
print(hashlib.sha256(p.read_bytes()).hexdigest())
PY
Paste the printed digest into inventory_sha256, and do not let a drafting script update that field automatically. An automatic hash update would hide the very edit the lock is meant to surface during review. Reviewers should look at the token diff first, then accept or reject the new digest as a separate decision. Only after that decision should anyone schedule a new orientation draft for the slots whose allowed tokens changed.
4. Draft only the orientation slots

Send the model the slot id, the word cap, and the allowed surfaces, and ask for that slot alone. Do not send a request that says to write the reference page, the introduction, and the examples in one pass. A narrow prompt reduces the chance that the model adds a sibling flag that merely sounds consistent with the table. Save the result under a slot file named for the block id, and leave every extract block untouched by that job.
Draft slot sequencing only.
Allowed surfaces: --out, --since, EXPORT_PARTIAL, partial export.
Do not name any other flag, environment variable, status, or error code.
Do not restate defaults, types, or status codes.
Maximum 120 words.
When a drafting tool is useful for this step, keep it on slot text only, after the inventory hash is already committed. MonkeyCode's free model access can draft an approved orientation slot, and the free server option can run the hash step and the checker.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
This article does not claim particular model names, quotas, hardware sizes, time limits, or benchmark results for that setup. The page remains useful without those options, because the manifest, the hash, and the checker are ordinary repository files. A local Python install is enough to rehearse the gate before any hosted runner is involved. The hosted option is a placement choice for steps that are already specified, not a source of interface facts.
5. Render the page from roles

A renderer should concatenate extracted tables and orientation files in manifest order, without asking a model to assemble the page. It should refuse to read a slot file for an extract block, because that file is evidence that a draft overwrote a locked region. It should also refuse to render an orient block whose slot file is missing, rather than filling the gap with generated prose. Missing prose is a visible gap a human can fill, while silent generation hides the fact that no approved slot text existed.
6. Run the token checker

The checker loads allowed surfaces, rejects unknown interface-like tokens in orientation files, and verifies the inventory hash before it reads prose. It is a proposed, unexecuted example, so treat the results as a specification for the gate rather than as measured outcomes. The script depends on PyYAML, uses only the standard library otherwise, and never contacts a model or a network service. Run it in the same job that builds the page, and block publication when the exit code is nonzero.
#!/usr/bin/env python3
"""Proposed, unexecuted checker for orientation slots.
Compares slot files with a locked manifest. It does not call a model.
Requires PyYAML.
"""
from __future__ import annotations
import hashlib
import json
import re
import sys
from pathlib import Path
import yaml
FLAG_HINT = re.compile(r"--[a-z0-9][-a-z0-9]*")
CONST_HINT = re.compile(r"\b[A-Z][A-Z0-9_]{2,}\b")
HTTP_HINT = re.compile(r"\bHTTP\s+\d{3}\b")
def load_manifest(path: Path) -> dict:
data = yaml.safe_load(path.read_text(encoding="utf-8"))
if not isinstance(data, dict):
raise SystemExit("manifest must be a mapping")
return data
def assert_hash(manifest: dict, token_path: Path) -> None:
expected = str(manifest.get("inventory_sha256", ""))
actual = hashlib.sha256(token_path.read_bytes()).hexdigest()
if expected != actual:
raise SystemExit(
f"inventory hash mismatch: manifest={expected} file={actual}"
)
def assert_ids(manifest: dict, token_path: Path) -> None:
payload = json.loads(token_path.read_text(encoding="utf-8"))
ids = {row["id"] for row in payload["tokens"]}
for row in manifest["tokens"]:
if row["id"] not in ids:
raise SystemExit(
f"manifest token missing from inventory: {row['id']}"
)
def surfaces(manifest: dict, allowed_ids: list[str]) -> list[str]:
by_id = {row["id"]: row["surface"] for row in manifest["tokens"]}
found: list[str] =
[] for token_id in allowed_ids:
if token_id not in by_id:
raise SystemExit(f"unknown token id in slot: {token_id}")
found.extend(by_id[token_id])
return found
def strip_allowed(text: str, allowed: list[str]) -> str:
cleaned = text
for phrase in sorted(set(allowed), key=len, reverse=True):
if phrase.startswith("--"):
pattern = re.escape(phrase) + r"(?![A-Za-z0-9-])"
elif " " in phrase:
pattern = re.escape(phrase)
else:
pattern = r"\b" + re.escape(phrase) + r"\b"
cleaned = re.sub(pattern, " ", cleaned, flags=re.IGNORECASE)
return cleaned
def unknown_hints(text: str, allowed: list[str]) -> list[str]:
leftovers = strip_allowed(text, allowed)
found: list[str] =
[] found.extend(FLAG_HINT.findall(leftovers))
found.extend(CONST_HINT.findall(leftovers))
found.extend(HTTP_HINT.findall(leftovers))
return found
def main() -> None:
if len(sys.argv) != 4:
raise SystemExit(
"usage: check_orientation_slots.py MANIFEST TOKENS SLOT_DIR"
)
manifest = load_manifest(Path(sys.argv[1]))
token_path = Path(sys.argv[2])
slot_dir = Path(sys.argv[3])
assert_hash(manifest, token_path)
assert_ids(manifest, token_path)
failures: list[str] =
[] for block in manifest["blocks"]:
slot_file = slot_dir / f"{block['id']}.md"
if block["role"] != "orient":
if slot_file.exists():
failures.append(f"extract block has a draft file: {slot_file}")
continue
if not slot_file.exists():
failures.append(f"missing orientation slot: {slot_file}")
continue
text = slot_file.read_text(encoding="utf-8")
words = text.split()
if len(words) > int(block["max_words"]):
failures.append(
f"{block['id']} has {len(words)} words; cap is {block['max_words']}"
)
allowed = surfaces(manifest, block["allowed_token_ids"])
for hint in unknown_hints(text, allowed):
failures.append(
f"{block['id']} names unlisted interface-like token: {hint}"
)
if failures:
print("\n".join(failures))
raise SystemExit(1)
print("orientation slots match the locked inventory")
if __name__ == "__main__":
main()
A passing slot uses only allowed surfaces and stays under the word cap recorded for that block in the manifest. A failing slot names a flag or an error constant that never appears in the surface list for that slot. The sample failure below uses a gzip flag and an export timeout constant, neither of which is present in the sample inventory. Against that failing fixture, the checker is specified to exit nonzero and to print each unlisted flag or constant.
Write the archive with --out, then narrow the window with --since.
If the job reports EXPORT_PARTIAL, rerun only after the source is complete.
Do not treat a partial export as a finished archive.

Pass --gzip when the archive is large, and set EXPORT_TIMEOUT if the network is slow.
python3 tools/check_orientation_slots.py \
docs/inventory/export-job.manifest.yaml \
docs/inventory/export-job.tokens.json \
docs/slots
How to read a checker failure

Read a failure as a routing decision, not as a suggestion to regenerate the whole page and hope the next draft is cleaner. The table below maps common signals to the role that must change and to the person or tool that should act. Use it during review so a model is not asked to repair a row that a human still has to own. If the table and the checker disagree, trust the inventory and fix the checker, rather than widening the prompt to silence the gate.
SignalRole that owns the fixRequired actionUnlisted flag or error constant in a slotorientReject the slot and redraft with the same allowed surfacesDefault, type, or status restated in proseorientDelete the restatement and render the field from the inventorySequencing advice that uses only allowed surfacesorientHuman edit for clarity, then acceptMissing deprecation marker or wrong defaultextractHuman updates the token file, then rehashesNew public error codeextractHuman adds the token, rehashes, then refreshes affected slotsExtract block has a slot fileextractDelete the slot file and investigate how it was writtenHash mismatchinventoryStop drafts until a human accepts the new digest
Limitations of the gate

Word-boundary and hint matching miss inventions that avoid tokens, such as a sentence that adds an optional path argument without naming it. A human still needs to read every orientation slot, because a real flag can be described incorrectly without introducing a new token. Phrase search can false-positive when a surface is a common word, so prefer a distinctive spelling whenever the interface gives you one. The checker does not review security claims, legal wording, support promises, or whether a public token should have been public.
Hash locking detects inventory drift, but it does not tell you whether the accepted inventory matched the binary shipped last week. You still need a release process that ties the capture command to a version, a commit, or another source you can reopen. The hint pattern is intentionally narrow, and a team with different token shapes should extend the tests before extending the regular expression. Widening the pattern without fixtures will create noise, and reviewers will start ignoring the gate that this workflow depends on.
Who should not use this approach

Teams with no specification, help capture, or other inventory source should not adopt this workflow yet. The checker would then bless a hand-typed list that nobody tied to a shipping interface, which recreates the original risk under a new filename. Marketing pages, incident narratives, and conceptual essays without stable tokens are a poor fit for extract and orient roles. Anyone who wants a model to discover the interface should stop, because discovery is an ownership task rather than a drafting task.
Large generated references that are already rendered entirely from a specification may not need orientation slots at all. Adding slots there only creates prose that can drift from the generator, so prefer improving the generator over adding a second writer. Solo maintainers who cannot review slot text should also skip the drafting step and keep the page fully extracted. A gate that nobody reads is worse than a short page, because it suggests a control that the repository does not actually practice.
Put the gate in the build

The practical split is small and strict: tools and humans own interface rows, and a model may write only inside preapproved orientation slots. Lock the inventory hash, render extract blocks without a model, and reject any slot that names an unlisted surface. If MonkeyCode's free server is already available to the team, run the hash step and the checker there before merging any free-model slot draft. That order keeps drafting convenience from becoming the source of the interface the page claims to document.#documentation #testing #devops #ai #software #coding #development #engineering #inclusive #community
Orientation Slots Are the Only Writable Region on a Reference Page

#devops #ai #community #software #coding #development
Dev Community @[email protected]
2026-10-13

Debug Totals Lie at Release: An Opt-Split Gate for C++ Patch Evals

The failure mode is ordinary, and it hides in a green log. A grader accepts a C++ patch because the debug binary prints 18432. A later release build of the same sources prints 18416. Nobody edited the prompt between those two runs. The optimizer met a signed addition that had already left the representable range.
That split is a silent regression. A single golden print, captured under one flag set, cannot see it.
This note describes an opt-split gate for AI C++ patches. One frozen input tape, one hand-written oracle, and the candidate built at -O0 and -O2. The gate fails when the oracle stays stable and the candidate does not. A prompt hash pin stops a later draft from changing the test without a recorded edit.
What a single print misses

Generated C++ often looks finished inside a driver that prints one number. The driver and the patch share a translation unit. Warnings are quiet, or they were never enabled. The build uses whatever optimization the local default happens to be, which is frequently closer to -O0 than to the flags a package will ship.
Signed overflow, a read of an uninitialized scalar, and a dependency on evaluation order can all survive that setup. A second flag set sometimes keeps the arithmetic inside the range the oracle demands. Sometimes it stops hiding the bug. The opt-split gate does not prove the absence of undefined behavior. It catches the cheaper mismatch: debug output and release output disagree, while a checked oracle prints the same bytes twice.
Files the gate owns

Four inputs define a run, and none of them is a dashboard screenshot. prompt.txt is the only prose the draft step may read. prompt.pin stores the SHA-256 of that file from the last accepted wording change. tape.txt lists signed integers, one token per line, and ends with END.
oracle.cpp is written by a person. candidate.cpp is the patch under test, and it must export the same signature the oracle exports. The contract stays narrow so the grade is about values, not about prose.
// contract.hpp
#pragma once
#include <cstdint>
#include <optional>
std::optional<std::int64_t> fold_checked(const std::int64_t* data, std::size_t n);
A returned integer is the exact sum. std::nullopt means the fold is rejected. The candidate is scored against the oracle decision, not against a golden string sitting in the prompt.
Step 1. Pin the prompt before a draft exists

A changed prompt is a changed test, even when every tape byte is identical. The pin check refuses to grade on drift. It does not try to be clever about whitespace.
python3 - << 'PY'
import hashlib, pathlib, sys
prompt = pathlib.Path("prompt.txt").read_bytes()
digest = hashlib.sha256(prompt).hexdigest()
pin = pathlib.Path("prompt.pin")
if not pin.exists():
pin.write_text(digest + "\n", encoding="ascii")
print("pinned", digest)
sys.exit(0)
expected = pin.read_text(encoding="ascii").strip()
if digest != expected:
print("prompt pin mismatch")
print("expected", expected)
print("actual ", digest)
sys.exit(2)
print("prompt pin ok", digest)
PY
Treat a mismatch as a harness failure, not as a patch failure. Update prompt.pin only in a commit that quotes the new wording. Nightly jobs must not rewrite the pin to make themselves green.
Step 2. Freeze the tape outside the driver

The driver reads tape.txt, calls fold_checked, and prints REJECT or a decimal integer. It contains no expected answers. A model that can see the golden line inside the driver will often echo it, and the echo will match at every optimization level.
// tape_driver.cpp
#include "contract.hpp"
#include <fstream>
#include <iostream>
#include <string>
#include <vector>
int main() {
std::ifstream in("tape.txt");
std::vector<std::int64_t> data;
std::string tok;
while (in >> tok) {
if (tok == "END") {
break;
}
data.push_back(std::stoll(tok));
}
std::optional<std::int64_t> got = fold_checked(data.data(), data.size());
if (!got) {
std::cout << "REJECT\n";
return 0;
}
std::cout << *got << "\n";
return 0;
}
Start with three tapes, and keep them in separate files. One sum fits in 64 bits. One sum crosses the top of the signed 64-bit range. One file contains only END, the empty range.
The samples below are unexecuted fixtures until a local compiler has produced the four binaries. The method does not depend on a hosted runner.
10
20
30
END
9223372036854775807
1
END
Step 3. Build both sides at two levels

Hold the standard, the warning set, and the include path fixed. Change only the optimization flag. Record the compiler identity before the first object file exists, so a later green row can be tied to a toolchain.
mkdir -p out
g++ --version | head -n 1 | tee out/compiler.txt
CXXFLAGS="-std=c++20 -Wall -Wextra -Werror -I."
g++ $CXXFLAGS -O0 -o oracle_o0 oracle.cpp tape_driver.cpp
g++ $CXXFLAGS -O2 -o oracle_o2 oracle.cpp tape_driver.cpp
g++ $CXXFLAGS -O0 -o cand_o0 candidate.cpp tape_driver.cpp
g++ $CXXFLAGS -O2 -o cand_o2 candidate.cpp tape_driver.cpp
A candidate that fails to compile receives a compile grade, and the matrix stops. Partial binaries are not compared. An oracle that fails to compile means the gate is misconfigured. That result is not a patch defect.
The oracle itself should be boring. This implementation is a proposal for the fixture, not a claim that a model emitted it, and not a measured result.
#include "contract.hpp"
#include <cstdint>
#include <limits>
#include <optional>
std::optional<std::int64_t> fold_checked(const std::int64_t* data, std::size_t n) {
__int128 acc = 0;
for (std::size_t i = 0; i < n; ++i) {
acc += data[i];
if (acc > std::numeric_limits<std::int64_t>::max()) {
return std::nullopt;
}
if (acc < std::numeric_limits<std::int64_t>::min()) {
return std::nullopt;
}
}
return static_cast<std::int64_t>(acc);
}
Run each binary from a directory that contains only the chosen tape.txt. Capture stdout bytes. Do not trim, and do not pass the text through a shell that rewrites newlines.
./oracle_o0 > out/oracle_o0.txt
./oracle_o2 > out/oracle_o2.txt
./cand_o0 > out/cand_o0.txt
./cand_o2 > out/cand_o2.txt
cmp -s out/oracle_o0.txt out/oracle_o2.txt
Step 4. Score a four-cell matrix

Compare the files with cmp. A missing newline is a mismatch. A numeric parser that forgives spaces will hide the drift this gate exists to catch.
Oracle -O0 vs -O2Candidate vs oracle at -O0Candidate vs oracle at -O2Gradedifferanyanyoracle unstable; do not score the patchmatchdifferanyfunctional missmatchmatchdifferopt-split failmatchmatchmatchpass for this tape

The third row is the silent regression. The patch met the contract in the debug build and broke it, or changed it, at -O2. Store the four stdout files beside the prompt hash. A later job that flips only cand_o2 is a regression even when the debug line still matches the earlier example.
Repeat the matrix for every tape. The patch passes the gate only when every tape passes. One opt-split fail fails the run. Do not average rows into a soft score. A mean will bury a release-only break under three comfortable debug matches.
Step 5. Attach a draft host without baking in its limits

Disclosure: This article was prepared as part of MonkeyCode's product outreach.
The matrix does not need a hosted model. A local editor can drop candidate.cpp into the tree, and the four g++ lines still apply. MonkeyCode enters only when a team wants a draft step or a remote build host. It is an open-source project. Operator materials describe free model access and a free server option. Both are availability claims, so confirm them in the current project docs on the day of the run.
Keep the draft blind. It may read prompt.txt and contract.hpp. It may not read oracle.cpp, the tapes, or earlier stdout. Showing the oracle to the model turns the grade into a transcription exercise, and transcription will pass at both optimization levels for the wrong reason.
Do not hard-code a token allowance, a machine size, or a rental duration into prompt.pin or the grader. Those numbers move, and a stale ceiling will outlive the plan that published it. If the account cannot start a build that day, fail the job closed and leave the previous pin untouched.
The free server, when it is actually available, is a fit for the four compile-and-run lines. The pass/fail rule should stay in a script the reviewer can diff on a laptop. A clean split is local pin check, remote compile, local grade. The host returns stdout files and the compiler version line, and the reviewer's copy of the matrix script assigns the grade.
Teams that already keep prompt.pin can send the frozen prompt to the current free model access and run this opt-split matrix on the free server, after reading that day's limits in the project docs. The grade script should still run unchanged if that host is offline.
Limitations, and who should skip the gate

Agreement across -O0 and -O2 is a screen, not a proof. Undefined behavior can survive both flag sets, and it can disappear under both. Sanitizer builds belong in a separate gate. This matrix also ignores throughput. A patch may pass every tape and still be too expensive to ship, so grade cost with a different fixture.
Skip the approach when the specification allows divergence under fast-math flags, or when the contract is approximate and a tolerance is the real oracle. Skip it when the oracle and the candidate are supposed to reject different edges. That mismatch is a spec argument, not a regression.
Skip it for patches that cannot be built as one extra translation unit, such as a plugin ABI or a generated lexer. The sample driver uses std::stoll and will throw on a corrupt tape. That is a harness bug, and the run should fail before any patch grade is written.
The __int128 accumulator is a GCC and Clang extension, not portable C++. On a toolchain without it, replace the oracle with a checked add written in standard C++. Pin that choice next to the compiler version line. Otherwise two machines will disagree about the oracle and the matrix will blame the patch.
What the log must hold

Each run record needs the prompt hash, the compiler version string, the four flag lines, and the stdout hashes. A green debug line without those fields is how a single total survives review. The release build was never graded, so nobody had a file to open.
The opt-split gate makes that file mandatory. Same tape, two optimization levels, and a human oracle that does not travel inside the prompt. When the release total moves and the oracle does not, the patch does not ship.#cpp #testing #ai #programming #software #coding #development #engineering #inclusive #community
Debug Totals Lie at Release: An Opt-Split Gate for C++ Patch Evals

#pragma #include #cpp #testing #inclusive #engineering
Dev Community @[email protected]
2026-10-12
A maintainer on a small library closed a pull request after the patch looked tidy and the model review sounded confident. The contributor had pasted a long log into chat, received praise, and opened the request without a public failing fixture. On a clean clone of the default branch, the reported command exited zero, so the patch had nothing to prove. That scene is a composite of common review failures, not a claim about one named project.
The same pattern appears when a model reviews a diff before anyone proves the bug on the upstream default branch. Agreement then becomes a style comment about the patch, rather than evidence that the failure existed at all. A useful workflow freezes a public bundle first, then invites a model to comment only on fields inside that bundle. The bundle, not the model's tone, decides whether the patch is ready for a human maintainer.
Freeze a public bundle the model is allowed to see

A review bundle is a directory committed on a scratch branch, or attached to the issue, before any model sees the patch. It holds the failing command, the expected exit status, a fixture file, and a short note about the default-branch commit. It also holds a license check result and a test command copied from the repository's own documentation. Secrets, tokens, and private hostnames stay out of that directory so the packet can be shared with a model.
Fields that make the packet checkable

The file base_sha.txt records the upstream commit where the failure was observed, and later reruns must detach at that commit.
The file exit_status.txt stores the status of the failing command, and a clean default-branch run must reproduce that status.
The file fixture.sha256 names the public input, so a review cannot swap in a quieter file after the model has spoken.
The file license_gate.txt records whether an SPDX identifier search succeeded, which is a gate and not a full legal review.

Build the bundle with a script that can be rerun

The following shell script is an unexecuted proposal, not a recorded run from this account, and paths must be adapted. It writes the base commit, copies one fixture, captures status streams, probes for an SPDX line, and hashes the packet. A contributor should replace the sample CLI invocation with the command already documented by the upstream project. The hash file is the handle a later review uses to prove the packet did not drift between machines.
#!/usr/bin/env bash
# Proposal: unexecuted example. Adapt paths before use.
set -euo pipefail
ROOT="$(git rev-parse --show-toplevel)"
SHA="$(git rev-parse HEAD)"
OUT="${ROOT}/.review-bundle"
mkdir -p "${OUT}/fixture"
echo "${SHA}" > "${OUT}/base_sha.txt"
cp "${ROOT}/repro/input.txt" "${OUT}/fixture/input.txt"
sha256sum "${OUT}/fixture/input.txt" > "${OUT}/fixture.sha256"
set +e
python3 -m project.cli --input "${OUT}/fixture/input.txt" \
> "${OUT}/stdout.txt" 2> "${OUT}/stderr.txt"
STATUS=$?
set -e
echo "${STATUS}" > "${OUT}/exit_status.txt"
if git grep -n "SPDX-License-Identifier" -- '*.py' > "${OUT}/spdx.txt"; then
echo "spdx-present" > "${OUT}/license_gate.txt"
else
echo "spdx-missing" > "${OUT}/license_gate.txt"
fi
find "${OUT}" -type f ! -name 'packet.sha256' -print0 \
| sort -z | xargs -0 sha256sum > "${OUT}/packet.sha256"
Running the script on a dirty worktree will mix unrelated edits into the story the packet tells the reviewer. A clean detached checkout of the recorded base commit is the honest place to capture the failing status. If that status is already zero, the contributor stops and does not ask a model to invent a bug. That stop is the point of the bundle, and it saves the maintainer from reviewing a speculative patch.
Climb three rungs before any model comment

Rung one runs the failing command on a clean checkout of the recorded base commit and expects the frozen exit status. Rung two applies the patch on that same commit and expects the command to exit zero. Rung three runs the repository test command, not a new test invented for the patch alone. A model should not be asked to invent those commands after the fact, because that invites a moving target.
Repeat the rungs on a second checkout

git fetch origin
git switch --detach "$(cat .review-bundle/base_sha.txt)"
bash scripts/oss-review-bundle.sh
test "$(cat .review-bundle/exit_status.txt)" = "1"
git apply --check patches/fix.diff
git apply patches/fix.diff
# Proposal: after the patch, rerun the documented command and expect status 0.
The first test line fails the workflow when the recorded status is not reproduced, which is the desired alarm. The apply check catches a patch that does not fit the recorded base commit, before anyone discusses coding style. After the patch applies, the same command should exit zero, and the repository test command should still pass. A model review that arrives before these three results exist is commentary, not evidence a maintainer can trust.
Minimize the fixture without deleting the failure

A long log invites a model to summarize noise and then miss the line that actually fails the command. The contributor should cut the fixture in halves, rerun the failing command, and keep the half that still fails. Each cut is recorded by appending the new fixture hash to a short minimization log stored inside the bundle. When a cut makes the status become zero, that cut is discarded and the previous fixture remains the public input.
Steps that keep the failure alive

Copy the original fixture into the bundle and record its hash, its line count, and the failing exit status.
Split that fixture into two halves and rerun the documented failing command against each half on the recorded commit.
Keep the smallest half that still returns the frozen status, and append that hash to the minimization log.
Stop when neither half fails, restore the previous fixture, and treat that file as the public input for review.
The log format below is a proposal with placeholder hashes, not measured output from a repository. A real run replaces those tokens with sha256sum output and the exit status just observed. The discarded row stays in the log so a reviewer can see which cut destroyed the failure. That trail matters more than a shorter file that no longer fails.
step=0 lines=48 sha256=placeholder-original status=1 kept
step=1 lines=24 sha256=placeholder-half status=1 kept
step=2 lines=12 sha256=placeholder-quarter status=0 discarded
Score review comments by citation

A model review earns trust only when each remark names a file that already sits inside the frozen bundle. A comment that calls the patch correct without naming exit_status.txt or fixture.sha256 scores zero for evidence. A comment that cites a packet file and quotes a line can be checked by a human in one pass. This scoring habit resists fluent agreement that replaces a failed reproduction on the recorded base commit itself.
The table below is a proposal for sorting comments, not a measured study from this account's history. Strong rows still require a human to open the cited file before the patch is allowed to move. Weak rows are corrections to the note, not reasons to merge the patch into the default branch. Empty rows are discarded even when the wording sounds confident, polite, and ready for a maintainer.
Model comment patternPacket citationHuman actionFluent praise with no filenameNoneDiscard the commentCites exit_status.txt but quotes the wrong digitWeakOpen the file and correct the noteCites fixture.sha256 and the minimization logStrongCompare hashes, then read the diffSuggests a test command absent from the repo docsOut of packetIgnore until a human adds that command
# Proposal: unexecuted. Scores whether review lines cite packet files.
from pathlib import Path
PACKET = {
"base_sha.txt",
"exit_status.txt",
"fixture.sha256",
"license_gate.txt",
"packet.sha256",
}
text = Path("model-review.md").read_text(encoding="utf-8")
cited = {name for name in PACKET if name in text}
missing = PACKET - cited
print(f"cited={sorted(cited)}")
print(f"missing={sorted(missing)}")
if not cited:
raise SystemExit("review cited no packet file")
The checker is also an unexecuted proposal, and it only tests whether filenames appear in the review text. It cannot tell whether the model understood the exit status, so a human still opens every cited file. If the checker exits because no packet file was named, the review is rejected without a second pleading prompt. A second prompt that begs for agreement would only recreate the original failure mode this bundle is meant to stop.
Use free model access only after the hash is frozen

MonkeyCode is relevant here only as an optional place to run the citation pass and to repeat the bundle script. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator-supplied claim used here is that free model access and a free server option currently exist. The operator also describes the project as open source, which readers should confirm in the repository before they depend on it.
This article does not state model names, token quotas, hardware sizes, or how long that access lasts. Readers should confirm current terms on the project page before relying on either option for a real review. A missing or changed offer does not break the ladder, because the packet can be reviewed by a person alone. Hosted help is a convenience after the hash exists, not a requirement for the reproduction or the patch.
A free model pass is useful after the packet hash is frozen, because the model then has a closed set of files to cite. A free server rerun is useful when the contributor's laptop has extra packages that the upstream image does not install. Both steps stay optional, and the ladder still works with a local model or with a second container on the same desk. The packet hash, not the model's tone, remains the acceptance criterion for moving the patch toward a human reviewer.
The prompt should ask for missing packet fields, mismatched hashes, and a license gate that printed spdx-missing. It should not ask for a verdict that the patch is ready to merge, because that verdict is a human decision. Output belongs in model-review.md beside the packet, so the filename checker can read it without extra parsing. The contributor then keeps only sentences that name a packet file before the maintainer sees the note.
The contributor deletes any sentence that does not name a packet file before the maintainer sees the note. That deletion is a human edit, and the pull request should say the model text was filtered by the checker. A maintainer can then review the patch, the packet, and a short list of cited gaps without rereading praise. The model does not appear as an author of the commit, because authorship stays with the person who can answer questions.
Limitations that should stay visible

This workflow does not prove that a patch is correct, and it does not replace the maintainer's review or the project's legal process. A minimized fixture can drop the real bug if the contributor cuts the input until the failure disappears. License header searches miss files that use a different SPDX line, or a license that lives only in a top-level file. A model can still cite the right filename and misread the exit status, so a human must open the cited file.
The free server option, where it is available, does not guarantee the same packages as the project's continuous integration image. A rerun that passes on a generic machine can still fail inside the upstream workflow file on the next push. Contributors should treat a green second machine as a portability hint, not as a substitute for the project's own checks. Quotas, model names, and how long free access lasts sit outside this article and must be read from current terms.
Who should skip the bundle

Skip this bundle when the repository forbids attached fixtures, or when the bug depends on private data that cannot be reduced. Skip it when the change is a documentation typo with no failing command, because a fake exit status would only add noise. Skip it when the contributor cannot read the test output well enough to reject a confident but wrong model comment. In those cases a short issue comment and a human review remain the honest path, and a model pass would not help.
What the maintainer should be able to replay

A maintainer should be able to detach at the recorded commit, rerun the bundle script, and see the same fixture hash. The patch should flip the frozen status, and the citation checker should show at least one packet file in the review note. If either hash differs, the conversation stops until the contributor publishes a new packet rather than explaining the drift in chat. That replay is the whole method, and it remains useful even if no hosted model or free server is used.
Readers who still want a clean second machine for that replay can check whether MonkeyCode's free server option is open. They should require the same packet hash on that machine before they trust the rerun at all. The free model access, if currently offered, fits only the citation pass after the hash file already exists. Neither option is a merge approval, and the project page is the place to confirm what is available that day.#opensource #testing #git #ai #software #coding #development #engineering #inclusive #community
A Citation Check for OSS Patches Starts With a Minimized Public Fixture
#opensource #ai #community #software #git #coding
Dev Community @[email protected]
2026-10-12
You open the incident channel on Monday and find three agent retries stacked against one failed checkout ticket. The original prompt is gone, replaced by a shorter note that a teammate typed just before lunch. Nobody can say which tool schema the first call used, or whether the second call changed the system text. Shared model access still shows the spend, yet the wiki names no owner for that sequence of calls.
That missing record is the defect you can close without arguing about which model sounds smarter this week. A retry that overwrites the failed prompt destroys the only input evidence the ticket still had left. You need a wiki handoff that freezes the prompt, names the roles, and blocks the next call until that page exists.
What the freeze records

A prompt freeze is a small file you write before anyone is allowed to send the same job again. It stores a hash of the prompt, a hash of the tool schema, the ticket id, and the name of the retry owner. It does not claim the model was correct, and it does not replace a review of the patch the agent proposed. You are keeping the input that produced a bad or unclear result, so the next person debugs that input instead of a rewritten memory.
Teams skip the file because a second call feels faster than a short write-up in the wiki. On shared free model access, that habit spends a common pool and leaves the next on-call engineer with nothing to diff. You can keep the write-up to one wiki page plus a JSON file that a local script is able to check.
Roles and the handoff rule

You name four roles at the top of the page, even when one person holds two of them on a small team. The prompt owner keeps the frozen text and may change it only by creating a new freeze id. The retry owner is the only person who may send the job again after the freeze file exists. The schema owner confirms that the tool definitions match the hashed file, including argument names and required fields.
The scribe pastes the numbered run into the ticket and links the freeze file so the handoff is visible. These roles fail when they stay implicit and everyone assumes someone else still has the original text. The retry owner must not edit the prompt in the same step used to resend the job. If the prompt owner wants a new instruction, they mint the next freeze id and the retry owner waits for the check to pass.
The numbered run you paste into the wiki

You keep the run short enough to paste under the ticket, then you link longer notes beneath that block. You number every step and you do not skip a line when a field is still unknown. An unknown field is a reason to stop the resend, not a reason to invent a plausible value.
Copy the exact prompt that failed into prompt.txt, including system text and user text, without tidying a single sentence.
Copy the tool schema the job actually loaded into tools.json, and ask the schema owner to confirm that filename in the ticket.
Run the freeze script so it writes freeze.json with both hashes, the ticket id, and a UTC timestamp from the machine clock.
Paste the freeze id, both hashes, and the four role names into the wiki page before anyone opens a model client again.
Let the retry owner resend only when the check command prints MATCH against those same two files.
If the prompt must change, stop, assign a new freeze id, and repeat from the first step instead of editing the old record.
You treat a missing file as a failed handoff rather than as a detail you can fill from memory. You do not reconstruct the prompt from chat recall and then call that reconstructed text a valid freeze. If the log viewer already dropped the text, you mark the page prompt_unavailable and you refuse the retry. You wait until someone recovers the text from client history or from version control before a new freeze exists.
Hash the prompt and the schema differently

The script below is an example you can copy into a scratch repository and run on your own machine. This example is unexecuted in this article, so you should run it on sample files before you adopt it. It does not call a model, and it is not a security boundary or an approval system for production changes.
You hash the prompt as raw bytes, because a changed space can change the model input even when the words look the same. You hash the schema from canonical JSON with sorted keys, so a cosmetic reorder does not look like a tool change. You should read every branch before you trust it, and you should keep this helper beside the wiki page the scribe linked.
#!/usr/bin/env python3
"""Example prompt freeze. Not a security boundary."""
import argparse
import hashlib
import json
import sys
from datetime import datetime, timezone
from pathlib import Path
def prompt_digest(path: Path) -> str:
return hashlib.sha256(path.read_bytes()).hexdigest()
def schema_digest(path: Path) -> str:
parsed = json.loads(path.read_text(encoding="utf-8"))
canonical = json.dumps(
parsed, sort_keys=True, separators=(",", ":"), ensure_ascii=False
)
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
def write_freeze(prompt: Path, schema: Path, ticket: str, owner: str, out: Path) -> None:
record = {
"ticket": ticket,
"retry_owner": owner,
"prompt_sha256": prompt_digest(prompt),
"schema_sha256": schema_digest(schema),
"frozen_at": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
"retry_allowed": False,
}
out.write_text(json.dumps(record, indent=2) + "\n", encoding="utf-8")
print(f"WROTE {out}")
def check_freeze(prompt: Path, schema: Path, freeze: Path) -> int:
record = json.loads(freeze.read_text(encoding="utf-8"))
ok_prompt = record.get("prompt_sha256") == prompt_digest(prompt)
ok_schema = record.get("schema_sha256") == schema_digest(schema)
allowed = record.get("retry_allowed") is True
if ok_prompt and ok_schema and allowed:
print("MATCH")
return 0
print("MISMATCH")
print(f"prompt_ok={ok_prompt} schema_ok={ok_schema} retry_allowed={allowed}")
return 1
def main() -> int:
parser = argparse.ArgumentParser(description="Freeze or check an agent prompt.")
parser.add_argument("--prompt", required=True, type=Path)
parser.add_argument("--schema", required=True, type=Path)
parser.add_argument("--ticket", default="unset")
parser.add_argument("--retry-owner", default="unset")
parser.add_argument("--out", type=Path)
parser.add_argument("--check", type=Path)
parser.add_argument("--allow-retry", action="store_true")
args = parser.parse_args()
if args.check:
return check_freeze(args.prompt, args.schema, args.check)
if args.out is None:
print("need --out or --check", file=sys.stderr)
return 2
write_freeze(args.prompt, args.schema, args.ticket, args.retry_owner, args.out)
if args.allow_retry:
record = json.loads(args.out.read_text(encoding="utf-8"))
record["retry_allowed"] = True
args.out.write_text(json.dumps(record, indent=2) + "\n", encoding="utf-8")
print("RETRY_FLAG_SET")
return 0
if __name__ == "__main__":
raise SystemExit(main())
You run the write step first, and you set the retry flag only after the wiki paste is actually in the ticket. The flag is a team convention inside the JSON file, not an access-control gate on the model client. You still depend on the retry owner to honor the flag, which is why the role is named on the page.
python freeze_prompt.py --prompt prompt.txt --schema tools.json \
--ticket CHKT-1842 --retry-owner "sam" --out freeze.json
python freeze_prompt.py --prompt prompt.txt --schema tools.json --check freeze.json
python freeze_prompt.py --prompt prompt.txt --schema tools.json \
--ticket CHKT-1842 --retry-owner "sam" --out freeze.json --allow-retry
python freeze_prompt.py --prompt prompt.txt --schema tools.json --check freeze.json
The check command should print MISMATCH until you create a fresh freeze that includes the allow-retry flag. That order is deliberate, because a copied command should not look successful before the handoff is done. If both hashes match and the flag is true, the script prints MATCH and the retry owner may send the single allowed call.
Sample files, not a captured job

The JSON shape below is a sample layout, not a capture from a live job, so you replace every field when you run the script. You can start from the tiny tool file, reorder its keys, and confirm the schema hash stays stable. You should not point this helper at YAML or at JSON with comments, because the parser will reject the file and the handoff will stop.
System: You may call lookup_order only. Do not invent an order status.
User: The checkout ticket says order 1842 failed after payment. Report the tool result only.
{
"name": "lookup_order",
"parameters": {
"type": "object",
"required": ["order_id"],
"properties": {
"order_id": {"type": "string"}
}
}
}
{
"ticket": "CHKT-1842",
"retry_owner": "sam",
"prompt_sha256": "<raw sha256 of prompt.txt>",
"schema_sha256": "<canonical sha256 of tools.json>",
"frozen_at": "2026-10-13T15:04:00Z",
"retry_allowed": false
}
Decision table for the retry owner

You do not resend the job just because a freeze file happens to sit in the working directory. You pick one row from the table and you record that row name in the ticket before the call. The table is a handoff aid, not a scoring model, and it never tells you to retry until a log turns green.
What you seeFreeze statusNext actionSame prompt, transport error, schema unchangedHashes match and retry flag is trueRetry owner sends once, then stops for a human readConfident but wrong tool callHashes matchPrompt owner writes a new freeze; do not resend the same textSchema file differs from what the job loadedSchema hash mismatchSchema owner repairs the file; nobody retriesPrompt text cannot be recoveredNo freezeScribe marks prompt_unavailable and stopsTicket holds secrets or unredacted customer textAnyStop; redaction is a separate review, not a retry

A later success does not prove that the first prompt was sound, especially when the text changed between calls. You use the freeze so a reviewer can diff the input that failed against the input that finally ran. If those hashes differ, you are looking at a new experiment, and you should label it as one.
Use free access without inventing a budget

Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode, in the facts supplied for this draft, provides free model access and a free server option you may use for this handoff. The retry owner can send the single allowed resend through that free model access, if the option is still offered when you run. You can keep the freeze script on the free server option when that host exists, rather than buying hardware for a hash. You do not write a token quota, a hardware size, a duration, or a permanence claim into the wiki unless current docs state them.
A stale number in the runbook will train the team to trust a budget that the product page no longer shows. You confirm the live terms before you link them, and you leave the quota line blank when the docs do not publish one. The useful constraint is the shared pool, because an undocumented retry spends attention and access that other tickets still need. If the free server option is absent on the day you need it, you still run the script anywhere Python 3 is installed.
You still refuse the resend until the wiki page and the freeze file both exist, even when the host is only a laptop. The product name is optional to the method, and the method should still work if you remove every mention of it. If you want one next step, add this page to the team wiki and point the retry owner at the check command.
Review the handoff in ten minutes

Open the freeze file and confirm the ticket id matches the page you pasted, including the prefix and the number.
Re-run the check command and keep the MATCH line in the ticket so a later reader sees the files were stable.
Ask the schema owner whether any tool argument changed after the hash, and stop if the answer is uncertain.
Write one sentence on what the new output did differently, and link it under the freeze id rather than over the old note.
You stop after one allowed resend unless a new freeze id exists, even when the output looks almost right. A second silent retry is how the Monday channel filled up with three calls and zero recoverable prompts. The page is done when a teammate who was absent can see who may call next and which files they must not edit.
Who should not use this page

This approach fails when the prompt is built in memory and never written to a file you can hash. It also fails when two jobs share one prompt file and you overwrite that file between the freeze and the check. You should not use the hash as a compliance archive, a secret scanner, or proof that an agent patch is safe to merge. Teams that already store immutable prompt versions in their orchestrator can skip the extra ritual unless retries drop that text.
You should skip any hosted model path when the ticket holds secrets, payment data, or text your policy forbids you to send. The freeze script can still run on redacted local fixtures, but the resend must wait for a route your policy allows. A matching hash is not permission to paste raw customer logs into a model, whether that model is free or paid. You also should not expect this page to catch a model that agrees too quickly with a flawed premise.
The freeze only keeps that premise stable long enough for a person to read the words before the next call. Judgment stays with the prompt owner and with the reviewer who inspects the tool call, not with the hash itself. If your team cannot spare a named retry owner, do not pretend the script will enforce the handoff for you.#python #ai #devops #tutorial #software #coding #development #engineering #inclusive #community
Freeze the Failed Prompt: A Wiki Handoff Before an Agent Retry
#devops #ai #community #software #coding #python
Dev Community @[email protected]
2026-10-12
An agent patch does not earn a flake freeze from one red run on a wide fixture. The freeze may open only after the property predicate, the stopped witness, and a disagreeing rerun series are all on the same record. A stable failure is a reject, and a stable pass is not a freeze.
That rule is narrower than rerunning a suite until it turns green. A rerun hides a regression when the input is still large, the seed changes, or the property text moves with the patch. Shrink first, then repeat, then classify.
A digest match on the original fixture is not the decision. The digest that matters is the one taken after shrinking stops, on the patch that actually ran.
Bind four identifiers before you classify

Every attempt carries a patch digest, a property id, a fixture digest, and a seed. Missing any one of them is a hold, not a pass and not a freeze. The property id must point at a predicate a reviewer can read without opening a model transcript. The fixture digest hashes the input that actually ran, not the directory the agent was given at the start.
A property is a pure check over an explicit fixture and the observed output. Wall clocks, network calls, and map iteration order stay out of the predicate unless they are fixture fields. If the patch edits the predicate, the property id changes. An oracle that the diff rewrote cannot certify the edit that rewrote it.
Hash the patch before the first run. If the worktree does not match that hash, stop. Later reruns that include a small fix are a different patch and need a new record. The gate does not inherit evidence across digests.
Shrink until the failure stops moving

A failing fixture is often larger than the bug. Unused flags, extra rows, and stale optional keys make the next run look random when one field is doing the work. Shrinking deletes or narrows a piece only when the same property still fails. The loop stops when the next candidate no longer fails, or when the candidate is identical to the current fixture.
The stopped fixture is the witness. Later reruns must match its digest. A rerun on the original wide file goes back to hold, even if the property id is unchanged. Otherwise the freeze record cites a test that was not the one under dispute.
This stop condition does not prove the witness is minimal in the mathematical sense. It proves only that the supplied shrink function had no further failing step within a fixed step budget. That is a bound on the tool. It is not a claim about every smaller input a person might invent.
Keep the failure identity stable while shrinking. If the property returns a reason code, a shrink step that changes the code is refused and the previous fixture remains the witness. A new reason code is a different failure, and it should not be folded into the original freeze discussion.
Decision table

The table below is the whole classifier. It has no hidden score.
Observed stateGatePatch digest, property id, fixture digest, or seed is emptyHoldShrink loop has not stopped, or the step budget ended without a local stopHoldFewer reruns than the policy minimum on that witnessHoldOutcome labels are outside pass and failHoldEvery recorded rerun passesPassEvery recorded rerun failsRejectReruns disagree, shrink stopped, expiry timestamp presentFreeze candidateReruns disagree, expiry timestamp emptyHold

The sample defaults are policy fields, not measurements.
Policy fieldValue in the sampleWhat it is notmin_reruns3a confidence interval or a false-accept ratemax_steps32a proof that no smaller failing input existsfreeze_expiresrequired non-empty stringa recommended duration

Three reruns is the smallest series that can show agreement, total failure, or disagreement without treating one run as a story. Teams that already require five reruns should pass that number in, and should not treat the default as a benchmark.
A freeze candidate is not a merge approval. It records that one witness was unstable under one seed and one digest. Expiry is an operator-supplied timestamp, and this article does not recommend a duration. An empty expiry keeps the candidate closed, so a freeze cannot become a permanent skip by omission.
Numbered procedure

Compute the patch digest and refuse to run if the worktree is dirty relative to it.
Select a property id the diff does not touch. If the predicate file is in the diff, stop and review the oracle before any rerun or shrink.
Execute the property once on the submitted fixture, with the seed recorded in the fixture header or the job spec. Store the outcome and the fixture digest.
On failure, shrink until a local stop or until the step budget ends. If the budget ends without a stop, hold. Do not freeze a witness the shrinker was still changing.
Hash the stopped witness. Rerun the same property on that digest with the same seed until the policy minimum is met. Do not draw a new seed between attempts.
Apply the decision table. Persist the four identifiers, the outcomes, and the expiry field in one record. A pull-request comment is not a substitute for that record.
Initial passes still need an explicit sample plan when the patch claims to fix a prior flake. One green run on a wide fixture does not retire a freeze candidate from an older digest. The old candidate expires on its own timestamp, or a reviewer closes it by pointing at a new stable pass on the same property id and a recorded witness. It does not vanish because the branch is green today.
Proposal code

The module below is a proposal. It was not executed against a production suite for this article. It does not call a model, and it does not open a network connection. fails and shrink_once are injected by the harness you already trust.
import hashlib
from dataclasses import dataclass
from enum import Enum
class Gate(str, Enum):
PASS = "pass"
REJECT = "reject"
HOLD = "hold"
FREEZE_CANDIDATE = "freeze_candidate"
@dataclass(frozen=True)
class Witness:
patch_digest: str
property_id: str
fixture_digest: str
seed: str
shrunk: bool
outcomes: tuple[str, ...]
freeze_expires: str
shrink_budget_hit: bool = False
def sha256_bytes(payload: bytes) -> str:
return hashlib.sha256(payload).hexdigest()
def bounds_check(fixture: dict, output: dict) -> bool:
return fixture["lo"] <= output["value"] <= fixture["hi"]
def drop_optional(fixture: dict):
if "optional_note" not in fixture:
return None
nxt = dict(fixture)
del nxt["optional_note"]
return nxt
def decide(w: Witness, min_reruns: int = 3) -> Gate:
if min_reruns < 2:
raise ValueError("min_reruns must compare more than one attempt")
required = (w.patch_digest, w.property_id, w.fixture_digest, w.seed)
if any(not item for item in required):
return Gate.HOLD
if w.shrink_budget_hit or not w.shrunk:
return Gate.HOLD
if len(w.outcomes) < min_reruns:
return Gate.HOLD
if any(item not in {"pass", "fail"} for item in w.outcomes):
return Gate.HOLD
unique = set(w.outcomes)
if unique == {"pass"}:
return Gate.PASS
if unique == {"fail"}:
return Gate.REJECT
if not w.freeze_expires:
return Gate.HOLD
return Gate.FREEZE_CANDIDATE
def shrink_until_local_stop(fixture, fails, shrink_once, max_steps=32):
current = fixture
for _ in range(max_steps):
if not fails(current):
return None
nxt = shrink_once(current)
if nxt is None or nxt == current or not fails(nxt):
return current
current = nxt
return current
bounds_check is only a shape example. Replace it with the predicate the patch claims to satisfy. drop_optional shows the shrink contract: return None when nothing optional remains, and never drop lo or hi in the same step. shrink_until_local_stop returns None when the current fixture does not fail, which signals an unsound step or a mismatched predicate. Callers should hold in that case rather than invent a witness.
A fixture header that can be hashed as canonical JSON looks like this. Sort keys before hashing so two equivalent objects do not produce two digests.
{
"patch_digest": "<64 lowercase hex chars>",
"property_id": "bounds_check",
"seed": "17",
"fixture": {"lo": 0, "hi": 10, "optional_note": "drop-me"}
}
A branch check can sit beside the module. The command is an example, not a reported run from a published build. The timestamp only shows that the expiry field is populated.
python - <<'PY'
from gate_witness import Witness, Gate, decide
base = dict(patch_digest="p"*64, property_id="bounds_check",
fixture_digest="f"*64, seed="17", shrunk=True,
shrink_budget_hit=False)
flake = Witness(**base, outcomes=("fail", "pass", "fail"),
freeze_expires="2026-10-19T00:00:00Z")
assert decide(flake) is Gate.FREEZE_CANDIDATE
stable = Witness(**base, outcomes=("fail", "fail", "fail"),
freeze_expires="2026-10-19T00:00:00Z")
assert decide(stable) is Gate.REJECT
no_end = Witness(**base, outcomes=("fail", "pass", "fail"), freeze_expires="")
assert decide(no_end) is Gate.HOLD
print("branches ok")
PY
sha256sum witness/minimal.json
Use the expiry your review policy already requires. Do not copy the sample timestamp into a real record.
Separate the draft from the rerun

Writing a predicate and executing it are different jobs. A fluent draft can look like a test while still encoding the bug it is supposed to catch. Keep generation on one side of the record and execution on the other.
Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode's free model access can draft a candidate predicate or a candidate shrink step as text. That text is untrusted input. It enters the repository only after the same review you would apply to a human patch, and it must not be a file the agent diff already rewrote.
MonkeyCode's free server option can host the rerun command so the witness digest is computed outside a worktree that still has uncommitted fixture edits. This article does not state quotas, hardware, model names, or how long either option remains available. If the remote job cannot print the patch digest it executed, discard the outcomes and hold.
Pin the command, the seed, and the witness file in the job spec. A runner that rebuilds a lookalike fixture is not repeating the witness. Similarity is not a digest match.
Limitations

The shrinker is unsound when shrink_once removes the field that caused the failure and fails still returns true for a different reason. The gate will then reject or freeze the wrong witness. Prefer shrink steps that delete one optional key at a time, and refuse a step whose reason code changes.
Disagreement across the minimum reruns does not name the flake source. The cause may be a race, hash-ordered iteration, or a clock the fixture forgot to pin. The candidate only blocks automatic merge. It is not a root-cause report.
A pass on the stopped witness says nothing about inputs the shrinker never produced. If the property is meant to cover a range, keep a separate broader sample. That sample must cite the same patch digest and the same property id, or it belongs to a different gate and cannot be added into this one.
No false-accept rate is claimed for the branch checks. The shell snippet locks control flow only. It does not inject faults, and it should not be cited as a detection percentage.
Who should leave this gate closed

Do not point a shrink loop at live third-party services. Repeated calls create load, and remote jitter masquerades as disagreement.
Do not use the table when the patch exists to change the property file. Review that oracle by hand, then mint a new property id.
Do not open a freeze during incident response if the failing check is the only signal on a known bad deploy. Incidental disagreement is not a reason to silence that signal. The stable-fail branch is the relevant output there, and a freeze is the wrong one.
Also skip the procedure if the four identifiers cannot be stored beside the result. A green badge without a witness digest is not evidence the property ran.
If a free server seat is already how reruns are hosted, print the patch digest and the witness digest before the gate word, and keep the shrink log in the same commit as that patch digest. The decision table still outranks the runner.#testing #python #ai #programming #software #coding #development #engineering #inclusive #community
Shrink the Witness Before a Flake Freeze Can Open
#testing #inclusive #engineering #development #python #coding
Dev Community @[email protected]
2026-10-11
A second model call is a budget decision, not a reflex, and this seventy-minute workshop teaches a local classifier that blocks the reflex. Students replay seven recorded failures, assign retry once, stop, or inspect, and leave with a standard-library script that runs without a network. The exercise fits a shared lab that already calls a free model endpoint and may attach a free server for class time. It does not prove that any host will keep those options, publish quotas, or name a stable model.
Why the label comes before the dashboard

The useful question is narrower than a general retry slogan, and it shows up whenever a lab client treats every error as temporary. A free endpoint still spends shared capacity and attention when the client repeats a request that the contract already rejected. A transport timeout and a schema rejection both print as failures, yet only the timeout can justify one more attempt. The workshop therefore grades the failure label, not the prose quality of whatever answer a model might have returned.
Outcomes students must be able to defend

By minute seventy, each student should be able to defend three actions with a record, a reason, and a second-call boolean. Retry once means a single extra attempt is allowed, and the next identical failure must stop. Stop means the client must not call again until a human changes the request, the credential, or the contract. Inspect means the row is incomplete, so automation should surface it instead of guessing a retry.
Clock for a prepared room

The clock assumes a prepared room, so instructors should paste the script into the workspace before students arrive. Each block has an exit check, and the class should not advance while that check is still red. Ten minutes of slack sits inside the middle blocks, which is enough to debug a path error or a Python version mismatch. The final ten minutes are for limitations, not for a live demo that the room has not measured.
Minutes 0-10: define retry once, stop, and inspect without opening the script.
Minutes 10-25: run the module and match the designed seven-line summary.
Minutes 25-45: write a defense for the schema row and the rate-limit row.
Minutes 45-60: trace the dry-run guard and add one unclassified status.
Minutes 60-70: submit a memo listing three cases the script must not decide.

Materials and honesty rules

The graded path uses Python 3.11 or newer and no third-party packages, which keeps the room independent of package mirrors. Students need a terminal, a text editor, and the fixture list inside the module they are about to save. A live key is out of scope, because the point is to classify records the instructor already captured or composed. Treat the script as a teaching artifact with a designed exit check, not as a measured benchmark from a production host.
Decision table students implement

The matrix below is the contract students implement, and the script is only a runnable copy of that contract. Status numbers are a teaching subset, so an unlisted code must land in inspect rather than in a silent default retry. A 429 is inspect on purpose, because a free shared pool may be enforcing a limit rather than reporting a short blip. Connection failures stay eligible for one retry, and the already-retried flag then forces stop on the following row.
SignalActionSecond callNoteerror is timeout, connection_reset, or dns_temporaryretry_onceyes, onceTransport may clear without a contract editstatus 408, 500, 502, 503, or 504retry_onceyes, onceStop instead if already_retried is truealready_retried is truestopnoThe one-attempt teaching budget is spentstatus 400, 401, 403, 404, 413, or 422stopnoThe same call will not repair the contractstatus 429inspectnoDo not inherit the transient retry pathstatus 200 with an empty bodyinspectnoA success status is not a complete answermissing or unlisted statusinspectnoGuessing a retry is a failed exit check

The classifier checks already_retried before status, so a repeated 503 stops because the one-attempt budget is already spent.
The classifier checks coarse transport errors before status, so a timeout with a missing status still qualifies for one retry.
The classifier checks 429 before the transient set, so a rate-limit row cannot slip into an automatic second call.

Worked module students rerun

Save the following module as retry_gate.py and run it with Python 3 before the discussion period begins. The example is local and unexecuted against any vendor, and the printed lines are the designed checks students should reproduce. If a row fails, fix the classifier rather than editing the expected label to match a convenient result. That rule keeps the lab honest when a later cohort wants to add records from a real log.
#!/usr/bin/env python3
'''Teaching fixture: classify a failure before a second model call.
Local and unexecuted against any vendor. Designed checks are not a benchmark.
'''
from __future__ import annotations
import sys
from dataclasses import dataclass
RETRY_ONCE = 'retry_once'
STOP = 'stop'
INSPECT = 'inspect'
STOP_STATUS = {400, 401, 403, 404, 413, 422}
RETRY_STATUS = {408, 500, 502, 503, 504}
TRANSPORT_ERRORS = {'timeout', 'connection_reset', 'dns_temporary'}
@dataclass(frozen=True)
class Decision:
action: str
reason: str
def classify(record: dict) -> Decision:
status = record.get('status')
error = (record.get('error') or '').lower()
body = record.get('body')
if record.get('already_retried'):
return Decision(STOP, 'retry budget already spent')
if error in TRANSPORT_ERRORS:
return Decision(RETRY_ONCE, 'transport error: %s' % error)
if status in STOP_STATUS:
return Decision(STOP, 'client or auth status %s' % status)
if status == 429:
return Decision(INSPECT, 'rate limit needs a human rule')
if status in RETRY_STATUS:
return Decision(RETRY_ONCE, 'transient status %s' % status)
if status == 200 and not body:
return Decision(INSPECT, 'empty success body')
if status is None:
return Decision(INSPECT, 'missing status')
return Decision(INSPECT, 'unclassified status %s' % status)
def allow_second_call(record: dict) -> bool:
decision = classify(record)
return decision.action == RETRY_ONCE and not record.get('already_retried')
def plan_next(record: dict) -> dict:
decision = classify(record)
if allow_second_call(record):
return {
'id': record['id'],
'plan': 'one_retry',
'reason': decision.reason,
}
return {
'id': record['id'],
'plan': 'skip',
'reason': decision.reason,
}
FIXTURES = [
{
'id': 't1',
'status': 503,
'error': '',
'body': None,
'already_retried': False,
'expect': RETRY_ONCE,
'expect_plan': 'one_retry',
},
{
'id': 't2',
'status': 401,
'error': '',
'body': {'message': 'unauthorized'},
'already_retried': False,
'expect': STOP,
'expect_plan': 'skip',
},
{
'id': 't3',
'status': None,
'error': 'timeout',
'body': None,
'already_retried': False,
'expect': RETRY_ONCE,
'expect_plan': 'one_retry',
},
{
'id': 't4',
'status': 429,
'error': '',
'body': None,
'already_retried': False,
'expect': INSPECT,
'expect_plan': 'skip',
},
{
'id': 't5',
'status': 503,
'error': '',
'body': None,
'already_retried': True,
'expect': STOP,
'expect_plan': 'skip',
},
{
'id': 't6',
'status': 200,
'error': '',
'body': None,
'already_retried': False,
'expect': INSPECT,
'expect_plan': 'skip',
},
{
'id': 't7',
'status': 422,
'error': '',
'body': {'message': 'schema'},
'already_retried': False,
'expect': STOP,
'expect_plan': 'skip',
},
]
def run() -> int:
failures = 0
for row in FIXTURES:
decision = classify(row)
plan = plan_next(row)
action_ok = decision.action == row['expect']
plan_ok = plan['plan'] == row['expect_plan']
if not (action_ok and plan_ok):
failures += 1
print(
'mismatch id=%s expected_action=%s expected_plan=%s'
% (row['id'], row['expect'], row['expect_plan']),
file=sys.stderr,
)
second = allow_second_call(row)
print(
'%s: action=%s second_call=%s plan=%s reason=%s'
% (row['id'], decision.action, second, plan['plan'], decision.reason)
)
print('checked=%s failed=%s' % (len(FIXTURES), failures))
return 1 if failures else 0
if __name__ == '__main__':
sys.exit(run())
python3 retry_gate.py
echo exit:$?
How to read the designed output

Read the designed output as a contract for the checker, not as a log captured from a running server. True on second_call appears only for t1 and t3, and every other row must print False and plan skip. Row t5 is the budget lesson, because the status is still 503 while already_retried forces an immediate stop. If your local run differs, treat the difference as a bug in the copy, not as a new research finding about endpoints.
t1: action=retry_once second_call=True plan=one_retry reason=transient status 503
t2: action=stop second_call=False plan=skip reason=client or auth status 401
t3: action=retry_once second_call=True plan=one_retry reason=transport error: timeout
t4: action=inspect second_call=False plan=skip reason=rate limit needs a human rule
t5: action=stop second_call=False plan=skip reason=retry budget already spent
t6: action=inspect second_call=False plan=skip reason=empty success body
t7: action=stop second_call=False plan=skip reason=client or auth status 422
checked=7 failed=0
The designed process exit is 0. That number only means the local expectations matched the classifier, and it says nothing about latency, token use, or host capacity.
Guided exercises
Minutes 0 to 10: name the actions

Ask each pair to rewrite the three definitions in their own words without looking at the script. Collect one example of a stop case from ordinary HTTP practice, such as a rejected schema or a missing credential. Do not accept a complaint about model quality as a class, because that sentence does not tell the client what to do next. End this block only when every pair can point to the boolean that would allow a second call.
Minutes 10 to 25: run the deck

Students run the module and compare seven lines with the decision table, including the second-call column. The designed summary is checked equals 7 and failed equals 0, which is a specification rather than a published performance number. If the room has no Python, an instructor can still walk the table row by row and mark the same booleans on paper. That paper path is slower, but it preserves the lesson when install rights are locked down by the lab image.
Minutes 25 to 45: defend stop and inspect

Each student picks the 422 row and writes why a second identical call cannot repair a schema mismatch. Each student also picks the 429 row and writes why an automatic retry could amplify load on a shared free pool. The discussion should stay on client behavior, not on blaming a model for a status code the client already understands. A useful note names the field that carried the evidence, such as status, error, or already_retried.
Minutes 45 to 60: trace the guard and add t8

Students trace plan_next and confirm that skip is the plan for every row except the first 503 and the timeout. They then add a t8 fixture with status 409, an expect action of inspect, and an expect plan of skip. The existing fall-through rule should accept that row without a new branch, which is the point of the unclassified path.
After that row is appended correctly, the designed summary becomes checked equals 8 and failed equals 0. No socket opens in this block, so a mistaken label cannot trigger a live call while the class is still learning. Keep the new fixture in the same file so the next cohort can rerun the extended deck without a separate handout.
{
'id': 't8',
'status': 409,
'error': '',
'body': {'message': 'conflict'},
'already_retried': False,
'expect': 'inspect',
'expect_plan': 'skip',
}
Minutes 60 to 70: write the limitation memo

Each student writes three cases the script must not decide, using complete sentences and naming the missing field. Acceptable cases include a non-idempotent tool call, a 429 that includes Retry-After, and a vendor code outside the teaching set. Unacceptable cases are vague complaints about model quality, cost, or speed, because this checker never saw those signals. Collect the memos before anyone opens a host dashboard, so the product discussion cannot replace the exit check.
Map one synthetic line

Use a synthetic line, not a production export, when you demonstrate the mapping on a projector. A synthetic line with id lab-17, status 503, an empty error, and retried no becomes the same shape as fixture t1. Students can append that mapped object to FIXTURES only after they also write the expected action and expected plan. If they forget the expected fields, the checker should fail closed rather than treat a missing expectation as success.
Where a host with free access fits

MonkeyCode enters the lesson only after the labels are stable, and only as one possible lab host. Disclosure: This article was prepared as part of MonkeyCode's product outreach. This article relies on two availability claims supplied for the draft: free model access and a free server option. This draft does not state quotas, hardware, duration, uptime, or benchmark scores, because those facts were not supplied for verification.
Students should read the host's current documentation before class, since availability claims can change and this page is not that documentation. A free server option may help a classroom share one practice host, but it does not change the failure labels or the one-retry budget. If documentation contradicts this workshop, follow the documentation and keep the local fixtures as the graded artifact.
Export rules if logs come later

If the instructor later exports logs, keep a local id, status, a coarse error tag, and an empty-body flag. Drop raw prompts, completions, authorization headers, and user identifiers before the file reaches any student machine. Map vendor error strings to a small tag set such as timeout, connection_reset, or dns_temporary, and leave unknown strings empty. An unknown string must not become a retry, because the classifier treats an empty error plus an unlisted status as inspect.
Limitations of the teaching contract

The status sets are incomplete, and a 409, 425, or vendor-specific code will fall through to inspect until an instructor adds a reviewed rule. One retry can still be the wrong policy for 503 storms, so the flag is a teaching brake rather than a production backoff schedule. The script does not measure tokens, latency, or queue depth, and it should not be cited as evidence about cost or speed. Empty body detection is naive, because a JSON null, a whitespace string, and a missing key are different defects in a real client.
A 429 with a Retry-After hint might be safe to defer, but these fixtures carry no such hint, so the honest label is inspect. Idempotency is assumed for the one retry, and a non-idempotent tool call should be stop even when the status looks transient. That case is listed in the closing memo because the script cannot see side effects that were not recorded in the row.
Who should not use this lab

Skip the lab if the course outcome is a tuned retry library with jitter, circuit breakers, and idempotency keys. Skip it if students must handle protected data, because the safe path here assumes fixtures that contain no personal content. Skip it if the goal is to compare model quality, since none of the rows include a completion to score. On-call playbooks also need vendor documents and ownership, which this seventy-minute room deliberately does not provide.
Facilitation notes

Instructors should resist turning the closing memo into a tool bake-off, because the room has no measured vendor results to compare. Grade the written reasons, the passing local checker, and the limitation list, not the number of live calls a student managed to send. Pairs are enough; a larger group hides whether each person can name the field that justified stop. If Python is unavailable, the paper walk still counts, provided the booleans match the table exactly.
Closing check

The core result is testable: seven local rows, three actions, and a guard that refuses an unearned second call. Run the module, confirm the designed summary, and only then consider pointing the same field map at a host log. If a lab already uses MonkeyCode's free model access and free server option, classify redacted error rows with these labels first. That pass is a review step, not a claim that the host produced the sample rows in this article.#python #ai #testing #tutorial #software #coding #development #engineering #inclusive #community
Workshop: Classify Endpoint Failures Before a Second Call in 70 Minutes
#usrbinenv #testing #inclusive #engineering #tutorial #development
Dev Community @[email protected]
2026-10-11
A ninety-minute spike ships only on local evidence. A remote draft never decides the ship bit. Mismatch the redacted digest, and the spike dies.
This note records one hypothesis and one clock. The artifact is a small local contract gate. You can rerun it without a network call.
The hypothesis

The hypothesis is narrow and easy to falsify. A free host is usable for drafting only after three local checks pass. Those checks are a secret scan, a digest match, and a timeout cap.
Fail one check and the process exits before any socket opens. The time box is ninety minutes from the note's start line. A pass after that minute still counts as a kill.
Late evidence is not evidence for this spike. One contract file is the only allowed input. One ship-or-kill bit closes the spike cleanly here.
Why the clock is the point

Public threads this week argue about retries and model mistakes. Treat those titles as untrusted topic signals only. They do not prove a gate for your repository.
This spike does not rank models against each other. It does not measure tokens, latency, or uptime. It answers whether your prompt contract is stable.
Split the ninety minutes into three fixed blocks. Use twenty minutes to redact and hash the fixture. Use forty minutes to run the harness and read the report.
Use the last thirty minutes to decide and stop. Do not borrow time from the decision block. A longer draft session means the spike already failed.
Stop writing notes when the clock says kill. A second edit after that minute is out of scope. Start a new note if the question changed.
Ship-or-kill rules

Five rules sit in the spike log before any call. A missing rule is a failed spike, not a partial pass. Print the list, then run the local command.
A secret scan must return zero pattern hits.
The prompt SHA-256 must equal the fixture hash.
The declared timeout must be fifteen seconds or less.
The report JSON must contain decision and digest.
Model text must not overwrite the decision field.
Rule one blocks obvious credentials before a hash. Rule two blocks silent prompt edits after review. Rule three blocks an unbounded wait inside the spike.
Rule four blocks a report you cannot parse later. Rule five blocks a draft that pretends to be authority. All five must pass, or the decision word is kill.
Fixture contract

Keep the checked-in fixture to four fields only. Extra fields would silently become a second hypothesis. Hash the redacted string, not the raw log.
{
"hypothesis_id": "digest-gate-001",
"timeout_seconds": 15,
"redacted_prompt": "Classify this redacted log line: <HOST> returned <CODE>.",
"expected_sha256": "replace-with-local-hash"
}
Replace live host names with stable placeholders first. Replace customer text before you compute the hash. Leave expected_sha256 empty until the command below runs.
Proposed harness

The harness below is a proposed, unexecuted example. It is not a benchmark and not a production client. Run it on a machine you already control.
#!/usr/bin/env python3
"""Local ship-or-kill gate. Network stays closed until this exits 0."""
import hashlib
import json
import re
import sys
from pathlib import Path
PATTERNS = [
re.compile(r"(?i)api[_-]?key\s*[:=]\s*\S+"),
re.compile(r"(?i)bearer\s+[a-z0-9\-._~+/]+=*"),
re.compile(r"(?i)password\s*[:=]\s*\S+"),
re.compile(r"sk-[a-zA-Z0-9]{16,}"),
]
def digest(text: str) -> str:
return hashlib.sha256(text.encode("utf-8")).hexdigest()
def main() -> int:
fixture = json.loads(Path(sys.argv[1]).read_text(encoding="utf-8"))
prompt = fixture["redacted_prompt"]
timeout = int(fixture["timeout_seconds"])
hits = [p.pattern for p in PATTERNS if p.search(prompt)]
actual = digest(prompt)
ok = (
not hits
and timeout <= 15
and actual == fixture["expected_sha256"]
)
report = {
"decision": "ship" if ok else "kill",
"digest": actual,
"secret_hits": hits,
"timeout_seconds": timeout,
"network_opened": False,
}
Path("spike-report.json").write_text(
json.dumps(report, indent=2) + "\n", encoding="utf-8"
)
print(report["decision"], actual)
return 0 if ok else 2
if __name__ == "__main__":
raise SystemExit(main())
Exit code zero means the local contract passed. Exit code two means you stop the spike. Do not catch that code and continue outward.
The report schema is part of the contract. The decision field may be only ship or kill. The network_opened flag must stay false in this file.
Commands to record

Compute the digest before you fill the fixture. Paste that hash into expected_sha256 by hand only. Then run the gate and record the exit.
python3 - <<'PY'
import hashlib
text = "Classify this redacted log line: <HOST> returned <CODE>."
print(hashlib.sha256(text.encode("utf-8")).hexdigest())
PY
python3 spike_gate.py contract.json
echo "exit=$?"
test -s spike-report.json
python3 -m json.tool spike-report.json >/dev/null
Write four lines into the spike note immediately. Record the start time in UTC on line one. Record the process exit code on line two.
Record the digest and the decision word next. If ninety minutes have elapsed, write kill anyway. Do not extend the clock for a cleaner draft.
The time box is part of the hypothesis. A helper who finishes at minute ninety-one still failed. Archive that note as a kill and stop.
Where a free host fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode enters only after the local gate exits zero. The operator describes it as an open-source project.
Operator notes also cite free model access today. The same notes cite a free server option. Those are availability claims, not a sizing sheet.
This draft states no model name and no token quota. This draft states no hardware size and no permanence. Confirm current terms in the official docs first.
Unread terms are a kill, even when the digest matches. Availability language is not a capacity plan here. Use the free server as a draft surface only.
Send the redacted prompt, never the raw log. Store the draft beside the report, not inside it. As of 12 October 2026, re-read the live terms before you rely on them.
A cached screenshot from an older week is not current evidence. Write the terms date beside the four-line clock log. A missing date is the same as unread terms.
After the gate exits zero

Follow the same five steps on every clean run. Skip none of them when the clock is still open. A skipped step turns a ship into a kill.
The local gate writes spike-report.json with network_opened false.
Open a free server session only after exit code zero.
Paste the redacted prompt from the contract file.
Save any draft text as draft-notes.md and nothing else.
Re-run the gate to prove the fixture stayed fixed.
If step two is down, keep the local report. The spike still answered the original hypothesis cleanly. A missing host is data, not a skipped gate.
Failure cases

Three failure cases belong in the design of this gate. Each one maps to a single kill reason. None of them require a remote call to detect.
A placeholder was left in the expected hash field.
A raw log was pasted where the redacted prompt belongs.
A decision word was copied from a model draft.
Failure one means you hashed the wrong string. Failure two means the secret scan should have fired. Failure three means the report schema was ignored.
Fix the fixture, then start a new ninety-minute note. Do not patch the report by hand and call it a ship. Hand-edited decisions are kills in this method too.
Acceptance checks

Run these checks in order on a clean directory. They are the acceptance tests for the harness. They do not require any remote host at all.
A clean placeholder prompt must exit with code zero.
A planted password string must exit with code two.
A timeout of thirty seconds must exit with code two.
A wrong expected hash must exit with code two.
The report file must exist after every single run.
Mark any skipped test as an incomplete spike. Incomplete spikes do not ship, even with a finished draft. Archive the report and stop at the time box.
Compare secret_hits with the planted line you used. An empty hit list on a planted password is a harness bug. Fix the pattern, then restart the ninety-minute clock.
Limits

The scanner is a short deny-list, not a product. A novel token shape can still slip through. Read the prompt by eye before you hash it.
The hash binds bytes, not the sentence meaning. A harmless paraphrase still fails the digest check. That failure is correct for this particular spike.
The timeout field is only a local declaration. It does not prove the remote side will obey. Enforce the same number in any client you add later.
This example never opens a socket on purpose. Add a client only after the report says ship. Until that later point, network_opened must stay false.
Python 3.11 or newer is enough for this file. No third-party package is required to run it. If your image lacks python3, the spike is blocked.
Who should skip this

Skip this spike if you cannot redact the source. Skip it if a named model build must be attested. Skip it if legal review needs a signed benchmark.
Skip it if fixture review needs more than ninety minutes. Skip it when today's free-server terms are unclear. Read the live limit yourself, outside this clock.
Also skip it for live production incident traffic. A live outage is the wrong place to test a digest gate. Use an already redacted sample from a closed ticket.
Evidence to keep

Keep four artifacts next to the hypothesis note. Keep contract.json with the expected hash filled in. Keep spike-report.json produced only by this local harness.
Keep draft-notes.md only if the gate already shipped. Keep a four-line clock log with the decision. None of those files should contain a live credential.
If a secret lands in a file, kill the spike. Delete the file and rotate that secret next. Do not commit the report until the scan is clean.
Close

The conclusion of this spike stays narrow on purpose. A free draft host is optional for the method. A matching digest is mandatory before any paste.
Kill the spike when the contract check fails. Kill it when the ninety-minute clock runs out. Kill it when the current terms were not read.
After a clean gate, a draft surface is optional. Check MonkeyCode's current free-access and free-server notes first. Rerun this harness before you paste any prompt.#testing #python #devops #ai #software #coding #development #engineering #inclusive #community
Ship Only If the Digest Matches
#usrbinenv #testing #inclusive #engineering #development #python
Dev Community @[email protected]
2026-10-11

A preempted stream is not a free retry. If the lane already emitted tokens, those tokens are spent, and starting the prompt over buys a prefix you already hold.
You feel the urge anyway. The socket drops, the free slot vanishes, and the editor still shows an unfinished function. The cheap instinct is to send the same messages again and hope the next slot is kinder. That instinct turns a short outage into a second bill for the same answer.
Think of a courier who already walked the parcel to your door, then got pulled off the route. You do not hire a second courier to repeat the walk from the warehouse. You keep what arrived and pay only for the last flight of stairs.
A model stream has that shape, with one wrinkle. Input context may have to be shown again, because a provider often will not hand you a receipt for cached prompt tokens. Output you already received must not be generated again. Those are different costs, and mixing them up is how the bill grows while the answer stays the same.
This note is a small ops habit for that split. It is a proposal, not a measured result, and the sample is unexecuted example code. It does not assume a model name, a quota, a machine size, or a promise that a free lane will exist next week.
Before the call, write a job row. Give it an id, a hash of the prompt, a token budget, and a cursor at zero. The hash is how you refuse to resume against a prompt you have since edited. If the hash does not match, the cursor belongs to a different question, and stitching it on would answer the wrong ask.
While tokens arrive, flush the cursor on a short interval, not only at the end. A flush at the end is a diary of success. A flush every few dozen tokens is a diary of the failure you actually have.
Store the assistant text, the count you treat as spent, and a monotonic revision. If the process dies between flushes, you lose only the gap. That gap is the acceptable loss. The prefix before it is not.
On the way back in, do not rebuild the original request and ask for a fresh completion. Build a continuation, and cap output at budget minus spent. If the API allows an assistant prefill, put the stored prefix there and ask it to continue without repeating itself.
If the API will not accept a prefill, send a short instruction plus the unfinished tail, not the entire original thread plus a demand for a new essay. If the remainder is below a floor you choose, abandon. A continuation that may spend the original budget again is just a replay wearing a new function name.
Here is the gate in plain Python. Save it as budget_cursor.py. Treat every number as a knob, not a measurement. The annotations assume a current Python 3.
import hashlib, json
from pathlib import Path
ROOT = Path(".jobstate")
def prompt_hash(messages: list[dict]) -> str:
blob = json.dumps(messages, sort_keys=True, ensure_ascii=False)
return hashlib.sha256(blob.encode()).hexdigest()
def open_job(job_id: str, messages: list[dict], budget_tokens: int) -> dict:
ROOT.mkdir(exist_ok=True)
path = ROOT / f"{job_id}.json"
if path.exists():
job = json.loads(path.read_text())
if job["prompt_hash"] != prompt_hash(messages):
raise SystemExit("prompt changed; refuse stale cursor")
return job
job = {
"job_id": job_id,
"prompt_hash": prompt_hash(messages),
"budget_tokens": budget_tokens,
"spent_tokens": 0,
"prefix": "",
"revision": 0,
"state": "open",
}
path.write_text(json.dumps(job))
return job
def flush(job: dict, piece: str, piece_tokens: int) -> dict:
job["prefix"] += piece
job["spent_tokens"] += piece_tokens
job["revision"] += 1
if job["spent_tokens"] > job["budget_tokens"]:
job["state"] = "over_budget"
path = ROOT / f"{job['job_id']}.json"
tmp = path.with_suffix(".json.tmp")
tmp.write_text(json.dumps(job))
tmp.replace(path)
return job
def continuation(job: dict, floor: int = 32) -> dict | None:
remain = job["budget_tokens"] - job["spent_tokens"]
if job["state"] != "open" or remain < floor or not job["prefix"]:
return None
return {
"assistant_prefill": job["prefix"],
"max_output_tokens": remain,
"instruction": "Continue from the prefill. Do not repeat it.",
}
The temp-file rename avoids a half-written JSON line if the process dies mid-flush on the same filesystem. It is not a distributed lock. Two workers sharing one job id will still clobber the cursor, so give each in-flight call its own id.
You can exercise the refusal path with no network and no key. The 12 below is a stand-in, not a tokenizer result. In a real call you store the usage figure for that chunk, not a guess from string length.
mkdir -p .jobstate
python - <<'PY'
from budget_cursor import open_job, flush, continuation
msgs = [{"role": "user", "content": "draft a migration note"}]
job = open_job("mig-14", msgs, budget_tokens=400)
flush(job, "Step 1. Freeze writes.\n", 12)
print(continuation(job))
print("remain", job["budget_tokens"] - job["spent_tokens"])
PY
If continuation returns None, you stop. You do not send the original call once more just in case. None means the prefix already consumed the useful budget, or there is no prefix worth resuming. A full replay would ignore the ending you just computed.
Watch the file if you want to see the cursor move under a fake stream. This loop is still local. It proves the flush, not a provider.
python - <<'PY'
import json
from budget_cursor import open_job, flush
job = open_job("mig-14", [{"role": "user", "content": "draft a migration note"}], 400)
for piece, n in [("Step 2. Copy rows.\n", 10), ("Step 3. Verify counts.\n", 11)]:
flush(job, piece, n)
print(json.dumps(job, indent=2))
PY
You have three exits, and you should pick one before the call, not during the panic. Abandon when the remainder is under the floor or the prefix is empty. Resume when the prefix is real and the remainder still fits the budget.
Spill to a lane you are willing to wait on only when the job cannot be dropped and the free slot has already failed. Spilling is not a loop. One spill, then a human, or a queue that can actually reserve the work. The reservation problem is older than this cursor, and this note does not solve it.
A free lane is a decent stage for that rehearsal, and a bad home for work you cannot drop. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The useful fit is narrow: MonkeyCode's free model access and a free server option can host a broken-stream drill, after you read the live limits in the product you have, and only for jobs you can abandon. This note does not record a quota, a card, a region, or a duration, because those figures move and no primary source for them was attached.
Free capacity is the wrong bet when the remainder is not optional. A migration note that can be rewritten later may die at a cursor. A payment capture, a mailbox send, an infrastructure apply, or a customer-visible webhook may not. Those need an idempotency key, and the side effect should run after the text is finished, not inside the stream. If a partial answer is worse than silence, do not emit it onto a lane that can vanish mid-sentence.
There is a quieter wrong bet than preemption: input replay. If the provider bills the prompt again on every continuation, a huge system prompt will swallow the savings from not regenerating the prefix. Log input tokens and output tokens separately on the resume call. If resumed input is nearly the original prompt plus the stored prefix, you moved the cost, you did not remove it.
Shrink the resumed context to the tail you still need, or stop using continuation for that prompt shape. The cursor still earns its keep when it blocks a second full completion. It will not make a bloated prompt cheap, and it will not refund tokens a vendor has already metered.
Skip this habit if you cannot persist the prefix outside the process. A cursor that lives only in memory dies with the preempt and leaves the replay you were trying to avoid. Skip it when the client hides token counts and you would be guessing piece_tokens. A guessed cursor is a lie in the ledger.
Skip it when the attempt must be byte-identical to the first sample. Continuation is a new draw, not a restored sampler, unless the provider documents otherwise. Do not claim determinism you did not buy. And skip it for any call that mutates the world before the stream ends, because a resume would be a second chance to mutate it.
The habit is small enough to keep. Open a job, flush while text arrives, and on death ask only for the remainder or ask for nothing. A free slot can host the drill. It cannot refund a prefix you already spent, and it cannot be the commitment under work you are unwilling to lose.#ai #devops #python #tutorial #software #coding #development #engineering #inclusive #community
Write the Prefix Down Before the Slot Dies

#inclusive #engineering #tutorial #development #python #coding
Dev Community @[email protected]
2026-10-11
You should reject an agent-written patch before it touches your tree, unless a written allowlist and a named test command both pass. Free generation makes extra files cheap to propose, while a wrong file remains expensive to undo. This case study walks you through a tiny Python gate you can run on any disposable workspace. The gate is a review tool, not a claim that any model or host is safe.
What you are protecting

You are protecting a three-file health service, not a production platform and not a secret-bearing repository. The service exposes /health, and you want a second route named /ready with one focused test. An agent can draft that change quickly, yet it can also edit deploy files, lockfiles, or a hidden env sample. Your job is to keep the applied diff inside a card you wrote before the session started.
Background

Picture a small folder that already contains app.py, test_app.py, and README.md, with no database and no cloud credentials. You ask an assistant to add readiness behavior, and you expect a unified diff plus a short report. The report must name the exact test command you will run after the patch applies cleanly. If either artifact drifts outside the card, you stop and you do not apply the patch.
An agent draft is still an untrusted patch, even when the prose around the diff sounds finished and careful. You need a local rule that does not depend on how confident the report appears in the chat window. This exercise does not rank models, and it does not borrow a scoreboard from a public thread. You are freezing one small contract that names allowed paths, forbidden paths, and a test command.
Goal

The goal is a fail-closed checker that exits nonzero unless three conditions hold at the same time. The diff may touch only paths on the allowlist, and it must not touch any forbidden path. The agent report must contain the exact test command that you already wrote on the card. You treat missing fields as failure, because a silent default is how an unreviewed file lands in the tree.
The session card

You write the card before you prompt, and you store it beside the checker rather than inside the agent's scratch space. You keep the card in the repository you control, so a later edit to the prompt cannot quietly widen the allowlist. You treat the byte cap as your own courtesy limit, not as a limit advertised by any host. You can change the goal line, but you should not add deploy files just because a draft looks tidy.
# session_card.yaml - proposal, not a production policy
goal: Add GET /ready that returns 200 and a ready status
allowed_paths:
- app.py
- test_app.py
forbidden_paths:
- .env
- .env.example
- credentials.json
- deploy/
- .git/
required_test_command: python -m unittest test_app.py
max_diff_bytes: 12000
Implementation

The checker below is a labeled, unexecuted example that you should read carefully before you run it. It uses only the Python standard library, so you do not add a dependency just to review a patch. It reads the card, the diff, and the report, then prints one JSON line for a completed check. You should treat a parser error or a bad argument count as a failed review before any patch is applied.
#!/usr/bin/env python3
"""session_gate.py - unexecuted example. Fail closed on an agent diff."""
import json
import sys
from pathlib import Path
def load_card(path: Path) -> dict:
card = {"allowed_paths": [], "forbidden_paths": []}
key = None
for raw in path.read_text(encoding="utf-8").splitlines():
line = raw.split("#", 1)[0].rstrip()
if not line.strip():
continue
if line.startswith(" - "):
if key is None:
raise SystemExit("list item before a key")
card.setdefault(key, []).append(line.strip()[2:].strip())
continue
if ":" not in line:
raise SystemExit("card line missing colon")
key, value = line.split(":", 1)
key, value = key.strip(), value.strip()
if value:
card[key] = int(value) if value.isdigit() else value
else:
card.setdefault(key, [])
required = ("goal", "required_test_command", "max_diff_bytes")
missing = [name for name in required if name not in card]
if missing or not card["allowed_paths"]:
raise SystemExit(f"incomplete card: {missing or 'allowed_paths'}")
return card
def changed_paths(diff_text: str) -> list[str]:
paths =
[] for line in diff_text.splitlines():
if line.startswith("+++ b/"):
path = line[6:].strip()
if path and path != "/dev/null":
paths.append(path)
return paths
def main() -> int:
if len(sys.argv) != 4:
print("usage: session_gate.py CARD DIFF REPORT", file=sys.stderr)
return 2
card = load_card(Path(sys.argv[1]))
diff_text = Path(sys.argv[2]).read_text(encoding="utf-8")
report = Path(sys.argv[3]).read_text(encoding="utf-8")
reasons =
[] if len(diff_text.encode("utf-8")) > int(card["max_diff_bytes"]):
reasons.append("diff exceeds max_diff_bytes")
allowed = set(card["allowed_paths"])
forbidden = tuple(card["forbidden_paths"])
for path in changed_paths(diff_text):
if path in forbidden or any(path.startswith(prefix) for prefix in forbidden):
reasons.append(f"forbidden path: {path}")
elif path not in allowed:
reasons.append(f"path outside allowlist: {path}")
command = str(card["required_test_command"])
report_lines = [line.strip() for line in report.splitlines()]
if command not in report_lines:
reasons.append("report missing required_test_command")
if "+++ " not in diff_text:
reasons.append("diff has no file headers")
result = {"ok": not reasons, "reasons": reasons, "command": command}
print(json.dumps(result, ensure_ascii=True))
return 0 if result["ok"] else 1
if __name__ == "__main__":
raise SystemExit(main())
Commands you run

You create two fixtures, one clean and one dirty, then you run the gate against each fixture. These commands are the reproducible artifact, and they do not call a network or a model. You keep the fixtures in a local fixtures directory, and you do not fetch them from an untrusted gist. You run each command in a clean shell so a previous exit code cannot confuse the result you record.
Local fixtures

Save this as fixtures/ready_ok.diff. It is a proposal hunk, not a patch this article applied to a live service.
--- a/app.py
+++ b/app.py
@@ -1,2 +1,5 @@
def health():
return {"status": "ok"}
+
+def ready():
+ return {"status": "ready"}
--- a/test_app.py
+++ b/test_app.py
@@ -1,2 +1,8 @@
import unittest
+from app import ready
+
+
+class ReadyTest(unittest.TestCase):
+ def test_ready(self):
+ self.assertEqual(ready()["status"], "ready")
Save fixtures/ready_ok.md as one exact line, with no extra words on that line.
python -m unittest test_app.py
Save fixtures/ready_bad.diff as a forbidden-path example. The token below is a placeholder, not a credential you should reuse anywhere.
--- a/.env
+++ b/.env
@@ -0,0 +1 @@
+API_TOKEN=example-not-real
A clean diff should mention only app.py and test_app.py, and the matching report should repeat python -m unittest test_app.py exactly. A dirty diff should add .env or deploy/app.yaml, or the matching report should omit the required command. You apply the patch only after the clean run prints "ok": true and you have read the diff yourself. You discard the dirty fixture even if the prose around it sounds confident, because the gate exists to ignore tone.
python3 session_gate.py session_card.yaml fixtures/ready_ok.diff fixtures/ready_ok.md
python3 session_gate.py session_card.yaml fixtures/ready_bad.diff fixtures/ready_bad.md
Apply only after a clean gate

patch -p1 --dry-run < fixtures/ready_ok.diff
patch -p1 < fixtures/ready_ok.diff
python -m unittest test_app.py
You use --dry-run first so a malformed hunk cannot rewrite a file before you have inspected the reject list. You run the unittest only after a real apply in that same workspace, not in a different checkout with different code. You stop immediately if the dry run, the apply, or the unittest fails, and you do not let the agent edit in place. You save the JSON line next to the diff so a later reviewer can see why you applied or refused the change.
Expected results
Specified exits

This section describes the specification of the example, not a measured production result and not a benchmark. A clean fixture should exit 0 and print a JSON object whose ok field is true. A dirty fixture should exit 1 and list either a forbidden path or a missing command. An incomplete card should exit through SystemExit before any diff is judged, because a missing allowlist must not become allow everything.
Decision table
FixtureWhat changedReportExpectedcleanapp.py, test_app.pyexact command lineexit 0forbiddenadds .envexact command lineexit 1silent reportallowed files onlycommand omittedexit 1broken cardany diffany reportabort before judgment

You can treat the table as the test plan, and you should not claim a pass until your machine prints the same exits. A row that aborts is still a failure, even when the diff itself would have been small and boring. You record the JSON beside the fixture name, so a later run can be compared without trusting memory. You should rerun the table after any parser edit, because a green memory of yesterday is not evidence about today's script.
Extra cases worth trying

You should also try an empty diff, which this checker rejects because it finds no file headers at all. A delete hunk that uses a null header can slip past this scan, so you should treat that pass as unfinished. If that null-header case passes, you revise the path parser before any agent draft is allowed near the tree. No latency number, token count, or pass rate is claimed here, because this example was not executed against a live service for this article.
Where free model access fits

Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode is relevant only as the place you might draft the patch and run the gate, if you choose that tool. The operator-supplied notes for this draft say that free model access and a free server option exist. This article does not name models, quotas, hardware, duration, or permanence, because those details were not verified against a primary source dated 2026-10-12. You should read the current product docs before you rely on any limit, and you should assume a free workspace can disappear or reset.
A split you can actually run

A practical split keeps the product mention from doing the review work that you still owe the tree. You may use free model access to propose app.py and test_app.py only, with the card pasted into the prompt as a constraint. You may use a free server as a disposable workspace to run session_gate.py and python -m unittest, away from your laptop secrets. You still copy the diff back by hand, and you still refuse the patch when the JSON says ok is false.
Confirm the docs first

If you want to try that split, start from the current MonkeyCode docs and confirm what the free options include today. Run the two fixture commands before you send a real repository, and stop if the docs disagree with this draft. That confirmation step is the only invitation this case study makes, and it is not a promise about price or capacity. You can ignore the product entirely and still keep the card, the checker, and the fail-closed rule.
Limitations

You should read these limits before you reuse the script, because a teaching checker hides several sharp edges. None of these limits is a reason to skip the gate, but each one is a reason not to overclaim what it proves. You can replace the parser later, and you should do that before anyone treats the script as team policy.
The tiny card parser is not a YAML library, and nested values or quoted colons will break it.
Path checks use the +++ b/ header only, so a clever or malformed diff can hide a rename.
A matching test command in the report does not prove the test passed or even exists on disk.
The byte cap is a local courtesy limit you chose, not a quota published by any vendor or host.
The gate cannot see side effects that an agent already performed on a server before you downloaded the diff.
Forbidden prefixes are literal strings, so a prefix such as deploy/ does not cover every obfuscated path.
The patch command is a local Unix assumption, so a Windows shell may need an equivalent apply tool you already trust.

Who should not use this approach

You should not use this gate as a security boundary for a hostile model, a malware sample, or a secret-filled repository. You should not point a free server at production credentials, private customer data, or a remote you cannot revoke. You should not skip the dry-run patch and the unittest just because the JSON line looks calm and short. Teams that need signed reviews, policy-as-code, or a real diff parser should use those systems instead of this teaching script.
Lessons

Write the allowlist before the prompt, because an agent cannot honestly invent your risk boundary after the fact.
Require the test command as an exact line, so a vague sentence about tests cannot pass the gate alone.
Fail closed on a missing card field, since a default of allow-all is the bug you are trying to remove.
Keep vendor claims out of the result line, and record only the exit code and JSON your own command printed.
Treat a free workspace as disposable, and never store long-lived keys there even for a short teaching demo.
You leave this exercise with a card, a checker, two fixture commands, and a rule you can explain to a reviewer in one minute. The rule is simple enough to repeat, and it says that no allowlist match and no named test means no applied patch. That habit stays useful even if you later replace the script with a stricter diff tool on your team. You can rerun the same fixtures next week, and any JSON change then belongs to your checker rather than a headline.#python #testing #ai #tutorial #software #coding #development #engineering #inclusive #community
Case Study: An Allowlist Gate That Rejects an Agent Patch Before You Apply It
#usrbinenv #testing #inclusive #engineering #tutorial #development
Dev Community @[email protected]
2026-10-11
The right agent seat is the one you can close without losing the only copy of the work. I would rather take a shorter paid lease, or run a box I operate, than keep a free chat I cannot audit. A seat is a temporary grant of model access, context, and side effects, not a teammate you quietly adopt for the week. If you cannot name the exit evidence before the first prompt, you have not chosen a tool yet.
Recent discussion keeps returning to a plain tension, even when the headlines change from week to week. Model quality can move in a week, while the release path does not automatically move with it. I am not treating any weekly roundup as evidence, because a trending title is only a topic signal. A topic signal is not a measurement I can cite, and it is not a reason to skip the lease card.
Start from the exit, not the invoice

Have you ever left a session open because closing it felt like throwing the reasoning away? That small hesitation is how a free hour becomes an unofficial path into the release branch. I now write the ending before I write the prompt, and I keep that ending in the repository rather than in a vendor tab. The invoice matters only after the ending is a file that a reviewer can open without me.
I have already used other gates for blast radius and review time, so this note does not rebuild those checks. The question here is narrower, and it is easy to dodge when the model feels helpful. Which seat can I actually end today, and which committed file proves that I ended it?
Ask four questions before the session exists

I use the same four questions for a bugfix, a documentation edit, and a spike that might be deleted. Each answer is a word or a small integer, because a paragraph of enthusiasm is not a decision. If any answer is mushy, I shrink the task until the answer fits on one line.
What data class may enter the prompt? I allow a public sample, internal code that contains no secrets, or a resident class that must stay on hardware I control. When the class is unclear, I stop, and I do not try a redacted version inside the same unbounded chat.
How many retries is the review time worth? I write an integer, usually one to three, and I stop when the counter reaches zero. Another retry without a new failing test is just a slower guess, even when the seat did not charge me.
Which artifact leaves with me? I want a diff, a failing test name, or a rejected hypothesis written on the lease card. A transcript that exists only in a browser tab is not an artifact I can hand to someone else.
Who can say no before the merge? If the honest answer is nobody, the seat is wrong for that change, including the seat that costs nothing. A free run can still ship a confident mistake, and the price of the mistake is not on the pricing page.
Would you still sign the diff if the model name were missing from the commit message? If the answer is no, the seat did not finish the job, and another hour will not finish it either.
Keep a lease card the branch can diff

I store the card beside the task notes, and I commit it when the choice itself needs a trail. The script below is a proposal, not a benchmark I executed for this article, so the printed labels are decisions rather than measurements. It refuses a recommendation when the exit fields are empty, which is the behavior I want on a tired afternoon.
#!/usr/bin/env python3
"""Proposal: label a one-task agent lease. Not a measured benchmark."""
from __future__ import annotations
import json
import sys
from dataclasses import dataclass
@dataclass(frozen=True)
class Lease:
data_class: str # public | internal | resident
retries: int
artifact: str # diff | test_name | hypothesis
reviewer: str # self | teammate | none
needs_long_context: bool
can_operate_a_server: bool
def choose(lease: Lease) -> str:
if lease.reviewer == "none" or not lease.artifact:
return "stop: no exit evidence"
if lease.data_class == "resident":
if lease.can_operate_a_server:
return "self-hosted: you accepted the restart duty"
return "stop: residency without an operator"
if lease.needs_long_context or lease.retries > 3:
return "paid: continuity is the product you are buying"
if lease.data_class == "public" and lease.retries <= 2:
return "free: short lease, keep the artifact, drop the transcript"
if lease.data_class == "internal" and lease.reviewer == "teammate":
return "paid or free-with-policy: only if current terms allow this data class"
return "stop: shrink the task or rewrite the card"
def main() -> int:
raw = json.load(sys.stdin)
lease = Lease(
data_class=raw["data_class"],
retries=int(raw["retries"]),
artifact=raw.get("artifact", ""),
reviewer=raw.get("reviewer", "none"),
needs_long_context=bool(raw.get("needs_long_context", False)),
can_operate_a_server=bool(raw.get("can_operate_a_server", False)),
)
print(choose(lease))
return 0
if __name__ == "__main__":
raise SystemExit(main())
A public fixture card can live in the issue, and it should be boring enough to review in a minute. I keep secrets out of the sample on purpose, because a teaching card that leaks a token is already a failed lease. If the real task cannot be reduced to this shape, the free row is the wrong row. The JSON is an example of the input, not a log from a session I ran.
{
"data_class": "public",
"retries": 2,
"artifact": "diff",
"reviewer": "self",
"needs_long_context": false,
"can_operate_a_server": false
}
Run the label before the first prompt

I hash the prompt and the card together, so a later edit cannot pretend it was the authorized task. These commands stay on my own machine, and they do not call a model provider at all. If the hash changes after I start, I treat that as a new lease rather than as a continuation.
python3 lease_seat.py < lease.json
sha256sum prompt.md lease.json | tee lease.sha256
git diff --stat
If the script prints stop, I do not open a session to see what the model might say. Curiosity without an artifact is how the retry counter becomes a suggestion instead of a rule. If the script prints free, I still copy the diff into the branch, and I keep whatever the card promised to keep. Does a larger context window change the hash I already wrote down beside the lease card?
The hash does not move when the window grows, and that mismatch is the useful part. The hash records the task I authorized, not the confidence the model expressed after it saw more files. I would rather fail the check in ten seconds than discover the scope creep during review. Have you watched a session helpfully open files that you never placed on the lease card?
Work the lease in five steps

I walk these five steps in order, and I do not skip to the model because the card feels obvious. The script is only a reminder, not a boss, but I still run it when I am tempted to improvise. Have you noticed that the improvised path is usually the one that touches one extra file?
I write the four answers on the card before I open any model tab, including a tab that is free.
I run the proposal script against that card, and I treat a printed stop line as a finished decision.
I hash the prompt beside the card, and I start only when the label matches the data class.
I copy the artifact into the branch, and I stop when the retry integer on the card hits zero.
I add the closing line under the card, and I do not reopen the session to polish a rejected hypothesis.

Read the three seats as fit criteria

Free model access fits a short public or synthetic task when the artifact is a diff or a named test and the retry count stays small. I do not put secrets, production credentials, or an irreversible migration on that seat, even for a minute. What is missing is not a slogan about quality, but a retention and uptime promise I have checked on the current terms page.
Paid access fits when I am buying continuity, a team audit trail, or a support path I can actually open. The same card still applies, because a larger invoice does not invent a reviewer who was absent at the start. If the current terms forbid my data class, the paid column is a stop, not a quieter upgrade. Why would I pay to repeat a data-class mistake I could have caught on paper?
Self-hosting fits when the data class is resident, or when prompts and logs must stay on hardware I operate. That fit includes patches, disk, and the hour when the process dies while I am offline. If I cannot name the person who restarts it, I am choosing a hope with extra steps, not self-hosting. The box can be cheaper on paper and still be the most expensive seat if nobody owns the restart.
I keep these tradeoffs in one table so the argument does not hide inside a paragraph. The cells are fit notes, not scores, and they assume you already answered the four questions. A blank cell means I would stop, rather than pretend that the seat is secretly fine. Can you place your real task into one row without quietly inventing a reviewer for it?
SignalFree seatPaid seatSelf-hostedData classPublic or synthetic sampleInternal code only if current terms allow itResident data on hardware you operateContinuityOne short session, then exitMulti-step context you are willing to pay forAs long as you can keep the process healthyExit evidenceDiff or test name you copy outSame evidence, plus a log you are allowed to exportLogs on disk you already controlRetry budgetOne or two, then stopThe integer on the card, not an open tabThe same integer, paid in your own timeSkip whenSecrets, migrations, or no reviewerTerms conflict, or nobody can say noNo named operator for patches and restarts
Place a free server only where the card already allowed it

Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode shows up in this method only as a candidate for the free row on the card. For this draft I am relying on two availability claims only: free model access, and a free server option. I am not stating a token quota, a hardware shape, a duration, or a comparison score. Those details were not verified for this draft, and product figures like that go stale quickly.
Before I rely on either claim, I read the current product terms and confirm that my data class is permitted. If the terms are silent or confusing, I leave the card on stop and edit in a local tool I already trust. I would not move a hosted seat into the resident row unless I control the deployment and have checked that path myself. A free server is still a lease with an owner, a log policy, and an end time you should assume can change.
If you want to rehearse the exit on a throwaway public fixture, a free model seat is enough for that drill. Save the diff, discard the transcript, and ignore the seat when the script told you to stop. I would not use that rehearsal as proof that the seat is safe for internal code. The rehearsal only proves that you can end a session you were willing to throw away.
Who should leave this card on the shelf

Skip this approach when you need a formal threat model, a compliance signature, or a measured comparison of model quality. The script does not score accuracy, latency, or money, and I did not execute it against a live provider while writing this note. Skip it when the change touches credentials, customer data, or a migration you cannot roll back before the day ends. Also skip it if you came for a vendor ranking, because I am not publishing one.
The limitation I watch most carefully is the false comfort that comes from a tidy label. A printed free line can still hide a sloppy prompt, a flaky test, or a terms page I skimmed too quickly. The card governs my behavior, and it is not a certificate that the model was right. If the label and the diff disagree, which one are you actually planning to trust in review?
Close the lease before the day ends

I end the session when the artifact is on the branch, or when the retry counter hits zero, whichever arrives first. Then I add one line under the card saying I kept the diff, or I rejected the hypothesis. That closing line remains useful even if every product name is later deleted from the page. The seat you can end cleanly is the only seat you were really allowed to start.#ai #devops #tutorial #python #software #coding #development #engineering #inclusive #community
Pick a Free, Paid, or Self-Hosted Agent Seat by the Lease You Can End
#usrbinenv #inclusive #engineering #tutorial #development #python
Dev Community @[email protected]
2026-10-11

The Friday patch looked small, tidy, and safe. An agent renamed a retry helper without touching public signatures. The first CI run returned a clean green badge.
The second run failed one idempotence property on identical inputs. The third run passed that same property again without a code change. The merge still landed before the Friday lunch break.
Monday's ledger showed duplicate charges on one customer account. The badge had treated unstable noise as proof of safety. This opening is a composite incident, not a measured outage recorded by this account.
A nearby testing question is whether a retry should count as proof. Better model drafts do not repair an oracle that flips on identical inputs. The gate has to separate those two failures before anyone merges.
Who may vote

Treat every automated check as a voter with one ballot. A property may block merge after two identical failures. A property may allow merge only after a stable pass.
A frozen flake may not cast either kind of vote. Its old failures stay filed as history, not as a fresh block. Its old passes also stay silent until a human repairs the oracle.
Example tests still help when you debug a single path. They do not certify behavior across generated inputs and boundaries. That split stays useful when an agent patch is wide.
Pin fixture bytes before you trust any oracle result. Hash the fixture pack and store the digest beside the patch. Commit that digest with the fixtures, outside the agent's unreviewed diff.
If that digest drifts, stop the run and refuse every vote. A moving fixture can counterfeit stability across two repeats. Drift is a stop condition, not a warning to click past.
Do not reuse a flake lease as this control. A lease expires and then invites another retry. A freeze removes the voter until a human repairs the oracle.
Those controls fail in different ways and should stay separate. Write the choice down so the next reviewer does not invent a third rule. The scorecard below is the written form of that choice.
Run the scorecard

The script is an unexecuted proposal, not a production result from this account. Copy it into a scratch repository and point it at your fixtures. Expect to replace the property names with invariants from your own service.
# proposal: unexecuted property gate
import hashlib, json, pathlib, subprocess, sys
ROOT = pathlib.Path(__file__).resolve().parent
FIX = ROOT / 'fixtures'
FREEZE = ROOT / 'freeze_set.json'
PROPS = ['idempotent_write', 'bounded_retry', 'no_secret_in_log']
def digest(folder: pathlib.Path) -> str:
h = hashlib.sha256()
for p in sorted(folder.rglob('*')):
if p.is_file():
h.update(p.relative_to(folder).as_posix().encode())
h.update(p.read_bytes())
return h.hexdigest()
def load_freeze() -> dict:
if not FREEZE.exists():
return {'frozen': []}
return json.loads(FREEZE.read_text())
def run_once(prop: str) -> int:
cmd = [sys.executable, '-m', 'pytest', f'properties/test_{prop}.py', '-q']
return subprocess.run(cmd, cwd=ROOT).returncode
def vote(prop: str, runs: int = 2) -> str:
frozen = set(load_freeze()['frozen'])
if prop in frozen:
return 'frozen'
codes = [run_once(prop) for _ in range(runs)]
if any(c != codes[0] for c in codes):
return 'freeze'
return 'pass' if codes[0] == 0 else 'fail'
def main() -> int:
expected = (ROOT / 'fixture_digest.txt').read_text().strip()
if digest(FIX) != expected:
print('fixture drift; no vote')
return 2
ballots = {p: vote(p) for p in PROPS}
freeze = load_freeze()
for name, ballot in ballots.items():
if ballot == 'freeze' and name not in freeze['frozen']:
freeze['frozen'].append(name)
FREEZE.write_text(json.dumps(freeze, indent=2) + '\n')
print(json.dumps(ballots, indent=2))
if any(v == 'fail' for v in ballots.values()):
return 1
if any(v in {'freeze', 'frozen'} for v in ballots.values()):
print('frozen voters abstain; human repair required')
return 3
return 0
if __name__ == '__main__':
raise SystemExit(main())
Pair the gate with one property that states a numeric cap. The example below is a proposal and has not been executed here. Swap the billing import before you treat a failure as real.
# proposal: unexecuted property, not a measured result
# import billing # replace with the module under test
def test_bounded_retry(monkeypatch):
calls =
[] def send():
calls.append(1)
raise TimeoutError('synthetic')
monkeypatch.setattr('billing.transport.send', send)
try:
billing.charge(amount=10, retries=3)
except TimeoutError:
pass
assert len(calls) == 3
Two repeats are a starting probe, not a statistical certificate. Raise the repeat count when the property touches clocks, networks, or files. Record that count in the scorecard so reviewers can see the budget.
A stable failure blocks the merge with exit code one. A mixed result freezes that property and writes its name. A frozen name abstains on every later run of the gate.
Abstention is not a pass and must not hide inside zero. The script returns three when any voter is frozen or newly frozen. CI should fail closed until that name is repaired in review.
Pin the digest before the agent starts editing the tree. Run the hasher once on a clean fixture directory. Store the printed line in fixture_digest.txt and commit it with the pack.
python - <<'PY'
import hashlib, pathlib
root = pathlib.Path('fixtures')
h = hashlib.sha256()
for p in sorted(root.rglob('*')):
if p.is_file():
h.update(p.relative_to(root).as_posix().encode())
h.update(p.read_bytes())
print(h.hexdigest())
PY
Then invoke the gate on the agent patch and print the exit status. Let the script own the repeat count on ordinary patch runs. Run an extra manual pass only when you are testing the gate itself.
python property_gate.py
echo exit:$?
The process speaks in four codes, like a traffic light with a closed lane. Code two means the fixture bytes moved, so nobody votes. Code one means a stable property failed, so the merge stops.
Code three means a voter abstained and still needs a human owner. Code zero means every voting property passed with no abstention left. Wire CI to accept only zero if the badge must mean clean.
Inspect the freeze file after a noisy night of reruns. Each name in that file needs a human owner and a repair note. Do not delete a name merely to paint the CI badge green.
python -m json.tool freeze_set.json
The script has no waiver flag and no hidden retry button. Removal of a frozen name belongs in a reviewed repair commit. That commit should show new stable runs beside the removal.
How the patch enters

Give the agent a sealed task, not a blank repository. State the invariants that must remain true after the edit. Forbid edits under properties/ and fixtures/ inside the same patch.
A patch that rewrites its own judge is not evidence of safety. If the agent touched a property, stop and split the change. Land the oracle fix with human tests before the behavior patch faces this gate.
A coding assistant can draft text without becoming a voter. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The operator describes MonkeyCode as an open-source assistant with free model access.
A free server option is the other availability claim supplied for this draft. This article does not state quotas, model names, hardware, or duration. It also states no benchmark numbers and no promise that the offer remains.
Use free model access as a drafter, not as a judge of the patch. Paste a stable failure log and ask for three hypotheses only. Pin any new property yourself before it joins the voter list.
The free server option can host the repeat probe away from a laptop. If that server is unavailable, the local scorecard still stands alone. Availability is a convenience claim, not a correctness proof.
Read a stable miss

A stable failure is a product signal, not an inconvenience to rerun away. Open the property file and name the broken invariant in one sentence. Compare the agent diff with that sentence before you blame the test.
Many agent patches pass narrow examples and still break a boundary. Suppose bounded_retry fails twice on the same fixture pack. The helper now retries four times instead of the pinned cap of three.
The example test used a stub that stopped early and hid the cap. The property called the real limit and failed on both repeats. Keep the property, reject the patch, and request a smaller diff.
Suppose that same property flips between pass and fail. Freeze it and stop counting it as a voter. The cap may be fine while the clock inside the oracle is not.
A sleep inside the oracle is a common source of mixed ballots. Replace the sleep with a fake clock and rerun the same probe. Unfreeze the name only after identical inputs agree twice.
Who should skip it

Skip the gate when you have no pinned fixtures to hash. A property without fixed inputs is a story, not a vote. Skip it when every check needs a live vendor network.
Jitter will fill the freeze file and hide real regressions underneath. Skip it for prototypes that will be deleted within the week. The ceremony then costs more than the patch itself is worth.
Teams with a mature mutation suite should not treat this gate as a replacement. It does not replace mutation work or production monitoring either. It answers whether this patch broke a named invariant or the oracle wobbled.
Limits to keep visible

Two repeats will miss rare flakes that appear only weekly. A freeze file can grow until almost nothing still votes. Review that file each week and assign a named owner.
A green exit with many abstentions is a warning, not a release certificate. Hashing fixtures will not catch a property that encodes the same bug. If the agent and the oracle share a wrong assumption, both runs pass.
Human review of every new property remains mandatory before it votes. Generated cases also need a seed logged beside the ballot. Without that seed, you cannot replay the failure on demand.
Install pytest before you trust any ballot from this script. A missing runner looks like a stable failure, not like noise. Fix the environment first, then rerun the same pinned fixtures.
Free model drafts can look precise and still be wrong. Free server runs can differ from production clocks and filesystems. Treat both as optional tools around the scorecard, not above it.
Need a second draft for one frozen property name. MonkeyCode's free model access can propose wording, not a ballot. Keep the vote on your tree, and show stable runs before unfreezing.#testing #python #devops #ai #software #coding #development #engineering #inclusive #community
Freeze Flakes Before Properties Vote

#testing #inclusive #engineering #development #python #coding
Dev Community @[email protected]
2026-10-11
The contract check failed once, then passed on a local rerun. The model in the pairing window offered a helper that retried the assertion three times and returned the first success. A senior on the same call did not accept the patch. The hour ended with a receipt file, a required seed, and a rule that the test process itself may not retry.
The scene is a reconstructed pairing hour, written as a worked example. It is not a production incident report, and it does not claim measured flake rates.
What the screen showed

A service under test exposes GET /orders/:id. The fixture inserts one order, the client waits 200 milliseconds, and the assertion expects status to equal packed. On the first run the body still said picking. The second run, started from the same shell without a clean database, returned packed and exit code 0.
That sequence is a familiar trap in a pairing room. A green rerun feels like proof. Often it is only leftover rows, a warm cache, or a worker that finished between the two commands.
The senior asked three questions before anyone edited the helper.
Which process owned the clock: the test, the app, or the database?
What was the seed, and was it written down before the second run?
If a second machine runs the same command once, does it fail the same way?
The model answered the first question with a guess about eventual consistency. No log line supported that phrase. Unsupported wording stayed out of the patch.
Dead end one: a retry inside the assertion

The first proposal wrapped the check in a loop and treated the first success as the result. The patch looked small. It also moved a timing policy into the test file, where the next editor could raise the limit and call the suite stable.
async function expectPacked(client, id) {
let last = null;
for (let attempt = 1; attempt <= 3; attempt += 1) {
last = await client.get(`/orders/${id}`);
if (last.body && last.body.status === "packed") return last;
await new Promise((resolve) => setTimeout(resolve, 200 * attempt));
}
throw new Error("status never reached packed");
}
The senior rejected that helper. A retry inside the assertion hides the attempt count from CI and from the pairing notes. The loop also couples a pass to whichever process still holds the previous row. Leftover state is not a contract.
Dead end two: stop because the model was free

The second proposal was to end the hour because a free model cannot be deterministic. That move throws away the only evidence on the table: one failing body, one passing body, and no seed file.
Model tier does not explain a status field. A free model and a paid model can both invent the same wrapper. The decision belongs to the harness. Free access can draft the next patch. It is not a reason to trust a green rerun, and it is not a reason to ignore a real mismatch.
The decision the hour kept

The senior kept a smaller rule. The pair wrote it down before the next edit.
The test process runs the check once.
A retry, if anyone still wants one, is a separate command with a written budget.
Each attempt records the seed, the command, the exit code, and a hash of the output.
A second process must reproduce the failure before application code changes.
The model may propose the receipt shape. It may not mark the receipt optional.
The script that follows is a proposed example for a scratch repository. It was not executed against a live service for this draft. Keep the file as retry-receipt.mjs so top-level await is valid, or set "type": "module" in a local package.json.
// retry-receipt.mjs
// Proposed example. Not a recorded production log.
import { spawn } from "node:child_process";
import { createHash } from "node:crypto";
import { writeFileSync } from "node:fs";
const budget = Number(process.env.RETRY_BUDGET || "1");
const seed = process.env.ORDER_SEED || "seed-not-set";
const cmd = process.env.CHECK_CMD || "node check-order.mjs";
function sha(text) {
return createHash("sha256").update(text).digest("hex");
}
function runOnce(attempt) {
return new Promise((resolve) => {
const child = spawn(cmd, {
shell: true,
env: { ...process.env, ORDER_SEED: seed, ATTEMPT: String(attempt) }
});
let out = "";
child.stdout.on("data", (chunk) => { out += chunk; });
child.stderr.on("data", (chunk) => { out += chunk; });
child.on("close", (code) => {
resolve({
attempt,
seed,
exit: code,
body_sha256: sha(out),
tail: out.slice(-240)
});
});
});
}
const attempts = [];
for (let i = 1; i <= budget; i += 1) {
attempts.push(await runOnce(i));
if (attempts.at(-1).exit === 0) break;
}
const receipt = {
decided: "retry_lives_outside_the_test",
budget,
seed,
attempts,
keep: attempts.some((row) => row.exit !== 0)
? "investigate_before_patch"
: "still_not_proof"
};
writeFileSync("retry-receipt.json", JSON.stringify(receipt, null, 2));
console.log(JSON.stringify({ path: "retry-receipt.json", keep: receipt.keep }));
The companion check fails closed when the seed was never set. Scoring a rerun without a seed is the failure mode the hour was trying to kill.
// check-order.mjs — single-shot sketch
const seed = process.env.ORDER_SEED;
if (!seed || seed === "seed-not-set") {
console.error("seed missing; refusing to score a rerun");
process.exit(2);
}
console.error(JSON.stringify({ seed, status: "picking" }));
process.exit(1);
Commands the notes kept

Run the receipt writer on the dirty shell first. Then run it again with a clean environment and a budget of one. The second command is the comparison, not a victory lap.
export ORDER_SEED="order-17-worker-not-run"
export RETRY_BUDGET="2"
export CHECK_CMD="node check-order.mjs"
node retry-receipt.mjs
env -i PATH="$PATH" HOME="$HOME" \
ORDER_SEED="$ORDER_SEED" RETRY_BUDGET=1 \
CHECK_CMD="node check-order.mjs" \
node retry-receipt.mjs
How to score the two files

A missing seed must exit 2. That exit means the harness refused to score, not that the product failed.
Budget 2 on a dirty shell may show a later pass. That pass does not authorize an application edit.
Budget 1 on a clean process is the row that matters.
If the clean process fails with picking and the dirty shell passed only on attempt 2, keep leftover state as the finding.
If both processes fail the same way, keep a missing worker step as the finding, and only then ask for a patch.
A receipt whose keep field is still_not_proof is still not approval. Exit code 0 does not upgrade itself.
Do not commit retry-receipt.json when the tail holds tokens, cookies, or customer payloads. Keep the hash. Drop the tail.
Replace the sketch's hard-coded picking line with a real HTTP call only after those seven checks pass on the stub. A live call added too early recreates the original argument in a noisier form.
Where free model access and a free server sit

Disclosure: This article was prepared as part of MonkeyCode's product outreach.
MonkeyCode is the assistant named for this outreach. The operator describes it as an open-source project with free model access and a free server option. Those two availability claims are operator-supplied context for the workflow. This draft does not name a model, a token quota, a machine size, or a duration. Figures like that go stale, and an older number would be the wrong fact to print.
Free model access, in this hour, is a way to draft the receipt schema and the failing-closed check without treating a paid balance as the reason to stop. A free server is a place to run the second process so the pass is not cached on the laptop that just went green. That split is the senior's third question. It is not a product verdict.
A free server is not a production twin. Free model access does not make a generated retry loop correct. If today's project docs do not list those options, drop the product and keep the receipt rule. The rule does not need the assistant.
When the docs do list them, a restrained use is to paste retry-receipt.json and ask which fields break the rule list. The human still sets keep.
Who should not use this

Do not point CHECK_CMD at untrusted input. The sample sets shell: true for a local scratch run, which is a sharp edge.
Do not share a database between the two processes. A second process on the same data volume recreates the trap.
Do not treat two runs as a flake-rate study. No sample size is claimed here.
Teams that need attested hardware, signed builds, or regulated evidence should not substitute a free server session for that evidence.
Teams whose worker is intentionally asynchronous should put the retry budget in the worker contract, with an owner and an alert, not inside expectPacked.
Reviewers who want a second green run to close the ticket should use another workflow. This one is built to refuse that closure.

What stayed in the notes

The model offered a short paragraph about resilience. The senior did not keep the paragraph. The kept decision was the receipt: seed required, budget outside the test, second process on a separate environment, and no application edit until both receipts name the same failure.
That split is enough for a pairing hour. It will not replace a load test, a trace, or a product decision about retries at the edge. It will stop a helper from laundering one lucky rerun into a merged patch.
Readers comparing assistants can check MonkeyCode's current notes on free model access and the free server option, then run the scratch script on a fixture they already trust. If those notes disagree with this paragraph, believe the notes.#testing #javascript #ai #devops #software #coding #development #engineering #inclusive #community
Outside the Assertion: The Retry Budget a Pairing Hour Refused to Inline
#testing #inclusive #engineering #development #coding #software
Dev Community @[email protected]
2026-10-11
An agent patch does not merge because the job is green. It merges when every changed symbol carries property credit, every fixture hash matches the lock, and every flake freeze contributes zero.
CI collapses three unlike signals into one bit. A property can pass. A fixture can replay. A known flake can be silenced. Those outcomes are not substitutes. The miss this ledger targets is specific: production behavior changed, no invariant covers the edited symbol, and a freeze is what kept the check quiet.
The scoring rules below are a proposal. The classifier is an unexecuted example, not a measured gate from a production repository.
What receives credit

Property credit is the only positive unit. A changed symbol earns one point when a predicate states an invariant, the predicate's source hash is stored in the receipt, and the predicate does not compare the result to a frozen document.
Fixture locks are mandatory and worth nothing. A matching hash shows that the input bytes are the bytes you reviewed. It does not show that the behavior is acceptable.
A flake freeze is a quarantine citation, also worth nothing. It may point at a baseline flake so an unrelated failure does not block review. It may not stand in for a missing predicate on a symbol this patch edited.
Coverage is credited symbols divided by changed symbols. Admission is exact, not approximate.
admit if coverage == 1.0
and fixture_status == "match"
and rejection_codes ==
[]reject otherwise
A coverage of 0.5 is not a soft warning. The receipt rejects, and the missing symbols go back to the author.
Where a Boolean gate stays silent

Agent diffs often include a new test next to the fix. That pattern looks careful. It fails the ledger when the assertion equals a fixture the same diff rewrote, or when a skip cites a freeze id that the patch itself created.
The job still passes. The receipt does not. R5 marks fixture equality dressed up as a check. R4 marks a freeze id absent from the baseline ledger. R3 marks a baseline freeze used as the only signal on a changed symbol.
A quieter miss is the helper that returns true after reading fixture state. The test name contains the word property. The predicate hash moves whenever the fixture moves, so the classifier withholds credit. A label in the test name is not an invariant.
Decision table
LaneCreditAdmit ruleRemote run allowed as proof?RejectionProperty on a changed symbol1 per symbolCoverage must be 1.0Only as confirmation after a local passR1 missing property, R5 fixture equalityFixture content hash0Exact match requiredNoR2 hash mismatchBaseline flake freeze0Not requiredNoR3 freeze as oracle, R4 unknown freeze idRemote-only log0Not acceptedNoR6 remote-only receipt

Read the table as the policy. The steps below only compute it.
Step 1: List changed symbols from the diff

Start from the patch, not from whichever tests the agent happened to touch. Count a symbol when its definition changes, or when a call site change alters an observable result.
git diff --unified=0 origin/main...HEAD -- '*.py' \
| awk '/^diff --git/{f=$0} /^@@/{print f, $0}'
Write one qualified name per line into changed_symbols.txt. An empty list ends the review early. A test-only edit can still carry a property if it claims to lock behavior, but it does not earn fixture credit for production files the diff never touched.
Keep the list short enough to review by hand. If a generated file explodes the symbol count, exclude that path in the command and say so in the receipt. Hidden exclusions are just another Boolean.
Step 2: Classify each touched assertion

Label every assertion in a touched test with one class.
property — a predicate over inputs and outputs that does not embed a full expected document.
fixture_eq — equality or snapshot comparison against a checked-in blob.
freeze_ref — skip, xfail, or an annotation that cites a flake id.
opaque — helpers, mocks, or assertions whose meaning is not visible in the diff.
Only class 1 can add credit. Class 2 may satisfy the fixture gate when the blob hash matches. Class 3 never adds credit. Class 4 adds nothing, and any changed symbol that has only class 4 evidence remains uncovered.
The vocabulary holds, implies, and invariant in the sample classifier is a local convention, not a language feature. If your suite uses different predicate names, change the hint before you trust a clearance.
Step 3: Hash fixture bytes, not paths

Path strings survive edits that replace the file. An agent can rewrite tests/fixtures/invoice.json and leave the path in the test unchanged.
python - << 'PY'
import hashlib, pathlib
root = pathlib.Path("tests/fixtures")
for path in sorted(root.rglob("*")):
if path.is_file():
digest = hashlib.sha256(path.read_bytes()).hexdigest()
print(f"{digest} {path.as_posix()}")
PY
Diff that listing against fixture.lock on the target branch. Any mismatch is R2. Updating the lock inside the patch is allowed only when a property still covers every changed symbol without using the new blob as the expected value.
If the fixture directory is absent, record fixture_status as not_applicable rather than match. Do not invent a hash to satisfy the gate.
Step 4: Cite freezes only from the baseline ledger

The quarantine file must already exist on the target branch. A patch-local freeze file is not a source of valid ids.
flakes:
- id: FLK-1044
signature: "TimeoutError: checkout_worker"
opened_on: "2026-09-02"
expires_on: "2026-10-16"
tests: ["tests/test_checkout.py::test_worker_drain"]
Those dates are example fields for the rule, not a report of a live incident. A citation is valid only when the id is in that baseline file, the signature still matches, and expires_on is later than the review clock. On a review clock of 2026-10-11, the example id above is still citable. An id that expired on 2026-10-01 is not.
Do not extend expiry in the behavior patch. Expiry is a policy edit and belongs in its own review. A freeze that merely keeps a changed symbol quiet is R3, even when the id is old and unexpired.
Step 5: Worked receipt

Take two changed symbols, billing.prorate and billing.invoice_total. The first test is assert holds(prorate(10, 0.5) >= 0). That is one credit. The second test is assert invoice_total(order) == load_fixture("invoice.json"). That is R5 and zero credit. An unrelated citation of baseline FLK-1044 adds no credit either.
{
"changed_symbols": ["billing.prorate", "billing.invoice_total"],
"credits": {"billing.prorate": 1, "billing.invoice_total": 0},
"coverage": 0.5,
"fixture_status": "match",
"freeze_citations": ["FLK-1044"],
"rejection_codes": ["R1", "R5"],
"admit": false
}
The repair is a predicate that does not read the fixture as an oracle, for example assert holds(invoice_total(order) == sum(line.net for line in order.lines)). After that predicate is local-green, coverage can reach 1.0 and R1 clears. R5 stays if the fixture equality remains in the file. Delete it, or keep it only as a non-oracle lock outside the credit column.
A rerun that stays green without the new predicate does not change this receipt. Green is not coverage.
Step 6: Classifier you can adapt

This module is unexecuted proposal code. It will mis-fire on suites that do not use the hint vocabulary. Run it on a scratch branch and compare its codes with a manual read before you wire it to merge.
#!/usr/bin/env python3
"""Unexecuted proposal: symbol credit for an agent-patch receipt."""
from __future__ import annotations
import hashlib
import json
import re
PROPERTY_HINT = re.compile(r"^\s*assert\s+(implies|invariant|holds)\b", re.M)
EQ_HINT = re.compile(r"assert\s+.+\s*==\s*load_fixture\b")
FREEZE_HINT = re.compile(r"flake-id:\s*(FLK-\d+)")
def sha256_text(text: str) -> str:
return hashlib.sha256(text.encode("utf-8")).hexdigest()
def classify_test(source: str) -> str:
if EQ_HINT.search(source):
return "fixture_eq"
if PROPERTY_HINT.search(source):
return "property"
if FREEZE_HINT.search(source):
return "freeze_ref"
return "opaque"
def score(changed: list[str], tests: dict[str, str], baseline_flakes: set[str]) -> dict:
credits = {name: 0 for name in changed}
codes: list[str] =
[] citations: list[str] =
[] for symbol, source in tests.items():
kind = classify_test(source)
found = FREEZE_HINT.findall(source)
citations.extend(found)
for flake_id in found:
if flake_id not in baseline_flakes:
codes.append("R4")
if symbol not in credits:
continue
if kind == "property":
credits[symbol] = 1
elif kind == "fixture_eq":
codes.append("R5")
elif kind == "freeze_ref" and credits[symbol] == 0:
codes.append("R3")
if any(credit == 0 for credit in credits.values()):
codes.append("R1")
coverage = 0.0 if not changed else sum(credits.values()) / len(changed)
reject = sorted(set(codes))
return {
"changed_symbols": changed,
"credits": credits,
"coverage": coverage,
"predicate_hashes": {name: sha256_text(body) for name, body in tests.items()},
"freeze_citations": sorted(set(citations)),
"rejection_codes": reject,
"admit": coverage == 1.0 and not reject,
}
if __name__ == "__main__":
sample = score(
changed=["billing.prorate", "billing.invoice_total"],
tests={
"billing.prorate": "def test_prorate():\n assert holds(prorate(10, 0.5) >= 0)\n",
"billing.invoice_total": "def test_total():\n assert invoice_total(order) == load_fixture('invoice.json')\n",
},
baseline_flakes={"FLK-1044"},
)
print(json.dumps(sample, indent=2))
The sample input is the worked case from Step 5, encoded as strings so the file runs without a repository. Replace those strings with real test bodies before you treat the output as a review artifact.
python tools/merge_ledger.py > receipt.json
python -c 'import json,sys; r=json.load(open("receipt.json")); sys.exit(0 if r["admit"] else 1)'
The process exit status is the mechanical gate. A reviewer still reads rejection_codes before any override. Overrides belong in the receipt as an explicit field, and they cannot convert freeze citations into credit.
The scorer above does not compute fixture_status, and it does not parse expiry. Intersect its admit flag with the Step 3 lock comparison, and feed it only flake ids that Step 4 already accepted. A true flag with a hash mismatch is still R2.
Step 7: Draft a missing predicate, then prove it locally

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access has one job here. After an R1 receipt lists uncovered symbols, you can ask a free model to draft candidate assert holds(...) lines from the diff. Those lines are untrusted text.
Keep a draft only if you can restate the invariant without the draft in front of you, and only after a local run executes it on the patched code. A draft that merely restates the fixture file is class fixture_eq, not credit.
The free server option has a second, separate job. Once the local property lane is green, rerun that pure lane remotely and attach the log as confirmation. Confirmation is not credit. It does not clear R2, R3, R4, or R6.
A receipt that contains only a remote log is rejected. Both options are operator-supplied availability claims for this workflow. This article does not state a model name, quota, hardware profile, duration, or permanence. If either option is unavailable, compute the receipt locally. admit must not flip because a remote lane was missing.
Step 8: Read four fields, in order

coverage equals 1.0. Anything lower returns to the author with the uncovered names.
rejection_codes is empty. A lingering R5 means a fixture equality is still in the diff, even if some other predicate scored a point.
Each id in freeze_citations is baseline, signature-matched, and unexpired against the CI clock printed in the receipt.
predicate_hashes match the test bodies in the diff. A hash that moved without a reviewed edit is a stale receipt, and stale receipts do not admit.
Merge only after those four checks. Do not lower the coverage bar to "most symbols" to clear a queue.
Limitations

The classifier is lexical. Helper-shaped properties become opaque and produce false R1 results. Vacuous forms such as assert holds(True) can look like credit unless you add a constant check. Add that check before clearance means anything.
Snapshot-heavy suites will collect R5 on almost every agent patch. That is a prompt to extract one invariant per change, not a prompt to disable the code. If you cannot name an invariant, this ledger does not fit that change.
Expiry uses the clock you pass in. A drifted clock admits dead freezes. Pin the clock to the CI timestamp and print it beside freeze_citations.
Nothing here is a formal proof. A fluent predicate can still encode the wrong business rule. Load, migration, and security reviews stay outside the receipt.
Who should skip it

Skip the ledger on a mechanical rename that your policy already treats as non-behavioral. Also skip it when the edited code has no deterministic predicate without live third-party state. In that setting the strategy manufactures R1 noise, and a recorded interaction contract is the honest artifact instead.
Do not adopt the ledger if you need the freeze file to grant merge credit. That need contradicts the rule. Do not use a free-model draft as the source of record, and do not file a free-server log as the only pull-request evidence.
Regulated teams should keep the local receipt, the lock file, and the CI timestamp in the archive their policy already names. On the next agent patch, start from the rejection codes rather than from a runner brand. The codes are the strategy. Remote confirmation stays optional.#testing #python #ai #devops #software #coding #development #engineering #inclusive #community
Changed Symbols Earn Property Credit, or the Agent Patch Stays Closed
#usrbinenv #testing #inclusive #engineering #development #python
Dev Community @[email protected]
2026-10-11
A temporary test ignore stays replayable only when three ledger fields still describe the patch under review. Those fields are the property identifier, the fixture contract hash, and an unexpired window for the same runner class. One green log does not create the match. A rewritten formula under the old identifier does not create it either.
Agent diffs break this quietly. The test, the fixture, and a copied ignore note arrive together. The note still looks specific. The bytes it names are no longer the bytes in the tree. Treat that as an invalid record, not as proof the check is flaky.
What each key is allowed to prove

A property identifier names one invariant. order_total_equals_sum_of_lines is an identifier. A path is not. Change the invariant, and you allocate a new identifier. The old row stays on the old behavior.
A fixture contract hash binds the inputs that invariant may read. Include schema version, field names, and fixed seeds. Exclude timestamps, hostnames, and process IDs. Those identify a run, not a contract.
The expiry field limits how long the ignore can be cited. It must name the runner class it was recorded for. A missing expiry is not open-ended approval. The decision is a reject.
These keys do not show the patch is correct. They show that a temporary ignore still points at the same invariant, the same input contract, and the same runner class. That narrower claim is the only one this workflow makes.
Canonical hashing before the digest is trusted

Hash the parsed contract, not the raw file. Load JSON, dump it with sorted keys and tight separators, then take SHA-256 of the UTF-8 bytes. A trailing newline must not move the digest. Reordered keys must not move it either.
Reject contracts that use floats for values the property treats as exact. Store money as integers in minor units, or as strings. Two parses of 0.1 are not a stable contract.
Object key order is normalized by the dump. Array order is not. Array order is data. If a list is semantically a set, sort it in the producer and record that rule by bumping schema. The bump changes the digest on purpose.
The digest is an equality check against the contract file in this patch. It is not a review of whether the contract is adequate. A clean hash can still describe the wrong inputs.
Use the same json.dumps settings on every side. The proposal leaves ensure_ascii at its default. Do not flip that flag in a one-off command, or non-ASCII field names will disagree for a reason that is not a fixture edit.
Decision table

Read left to right. The first failed key wins. Do not average a near-miss into a partial allow.
Property IDFixture hashWindowActionEqual to the cited IDEqual to this contractUnexpired, same runner class, failure class environmentallow_cite. Do not extend the window in this patch.EqualDifferentAnyreject:fixture_hash. Review the contract edit as a behavior change.UnequalAnyAnyreject:property_id. Leave the old row on the old identifier.EqualEqualMissing, expired, or runner class differsreject:window. Do not copy the old timestamp forward.Absent from the test moduleAnyAnyreject:unnamed. The search command covers this row.

An expiry equal to --now is already closed. Matching keys with a failure class other than environment still reject. The checker returns reject:not_environment for that case. allow_cite is not a merge approval. The named assertion still has to run.
Five steps

Put the identifier in the test module. The ledger string must be searchable in that file. A pull-request comment is not an identifier.
Write a contract file from inputs the assertion actually reads. Keep schema, field names, and the seed. Omit clocks and host identity.
Classify the miss before the row is eligible. Use invariant when the named property fails on the hashed contract. Use environment for timeouts, connection resets, and missing optional services. Only environment may later return allow_cite.
Compare the row at a timestamp you pass in. --now is an argument, not a clock read, and not a timestamp copied from a remote log.
Keep the decision string with the diff. allow_cite means the existing row may stay referenced. Any reject:* value means the reference comes out. This checker does not mint a replacement row.

Proposed test and proposed checker

Both listings are unexecuted proposals. They are not a production run, and they are not measurements.
def test_order_total_equals_sum_of_lines(contract):
"""property_id: order_total_equals_sum_of_lines"""
lines = contract["lines"]
assert order_total(lines) == sum(row["qty"] * row["price"] for row in lines)
The assertion is the invariant. The docstring is the identifier. If the formula changes, the identifier should change with it. Keeping the name while editing the formula is how an ignore survives a behavior change.
#!/usr/bin/env python3
"""Proposed checker for one ignore record. Unexecuted example."""
import argparse
import hashlib
import json
from datetime import datetime, timezone
def contract_hash(path: str) -> str:
with open(path, "r", encoding="utf-8") as handle:
payload = json.load(handle)
canonical = json.dumps(payload, sort_keys=True, separators=(",", ":"))
return hashlib.sha256(canonical.encode("utf-8")).hexdigest()
def parse_zulu(value: str) -> datetime:
stamp = datetime.fromisoformat(value.replace("Z", "+00:00"))
return stamp.astimezone(timezone.utc)
def decide(record: dict, digest: str, now: datetime) -> str:
if not record.get("property_id"):
return "reject:unnamed"
if record.get("property_id") != record.get("cited_property_id"):
return "reject:property_id"
if record.get("fixture_hash") != digest:
return "reject:fixture_hash"
if record.get("runner_class") != record.get("cited_runner_class"):
return "reject:window"
if record.get("failure_class") != "environment":
return "reject:not_environment"
expires = record.get("expires_at")
if not expires or parse_zulu(expires) <= now:
return "reject:window"
return "allow_cite"
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--ledger", required=True)
parser.add_argument("--contract", required=True)
parser.add_argument("--now", required=True)
args = parser.parse_args()
with open(args.ledger, "r", encoding="utf-8") as handle:
record = json.load(handle)
print(decide(record, contract_hash(args.contract), parse_zulu(args.now)))
if __name__ == "__main__":
main()
The ledger below illustrates field shape for an evaluation date of 2026-10-11. The expiry value is a format example only. It is not a recommended window, and it is not an observed flake interval.
{
"property_id": "order_total_equals_sum_of_lines",
"cited_property_id": "order_total_equals_sum_of_lines",
"fixture_hash": "replace-with-digest",
"runner_class": "local-fixture",
"cited_runner_class": "local-fixture",
"failure_class": "environment",
"expires_at": "2026-10-18T00:00:00Z"
}
{
"schema": 1,
"fields": ["qty", "price"],
"seed": 17,
"lines": [{"qty": 2, "price": 500}]
}
The sample ledger fails closed until fixture_hash is replaced with the digest of that contract. That is intentional.
Commands

Print the digest with the same canonicalization, then replay the decision at a fixed time. After that, confirm the identifier is still in the test module.
python3 - <<'PY'
import hashlib, json
with open("fixture_contract.json", encoding="utf-8") as handle:
payload = json.load(handle)
canonical = json.dumps(payload, sort_keys=True, separators=(",", ":"))
print(hashlib.sha256(canonical.encode("utf-8")).hexdigest())
PY
python3 freeze_keys.py --ledger freeze_record.json --contract fixture_contract.json --now 2026-10-11T00:00:00Z
rg -n "property_id: order_total_equals_sum_of_lines" tests/test_order_total.py
The Python proposal checks the ledger against the contract file. The search checks the test module. Both must pass. If the digest mismatches, stop. Changing expires_at does not repair a hash. If the search fails, stop. The row points at a name the patch does not contain.
Redact before any draft

A draft is only as usable as the excerpt you paste. Strip hostnames, account identifiers, tokens, and customer payloads first. Keep the exception class, the property identifier, and the contract field names.
Do not paste fixture values when they are production records. The contract in the repo should already be synthetic. If it is not, stop this workflow. Replace the data, then hash the replacement. The new digest is a new contract.
The scratch note may store a drafted label. It may not store secrets that survived a weak redact. When redaction is uncertain, skip the draft and set failure_class by reading the assertion locally.
Where a draft and a server sample may enter

Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode's free model access fits step 3 as a drafting aid, not as a label the checker stores. Provide the identifier, the contract field names, and the redacted excerpt. Ask for one label, invariant or environment, plus one sentence of reason. Leave that text in the scratch note. decide never reads it.
MonkeyCode's free server option fits step 4 as one place to collect a sample. It is not a source of --now. If you use it, write the class string you actually ran, for example free-server, into cited_runner_class. A differing class string fails the window key. The log can sit beside a new, unapproved row. It cannot flip an existing row to allow_cite.
Neither availability claim states a quota, a hardware shape, a retention period, or a standing waiver of review. If either option is unreachable, leave the scratch note empty and run the local checker. The allow path does not call them.
When a redacted excerpt and a contract file are already on disk, free model access and the free server option are sufficient to draft the class label and to attach one labeled sample before anyone edits the expiry field.
Who should not use this

Skip the workflow when the suite cannot name a pure invariant. Full-page snapshot diffs, tests that pass by sleeping, and patches that replace the runner are out of scope. A hash would still compute. The action code would not mean what a reviewer expects.
Do not use allow_cite to retain a failing invariant. The environment class is a precondition. Do not source --now from an untrusted log line. Do not treat digest equality as proof the contract is the right contract.
This article reports no flake rate, no runtime, and no count of patched tests. Borrowing those figures from an older write-up would not make decide more correct.
What the patch should still contain

Ship the named assertion, the contract file, and the decision string for the timestamp you chose. Remove every ignore reference that returns reject:*. Drafts and server samples can remain in the discussion if a reviewer wants context. They stay off the allow path.
The gate is local and replayable. Three keys, one caller-supplied timestamp, and an environment class are the whole decision. Context that cannot change decide does not belong in the record.#testing #python #devops #ai #software #coding #development #engineering #inclusive #community
Replay a Temporary Test Ignore Only When Three Ledger Keys Still Match
#usrbinenv #testing #inclusive #engineering #development #python