A small platform team cloned a coding-agent repository on Thursday and started the host process before lunch. The health route returned a short success body, and a model call through the free access path also returned text. The channel treated those two green signals as a release note and moved the change toward review. Three days later the same sequence failed in a pull-request check because a tool field had been renamed.
The port still answered, which made the failure look like a flaky test rather than a broken tool trace. That confusion is common when a smoke curl is asked to stand in for a closed workflow loop. A curl proves that some process accepted bytes, much as a doorbell proves that a building has electricity. It does not prove who opened the door, which room was reached, or whether the same visitor would be recognized tomorrow.
MonkeyCode is relevant here only as one place where a free server option and free model access can host that early smoke path. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Only two product facts are used here: a free server option and free model access, with no model name or quota attached. Readers should confirm both facts in the current project docs, because access limits can change and this article does not freeze them.
Recent public discussion has noted that model quality can rise while everyday software practice stays loose. That observation fits this failure, because a better reply does not tighten an unpinned host, an unpinned schema, or an unpinned tree. The practical question is which claim each separate probe is allowed to support in a review. A useful workflow separates reachability, execution, and replay before anyone writes a summary sentence for the channel.
Myth: a listening port means the loop is closed
Developers repeat this claim because local demos reward the first successful status code more than the second differing one. The corrected model treats reachability as a gate, not as a verdict about tools, files, or repeated behavior. Record the process command, the bound port, and the status code, then stop the claim at that boundary. Anything past the boundary belongs to a different probe with its own fixture and its own failure name.
On Thursday the health body was enough to satisfy the channel, and that social fact is not technical evidence. A process can listen while its tool registry still points at yesterday's field names and yesterday's paths. The analogy is a shop with the lights on and the cash drawer disconnected from the inventory system. Customers can enter the shop, and the inventory books can still be wrong at the close of day.
Myth: a model paragraph means the tool actually ran
A fluent reply can describe a file change without the host ever having executed the described tool call. That gap resembles a meeting note that summarizes a decision nobody entered into the issue tracker. Evidence for execution is a structured tool result, a schema version, and a diff against the fixture repository. If any of those three artifacts is missing, the paragraph remains narration rather than an integration result.
Free model access does not change that standard, because availability of a model path is not identity of a schema. The same caution applies to the free server option, which can serve the process without freezing the registry inside it. Ask the run for a machine-readable tool result, and keep the prose reply in a separate field. Reviewers can then reject narration that lacks the structured sibling and the schema version it claims to match.
Myth: sending the same prompt again counts as a replay
A second prompt can hit a different tool schema, a different working tree, or a different unspoken default. Replay requires the same request body, the same repository revision, and the same recorded tool responses. Without those pins, the second run is a new experiment that happens to share some wording with the first. Store the pins beside the output so a later reader can see what was held constant and what was not.
Retry culture makes this myth sticky, because a green second attempt feels like confirmation rather than a new sample. Confirmation needs a recorded first sample and a diff of the pins, not a similar sentence in the terminal. If the schema version changed between attempts, label the second attempt as a migration check instead. If the revision changed, label it as a new build, even when the prompt text matches.
A corrected mental model
Think of the free server as a shared workbench and the free model path as a shared measuring tape. Both can be useful on the first afternoon, and neither becomes a calibrated instrument by being available. Calibration is the evidence file you write: revision, schema version, probe name, and a named outcome. When that file is absent, a green curl should stay in the lab notebook as a reachability note only.
The closed loop is the path from pinned tree, through a named tool call, to a stored result that a second machine can classify. Reachability opens the bench, the execution probe checks the measuring marks, and replay checks whether the marks still agree. Collapsing those three checks into one status code is how Thursday's note became Monday's review surprise. Naming the three checks separately is slower by a few minutes and cheaper than a reverted review thread.
An unexecuted classifier you can adapt
The following checker is a proposal, not a captured run from this account, and it invents no product metrics. It reads a local JSON fixture that you produce, then classifies each probe before any human summary is written. Adapt the field names to your host, and do not treat the sample paths as documented product endpoints. The script fails closed when a required pin is missing, which is the behavior a smoke curl usually lacks.
#!/usr/bin/env python3
"""Proposal only: classify a local evidence fixture. Not a recorded product run."""
import json
import sys
REQUIRED = ("git_revision", "schema_version", "probe", "outcome")
PROBES = {
"reachability": {"allowed_outcomes": {"listening", "refused", "timeout"}},
"execution": {"allowed_outcomes": {"executed_match", "executed_mismatch", "tool_missing"}},
"replay": {"allowed_outcomes": {"pins_match", "pins_drift", "incomplete"}},
}
def classify(row: dict) -> dict:
missing = [key for key in REQUIRED if not row.get(key)]
probe = row.get("probe")
outcome = row.get("outcome")
if missing or probe not in PROBES:
return {"verdict": "incomplete", "missing": missing or ["probe"], "claim_scope": "none"}
allowed = PROBES[probe]["allowed_outcomes"]
if outcome not in allowed:
return {"verdict": "invalid_outcome", "probe": probe, "claim_scope": "none"}
scope = {
"reachability": "port_only",
"execution": "tool_execution_only",
"replay": "pinned_comparison_only",
}
[probe] passed = outcome in {"listening", "executed_match", "pins_match"}
return {"verdict": "pass" if passed else "fail", "probe": probe, "claim_scope": scope}
def main() -> int:
document = json.load(sys.stdin)
rows = document.get("probes", [])
report = [classify(row) for row in rows]
scopes = {item["claim_scope"] for item in report if item["verdict"] == "pass"}
needed = {"port_only", "tool_execution_only", "pinned_comparison_only"}
closed = needed <= scopes
json.dump({"probes": report, "closed_loop": closed}, sys.stdout, indent=2)
sys.stdout.write("\n")
return 0 if report and all(item["verdict"] == "pass" for item in report) else 1
if __name__ == "__main__":
raise SystemExit(main())
A reader can generate the surrounding pins with ordinary commands, then fill the fixture by hand from observed responses. The commands below record identity of the tree and the clock, not the performance of any host or model. They are safe to run in a disposable clone, and they still do not prove a tool call happened. Paste their output into the fixture fields rather than into a chat summary that will be forgotten.
git rev-parse HEAD
git status --porcelain=v1
python3 --version
date -u +%Y-%m-%dT%H:%M:%SZ
{
"git_revision": "replace-with-git-rev-parse",
"schema_version": "replace-with-tool-schema-id",
"probes": [
{
"git_revision": "replace-with-git-rev-parse",
"schema_version": "replace-with-tool-schema-id",
"probe": "reachability",
"outcome": "listening"
},
{
"git_revision": "replace-with-git-rev-parse",
"schema_version": "replace-with-tool-schema-id",
"probe": "execution",
"outcome": "executed_mismatch"
},
{
"git_revision": "replace-with-git-rev-parse",
"schema_version": "replace-with-tool-schema-id",
"probe": "replay",
"outcome": "incomplete"
}
]
}
python3 classify_probes.py < evidence.json
Read the machine-readable report before you accept a prose recap of that same afternoon in the team channel. A pass on reachability with an executed_mismatch is the Thursday shape: the door opened, and the inventory field did not match. An incomplete replay means you may not claim that the second attempt confirmed the first recorded sample. The closed_loop flag becomes true only when all three scopes have passed, which the sample fixture above should refuse.
What the Thursday notes should have contained
The useful note would have said the port listened, the schema id was unread, and no replay pins were stored. That sentence is less exciting than a release claim, and it is the one a later reviewer can trust. Add the git revision from the disposable branch and the tool schema identifier from the host response. Leave model marketing names out of the note unless the current docs and the response metadata agree.
If the execution probe returns executed_mismatch, stop and diff the fixture schema against the live registry. Do not paper over that diff with a second prompt that asks the model to explain the error. An explanation can be helpful after the mismatch is recorded, and it is not a substitute for the mismatch record. The corrected habit is to file the structured failure first and to write the narrative only after that file exists.
Limitations and who should skip this
This approach does not load-test the host, price a loop, or prove that a warm process kept state stable. It also does not verify authentication, secret redaction, or multi-tenant isolation on a shared free server. Teams that need those guarantees should use their own audited environment and their own written acceptance tests. People debugging a one-off prompt with no repository fixture will get little value from the extra pins.
Skip this workflow if you cannot record a repository revision, because the classifier will refuse to pass. Skip it if your question is capacity or cost, since those questions need different instruments and different fixtures. Skip it if you need a vendor benchmark, because this article records no timings and names no model. The sample outcomes are labels for a local file, not measurements collected from any public or shared run.
If the free server option is already available to you, point the three probes at a disposable fixture branch before you announce a result. Read the current docs for access limits before you quote any allowance in a design review. That habit keeps a Thursday smoke signal from turning into an accidental release claim on the following Monday.#testing #devops #python #ai #software #coding #development #engineering #inclusive #community
A Smoke Curl Is Not a Closed Loop: A Myth-Busting FAQ