I wired up a free model endpoint last month. I sent one request. It returned 200 OK. I shipped the feature. Then the tool calls never happened. The endpoint accepted the request. It just ignored half of it. Sound familiar?
Here is my question. What does a 200 actually prove? My answer: almost nothing about capabilities. It proves the transport worked. That is it.
So I stopped trusting status codes and started probing the contract. This post is the FAQ I wish I had read first. It ends with a script you can run in about two minutes.
Myth 1: A 200 means the model supports my feature
The server can accept a tools array and silently ignore it. The model then answers in prose. No error. No warning. Your code sees a normal response and skips the branch. Is that a bug in the model? No. It is a bug in your assumption.
The corrected model: support is a property of the endpoint, not of the model name. Probe it. Record it. Re-probe when anything changes.
Myth 2: finish_reason is optional metadata
It is the most important field in the response. length means the model ran out of output budget. Your answer looks fine and is probably cut in half. I have watched a parser happily eat a truncated JSON object.
{"choices":[{"finish_reason":"length","message":{"content":"{\"items\": [1, 2, 3, 4"}}]}
If you do not branch on finish_reason, you are parsing whatever bytes arrived. That is not a feature. That is a coin flip.
Myth 3: Streaming and non-streaming are interchangeable
They are two different code paths. In streaming, finish_reason arrives in the final chunk, not the first. Some servers only report token usage when you ask for it explicitly. Some omit it entirely.
Ask yourself: does my client read the last event? Mine did not, for a week. I was logging None as the finish reason every single time.
Myth 4: JSON mode means my schema
It means valid JSON. It does not mean your keys. It does not mean your types. Validate every response against a schema before it touches business logic. Treat model output like user input, because that is what it is.
Myth 5: Retries are always safe
Retry a failed read, fine. Retry a tool call that already wrote a row, and now you have two rows. I learned this from a duplicate charge, not a blog post.
Add an idempotency key your tool honors. Make the retry decision depend on the operation, not on the exception class.
The artifact: a five-minute capability probe
This script uses only the Python standard library. It assumes a chat-completions-shaped endpoint. Adjust the paths if yours differs.
# probe_capabilities.py — smoke test only, not a benchmark
import json, os, time, urllib.request, urllib.error, concurrent.futures as cf
BASE = os.environ["BASE_URL"].rstrip("/")
KEY = os.environ["API_KEY"]
MODEL = os.environ["MODEL"]
def call(payload, stream=False, timeout=60):
req = urllib.request.Request(
BASE + "/chat/completions",
data=json.dumps(payload).encode(),
headers={"Content-Type": "application/json",
"Authorization": f"Bearer {KEY}",
"Accept": "text/event-stream" if stream else "application/json"},
method="POST")
t0 = time.time()
try:
with urllib.request.urlopen(req, timeout=timeout) as r:
return {"status": r.status, "ms": int((time.time()-t0)*1000),
"raw": r.read().decode("utf-8", "replace")}
except urllib.error.HTTPError as e:
return {"status": e.code, "ms": int((time.time()-t0)*1000),
"raw": e.read().decode("utf-8", "replace")}
def probe_truncation():
p = {"model": MODEL, "max_tokens": 8,
"messages": [{"role": "user", "content": "List ten colors."}]}
r = call(p)
if r["status"] != 200:
return {"probe": "truncation", "ok": False, "why": r["status"]}
ch = json.loads(r["raw"])["choices"]
[0] return {"probe": "truncation", "ok": True,
"finish_reason": ch.get("finish_reason"),
"reports_usage": "usage" in json.loads(r["raw"])}
def probe_tools():
p = {"model": MODEL, "max_tokens": 128,
"messages": [{"role": "user", "content": "Read ./README.md and report its first line."}],
"tools": [{"type": "function", "function": {
"name": "read_file",
"description": "Read a file from disk.",
"parameters": {"type": "object",
"properties": {"path": {"type": "string"}},
"required": ["path"]}}}]}
r = call(p)
if r["status"] != 200:
return {"probe": "tools", "ok": False, "why": r["status"]}
msg = json.loads(r["raw"])["choices"][0]["message"]
return {"probe": "tools", "ok": True,
"returned_tool_calls": bool(msg.get("tool_calls"))}
def probe_json():
p = {"model": MODEL, "max_tokens": 128,
"messages": [{"role": "user",
"content": 'Reply with only JSON: {"status":"ok","count":3}'}]}
r = call(p)
if r["status"] != 200:
return {"probe": "json", "ok": False, "why": r["status"]}
text = json.loads(r["raw"])["choices"][0]["message"]["content"]
try:
obj = json.loads(text)
return {"probe": "json", "ok": True,
"keys_match": sorted(obj) == ["count", "status"]}
except json.JSONDecodeError:
return {"probe": "json", "ok": True, "keys_match": False,
"note": "not parseable as JSON"}
def probe_concurrency(n=2):
p = {"model": MODEL, "max_tokens": 16,
"messages": [{"role": "user", "content": "Say ok."}]}
with cf.ThreadPoolExecutor(max_workers=n) as ex:
codes = list(ex.map(lambda _: call(p)["status"], range(n)))
return {"probe": "concurrency", "ok": True, "statuses": codes,
"saw_429": 429 in codes}
if __name__ == "__main__":
for fn in (probe_truncation, probe_tools, probe_json, probe_concurrency):
print(json.dumps(fn(), sort_keys=True))
Run it like this. Keep the key in the environment, never in the file.
export BASE_URL="$YOUR_ENDPOINT"
export API_KEY="$YOUR_KEY"
export MODEL="$YOUR_MODEL"
python3 probe_capabilities.py
Reading the output: a decision table
Probe resultWhat it meansWhat I do next
returned_tool_calls: falseTools accepted, not acted onDrop the agent branch, or change endpoint
finish_reason: "length" on an 8-token capNormal, expectedKeep the check; raise budget in real calls
finish_reason: null in streamingClient is not reading the final eventFix the parser before anything else
reports_usage: falseNo token accounting from serverCount tokens client-side, or accept blind cost
keys_match: falseValid JSON, wrong shapeAdd schema validation at the boundary
saw_429: true at n=2Very low concurrency ceilingQueue work; never fan out blindly
Nothing here measures quality. Nothing here measures speed. Those are different experiments with different controls.
Where I run this
I run probes from a throwaway box, not my laptop. Same reason I do not test migrations on my main database. The probe should not touch anything I care about.
MonkeyCode's free model access and free server option are the two pieces I use for that. The free model access gives the probe something to talk to. The free server option gives it a disposable place to run. Disclosure: This article was prepared as part of MonkeyCode's product outreach.
I am not claiming the free tier is fast, generous, or permanent. I do not know its limits, and I am not going to guess at them. Run the concurrency probe and read your own numbers. That is the whole point of this post.
Limitations and who should skip this
- This is a smoke test. It is not a benchmark and not a load test.
- Results describe one endpoint, one model, one moment. They expire.
- Paid tiers can behave differently from free ones. Re-probe after switching.
- Only probe endpoints you own or are authorized to test.
- Do not use this to grade answer quality. It checks contract shape only.
Skip this approach if you need contractual latency SLOs, deterministic output, or guaranteed tool-call behavior. Buy a plan with a written contract. A probe will not save you from a missing SLA.
The corrected mental model
Treat every free endpoint as a set of separate promises. Transport, truncation, tool calls, JSON shape, and concurrency are five different promises. A 200 only confirms the first one.
Which promise broke on your last integration? I keep a one-page probe log per endpoint now. It has saved me two weekends so far.#ai #api #testing #python #software #coding #development #engineering #inclusive #community
200 OK Proves Nothing About Tool Calls: A Free-Model Capability FAQ With a Probe
