1000 results for “Engin”
A campaign lands in your lap and the numbers are flat. The button says "Submit." The page is three paragraphs of feature copy above the fold. You have one experiment slot this sprint, one shared traffic budget, and two candidate changes that both look obviously wrong.
So which one do you ship first?
The useful answer is neither "test both" nor a coin flip. The two changes answer different questions, and one of them is usually wasted if you run it in the wrong order.
The two problems look identical in a dashboard
Both failures surface as the same line: campaign traffic arrives, signups don't happen. To separate them, split the funnel into two questions.
The first question is about the landing page. The second is about the call to action. A dashboard that only tracks sessions and signups cannot tell you which one broke.
A diagnostic table before you write any code
What the data showsLikely causeWhere to startMost sessions end within a few seconds; low scroll depthThe page does not match the promise in the adLanding pageDeep scroll, pricing or docs section viewed, few form startsThe ask is vague, buried, or costlyCall to actionMany form starts, few completionsForm frictionNeither. Fix the formLow engagement and low action, traffic source unclearTargeting or message mismatch upstreamNeither. Fix the campaign
That last row matters more than it looks. If the audience arriving from the campaign was never a fit, no amount of button rewording or hero-copy polishing will rescue the page. Fixing the targeting is cheaper than any experiment.
Why the order matters more than the individual change
A call to action is read in the context of the headline, the proof, the price, and the surrounding copy.
That produces two ordering rules.
Rule one: a button cannot fix a page that never establishes value. If visitors leave before they understand what they are being asked to do, a sharper label has nothing to attach to. You will measure noise and conclude that labels do not matter.
Rule two: do not optimize a component you are about to replace. If a page rewrite is already on the roadmap for this quarter, running a careful label experiment on the current page is a throwaway. The result will not transfer to the new layout, because the context that produced it no longer exists.
The second rule is the one teams break most often, because label experiments are fast to ship and feel productive.
Do the math before you commit to either
A small change usually produces a small effect, and a small effect demands a large sample. A page rewrite can plausibly shift behavior by several points; a button label change often moves it by a fraction of one.
A quick calculation, runnable in any Python environment, shows how steep that difference is:
import math
def sessions_needed(p_control, p_treatment, alpha=0.05, power=0.80):
z_a = 1.959963985
z_b = 0.8416212336
p_bar = (p_control + p_treatment) / 2
spread = (z_a * math.sqrt(2 * p_bar * (1 - p_bar))
+ z_b * math.sqrt(p_control * (1 - p_control) + p_treatment * (1 - p_treatment))) ** 2
print(math.ceil(spread / (p_treatment - p_control) ** 2))
sessions_needed(0.04, 0.045) # a 0.5-point lift on a 4% baseline
sessions_needed(0.04, 0.06) # a 2-point lift on the same baseline
On a 4% baseline, detecting a 0.5-point improvement at 80% power takes roughly 25,000 sessions per arm. Detecting a 2-point improvement takes roughly 1,900. Same campaign, same page, a thirteen-fold difference in required traffic.
Two conclusions follow. First, if your campaign delivers a few thousand sessions a month, a micro-optimization of the button label is probably not measurable, no matter how good the label is. Second, the size of the expected effect should drive the choice of experiment far more than how easy the change is to ship.
A default order that holds up in practice
Step 4 is where a lot of teams want to start, and it is a legitimate starting point, but only when the diagnostic in the table above says the page is already doing its job.
Track what happens after the action, not just the action
A call to action that increases signups while filling the pipeline with people who never come back is a net loss. Before you run either experiment, decide on guardrail events and log them alongside the primary metric:
// One exposure event per session, emitted once and cached client-side
trackEvent("experiment_exposure", {
experiment_id: "cta_copy_q3",
arm: "specific-verb",
campaign_id: "launch_aug",
});
// Downstream quality signals, tied to the same session
// activation_started, first_key_action, repeat_usage_7d
If the primary metric moves up and a guardrail moves down, that is a trade-off that needs a decision. Guardrails are especially important for landing page experiments, because a page that over-promises will inflate the action rate and damage downstream quality.
Do not change both at once
Running a page experiment and a call-to-action experiment simultaneously on the same traffic makes the result unreadable. You cannot tell whether the change came from the page, the ask, or the interaction between them.
If you genuinely need both answers and have the traffic to support it, use a factorial design: four arms, so every page version is paired with every call to action. The interaction term is often the most interesting number in the output, because it tells you whether a strong page makes the label irrelevant. The cost is roughly four times the traffic of a two-arm experiment.
For most campaigns, sequential experiments are the practical choice: page first, then the ask, with each experiment consuming a pre-committed block of traffic. Avoid switching between them mid-run based on how the numbers look.
Link hygiene when campaign traffic is split
If you are routing campaign traffic between page builds and call-to-action experiments, the link you publish becomes part of the workflow. Campaign copy already competes for space with a headline and a button, and long parameter-heavy URLs wrap badly in chat apps and social posts. A short link such as LinkBit keeps the destination compact and readable in the creative. That is a formatting convenience. Your experiment assignment should live in your own instrumentation, not in the link.
The short answer
Run the landing page experiment first when the funnel shows that visitors never engage, when message match is broken, or when a rewrite is already planned. Run the call to action first when the page is doing its job and the ask is the weak link, and only when your traffic is large enough to detect the smaller effect you are likely to see.
Before either, confirm what your instrumentation measures. Most experiment programs that stall are running careful changes on a funnel they cannot see.#experimentation #conversionrate #landingpages #webanalytics #software #coding #development #engineering #inclusive #community
Should You A/B Test Your Call to Action or Your Landing Page First?
BeeLadybug, my open-source debug overlay, has always shown what's happening in real time: FPS, logs, !important, z-index traps, clicks being swallowed by the wrong element.
The problem is the overlay. Make it big and it covers the game or the page you're debugging. Make it small, the way it is now, and you're squinting at a phone-sized box on the left of the screen while the rows scroll past.
One evening I gave up and asked myself a simple question: why can't I just press a key and get everything on one page, to read calmly?
One key, one file
Version 0.6.1 adds exportReport(). Press F8, or the EXPORT button, and you get report-YYYYMMDD-HHMM.html:
!important, stacking contexts, intercepted clicks, overflow), long tasks, slow resources, Python metricsIt's plain HTML with a dark theme. Ctrl+P turns it into a PDF. You can also get json (for CI or an AI agent) or csv (for a spreadsheet).
import { BeeLadybugCore, exportReport } from 'bee-ladybug';
const bee = new BeeLadybugCore();
// ...attach adapters...
exportReport(bee, { format: 'html' }); // downloads the file
In headless mode (ui: false) nothing is downloaded. The function just returns the data, so it works in Node and in bots too.
How it fits a "blind" Core
The Core still knows nothing about Canvas, the DOM or Python. Each adapter can now register an optional snapshot() hook that returns rows like { severity, selector, rule, value, source }. The exporter asks every registered adapter for its snapshot, and if one throws, it becomes a single warn row while the rest of the report still renders. A new adapter only has to implement snapshot() to show up in the report.
Then I pointed it at real sites
First, my own games portal. It found:
.entry-content creating a stacking context that traps the z-index of the game gridThen, for fun, a big professional games portal. 25 errors, 51 warnings 😄 To be fair, most of them are intentional tricks: negative margins so a carousel can bleed to the edges, and content clipped on purpose inside cards. Their site works great. It still averaged ~52 FPS with 3× my DOM nodes, which taught me more than the error count did.
…and it found bugs in itself
The best part of testing on real pages: the report exposed its own weak spots.
×NAll three are being fixed. A debugger that finds bugs in itself on its first real run felt like a good sign.
Try it
Install it in your project:
npm i bee-ladybug
Or try it right now without installing: open any page, press F12 and paste this in the console, then press F8 for the report.
import('https://cdn.jsdelivr.net/npm/[email protected]/src/index.js')
.then(({ BeeLadybugCore, WebDOMAdapter, CSSAdapter }) => {
const bee = new BeeLadybugCore();
new WebDOMAdapter(bee, { root: document.body }).attach();
new CSSAdapter(bee).attach();
window.bee = bee;
});
Feedback welcome, especially from people debugging Canvas games and DOM-heavy pages.#javascript #webdev #debugging #opensource #software #coding #development #engineering #inclusive #community
I got tired of reading CSS bugs in a tiny overlay, so I made my debugger print a report
I wired up a free model endpoint last month. I sent one request. It returned 200 OK. I shipped the feature. Then the tool calls never happened. The endpoint accepted the request. It just ignored half of it. Sound familiar?
Here is my question. What does a 200 actually prove? My answer: almost nothing about capabilities. It proves the transport worked. That is it.
So I stopped trusting status codes and started probing the contract. This post is the FAQ I wish I had read first. It ends with a script you can run in about two minutes.
Myth 1: A 200 means the model supports my feature
The server can accept a tools array and silently ignore it. The model then answers in prose. No error. No warning. Your code sees a normal response and skips the branch. Is that a bug in the model? No. It is a bug in your assumption.
The corrected model: support is a property of the endpoint, not of the model name. Probe it. Record it. Re-probe when anything changes.
Myth 2: finish_reason is optional metadata
It is the most important field in the response. length means the model ran out of output budget. Your answer looks fine and is probably cut in half. I have watched a parser happily eat a truncated JSON object.
{"choices":[{"finish_reason":"length","message":{"content":"{\"items\": [1, 2, 3, 4"}}]}
If you do not branch on finish_reason, you are parsing whatever bytes arrived. That is not a feature. That is a coin flip.
Myth 3: Streaming and non-streaming are interchangeable
They are two different code paths. In streaming, finish_reason arrives in the final chunk, not the first. Some servers only report token usage when you ask for it explicitly. Some omit it entirely.
Ask yourself: does my client read the last event? Mine did not, for a week. I was logging None as the finish reason every single time.
Myth 4: JSON mode means my schema
It means valid JSON. It does not mean your keys. It does not mean your types. Validate every response against a schema before it touches business logic. Treat model output like user input, because that is what it is.
Myth 5: Retries are always safe
Retry a failed read, fine. Retry a tool call that already wrote a row, and now you have two rows. I learned this from a duplicate charge, not a blog post.
Add an idempotency key your tool honors. Make the retry decision depend on the operation, not on the exception class.
The artifact: a five-minute capability probe
This script uses only the Python standard library. It assumes a chat-completions-shaped endpoint. Adjust the paths if yours differs.
# probe_capabilities.py — smoke test only, not a benchmark
import json, os, time, urllib.request, urllib.error, concurrent.futures as cf
BASE = os.environ["BASE_URL"].rstrip("/")
KEY = os.environ["API_KEY"]
MODEL = os.environ["MODEL"]
def call(payload, stream=False, timeout=60):
req = urllib.request.Request(
BASE + "/chat/completions",
data=json.dumps(payload).encode(),
headers={"Content-Type": "application/json",
"Authorization": f"Bearer {KEY}",
"Accept": "text/event-stream" if stream else "application/json"},
method="POST")
t0 = time.time()
try:
with urllib.request.urlopen(req, timeout=timeout) as r:
return {"status": r.status, "ms": int((time.time()-t0)*1000),
"raw": r.read().decode("utf-8", "replace")}
except urllib.error.HTTPError as e:
return {"status": e.code, "ms": int((time.time()-t0)*1000),
"raw": e.read().decode("utf-8", "replace")}
def probe_truncation():
p = {"model": MODEL, "max_tokens": 8,
"messages": [{"role": "user", "content": "List ten colors."}]}
r = call(p)
if r["status"] != 200:
return {"probe": "truncation", "ok": False, "why": r["status"]}
ch = json.loads(r["raw"])["choices"]
[0] return {"probe": "truncation", "ok": True,
"finish_reason": ch.get("finish_reason"),
"reports_usage": "usage" in json.loads(r["raw"])}
def probe_tools():
p = {"model": MODEL, "max_tokens": 128,
"messages": [{"role": "user", "content": "Read ./README.md and report its first line."}],
"tools": [{"type": "function", "function": {
"name": "read_file",
"description": "Read a file from disk.",
"parameters": {"type": "object",
"properties": {"path": {"type": "string"}},
"required": ["path"]}}}]}
r = call(p)
if r["status"] != 200:
return {"probe": "tools", "ok": False, "why": r["status"]}
msg = json.loads(r["raw"])["choices"][0]["message"]
return {"probe": "tools", "ok": True,
"returned_tool_calls": bool(msg.get("tool_calls"))}
def probe_json():
p = {"model": MODEL, "max_tokens": 128,
"messages": [{"role": "user",
"content": 'Reply with only JSON: {"status":"ok","count":3}'}]}
r = call(p)
if r["status"] != 200:
return {"probe": "json", "ok": False, "why": r["status"]}
text = json.loads(r["raw"])["choices"][0]["message"]["content"]
try:
obj = json.loads(text)
return {"probe": "json", "ok": True,
"keys_match": sorted(obj) == ["count", "status"]}
except json.JSONDecodeError:
return {"probe": "json", "ok": True, "keys_match": False,
"note": "not parseable as JSON"}
def probe_concurrency(n=2):
p = {"model": MODEL, "max_tokens": 16,
"messages": [{"role": "user", "content": "Say ok."}]}
with cf.ThreadPoolExecutor(max_workers=n) as ex:
codes = list(ex.map(lambda _: call(p)["status"], range(n)))
return {"probe": "concurrency", "ok": True, "statuses": codes,
"saw_429": 429 in codes}
if __name__ == "__main__":
for fn in (probe_truncation, probe_tools, probe_json, probe_concurrency):
print(json.dumps(fn(), sort_keys=True))
Run it like this. Keep the key in the environment, never in the file.
export BASE_URL="$YOUR_ENDPOINT"
export API_KEY="$YOUR_KEY"
export MODEL="$YOUR_MODEL"
python3 probe_capabilities.py
Reading the output: a decision table
Probe resultWhat it meansWhat I do nextreturned_tool_calls: falseTools accepted, not acted onDrop the agent branch, or change endpointfinish_reason: "length" on an 8-token capNormal, expectedKeep the check; raise budget in real callsfinish_reason: null in streamingClient is not reading the final eventFix the parser before anything elsereports_usage: falseNo token accounting from serverCount tokens client-side, or accept blind costkeys_match: falseValid JSON, wrong shapeAdd schema validation at the boundarysaw_429: true at n=2Very low concurrency ceilingQueue work; never fan out blindly
Nothing here measures quality. Nothing here measures speed. Those are different experiments with different controls.
Where I run this
I run probes from a throwaway box, not my laptop. Same reason I do not test migrations on my main database. The probe should not touch anything I care about.
MonkeyCode's free model access and free server option are the two pieces I use for that. The free model access gives the probe something to talk to. The free server option gives it a disposable place to run. Disclosure: This article was prepared as part of MonkeyCode's product outreach.
I am not claiming the free tier is fast, generous, or permanent. I do not know its limits, and I am not going to guess at them. Run the concurrency probe and read your own numbers. That is the whole point of this post.
Limitations and who should skip this
Skip this approach if you need contractual latency SLOs, deterministic output, or guaranteed tool-call behavior. Buy a plan with a written contract. A probe will not save you from a missing SLA.
The corrected mental model
Treat every free endpoint as a set of separate promises. Transport, truncation, tool calls, JSON shape, and concurrency are five different promises. A 200 only confirms the first one.
Which promise broke on your last integration? I keep a one-page probe log per endpoint now. It has saved me two weekends so far.#ai #api #testing #python #software #coding #development #engineering #inclusive #community
200 OK Proves Nothing About Tool Calls: A Free-Model Capability FAQ With a Probe