Examples and counterexamples

What it is and why it exists

An example in a coding request is a concrete observation you want from a particular input and state. It might map a function argument to a return value, an HTTP request to a response, or an event to a state transition. The useful part is not sample syntax; it is the exact behavior the sample commits to.

A counterexample is a case that disproves a tempting but unwanted rule. If “stock 4 with a reorder threshold of 8 means reorder” is the only example, an implementation can hard-code a threshold of 10 and still look correct. “Stock 8 with a threshold of 5 means hold” rejects that generalization.

Counterexamples do not have to be invalid inputs. A valid order just below a discount boundary, an administrator viewing an archived record, or an empty but permitted collection can all be counterexamples to rules that are too broad. Invalid inputs are one important category, but they answer a separate question: how the interface rejects values outside its domain.

The pair belongs inside an API contract when callers depend on it. The example identifies behavior that must occur; the counterexample identifies similar behavior that must not occur. Together they reduce the set of implementations that can plausibly claim to satisfy the request.

This matters with coding agents because a model must fill every gap well enough to produce complete code. It often chooses a familiar default: truthiness for validation, an exclusive boundary, normalization instead of rejection, or the first branch order suggested by prose. A fluent implementation can hide those decisions until a nearby production case behaves differently.

Examples are still finite evidence, not a proof of every input. Ten cases from the same comfortable region may constrain less than two values on opposite sides of a boundary. Selection matters more than volume.

You use this technique when requesting a new function, changing legacy behavior, defining a parser, reviewing generated tests, or explaining a defect. It is especially valuable when ordinary words such as “valid,” “nearby,” “active,” “empty,” or “over” admit several reasonable interpretations.

The goal is not to predict every implementation detail. It is to expose product decisions at the stable observable boundary while leaving internal structure open. A good case rejects a wrong behavior without forcing an irrelevant helper, loop, or data structure.

How it works

Start by writing the request as a behavior rule with named inputs and observable results. Then list the dimensions that can change the answer: numeric ranges, categories, state, permissions, timing, absence, and failure channels. Those dimensions are the search space from which representative examples and counterexamples are selected.

The anatomy of a useful case

Each case needs a reason for existing. Without one, future maintainers cannot tell whether a value is intentional or incidental, and an agent may preserve noise while missing the actual rule.

Names should describe the distinction, not merely repeat an input. “Case 4” and “stock equals 8” make the reader reconstruct the reason. “Active item at the inclusive threshold” states why equality appears.

Expected output must be independent enough to act as a test oracle . A literal value, a stable error type, or a business invariant can provide that independence. Computing the expected value with the same condition intended for production merely duplicates a possible misunderstanding.

Representative examples establish the center

A representative example stands for a meaningful region in the input space. For a reorder rule, one ordinary active item below its threshold is a useful representative. For a parser, one canonical valid string establishes the accepted form and result type.

Choose representatives from business categories, not arbitrary numerical spread. 10, 20, and 30 add little if every value is an ordinary valid quantity. An active item, a discontinued item, and an unknown item state are different because the contract treats them differently.

Positive examples also establish output shape. 25 -> 25 distinguishes a parser that returns a number from one that returns the original string. An HTTP example should likewise name the status, stable fields, and relevant side effects rather than saying only “succeeds.”

Counterexamples cut off unwanted rules

A counterexample is relative to a candidate rule. First say what an agent might infer, then choose the smallest case on which that inference differs from the intended behavior. This makes the case diagnostic instead of merely unusual.

For stock <= reorderAt, the equality case distinguishes an inclusive comparison from a strict comparison. A custom threshold distinguishes a per-item threshold from a hard-coded constant. A discontinued item distinguishes the complete policy from a threshold-only rule.

Nearby cases are powerful because they change one meaningful fact. When stock: 7 and stock: 8 differ at a threshold of 7, unrelated fields remain fixed, so the reason for the changed result is visible. If every field changes, several rules can explain the contrast.

Not every counterexample must be numerically adjacent. For categorical behavior, change one category; for stateful behavior, repeat the operation; for permissions, hold the resource fixed and change the actor or state. The principle is controlled contrast.

Boundaries, partitions, and interactions

Build the case set in a deliberate order:

1. Pick one ordinary valid case for each behavior category.
2. Add exact values on both sides of each inclusive or exclusive boundary.
3. Add empty, absent, malformed, and out-of-range inputs with distinct expected failures.
4. Add a case where two rules apply at once to establish precedence.
5. Add a repeated or concurrent action when history can affect the result.
6. Add a forbidden side effect when returning the right value is not enough.

The fourth step often needs a decision table . Single-condition examples can prove each rule exists while still leaving their interaction ambiguous. One overlapping row can reveal whether “archived” overrides “administrator,” or the other way around.

A forbidden behavior is part of the output. “Return deny and do not emit an audit-success event” says more than “deny.” For a failure case, name whether state remains unchanged, whether retry is safe, and which error channel the caller observes.

Keep unknowns explicit

An omitted case is not an instruction to choose freely. If the product owner has not decided whether whitespace is trimmed, write “unresolved: surrounding whitespace” and ask for a decision. That marker prevents a generated implementation and generated test from silently agreeing on an invention.

Separate current behavior from desired behavior in legacy code. A characterization example may say that "001" currently parses as 1; the new contract may require rejection. Label both states so the agent does not preserve the old behavior as an accidental compatibility promise.

You have enough cases when each important wrong interpretation is rejected by at least one case and each required behavior has a representative. “Enough” is risk-based: authorization and money calculations need more hostile cases than a private display formatter. The set can stay small when every case has a distinct job.

Examples

The examples use Node’s strict assertions, but their main subject is case selection rather than the assertion library. Each file is self-contained and was run with Node v24.14.0; the output shows how each added case removes ambiguity.

Eliminate plausible inventory rules

The first low-stock example fits four implementations. Each counterexample changes one decision-bearing fact, so incorrect candidates disappear for a known reason.

The representative case alone confirms almost nothing about equality, configuration, or lifecycle state. It gives the agent a useful center, but all four rules agree there. Agreement on one point is not evidence that they agree elsewhere.

The equality case kills only strictBoundary; the custom threshold kills only fixedThreshold; the discontinued case kills the remaining threshold-only rule. This is a distinguishing set because every row separates intended behavior from at least one credible mistake.

The file compares candidate implementations to make the narrowing visible. A real prompt does not need to enumerate code candidates, but it should name the misconceptions in prose: inclusive boundary, item-specific threshold, and discontinued-item override.

Specify accepted form and failure categories

“Parse a batch size such as 25” leaves coercion, whitespace, leading zeroes, suffixes, and range errors open. These examples state one canonical input and four counterexamples with caller-visible error types.

import assert from "node:assert/strict";

function parseBatchSize(raw) {
  if (typeof raw !== "string") {
    throw new TypeError("batch size must be a string");
  }
  if (!/^[1-9]\d{0,2}$/.test(raw)) {
    throw new TypeError("canonical decimal string required");
  }
  const value = Number(raw);
  if (value > 100) {
    throw new RangeError("batch size must be at most 100");
  }
  return value;
}

const cases = [
  { input: "25", expected: 25 },
  { input: "001", error: TypeError },
  { input: "25 items", error: TypeError },
  { input: 25, error: TypeError },
  { input: "101", error: RangeError },
];

for (const example of cases) {
  try {
    const actual = parseBatchSize(example.input);
    assert.equal(actual, example.expected);
    console.log(`${JSON.stringify(example.input)} -> ${actual}`);
  } catch (error) {
    assert.ok(error instanceof example.error);
    console.log(`${JSON.stringify(example.input)} -> ${error.name}`);
  }
}

"25" -> 25
"001" -> TypeError
"25 items" -> TypeError
25 -> TypeError
"101" -> RangeError

The leading-zero case rules out permissive numeric coercion even though "001" contains only digits. The suffixed string rules out partial parsing, while the numeric input prevents an agent from accepting multiple types merely because JavaScript makes that convenient.

The two error classes preserve a useful distinction. TypeError means the representation is outside the accepted form; RangeError means the form is valid but the value exceeds the supported domain. If callers do not use that distinction, one stable validation error might be a simpler contract.

The set still leaves "1" and "100" implicit boundary cases. For a production parser, add them, plus the below-range form "0". Counterexamples make remaining gaps visible; they do not automatically close them.

Cover interactions with a decision table

Role, ownership, and archive state each sound simple alone. The counterexamples below activate them together so branch precedence becomes observable.

import assert from "node:assert/strict";

function documentPermission({ role, isOwner, archived }) {
  if (!new Set(["admin", "member", "viewer"]).has(role)) {
    throw new TypeError("unknown role");
  }
  if (archived) return role === "admin" ? "view" : "deny";
  if (role === "admin") return "edit";
  if (role === "member" && isOwner) return "edit";
  return "view";
}

const cases = [
  {
    name: "admin edits an active document",
    input: { role: "admin", isOwner: false, archived: false },
    expected: "edit",
  },
  {
    name: "member edits an owned active document",
    input: { role: "member", isOwner: true, archived: false },
    expected: "edit",
  },
  {
    name: "member only views another active document",
    input: { role: "member", isOwner: false, archived: false },
    expected: "view",
  },
  {
    name: "viewer ownership does not grant editing",
    input: { role: "viewer", isOwner: true, archived: false },
    expected: "view",
  },
  {
    name: "archive removes admin editing",
    input: { role: "admin", isOwner: true, archived: true },
    expected: "view",
  },
  {
    name: "archive blocks an owning member",
    input: { role: "member", isOwner: true, archived: true },
    expected: "deny",
  },
];

for (const example of cases) {
  const actual = documentPermission(example.input);
  assert.equal(actual, example.expected, example.name);
  console.log(`${example.name}: ${actual}`);
}

admin edits an active document: edit
member edits an owned active document: edit
member only views another active document: view
viewer ownership does not grant editing: view
archive removes admin editing: view
archive blocks an owning member: deny

The first two rows establish ordinary edit paths. The viewer-owner row is a counterexample to “owners can edit,” because ownership grants editing only to members. The archived rows establish that archive state overrides editing rights, while administrators retain read access.

This is not the complete Cartesian product of three roles, two ownership states, and two archive states. It is a selected matrix in which each omitted row should follow from a reviewed rule. Generate the full product when risk is high or the rule implementation is difficult to inspect.

Authorization needs enforcement below the user interface and tests at the real policy boundary. These cases specify the decision function, but they do not prove that every route calls it or that data reads enforce the same scope.

Pitfalls

Showing only one happy path

Fix: write down two plausible wrong rules before adding cases. Keep the representative example, then add the smallest counterexample that makes each wrong rule produce a different observable result.

Adding many correlated examples

Fix: make a dimension table and mark which values each case covers. Replace redundant rows with boundary pairs, category changes, and at least one overlap between rules.

Calling every invalid input a counterexample

Fix: label cases as representative valid behavior, valid counterexample, or invalid-input rejection. For rejected inputs, specify the error channel and forbidden state changes separately.

Letting the implementation create the oracle

Fix: approve literal outcomes and error categories before generation, or derive them from an independent policy source. Mutate a boundary or precedence rule and confirm that at least one case fails.

Overspecifying incidental details

Fix: assert results, stable failures, state transitions, and required effects at a public boundary. Mention an internal interaction only when that interaction is itself the contract, such as charging at most once.

Treating omitted cases as defaults

Fix: add an unresolved-cases list to the prompt and require questions before implementation. After decisions are made, convert each consequential answer into a named case or explicit rule.

Further reading

• GitHub Copilot documentation: Prompt engineering
• Cucumber documentation: Gherkin reference
• Node.js 24 documentation: Strict assertion mode
• Testing Library: Guiding principles
• Google Testing Blog: Change-detector tests considered harmful