# AI and LLM engineering rules

Follow these CodeWiki-derived rules when you work in this project.

- Generated code often reads `response.content[0].text` directly.
  Why: It fails or silently drops content when the first block is a tool call, the response has several text blocks, or an enabled feature introduces another block type.
  Source: [Claude API](https://codewiki.com/ai/claude-api/)
- Sending only the latest user text removes earlier context.
  Why: In a tool loop, sending only `tool_result` also drops the assistant `tool_use` block that the result must match.
  Source: [Claude API](https://codewiki.com/ai/claude-api/)
- `tool_use.input` remains untrusted even when it passes JSON Schema.
  Why: A model choosing `refund_order` and producing a valid order ID doesn't prove the current user may issue the refund.
  Source: [Claude API](https://codewiki.com/ai/claude-api/)
- An SDK or proxy can retry timeouts, rate limits, and server errors.
  Why: If the same loop also repeats a payment, email, or file deletion, network uncertainty can become a duplicated side effect.
  Source: [Claude API](https://codewiki.com/ai/claude-api/)
- Old examples may wait for `data: [DONE]` or concatenate only `text_delta`.
  Why: Current Messages SSE ends with `message_stop` and may interleave `ping`, unknown events, or an in-stream `error`.
  Source: [Claude API](https://codewiki.com/ai/claude-api/)
- Updating `anthropic-version` doesn't upgrade the model, and copying an old model ID doesn't select a newer protocol.
  Why: Scattering both values through business code turns migration into an unauditable search-and-replace operation.
  Source: [Claude API](https://codewiki.com/ai/claude-api/)
- Writing only `from_pretrained("org/model")` resolves the repository's default branch on a cache miss, so the same deployment code can load different artifacts after an update.
  Source: [Hugging Face](https://codewiki.com/ai/huggingface/)
- Do not assume this is safe: `pipeline("text-classification")` can select a default model, but that choice is not an application dependency contract and can trigger an unexpected download.
  Source: [Hugging Face](https://codewiki.com/ai/huggingface/)
- Generated code sometimes loads a tokenizer from one repository and a model from another, or pins a revision for only one of them.
  Source: [Hugging Face](https://codewiki.com/ai/huggingface/)
- Adding `trust_remote_code=True` to bypass an unknown-architecture error allows custom Python from the repository to execute inside the loading process.
  Source: [Hugging Face](https://codewiki.com/ai/huggingface/)
- Putting `token="hf_..."` in an example, exception, or build log exposes repository credentials to version history and logging systems.
  Source: [Hugging Face](https://codewiki.com/ai/huggingface/)
- Do not assume this is safe: one successful load on a development machine may reuse old files scattered through its cache; it does not prove that a new environment has one complete, consistent snapshot.
  Source: [Hugging Face](https://codewiki.com/ai/huggingface/)
- A long pipe can connect components whose nominal runnable interfaces hide incompatible values, such as an `AIMessage` flowing into a function that expects a string.
  Source: [LangChain](https://codewiki.com/ai/langchain/)
- Generated code often copies `LLMChain`, `ConversationBufferMemory`, `initialize_agent`, or `create_react_agent` imports from older tutorials into a LangChain 1.x project.
  Source: [LangChain](https://codewiki.com/ai/langchain/)
- Do not assume this is safe: calling `.stream()` or `.batch()` does not guarantee token-level streaming or one provider-side batch request for every runnable.
  Source: [LangChain](https://codewiki.com/ai/langchain/)
- Do not assume this is safe: a module-level dictionary of message histories can leak one user's context into another session and disappears when the process restarts.
  Source: [LangChain](https://codewiki.com/ai/langchain/)
- Retrying an entire agent after a timeout can repeat a tool action even when the first payment, email, or ticket creation succeeded remotely.
  Source: [LangChain](https://codewiki.com/ai/langchain/)
- Swapping a model class can preserve method names while changing tool selection, structured-output support, content blocks, token usage, and error behavior.
  Source: [LangChain](https://codewiki.com/ai/langchain/)
- Do not treat fluent output as a trusted fact or authorized instruction.
  Why: A model can invent fields, omit qualifications, or generate an action the application never allowed.
  Source: [LLM application basics](https://codewiki.com/ai/getting-started/)
- Concatenating user text, retrieved documents, or tool results into privileged instructions.
  Why: Prompt injection in that content can steer model output, and delimiters don't form a security boundary.
  Source: [LLM application basics](https://codewiki.com/ai/getting-started/)
- Testing only a few happy cases written by the team.
  Why: Empty text, mixed intent, language changes, long content, and adversarial sentences from real traffic quickly break that demo.
  Source: [LLM application basics](https://codewiki.com/ai/getting-started/)
- Do not assume this is safe: changing a prompt and model without recording versions, or changing both at once.
  Why: When behavior moves, the team cannot isolate the cause or roll back reliably.
  Source: [LLM application basics](https://codewiki.com/ai/getting-started/)
- Logging complete prompts, responses, and user data for debugging.
  Why: Logs then accumulate personal information, business data, access tokens, and secrets the model may have repeated.
  Source: [LLM application basics](https://codewiki.com/ai/getting-started/)
- Do not assume this is safe: "Runs on my machine" does not automatically mean "data never leaves the machine." A runner may support cloud models or network features, while application logs and system backups may copy prompts and responses elsewhere.
  Source: [Local large language models](https://codewiki.com/ai/local-llms/)
- Selecting a quantization by parameter count or filename alone.
  Why: Architectures, templates, quantization methods, and runner backends can differ at the same parameter count. Fitting in memory proves neither quality nor latency.
  Source: [Local large language models](https://codewiki.com/ai/local-llms/)
- Do not treat quantized weight size as total memory demand.
  Why: Long contexts, concurrent slots, runtime buffers, and partial CPU offload all change memory use and latency. Near the capacity limit, jitter or load failure becomes likely.
  Source: [Local large language models](https://codewiki.com/ai/local-llms/)
- Do not treat a clean stream close as a complete generation, or concatenating text while discarding the final `done_reason`.
  Why: Length truncation, service errors, and mid-stream disconnects can then send half an answer downstream.
  Source: [Local large language models](https://codewiki.com/ai/local-llms/)
- Presenting a warm-cache, single-request, short-prompt demonstration as production performance.
  Why: Initial loading, long-prompt prefill, request queues, and CPU offload can each be the bottleneck, which one average tokens/s figure hides.
  Source: [Local large language models](https://codewiki.com/ai/local-llms/)
- Do not treat "cleaning" as lossless removes signals the task may need.
  Why: A stop-word list can delete negation, punctuation filters erase questions and sentence boundaries, and NFKC folds compatibility characters. Fix: Keep the source text and store normalization as a derived field. Run an ablation for each step, then put the chosen normalization form and order in code shared by training and inference.
  Source: [Natural language processing](https://codewiki.com/ai/nlp/)
- Do not assume this is safe: normalizing first and applying the resulting model spans to the source produces wrong highlights or truncation.
  Why: Case folding, compatibility normalization, and whitespace compression can all change length. Fix: Use a tokenizer that returns source-text offsets, or preserve an explicit position map during transformation. Cross-service contracts must name the coordinate unit and end-position rule. Test combining characters and emoji.
  Source: [Natural language processing](https://codewiki.com/ai/nlp/)
- Fitting a vocabulary, IDF weights, feature selector, or normalization statistics on all data before creating the test split causes data leakage.
  Why: Row-level random splits also leak near-duplicate texts, users, or sessions. Fix: Make exclusive groups by user, source, or time first, then fit transformations on training data only. Run near-duplicate detection before the split and lock the final test set as read-only data.
  Source: [Natural language processing](https://codewiki.com/ai/nlp/)
- Reporting only aggregate accuracy can hide complete failure on a minority class.
  Why: Token-level scoring for sequence labels can also count a large number of correct `O` labels while missing every important entity. Fix: Report per-class precision, recall, F1, and support. For spans, use exact-boundary scoring plus any task-approved partial rule, and inspect slices by language, length, and source.
  Source: [Natural language processing](https://codewiki.com/ai/nlp/)
- Using different tokenizer versions, vocabularies, or special-token settings in training and inference silently changes input ids.
  Why: The code can still run even though its outputs no longer correspond to the learned representation. Fix: Release model weights, tokenizer files, normalization configuration, label map, and decoding thresholds as one artifact. Check an immutable revision or digest on load and run an end-to-end smoke test with fixed examples.
  Source: [Natural language processing](https://codewiki.com/ai/nlp/)
- Generated code often puts `OPENAI_API_KEY` in a frontend environment file or calls OpenAI directly from the browser.
  Why: A build tool can include a value that looks like an environment variable in the public JavaScript bundle, where any visitor can extract and abuse it.
  Source: [OpenAI API](https://codewiki.com/ai/openai-api/)
- `response.output[0].content[0].text` only happens to work for some simple responses.
  Why: Reasoning items, tool calls, multiple messages, or future output types can change the order and shape.
  Source: [OpenAI API](https://codewiki.com/ai/openai-api/)
- Passing `previous_response_id` supplies context from the earlier response, but the preceding `instructions` don't automatically enter the next turn.
  Why: Generated code that sends only new user text can silently lose product rules and output constraints.
  Source: [OpenAI API](https://codewiki.com/ai/openai-api/)
- A 429 can mean a temporary request-rate limit, but it can also mean exhausted credit or a spend limit.
  Why: Authentication failures, invalid parameters, and exhausted budgets don't recover when an identical request is resent; blind backoff only adds traffic and delay.
  Source: [OpenAI API](https://codewiki.com/ai/openai-api/)
- An SDK, proxy, or job runner can retry a model request.
  Why: If the same retry block also charges a card, sends mail, or writes a database, a temporary transport failure can duplicate a side effect.
  Source: [OpenAI API](https://codewiki.com/ai/openai-api/)
- Receiving HTTP 200 or several text deltas doesn't prove that generation ended successfully.
  Why: The network can disconnect before `response.completed`, and the API can also send an error event within the stream.
  Source: [OpenAI API](https://codewiki.com/ai/openai-api/)