AI and LLM engineering interview bank

Use a direct model call when the operation is one request and the surrounding control flow is ordinary application code. Use a runnable sequence, parallel mapping, or branch when your program can determine the steps and you want consistent invocation, composition, and tracing. Use create_agent only when the model must choose tools or decide how many iterations a task requires. Each move adds flexibility and a larger failure surface, so start with the smallest control structure that expresses the behavior.

Name the concrete value crossing every edge: mapping, prompt value, message, string, or domain object. Invoke the smallest upstream prefix and inspect its result before adding the next component. Check required dictionary keys and whether a parser intentionally converts a message into text or structured data. Schema and graph inspection can help, but they do not prove provider content behavior. Keep a contract test at each unstable boundary so an integration upgrade fails close to the component that changed.

The schema checks argument shape, not authority or business policy. The tool must authenticate runtime identity, authorize the requested resource and action, constrain paths and amounts, and avoid exposing secrets in results or traces. Side-effecting tools also need idempotency keys and durable outcome records because model, network, or graph retries can repeat a call. Treat model-selected arguments as untrusted input, keep consequential tools on narrow allowlists, and place human approval before actions whose cost or reversibility requires it.

When stop_reason is tool_use, enumerate every tool_use block rather than assuming the first block is the only call. Resolve names from an allowlist, validate arguments, authenticate the principal, authorize the resource and action, and execute the handler. Append the assistant content unchanged, then add one user-role tool_result for each tool_use_id, including explicit error results when needed. Send the same tool definitions on the continuation request. Side-effecting handlers also need durable idempotency keys and outcomes, because model, network, or process retries can otherwise repeat an action.

Inventory every imported symbol before changing behavior. LangChain 1.x centers the main package on create_agent, while many legacy chains and helpers moved to langchain-classic; do not solve each missing import by blindly adding that package. Replace fixed legacy chains with runnable composition where it clarifies data flow, and replace older agent constructors with create_agent. Revisit memory as thread-scoped checkpointed state rather than a process-global history object. Finally, run provider, tool-call, streaming, and persistence tests because an import-clean migration can still change runtime behavior.

Each request contains the conversation state needed to generate the next assistant message; the API does not retain an application chat session between calls. The application therefore owns persistence, history trimming, tenant isolation, and replay. For a normal next turn, resend the relevant user and assistant messages in order. During client tool use, preserve the assistant content containing every tool_use block and follow it with user-role tool_result blocks whose IDs match. Statelessness makes behavior auditable, but only if the stored history is complete and correctly scoped.

The content array is a tagged union, not a text field. One response can contain text, one or more tool_use blocks, or blocks from other enabled capabilities, so code should dispatch every block by type and tolerate documented extensions. stop_reason describes the turn-level outcome: end_turn is normal, max_tokens can mean incomplete output, and tool_use requires another application-controlled round trip. Looking at content alone misses termination state; looking only at stop_reason misses the work encoded in individual blocks. Unknown values should produce safe diagnostics rather than an unsafe default.

Retry only transient HTTP failures with bounded backoff, accounting for retries the SDK already performs. Authentication and invalid-request errors require a fix, not another identical call. A stream can fail after HTTP 200, so partial text stays incomplete until message_stop; a new request is a new generation and must not be concatenated onto the old one as if it resumed. Tool execution is a separate transaction boundary. Persist its call ID, idempotency key, and outcome before retrying, then query the existing result after an ambiguous timeout to avoid duplicate effects.

Parse each protocol event incrementally and apply backpressure instead of buffering the complete response. Keep received text provisional until the documented terminal event arrives, then inspect its stop reason and usage or timing fields. A connection that closes first produces incomplete output, even if the text looks fluent. Bound connection and read timeouts and retry only under an explicit policy. A retry starts a new generation and cannot be concatenated as a continuation of the old stream. Before any side effect, validate the completed output and use an application-level idempotency key.

Keep streamed text provisional until an explicit terminal event reports an acceptable response status. A disconnect after HTTP 200 is still incomplete, and a retry starts a new generation that must not be concatenated as if it resumed the old stream. Classify errors before retrying: temporary throttling may honor Retry-After, while authentication, invalid requests, and spend limits need remediation. Count SDK retries inside the total attempt budget. After validation, commit each business side effect through a separate authorized transaction with a durable idempotency key and recorded outcome.

The boundary starts before the provider request and ends after an application decision. It builds a versioned request from trusted instructions and clearly identified untrusted data, sets model and output limits, and records correlation metadata. On return, it checks completion state, parses content, validates schema and domain constraints, and chooses an explicit fallback. Permissions and side effects remain outside the model adapter: application code authenticates the caller, authorizes actions, and handles idempotency. Logs retain approved metadata and redacted diagnostics rather than complete sensitive prompts.

A schema proves that data has the expected shape; it does not prove that the facts are correct, the classification is appropriate, or the requested action is authorized. A valid amount can still exceed a business limit, and a valid account ID can belong to another tenant. Treat schema validation as one gate. Follow it with allowlists, domain invariants, provenance checks where claims need evidence, and authorization based on runtime identity. For consequential actions, require approval and an idempotency key rather than asking the model to confirm itself.

Separation records provenance and helps the model distinguish intent, but both instruction and data channels still enter one model context. A malicious user message or retrieved document can influence generation despite labels or delimiters. The dependable controls sit outside that persuasion problem: limit what the model can select, validate outputs, authorize every resource and action at execution time, and give tools least privilege. High-impact actions need an independent policy or human gate. Test direct and indirect injection cases, including hostile content from documents and tool results.

First preserve the old configuration as a baseline on a versioned set of representative, boundary, and adversarial cases. Change one major variable so any regression is attributable. Run both configurations, keep raw and parsed outputs, and inspect case-level differences rather than only an aggregate score. Domain owners should resolve disputed expected behavior. After offline acceptance, use a limited rollout to watch validation failures, human overrides, fallbacks, and user corrections. Keep the previous configuration available so a production regression has a tested rollback path.

Choose a split boundary that matches deployment, such as user, conversation, source document, or time, and keep each group on one side. Detect exact and near duplicates before splitting. Fit vocabularies, IDF weights, feature selection, calibration, and every learned preprocessing step on training data only; validation and test data may only be transformed. Use validation data for routine model and threshold choices. Keep the final test set read-only, record every use, and investigate an implausibly strong baseline before trusting a complex model.

Accuracy gives every example equal weight, so a majority-only predictor can look acceptable while never detecting a rare but costly class. Report the confusion matrix and per-class precision, recall, F1, and support. Macro averages give each class equal weight; micro averages pool decisions and are usually dominated by common classes. Select thresholds on validation data using the cost of false positives, false negatives, and abstention. Then inspect failed examples in slices such as language, channel, length, and time period, because one average cannot locate the failure.

Use Pipeline to validate a supported task quickly when its preprocessing, model call, and standard postprocessing match your needs. It returns task-oriented Python values and keeps setup small. Use AutoTokenizer with a task-specific AutoModel class when you need explicit batching, tensor access, custom pooling, device placement, or postprocessing. In either case, provide the repository ID, task, and commit revision explicitly. Pipeline is a convenience layer over the same artifacts, not a separate model format, and successful execution does not validate label semantics or business policy.

Resolve the selected branch or release tag to a full commit hash and pass that revision to every tokenizer, processor, configuration, and model load. Build from an empty cache, instantiate the exact target class, then repeat startup offline with local_files_only enabled. Record the repository hash alongside locked Transformers, huggingface_hub, tensor-backend, and Python versions, plus the device and dtype policy. Save an artifact manifest and evaluation result with the application release. A pinned model commit alone is insufficient because loader code and numerical kernels can still change behavior.

Start with the model card: intended use, limitations, training context, evaluation evidence, and license. Confirm that the repository task and architecture match your loader, then inspect the pinned revision’s file list, sizes, weight formats, and any custom Python. Prefer reviewed Safetensors weights and leave trust_remote_code disabled unless pinned code has been audited and isolated. Check gated-access obligations and test the model on representative, boundary, and adversarial inputs from your domain. Popularity and download counts help discovery, but they are neither a security review nor evidence that the model meets your requirements.

Supply a narrowly scoped read token through HF_TOKEN or managed secret injection, never source code or request data. Remote Hub authorization protects download access, but a downloaded snapshot is then governed by local filesystem permissions. A shared cache can expose restricted artifacts across service identities or tenants, so set explicit ownership and mount policy. Separate token rotation from cache removal because revoking a token does not erase local files. Logs should retain repository ID, commit hash, and request ID while excluding credentials and sensitive paths. Test both authorized cold-cache download and credential-free offline startup.

Choose local inference when the workload has a firm data-residency or offline requirement, a model that passes your domain evaluation can run on available hardware, and the expected load justifies operating that hardware. Compare it with a hosted API on the same quality cases, latency targets, and full cost model. Local is a poor default when you need the strongest hosted capability, rapid elasticity, or a provider service-level agreement. It also transfers patching, capacity, monitoring, access control, and recovery to your team, so include those duties in the decision.

Start with parameter count times bits per weight as a lower bound, then inspect the actual artifact and runner metadata. Add memory for runtime buffers, the computation graph, drivers, and the KV cache. Cache demand changes with model architecture, context length, cache type, batch size, and concurrent sequences, so file size alone is not enough. Run the exact artifact, runner version, backend, maximum permitted input and output, and target concurrency on the real machine. Record peak system and accelerator memory, offload behavior, load failures, and latency with operating headroom.

Record the model source and immutable revision or file digest, quantization type, tokenizer, chat template, runner version, launch arguments, hardware backend, and relevant driver versions. Attach the domain evaluation set and its results to that exact combination. A mutable model tag is insufficient because it may later resolve to different bytes, and pinning weights alone does not pin template or runner behavior. Build once from an empty cache, verify startup with network access blocked, and retain the reviewed artifact bundle. Rollback should restore the tested combination rather than search for a similarly named model.

Start with the decision the product must make, then define the input unit and output shape: document labels, token labels, source spans, ranked candidates, or generated text. Specify the label set, abstention path, span-boundary convention, and business cost of each error. Write an annotation guide and test whether independent annotators can apply it consistently. Finally, choose metrics and evaluation slices that match the decision. Model selection comes later; a stronger model cannot repair overlapping labels or an output nobody can verify.

Normalization is a task-specific view, not a lossless replacement. NFKC, case folding, whitespace compression, or punctuation removal can change content and string length. A model span measured on that view may no longer slice the source correctly, and the source is still needed for display, audits, and relabeling. Store the original unchanged, version each transformation, and use a tokenizer-provided offset map or maintain one explicitly. Across services, state whether offsets count bytes, Unicode code points, UTF-16 code units, grapheme clusters, or model tokens.

Use the Responses API as the direct model-call boundary for a new text-generation application, especially when the result may include reasoning items, tool calls, or streamed events. The API owns model inference and the documented response protocol. Your application still owns authentication of its users, authorization, input limits, tenant isolation, response validation, business policy, and side-effect control. Keep those controls around a small provider adapter. A successful completed response is model output, not permission to publish, pay, delete, or update a record.

The output array is a typed sequence, not a guaranteed one-message envelope. A reasoning item or tool call can precede text, a response can contain multiple message items, and message content can contain several block types. For a narrow text-only path, first require an acceptable terminal status and then use the SDK output_text aggregation helper. For protocol-level handling, iterate every output item and content block by type, preserve supported non-text items, and route unknown variants safely. Incomplete or failed responses need an explicit fallback rather than best-effort indexing.

Treat response and conversation IDs as tenant-scoped resource references, not opaque values a client may attach freely. A continuation using previous_response_id receives earlier response context, but it must send the current trusted instructions again. Choose response chains or Conversation objects only after deciding retention, deletion, and data-residency policy, because their persistence differs. Track context growth and define a tested reject, compact, summarize, or restart path. Keep canonical business state in your own database; provider-side conversation state is inference context, not the sole record of an approved decision.