Silverberry AI Engineering Guide

Silverberry's AI Engineering Guide organizes the discipline that sits between a prompt and a reliable production system into ten areas, all laddering up to four levers: latency, quality, cost, and reliability. ### The Ten Disciplines 1. Serving architecture — Inference splits into prefill (compute-bound) and decode (memory-bandwidth-bound). Continuous batching and paged attention keep the GPU saturated and eliminate KV cache fragmentation. 2. Caching strategy — Prompt caching is mathematically safe; semantic caching lifts hit rate but risks stale answers. Manage the KV cache like a memory hierarchy across VRAM, RAM, and NVMe. 3. Model compression — Quantization for cost, speculative decoding for latency, distillation only for stable tasks. Four-bit can collapse on multi-step reasoning, so benchmark on your own distribution. 4. Output reliability — Generate, parse, validate, repair. After bounded retries, cascade to simpler schemas or human review. Treat tool calls like public APIs with idempotency keys and timeouts. 5. Agent systems — Nest budgets across run, loop, and per-tool. A no-progress detector catches agents stuck repeating identical states. Model routing sends easy traffic to small models. 6. Retrieval-augmented generation — A seven-stage pipeline where most hallucinations are actually retrieval failures. See the RAG pipeline deep dive for where quality breaks. 7. Evaluation — Golden sets, regression, adversarial, LLM-as-judge, human evals. Watch for calibration drift, position bias, and Goodhart's Law. 8. Observability — Structured traces with spans for retrieve, rerank, and LLM calls. Cost attribution per user journey turns totals into actionable numbers. 9. Safety and isolation — Five-layer prompt-injection defense and tenant IDs in every cache key to prevent cross-tenant leakage. 10. The four-way tradeoff — Every choice moves latency, quality, cost, and reliability simultaneously. No row is all green. ### Six Failure Modes to Design For Mature teams treat hallucinated tool calls, malformed JSON, stale retrieval, runaway agents, silent eval regression, and cross-tenant context contamination as design inputs from day one — not surprises to firefight later. For a faster on-ramp, see how to ship your first production LLM feature in a weekend or the LLM cost control FAQ.