Production Claude Application Capstone
Ship a production Claude application. Cover environment config, observability, safety layers, deployment patterns like canary and feature flags, and incident response runbooks, then bring every technique from the AI Engineer path together in a portfolio-grade end-to-end capstone.
View badge details
About This Course
Ship a production Claude application. Environment config, observability, safety layers, deployment patterns (canary, feature flags), incident response runbooks, and the final end-to-end Orion assistant capstone that brings together every technique from CLD-AI-101 through CLD-AI-111.By the end of this course you will have built, deployed, and instrumented a production-grade Claude application for Orion Analytics one you can point at as your portfolio piece and reference architecture.
Course Curriculum
10 Lessons
Production readiness checklist
By the end of this lesson you will know the 12-item production-readiness checklist every Claude app should pass before ship: secrets management, eval-gated deploys, cost caps, structured logging + trace_id, retry-with-backoff, safety layer (injection + PII + refusal), feature flags for model versions + prompts, alerts on 429 + cost + refusal-rate deltas, runbooks for common incidents, docs. Anchors every remaining CLD-AI-112 hands-on lab and the final Orion capstone.
Build the app shell with config and healthz - Lab Exercises
By the end of this hands-on lab you will have wired the Orion Analytics assistant baseline — a typed config loaded from env vars (ANTHROPIC keys injected by the Claude API proxy, plus feature flags), a healthz endpoint that reports config validity + Anthropic reachability, and a first Claude call gated on healthz-green so the app refuses to serve requests when misconfigured. Uses the PythonAI container + Claude API proxy — the app-shell shape you'll layer safety, observability, and feature flags into across L4-L10.
Observability — structured logs, traces, metrics
By the end of this lesson you will know the observability discipline every production Claude app needs — a stable log-record schema (trace_id, request_id, agent_role, tokens_in, tokens_out, latency_ms, cost_usd, cache_hit, refusal), trace_id propagation across every downstream call, per-request metric emission for dashboards, and the guiding rule 'what you can't measure you can't ship'. Anchors the L4 hands-on where you wrap the Anthropic client with the emitter.
Instrument Claude calls with structured logs - Lab Exercises
By the end of this hands-on lab you will have wrapped the Orion Anthropic client so every Claude call emits one structured JSON log record carrying trace_id, latency_ms, tokens_in, tokens_out, computed cost_usd, cache-read/write hit counters, and a refusal signal — ready to ship into OpenTelemetry, Datadog, or Splunk. Uses the PythonAI container + Claude API proxy. Verifies every required field is present on a real /chat call before moving on.
Safety layer — injection defense, PII, content filters
By the end of this lesson you will know the four-layer safety stack every production Claude app needs — ingress prompt-injection detection (before Claude sees the input), PII redaction before logging (so trace records don't leak SSNs or emails), refusal patterns for out-of-scope requests, and a post-hoc content-safety check on Claude's output — plus where each layer catches which class of attack. Sets up L6 hands-on where you wire all four layers into the Orion app.
Add the safety layer to the Orion app - Lab Exercises
By the end of this hands-on lab you will have layered the full safety stack onto the Orion assistant — ingress prompt-injection detection that blocks obviously-adversarial inputs before Claude sees them, PII redaction on log records (SSN + email + credit-card patterns), and output PII scrubbing before responses reach the user — then verified three test cases (clean, injection, PII-in-output) behave correctly end-to-end. Uses the PythonAI container + Claude API proxy.
Deploy safely — feature flags, canary, rollback
By the end of this lesson you will know how to deploy a Claude app safely — hash-based feature flags that route a fraction of traffic to a canary variant (5% → 25% → 100%), per-variant metric aggregation (latency, cost, refusal rate) so you can compare before promoting, instant rollback via a flag flip when the canary breaches thresholds, and the promote/rollback decision rule as a machine-parseable scorecard. Sets up L8 hands-on where you ship a real canary.
Ship a canary with metric-gated promotion - Lab Exercises
By the end of this hands-on lab you will have routed 5% of simulated Orion assistant traffic to a canary prompt variant using a stable hash-based feature flag, emitted per-variant metrics (p95 latency, refusal rate, cost), and applied a deterministic promote/rollback rule — canary promotes only if p95 delta < 20% and refusal rate delta < 2pp against baseline. Uses the PythonAI container + Claude API proxy. Proves canary deployment with real numbers.
Incident response — dashboards + runbooks
By the end of this lesson you will know the five most common Claude-app incidents and how to respond: cost spikes (a runaway loop or a bad prompt burning tokens), refusal-rate spikes (a safety-layer overshoot after a prompt change), 429 storms (rate limit exceeded, backpressure your ingress), prompt-injection outbreaks (adversarial traffic surge, tighten ingress filter), upstream Anthropic outages (fail closed vs degrade). Each incident gets a checklist runbook. Sets up the L10 capstone.
Orion end-to-end assistant capstone - Lab Exercises
By the end of this hands-on final capstone you will have shipped the complete production-shaped Orion Analytics assistant — prompt caching (from CLD-AI-111), the four-layer safety stack, model routing (Haiku classify + Sonnet generate), structured JSON logs with trace_id, hash-based feature flags for canary rollouts, PII scrubbing on ingress and egress, and end-to-end traces validated by an audit-trail check. Uses the PythonAI container + Claude API proxy — the portfolio piece for the whole Claude AI Engineer path.