Agentic AISystem Design.

DesignCompare deterministic workflows, retrieval and agents. Define service-level objectives, allowed actions, failure costs and a baseline before choosing a framework.

BuildMap an enterprise incident-triage assistant: users, evidence sources, tools, approval boundary and fallback path.

ProveDefend one non-agent alternative and specify measurable task success, latency and cost budgets.

DesignTrace a large language model (LLM), typed state, structured outputs and tool calls. Separate model proposals from host-controlled execution and stop conditions.

BuildImplement a minimal Python agent loop with validated arguments, a step budget and a deterministic fake-model adapter.

ProveMalformed tool arguments and repeated calls terminate safely without executing an unauthorized action.

DesignDesign narrow tools with authorization, deadlines and idempotency keys. Distinguish read retries from side-effect retries.

BuildAdd read-only incident lookup and a simulated change-request tool with duplicate suppression.

ProveReplaying an identical request creates exactly one simulated change request; an unauthorized request creates none.

DesignDesign retrieval-augmented generation (RAG): ingestion, chunk identity, hybrid search, reranking and citations. Enforce access filters before retrieval.

BuildIndex synthetic runbooks for two tenants and compare lexical and embedding retrieval on a small labeled query set.

ProveReport retrieval relevance and verify that cross-tenant documents never enter the candidate set.

DesignAssemble evidence, instructions and tool outputs with provenance. Compare truncation, summarization and selective retrieval; handle conflicting and missing evidence.

BuildBuild a token-budgeted context assembler and inject outdated and contradictory runbooks.

ProveEvery recommendation cites available evidence or abstains; document what information summarization loses.

DesignSeparate execution state, conversation context and durable user memory. Define scope, expiry, consent, correction and deletion semantics.

BuildAdd scoped memory with timestamps and explicit writes; replay a corrected user preference and an expired incident fact.

ProveA correction supersedes the older value, deletion persists, and another tenant cannot retrieve it.

DesignModel conditional branches, checkpoints and deterministic routing with LangGraph. Keep business invariants outside model prompts.

BuildMove the baseline into a typed graph with evidence collection, proposal and review states.

ProveA persisted checkpoint resumes at the intended node without losing the approved state.

DesignCompare fixed plans, dynamic planning and parallel evidence gathering. Bound fan-out, recursion, deadlines and shared-state mutation.

BuildRun independent log and runbook lookups concurrently, then simulate one slow or unavailable source.

ProveThe run respects its deadline and produces a partial-evidence response with an explicit limitation.

DesignCompare supervisor, handoff and worker patterns against a single-agent baseline. Inspect coordination cost, duplicated work and failure propagation.

BuildEvaluate a two-worker variant on the same fixtures, then retain or reject it using measured task quality, latency and cost.

ProveSubmit a trade-off table; a more complex architecture is accepted only with supporting evidence.

DesignUse the Model Context Protocol (MCP) for tool discovery and invocation. Separate protocol transport, identity, authorization and tool semantics.

BuildExpose a read-only incident tool through a local MCP server and validate responses at the client boundary.

ProveUnknown tools, invalid responses and unavailable servers produce explicit recoverable errors.

DesignDesign human-in-the-loop approval, least privilege and audit trails. Bind approval to exact action arguments, identity and expiry.

BuildPause a proposed change, approve its immutable payload and attempt to alter it before execution.

ProveChanged or expired approvals cannot execute; the audit trail distinguishes proposal, approval and execution.

DesignTreat retrieved documents and tool output as untrusted. Combine policy enforcement, network restrictions, sandboxing and secret isolation; prompts alone are not a security boundary.

BuildInject hostile runbook text into a synthetic environment and test tool allowlists and simulated egress controls.

ProveThe protected action is blocked by application controls even if the model follows the hostile text.

DesignSeparate unit, tool-contract, retrieval and end-to-end evaluations. Define development and held-out sets, representative failure cases and annotation criteria.

BuildCreate a versioned incident fixture set with required evidence, allowed actions and explicit failure labels.

ProveA known-bad implementation fails an exact assertion; held-out fixtures are not used to tune prompts.

DesignCombine deterministic checks with human review and model judges. Inspect disagreement, rubric leakage and correlated errors.

BuildCompare judge decisions with a small human-labeled sample and trace a failure back to retrieval, planning or execution.

ProveReport agreement and disagreements without presenting a small sample as proof of reliability.

DesignMeasure task success, per-stage latency, token usage and cost under equivalent workloads. Compare model routing, caching and context strategies.

BuildRun matched baseline and candidate experiments, including cache invalidation and a changed evidence source.

ProvePublish actual measurements with workload and sample size; reject gains that violate correctness or freshness.

DesignSeparate request handling from durable workers. Compare queues, workflow engines and database checkpoints; reason about at-least-once delivery.

BuildPersist a run and an outbox event atomically, then replay a worker after a simulated crash.

ProveNo committed work is lost and duplicate delivery cannot duplicate the simulated external effect.

DesignBound queues and worker concurrency. Handle rate limits, competing updates, cancellation and cleanup across tool calls.

BuildRun competing jobs against a rate-limited fake provider and cancel one while it is waiting.

ProveThe cancelled run performs no new side effects; active work stays within configured concurrency limits.

DesignUse timeouts, retry budgets, circuit breakers and fallback policies. Distinguish retryable transport errors from rejected business actions.

BuildInject provider timeouts and malformed tool results; fall back to a read-only evidence summary.

ProveThe system finishes within its deadline and never reports a failed action as successful.

DesignBuild a FastAPI boundary around durable runs. Design authentication, tenant isolation, secrets handling and versioned request/response contracts.

BuildContainerize the service with a separate worker and persistent database using Docker Compose.

ProveA clean checkout can start the system; health checks distinguish serving health from provider availability.

DesignCorrelate requests, runs and tool spans with OpenTelemetry. Define redaction, retention, cost attribution and actionable alerts.

BuildTrace a slow request through its components and remove synthetic sensitive values before export.

ProveLocate the bottleneck from one trace and verify that prohibited fields are absent from emitted telemetry.

DesignVersion prompts, models, tools and fixtures together. Build continuous integration and continuous delivery (CI/CD) checks, canary criteria and compatible rollback.

BuildRelease an intentionally regressed candidate into a local canary simulation, then revert it.

ProveThe gate rejects the regression, and rollback preserves existing run and approval state.

DesignIntegrate the incident-triage system: permissioned retrieval, bounded planning, durable execution, approval and observability.

BuildComplete one end-to-end scenario using synthetic incidents and a simulated change-management service.

ProveA reviewer can reproduce the scenario from the README and follow the evidence to the approved action.

DesignExercise the operating envelope, not just average latency. Test authorization, stale approvals, restarts, load and provider failure.

BuildRun the shared failure-injection suite and record task outcomes, tail latency, cost and unresolved risks.

ProveReport the tested envelope and limits honestly; failed checks become documented fixes or release blockers.

DesignDefend requirements, alternatives, failure boundaries and cost decisions. Explain how the system should evolve under changed constraints.

BuildPresent the capstone, respond to a new scale or privacy requirement and hand over the operating runbook.

ProveDeliver evidence for each rubric dimension and distinguish demonstrated capability from future work.