SupportPilot — AI Engineering Companion System
A reproducible AI-engineering reference system built across eight maturity stages: risk/product contracts, retrieval, evaluation, provenance, approvals, observability, security and progressive delivery without provider lock-in.
- Role
- AI / Agent Engineering
- Status
- Research
SupportPilot is the applied AI-engineering system behind the book. It is intentionally not presented as a magic assistant: it is a reference architecture for turning probabilistic model output into controlled, testable software behavior.
The core lesson is simple and useful in production: the model proposes, the system grants authority.
- ▸Make AI behavior reproducible without depending on paid provider APIs
- ▸Separate probabilistic model output from system authority
- ▸Track evidence and provenance for generated answers
- ▸Define evaluation gates before releases
- ▸Model risk, approval and escalation paths explicitly
- ▸Keep examples small enough to study, but realistic enough to transfer
- ▸Educational reference system, not a SaaS product
- ▸No hidden API keys or vendor-specific assumptions required for the core learning path
- ▸Examples must remain deterministic enough for readers and reviewers to reproduce
- ▸Reproducibility vs. live-model excitement: deterministic paths make the system easier to verify.
- ▸Architecture density vs. beginner approachability: staged V0-V7 increments keep the full system understandable.
- ▸Explicit approvals vs. automation speed: authority stays with the system and operator, not the model.
- ▸Risk contracts define which actions require approval
- ▸Tool boundaries separate recommendation from authority
- ▸Provenance and evidence paths reduce untraceable generated claims
- ▸Evaluation gates catch regressions before release
- ▸Deterministic fixtures stabilize examples across machines and time
- ▸Progressive delivery keeps model/system changes reversible
- ▸Machine-checkable gates for staged examples
- ▸Evaluation scenarios for answer quality, provenance and escalation
- ▸Failure cases documented as part of the system design
- ▸Reference system designed to run without required provider keys
- ▸Progressive delivery pattern described for moving from demo to production
- ▸Track prompts, evidence, decisions, approvals and release-gate outcomes
- ▸Measure AI behavior as system telemetry, not vibes
- ▸Published as the practical companion system for an 18-chapter AI-engineering book
- ▸Connects product risk, architecture, evaluation, operations and documentation into one teachable system
- ▸Gives the portfolio a concrete AI/Agent Engineering proof without pretending it is a production SaaS
- ▸Research/book artifact: Become an AI Engineer — 18 chapters, 5 parts, SupportPilot V0-V7
- ▸Book frontmatter documents reproducible fixtures and no required provider API keys
- ▸Related research: Graph Engineering for AI Agent Systems
- ▸The model is not the system; the system decides what the model is allowed to influence.
- ▸Evaluation and provenance are product features when AI output becomes user-facing.
problem
Most AI demos optimize for impressive output, not for accountability. Production AI systems need contracts, evidence, provenance, evaluation, approval paths and release gates before a model is allowed to affect users or business state.
context
SupportPilot is the companion system behind the book 'Become an AI Engineer'. It turns AI-engineering principles into a staged reference architecture from V0 to V7: each stage adds a production concern such as retrieval, evaluation, security, observability or delivery control.
requirements
constraints
architecture
SupportPilot is organized as staged system maturity: product/risk contract first, then deterministic fixtures, retrieval and provenance, evaluation gates, tool/agent boundaries, human approval, observability, security and progressive delivery. The architecture treats the model as one component inside a larger controlled system.
key decisions
Why: Agent behavior only becomes operable when success, failure, authority and escalation are defined outside the model prompt.
Slower than building a flashy chat demo, but produces a system that can be reviewed, tested and evolved.
Why: Readers and reviewers need repeatable behavior; provider drift would make examples harder to trust.
Less impressive than live-model demos, but much better for verification, CI and teaching.
Why: A useful AI answer must carry where it came from, what was used and what remains uncertain.
Adds storage and UI complexity, but prevents untraceable claims from looking authoritative.