Chapter 5: Setting Up Your Agent Stack

1. Prompt Engineering & Template Systems

Prompting is the primary control surface of an agent. Production systems separate system prompts (role, constraints) from task prompts (objectives, context).

system_prompt:
  role: "planner"
  invariants:
    - cost_budget
    - tool_scope
    - safety_policy

task_prompt:
  objective: ${user_goal}
  context: ${retrieved_memory}
  expected_output: plan

Key Patterns:

• Declarative Schemas: Treat prompts as structured, versioned artifacts
• Pattern-Based Prompting: Reusable templates for planner, critic, router, executor
• System vs Task Separation: Stable role definitions vs dynamic context

2. LLM Model Selection

Agent stacks benefit from intentional heterogeneity rather than reliance on a single model.

if task.type == "classification":
    model = lightweight_model
elif task.requires_reasoning:
    model = generalist_llm
else:
    model = fallback_model

Model Routing Patterns:

• Generalist Models: Handle reasoning, planning, summarization
• Specialist Models: Excel at classification, extraction, code generation
• Dynamic Routing: Based on task intent, latency budget, cost constraints

3. Memory Architecture

Memory enables agents to improve over time. Production systems use layered memory with distinct lifecycles:

Tri-Level Memory Design

• Episodic Memory: Concrete events, interactions, execution outcomes (write-heavy)
• Semantic Memory: Stabilized knowledge, facts, documents, embeddings (retrieval-optimized)
• Procedural Memory: Executable know-how, skills, workflows (callable modules)

episodic.log(event, outcome)
summary = summarize(episodic_window)
semantic.store(summary)

if plan.requires_skill:
    procedural.invoke(skill_id)

4. Vector & Graph Databases

Hybrid retrieval patterns balance semantic similarity with policy constraints:

# Step 1: Vector-based candidate retrieval
candidates = vector_db.search(
    query_embedding,
    top_k=50
)

# Step 2: Graph-based constraint filtering
filtered = graph_db.filter(
    nodes=candidates,
    constraints={
        "jurisdiction": "EU",
        "document_status": "approved",
        "policy_version": ">= v3"
    }
)

# Step 3: Ranked handoff to agent
results = rank(filtered)
return results

Key Considerations:

• Metadata-Driven Routing: Domain scoping, versioning, trust boundaries
• Consistency & Freshness: Time-aware metadata and controlled reindexing
• Hybrid Search: Combine keyword + vector + graph constraints

5. Orchestration Runtimes

Modern orchestration defines how agent logic is composed, executed, and observed:

node(plan)
node(retrieve)
node(execute)
node(evaluate)

plan -> retrieve
retrieve -> execute
execute -> evaluate
evaluate -> execute  # retry loop

Key Frameworks:

• LangChain: Chain-based composition, standardized abstractions
• CrewAI: Role-based collaboration and delegation
• AutoGen: Conversational agent coordination
• LangGraph: Graph-based control flow with branching and state

6. Tool Use, Function Calling & MCP

Federated tool servers with explicit contracts enable scalable, governed tool integration:

• Independent Deployment: Tools versioned and deployed separately
• Centralized Authentication: Credential management via service gateways
• Policy Enforcement: Controls through existing infrastructure
• Clear Ownership: Lifecycle management and responsibility