The agent is the easy part — Engineering @Sidetrade

Model, prompt and tools are enough for a demo. Production needs more.

flowchart LR
  A[Customer data] --> B[Agent] --> C[Tool] --> D[Action]
  classDef agent fill:#E8F4FB,stroke:#4B9AEF,color:#0C1B45
  class B agent

fig. 1 — The demo version: a straight line from data to action.

A collections agent is easy to imagine. Give a model access to the right customer data. Let it inspect overdue invoices. Connect it to email, CRM and ERP tools. Ask it to recommend the next best action. In a demo, that can look complete. An invoice is 30 days overdue. There is no dispute. The customer has not responded. The agent drafts a reminder, records the activity and moves on.

It feels like the difficult part has been solved. Far from it. The difficult part begins when that same agent is expected to operate across thousands of real customer accounts, each with different histories, policies, risk levels, contractual relationships and human stakeholders. That is where the agent harness starts to matter.

The demo version is deceptively simple

The mental model for an AI agent often looks something like this. Customer data feeds the agent, the agent calls a tool, the tool produces an action. That is enough to prove the model can reason, call tools and produce a useful result. Enterprise collections is dense with constraints: disputes, holds, approval thresholds, jurisdictions.

Collections is full of exceptions

Consider a $75,000 invoice that is 45 days overdue. At first glance, the next action seems obvious: contact the customer.

But what if the invoice is already in dispute? What if the dispute relates to a service failure, the account is strategically important, there is an active promise to pay, or the customer was contacted yesterday? What if the account manager has requested a temporary hold, the customer sits in a jurisdiction with different communication requirements, a payment plan can be offered but only within an approved threshold, or the ERP says the invoice is overdue while the CRM notes that payment is already in process?

The model can still reason about this. But the enterprise cannot rely on the model to invent the rules of engagement as it goes.

That is the distinction between an impressive agent and a production system.

The model is only one part of the system

The agent may decide what it thinks should happen next. The surrounding system has to decide what the agent may know, what it may do, whether a human must approve, and how the decision is recorded and later evaluated. That surrounding architecture is what I mean by the agent harness. The agent sits inside the harness. The harness is everything around it.

flowchart LR
  IN[Customer / collector / event]
  subgraph HARNESS[AGENT HARNESS]
    direction LR
    CTX[Context] --> AG[Agent]
    ID[Identity] --> AG
    POL[Policy] --> AG
    AG --> OBS[Observability] --> EVAL[Evaluation]
    AG --> ST[State]
    AG --> TOOLS[Tools]
  end
  IN --> CTX
  TOOLS --> SYS[ERP / CRM / Email / Payments]
  classDef agent fill:#ffe9e0,stroke:#ff5a28,color:#0C1B45
  classDef outside fill:#ffffff,stroke:#0C1B45,color:#0C1B45
  class AG agent
  class IN,SYS outside
  style HARNESS fill:#fff3ee,stroke:#ff5a28,stroke-dasharray:4 3

fig. 2 — The agent sits inside the harness: context, identity and policy feed it; it acts through tools and state; observability closes the loop into evaluation.

Intelligence and control are different jobs

This distinction matters because enterprise agents are not supposed to make every decision probabilistically. Some things benefit from judgment. Others should remain deterministic.

A collections agent might use AI to interpret a customer's reply, classify the reason for non-payment, summarize an account history, choose the most relevant playbook, recommend a next action, or adapt the tone of a communication. But it should not be left to probabilistic judgment to decide whether a user is authorized to access an account, whether a $100,000 write-off is permitted, whether a mandatory compliance step can be skipped, whether a payment plan exceeds an agreed threshold, or whether a restricted field can be sent to an external recipient.

Enterprise agents are probabilistic systems operating inside deterministic boundaries.

A real collections flow looks more like this

A production flow is a sequence of checks:

%%{init: {'flowchart': {'nodeSpacing': 18, 'rankSpacing': 26, 'padding': 8, 'wrappingWidth': 140}}}%%
flowchart LR
  subgraph P1[Prepare]
    direction TB
    A["Overdue account<br/>enters queue"] --> B["Assemble<br/>account context"] --> C{"Open<br/>dispute?"}
    C -->|Yes| D["Select collections<br/>playbook"]
    C -->|No| D
  end
  subgraph P2[Decide and act]
    direction TB
    E{"Action<br/>permitted?"} -->|Yes| F["Agent recommends<br/>next action"] --> G{"Escalation<br/>required?"}
    E -->|No| X["Blocked: nothing<br/>executes"]
    G -->|Yes| H["Escalate:<br/>suspend + resume"] --> I["Execute via<br/>scoped tool access"]
    G -->|No| I
  end
  subgraph P3[Record and learn]
    direction TB
    J["Record state +<br/>audit trail"] --> K["Observe<br/>outcome"] --> L["Feed scenario back<br/>into evaluation"]
  end
  P1 --> P2
  P2 --> P3
  classDef agent fill:#ffe9e0,stroke:#ff5a28,color:#0C1B45
  class F agent
  classDef phase fill:#ffffff,stroke:#BFC7D5,color:#8E97AE
  class P1,P2,P3 phase

fig. 3 — A production collections flow: context, a permission check, an escalation branch, and a feedback loop into evaluation.

Notice how little of this is the model. The agent matters, but the reliability of the system comes from everything around it.

One step deserves a second look.

EscalationThe run itself pauses mid-task, with everything it has already worked out still intact, and resumes from that exact point once a human decides. There is no separate system to keep in sync, and no state to reconstruct after the fact.

Context is part of the harness

Even the apparently simple question of what the agent should know becomes architectural. A collections agent may need a dozen sources at once. ERP and CRM records, recent contact, the dispute file, and the account's policies and playbooks.

Simply exposing all of that information to the model is not a strategy. The system needs to assemble the right context for the current decision. Too little context and the agent makes a poor decision. Too much context and you create cost, latency, noise and potentially unnecessary data exposure.

This is why retrieval-augmented generation is only a small part of the enterprise context problem.

Policy is part of the harness

The collections function already has rules. The agent does not remove the need for them. If anything, greater autonomy makes them more important. Rules come from several places, such as regulation, internal risk policy, customer segmentation, an SOP, or a playbook that guides judgment rather than enforcing a hard rule.

Those concepts are different, and the harness needs to understand that difference.

A policy

Payment plans above $50,000 require approval. Enforced by the runtime itself, deterministic, checked before the action executes.

An SOP

Verify the account, inspect open invoices, check disputes, review recent contact, then decide. Guidance the agent consults, not a rule the system enforces.

A playbook

For strategic customers with service-related disputes, prioritize resolution and account-team alignment before escalating. Judgment, not a rule.

Evaluation is part of the harness too

How do we know the agent is good?

The question is not whether the email sounded convincing or the tool call was valid. It is whether the system behaved correctly. For the $75,000 disputed invoice, we might want to evaluate whether the agent identified the active dispute, used the correct playbook, avoided an inappropriate collections escalation, involved the right stakeholder, respected the approval boundary and recorded the decision correctly.

Some of those are deterministic. Did it respect the approval boundary? Was the decision recorded? Those can be checked without ever calling a model again. Others are a judgment call. Was it the right playbook? Was the escalation appropriate? Those need a model, or a person, to grade them. Treating both as the same kind of test is how evaluation becomes theater.

These are questions about business behavior. And if the model, prompt, tool, knowledge source or policy changes tomorrow, we need to know whether that behavior has regressed. This is why evaluation cannot be something added after the agent has been built. It is part of the architecture.

The shift is from agents to agent systems

The first generation of enterprise AI focused on getting models to do useful things.

The next challenge is making those things safe, repeatable and operable at scale.

Agent

The model that reasons about what to do.

Specification

What the agent is actually supposed to do, written down and testable.

Evaluation

Whether the behavior was correct, checked before it ships and after.

Policy

The deterministic boundaries the agent cannot reason its way around.

Identity

Who or what is acting, and on whose authority.

Authorization

What that identity is actually allowed to do.

Context

The right information for the decision in front of it, no more.

Harness

All of it, working together, is what makes an agent usable in the enterprise.

The model is the visible part of the agent. The harness is what makes it usable in the enterprise.

Next: enterprise agents need specifications. Part 1 of 15

written by

Luke Hennerley

VP of AI Operations at Sidetrade. Responsible for how Sidetrade runs enterprise AI agents, in our own operations and our customers', including the industry's first AI Cash Collection Agent.