AI agents for tradespeople: what works (and what doesn't) in production

The first AI agent we built for a tradespeople platform had a beauty demo: ask a question, get a perfect answer right away, founder happy, investor happy. Three weeks later in production: 40% of questions got a generic "as a layperson I'd call an expert" answer. The LLM knew nothing about the company's own knowledge base, nothing about CAO rules, nothing about prices. Production is something different than a demo.

Lesson one: RAG on domain data is not optional. For tradespeople the chatbot only works if it answers the way the industry works, including regional differences, supplier names and the quirks of CAO interpretations. We always run an embedding layer (OpenAI text-embedding-3 or Cohere embed v3) over the client's own knowledge base. That alone moves the success rate from 40% to ~80%.

Lesson two: agents with tools beat plain RAG. Pulling out raw text is the baseline. The real difference comes when the agent is allowed to call functions: "check availability in the calendar", "fetch the quote template", "forward this to office@". Claude Sonnet 4.7 with tool-use plus `computer_use` for fieldwork cases is one of the most productive combos we know.

Lesson three: cost-per-interaction wins or loses the project. One tradesperson asks the agent 12 questions a day on average. 300 tradespeople × 12 × 30 days = 108,000 requests per month. If each request burns 4k tokens of output on Claude Sonnet at € 3 per million input / € 15 per million output, that's fast money. We cache aggressively (prompt-caching, 1h TTL) and route to Haiku for simple questions. A factor of 3-5× cheaper than "everything on Sonnet".

Lesson four: escape the demo syndrome with boring metrics. Success rate is not enough. Also measure "answer without the user having to rephrase", "answer that leads to action" and "answer that doesn't end in a fallback". Those three count more than top-line accuracy. This is where Tradevo-style platforms live or die.

Lesson five: let the client own the knowledge base. Every time we built the "agent" for a client and kept the knowledge on our infrastructure, migration ended up awkward. Ever since we put the knowledge base and embeddings in the client's own Supabase or Postgres+pgvector, it's transferable. Agent logic stays with us, data stays with them. Boring but crucial.

How do you start? A three-week PoC on one question category, RAG on one dataset, one tool. If the conversion rate from the PoC is above 70%: scale. If not: you need a different idea. After that, expand iteratively with more tools and more industry knowledge. No big-bang release.