AI / Automation
Case 06 / 13
What 3,000 outbound emails taught me about list quality
An agentic outbound workflow that researches companies, finds contacts, writes the email, and follows up on its own — and what measuring it honestly showed about list quality.
Crave Robotics · 2026 · Sole designer and builder
- An agent researches target companies, identifies contacts, writes the conversation points, and sends and follows up.
- 3,000+ emails sent, 3.1% reply rate.
- Measuring real conversations, not just replies, showed the bottleneck was list quality — fabricated domains and wrong roles — not the copy.
- Including the diagnosis because it is the transferable part.
The numbers
- 3,000+Outbound emails sent
- 3.1%Reply rate
- ~1.5%True conversation rate, first 2,000 sends
- Apollo.io
- LLM APIs
- Resend
- Gmail API
- Python
- Railway
Run a list through the pipeline
Interactive — synthetic dataToggle the checks below and watch which rows survive to a send.
| Company | Domain | Contact | Outcome |
|---|---|---|---|
| Harbourline Foods | harbourline.example | Facilities Lead | Deliverable |
| Verity Campus Group | veritycampus.example | Ops Director | Dead DNS record |
| Northgate Union | northgate-union.example | Auxiliary Services | Deliverable |
| Piedmont Health | piedmonthealth.example | VP Marketing | Wrong role at company |
| Ashford Labs | ashfordlabs-inc.example | Workplace Manager | Fabricated domain |
| Cobalt Works | cobaltworks.example | Office Manager | Deliverable |
| Ridgeway Institute | ridgeway-edu.example | Dining Services | Dead DNS record |
| Lantern Manufacturing | lanternmfg.example | Plant Ops | Deliverable |
- Emails sent
- 8
- Reached intended recipient
- 4
- Wasted sends
- 4
Validation does not fix the wrong-role sends. Those need a defined ICP and qualification criteria, which is the upstream half of the same problem.
Synthetic lead list. Company names and domains are fictional. The failure modes shown are the ones the real diagnostic surfaced.
How it fits together
- 1Company research
- 2Contact identification
- 3Validation (added later)
- 4LLM conversation points
- 5Send and follow-up
- 6Engagement capture
- 7Reply classification
Read the full case study Hide the case study
The problem
Field sales needed outbound pipeline for new machine placements and had no systematic prospecting motion — manual list-building and one-off emails.
Why it mattered
Machine placement drives revenue directly. A repeatable outbound motion was worth building, and it was a natural place to apply LLMs to a real commercial workflow rather than a demo.
What I built
An agentic outbound workflow. It researches target companies, identifies the relevant contacts through Apollo.io, has an LLM draft personalized conversation points, and then sends and follows up through Resend and the Gmail API, with engagement captured at each step. It runs as a deployed service with scheduled execution.
Then I measured it properly. Across 3,000+ emails the reply rate is 3.1%. But replies are not conversations, and when I classified the first 2,000 sends by whether a real person engaged, the true conversation rate was about 1.5% — and the conversations that did happen mostly stayed in email and died before reaching a meeting.
What actually went wrong
The instrumentation is the reason this page exists. The headline reply rate looked survivable; the conversation rate was half of it, and the gap between them was the finding. Tracing it back, the enriched lists contained fabricated domains and dead DNS records, so a meaningful share of sends were reaching unintended recipients or nobody at all. Others reached real people in the wrong role at the right company.
The constraint was never message generation. LLM personalization worked fine — the copy was good. The failure sat one step upstream, in list quality and ICP definition, and no amount of generation quality compensates for sending to the wrong person.
I am including this project rather than quietly dropping it because the diagnosis is the transferable part. The demo below lets you add the validation gate that was missing and watch what it catches — and, more usefully, what it does not.
What I learned
Instrumenting for the real success metric mattered more than anything else I built. A reply-rate number alone looks healthy; measuring genuine conversations showed how much of it was not. I also learned that AI-generated volume amplifies bad inputs rather than compensating for them — a thousand well-written emails to poorly-qualified contacts is worse than a hundred to good ones, because it spends domain reputation to do it.
What I would do next
Domain and deliverability validation as a hard gate before send. A tightly-defined ICP with qualification criteria instead of broad sourcing. Sequences designed to reach a booked meeting rather than a reply. Volume traded down for list quality.
Described without confidential information. The demo and any figures inside it use synthetic data.