The short version: SMBs with no dedicated technical bench should hire. Mid-market companies with a developer + ops person + product manager who can commit 30+ hours per week to AI should build. Most successful programs are actually hybrid, agency ships the first 2-4 playbooks while training an internal owner.
When in-house wins
- You already have a developer + an ops person + a product manager who can dedicate 30+ hours per week combined
- Your AI use case is unusual enough that no agency has shipped your exact thing before
- You're building IP that's a competitive differentiator (proprietary models, novel architectures)
- You're an enterprise with multi-year AI roadmap and a CTO who wants AI native
When an agency wins
- You're SMB-sized and don't have the bench to commit 30+ hours per week for 90 days
- Your AI use case is something agencies have shipped many times (receptionist, missed-call text-back, lead qualifier, scheduling, review automation)
- You want speed-to-live (14 to 90 days) without paying the in-house learning curve
- You want someone whose full-time job is staying current on AI platforms (the space changes weekly)
- You'd rather pay $200 to $1,500 a month for a working system than $120,000 a year for an ML hire
The cost math
| Agency (Mastodon) | In-house build | |
|---|---|---|
| Setup (per playbook) | $3K, $15K | $15K, $50K of internal time |
| Time to live | 14 to 30 days | 60 to 180 days |
| Monthly cost | $200, $1.5K (platform + tuning) | $100, $800 platform + internal-owner time |
| First hire to enable in-house | None | $120K+ ML or full-stack engineer |
| Risk of pilot stalling | Low (agency has shipped 50+ before) | High (first-build learning curve) |
| Risk of post-launch drift | Low (agency owns tuning) | High (internal owner gets pulled to other work) |
| IP ownership | You own it; agency owns the playbook | You own everything |
| Speed to next use case | 2 to 4 weeks (next playbook) | Months (re-scope, re-learn) |
Hybrid as a third option
Most successful SMB AI programs are hybrid: agency does the first 2 to 4 playbooks while training an internal owner. After 12 months the internal owner runs the routine work and the agency handles new platforms, complex integrations, and quarterly tuning. This is what we recommend most often.
Questions to ask either way
- Who specifically owns this system at day 91?
- What's our written escalation path when the AI gets it wrong?
- How do we measure whether this worked, in dollars or hours?
- What does a vendor (or internal team) handoff look like if priorities change?
- Where does the data live? Who controls it?